Professional World / Build / active

Local Computer Use Agent

Local Computer Use Agent

A supervised Windows desktop automation agent with OCR, template matching, structured logs, local/OpenAI routing, and an operator-facing GUI.

Problem

Desktop automation usually breaks at the exact moment operators need clarity, especially when a workflow mixes visual recognition, recovery logic, and human review.

Why it matters

This project shows a real operator-first automation philosophy: observable state, recovery ladders, logging, and controlled execution instead of blind automation theater.

System description

The agent captures screens, runs OCR and template logic, classifies the visible state, and executes bounded actions while keeping the operator in the loop through logs, dry-run modes, browser support, and supervised fallbacks.

Operating context

Built out of the ce_automator workspace, this system has to survive messy real Windows state, not ideal screenshots. OCR, template matching, logs, and recovery paths stay visible in the dossier because supervised control is the point of the build.

Tools / methods

PythonTkinterOCRTemplate matchingStructured runtime logsLocal/OpenAI routingOperator review gates

Constraints

  • Needs to recover from visual ambiguity without pretending certainty.
  • Has to stay supervised and auditable.
  • Must tolerate real Windows UI drift and timing issues.

Workspace source

Windows desktop computer-use agent with OCR, template matching, local/OpenAI model routing, structured logs, and a Tkinter operator GUI.

Source / ce_automator/README.md

Workspace / ce_automator

Detected files / 10

.env.env.exampleaction_controller.pyagent_runtime.pyautomation_controller.pybrowser_controller.pybrowser_llm_handler.pybrowser_support_window.pychoice_matcher.pyconfig.py

Cross-links