Observe
Read within approved paths, inspect runtime status, and reason. No desktop mutation.
Local inference / Windows + browser
Computer automation you can inspect.
DjenisAiAgent reads the current interface, asks a model running on your machine for one bounded action, and checks the result before moving on. No provider key, remote-model fallback, or hidden runtime download.
01objective
02observe
03act
04verify
05finish
01 / System
See the state.
Choose one action.
Check the result.
The loop stays deliberately legible. Screenshots and accessibility data go only to the configured local endpoint; the model returns one structured function call; the host executes only what the active runtime and permission tier expose.
Architecture
A short path from intent to evidence.
- 01Observe
Capture the screen and a structured UI snapshot.
- 02Reason locally
Ask the pre-provisioned local model for exactly one declared function call.
- 03Act
Run a tool allowed by the runtime and operator policy.
- 04Verify
Feed the outcome into the next turn before claiming success.
02 / Guided demo
A complete turn,
without the black box.
This deterministic walkthrough runs entirely in your browser. It demonstrates the orchestration contract; it is not connected to a live desktop or model server.
Operator objective01 / 04
“Open Calculator and evaluate 12 × 8.”
The operator defines one bounded outcome. The task starts with no implied permission to write files or run shell commands.
Capability matrix
The runtime determines the tools.
CapabilityWindows nativeDocker / remote browser
Desktop UI automationWindows nativeSupportedDocker / remote browserUnavailable
Browser DOM toolsWindows nativeLocal debuggerDocker / remote browserRemote Selenium
Host screen captureWindows nativeSupportedDocker / remote browserUnavailable
Authenticated web consoleWindows nativeSupportedDocker / remote browserSupported
Local inferenceWindows nativeLoopback endpointDocker / remote browserInternal Ollama sidecar
03 / Security model
Useful by choice.
Restricted by default.
The default tier can inspect approved files and reason about the current state. Interaction and system control are separate opt-ins. In Docker, agent and model stay internal; a hardened loopback gateway exposes the console, while browser egress remains separate.
Interact
Use approved UI and browser controls without unlocking shell or file writes.
System
Shell, file writes, app launch, and saved screenshots require the system tier plus explicit dangerous-action confirmation.
04 / Install v0.3.0
One release.
One verified image.
The local web stack.
Current releasev0.3.0
The v0.3.0 web and remote-browser runtime is distributed as a container image through GHCR. There is no desktop installer or downloadable release archive. The canonical release guide provides the source-bound Compose files, immutable image digest, and explicit model-provisioning command.
- 01
Check the host. You need Docker Compose and enough local capacity for the model and its configured context window.
- 02
Follow the exact release guide. Fetch the Compose definition from its authorized source commit and select the verified GHCR digest.
- 03
Provision, then start. Pull the model through the one-time provisioning profile, launch the full stack, and wait for
/readybefore enabling interaction.
The walkthrough above explains the orchestration contract. Use the Compose stack for the authenticated web console and remote browser. For Windows-native desktop control, clone the exact release with git clone --branch v0.3.0 --depth 1 https://github.com/ejupi-djenis30/DjenisAiAgent.git, then follow its Windows quick start.