repowiserepowise

Command-line tool

vercel-labs / agent-desktop

Computer use CLI for AI agents

★ 23,562stars on GitHubRustApache-2.0
Install agent-desktopView on GitHub★ 23.6kSee this for your own repoAsk this codebase

Repowise maps public repos from their source. Not affiliated with vercel-labs. What is Repowise?

github.com/vercel-labs/agent-desktopIndexed at 8e8dd7c

What is agent-desktop?

agent-desktop is a command-line tool and C library that lets AI agents see and operate desktop apps on macOS, Windows, and Linux. It reads the accessibility tree of an app and returns it as JSON, with a ref for each element the agent can act on, so no screenshots or pixel matching are needed. The agent's own model decides each step, since agent-desktop has no model inside. The core is written in Rust. To install it, run npm install -g agent-desktop and follow the steps below. Its code health is 9.3 out of 10 (Excellent), ranked #4 of 4,000+ indexed repos. The last commit to agent-desktop was on Nov 9, 2026, and 6 people have contributed.

As of commit 8e8dd7c, indexed Nov 11, 2026. What it is comes from a model reading the README; the rest is measured from the source and git history, not from the maintainer.

For agentsagent-desktop.md

Repo facts: type, license, health, rank, contributors
agent-desktop facts
TypeCommand-line tool
LicenseApache-2.0
Code health9.3 / 10 (Excellent)
Health rank#4 of 4,000+ indexed repos
Last commitNov 9, 2026
Contributors6
Bus factor1
Size3,449 files, 367,581 lines

Install agent-desktop

Commands copied from the README

  1. 1Install the package globally with npm, which includes native binaries and platform helpers.
    npm install -g agent-desktop
  2. 2Check the platform, permissions and supported surfaces.
    agent-desktop status
  3. 3On macOS, walk through the Accessibility permission grant.
    agent-desktop permissions --guide
  4. 4Take an interactive snapshot of Finder, which lists each element with a ref.
    agent-desktop snapshot --app Finder -i

Releases on GitHub (latest v1.3.1, Nov 13, 2026)

Full install notes on GitHub

Explore agent-desktop

Ask about this code in your agent

This connects Repowise to agent-desktop's code. It does not install agent-desktop. To install agent-desktop, use the install steps.

Useful if you want to change or fork agent-desktop: your agent can ask how it is put together and get answers that cite its files.

Show the connect command (Claude Code, Cursor, Codex, VS Code)

Run it in a terminal, then type /mcp in Claude Code and sign in with Repowise. No key to paste.

Docs
bash
claude mcp add --transport http repowise https://api.repowise.dev/mcp/vercel-labs/agent-desktop
Claude Code, Claude, Cursor and VS Code sign you in on first use, free. Codex and Windsurf need an API key: sign in to create one.Sign in for a key

This repo is public, so anyone with a free Repowise account can connect.

Try asking

  • get_risk(["src/main.rs"])
  • get_answer("how is agent-desktop put together?")
  • get_why("why is agent-desktop structured this way?")

Ask about agent-desktop

Answers come from agent-desktop's README and source, with the files they cite. Two free answers a day without an account.

Featured answer

Give me an overview of agent-desktop

agent-desktop is a command-line tool and C library that lets AI agents see and operate desktop apps on macOS, Windows, and Linux. It reads each app's accessibility tree and returns it as JSON, with a ref for each element the agent can act on, so no screenshots or pixel matching are needed. The agent's own model decides each step, since agent-desktop has no model inside. Input enters through the CLI in src, which can also run as an MCP server via the mcp subcommand, and both the CLI and the C library run the same Rust core in crates/core. crates/ffi exposes that core as a C ABI shared library, and crates/linux uses AT-SPI to act on Linux apps. To start reading, open crates/core/src/lib.rs, the entry point of the core.

Files per layer in agent-desktop6 layers.

Basis: Inferred · knowledge graph· as of 8e8dd7c

Want this for your own codebase? Map your repo free

About this page

Get a page like this for your own codebase

Repowise built this map of vercel-labs/agent-desktop from its source and git history. It parses every symbol, draws the dependency graph, scores each file's health from complexity, duplication, coverage and churn, finds hotspots and owners in the history, and writes documentation for every module. Your coding agent reads the same index over MCP, so its answers cite real files.

Free for 2 public repos, with a page like this one. No card. See how Repowise works

Prefer local? pip install repowise && repowise init indexes your own checkout, no account needed. Star Repowise on GitHub

vercel-labs/agent-desktop FAQ

What is agent-desktop?

agent-desktop is a command-line tool and C library that lets AI agents see and operate desktop apps on macOS, Windows, and Linux. It returns each app's accessibility tree as JSON, with refs for the elements the agent can act on.

How does agent-desktop work?

agent-desktop reads the accessibility tree of a desktop app and returns it as JSON, with a ref for each actionable element. The agent clicks or types on a ref, reads the UI again to check the result, and repeats, with its own model choosing each step.

How do I install agent-desktop?

agent-desktop installs from npm with npm install -g agent-desktop, and the package includes native binaries for x64 and ARM64. After installing, agent-desktop status shows the platform, permissions and supported surfaces. See the install steps

What language is agent-desktop written in?

agent-desktop's primary language is Rust. The repository also contains Python, shell, JavaScript, TypeScript and C# files.

How is the agent-desktop code organized?

agent-desktop keeps its Rust crates in crates, including core, ffi and linux, and its command-line tool in src. Tests live in tests, the documentation website in apps/docs, and helper scripts in scripts.

Who is agent-desktop for?

agent-desktop is for AI agents that need to see and operate desktop apps. An agent can take a snapshot of an app, then click or type on refs from that snapshot to act on its controls.

What license does agent-desktop use?

agent-desktop is released under the Apache-2.0 license.

Code health

9.3out of 10Excellent

Hotspots average 8.6, the place fixes pay off most.

Full health report

On the leaderboards:#1 of 21 Rust repos

Key features of agent-desktop

  • Reads the accessibility tree of a desktop app and returns it as JSON, with a ref for each element the agent can act on, without using screenshots or pixel matching.
  • Leaves the decision loop to the agent's own model, since agent-desktop has no model inside.
  • Runs ref actions headless by default, using the accessibility API without moving the pointer or taking focus, and allows physical input only with the global headed flag.
  • Serves every command as an MCP tool over stdio when started with the mcp subcommand.
  • Ships a C library, libagent_desktop_ffi, so a program can call agent-desktop in process.
  • Works on macOS, Windows, and Linux through each system's accessibility API.

How vercel-labs/agent-desktop works

Input enters through the agent-desktop command-line tool in src, which the same binary can also run as an MCP server when started with the mcp subcommand. Both the CLI and the C library run the same core, found in crates/core, a Rust library behind the commands that click, type, read and wait on controls, with its entry point in crates/core/src/lib.rs. The crates/ffi crate exposes that core as a C ABI shared library, so other programs can run commands in process instead of starting the CLI for each command. Each binary is built for one operating system and talks to that system's accessibility API, and crates/linux reads and acts on Linux applications through AT-SPI, the accessibility bus.

The codebase has 3,449 files and 37,087 symbols in 8 modules, led by Rust, Python and Shell. Code health is 9.3 out of 10, rated excellent.