# vercel-labs/agent-desktop

> agent-desktop is a command-line tool and C library that lets AI agents see and operate desktop apps on macOS, Windows, and Linux. It reads the accessibility tree of an app and returns it as JSON, with a ref for each element the agent can act on, so no screenshots or pixel matching are needed. The agent's own model decides each step, since agent-desktop has no model inside. The core is written in Rust.

Source: https://github.com/vercel-labs/agent-desktop
Indexed: commit 8e8dd7c on 2026-11-11 by Repowise. Facts come from the source and git history, not from the maintainer.

## Facts

- Type: Command-line tool
- Language: Rust
- Stars: 23,619 at index time
- License: Apache-2.0
- Code health: 9.3/10, Excellent
- Last commit: 2026-11-09
- Contributors: 6

## Install

From the README: https://github.com/vercel-labs/agent-desktop#install

Install the package globally with npm, which includes native binaries and platform helpers.
    npm install -g agent-desktop

Check the platform, permissions and supported surfaces.
    agent-desktop status

On macOS, walk through the Accessibility permission grant.
    agent-desktop permissions --guide

Take an interactive snapshot of Finder, which lists each element with a ref.
    agent-desktop snapshot --app Finder -i

## How it works

Input enters through the agent-desktop command-line tool in src, which the same binary can also run as an MCP server when started with the mcp subcommand. Both the CLI and the C library run the same core, found in crates/core, a Rust library behind the commands that click, type, read and wait on controls, with its entry point in crates/core/src/lib.rs. The crates/ffi crate exposes that core as a C ABI shared library, so other programs can run commands in process instead of starting the CLI for each command. Each binary is built for one operating system and talks to that system's accessibility API, and crates/linux reads and acts on Linux applications through AT-SPI, the accessibility bus.

- crates/core/src/lib.rs: entry point
- crates/ffi/src/convert/app.rs: entry point
- crates/ffi/src/lib.rs: entry point
- crates/linux/src/bin/agent-desktop-linux-clipboard-owner.rs: entry point
- crates/linux/src/lib.rs: entry point
Modules: crates, tests, src, probes, scripts, apps, npm, .githooks. 3,449 files, 367,581 lines, 37,087 symbols.

## Code health

- Score: 9.3/10 (Excellent)
- Trend: 9.3 on 2026-11-07, 9.3 on 2026-11-11
- Rank: #4 of 4,000+ indexed repos
- Hotspots: crates/core/src/snapshot/mod.rs, crates/ffi/include/agent_desktop.h, crates/macos/src/tree/resolve/mod.rs, crates/macos/src/actions/dispatch.rs, src/dispatch/mod.rs
- Lowest file: probes/windows/26-shell-surfaces/shell-probe.cs
- Dead code: 8 findings

## Maintenance

- Last commit 2026-11-09, first commit 2026-02-19, 304 commits
- 6 contributors, bus factor 1

## Security

- Security findings are shown to signed-in users at https://repowise.dev/repo/vercel-labs/agent-desktop/security

## More

- Page: https://repowise.dev/repo/vercel-labs/agent-desktop
- Ask this codebase: https://repowise.dev/repo/vercel-labs/agent-desktop/chat
- Docs: https://repowise.dev/repo/vercel-labs/agent-desktop/docs
- MCP (signs in with Repowise on first use):
    claude mcp add --transport http repowise https://api.repowise.dev/mcp/vercel-labs/agent-desktop