Upgrades `agent-client-protocol` from 3.6.0 (schema 1.13.0) to 5.0.0 (schema 1.15.1), and brings Zed's v1 and v2 ACP clients in line with the spec and schema changes made since then. ## SDK 5 port - Schema types move from `schema::v1`/`schema::v2` to `v1::schema`/`v2::schema`, builders to `Client.v1()`, and v2 handle types to `agent_client_protocol::v2`. - `agent_client_protocol::Error` is now the SDK's own type, not the v1 schema error. Zed uses it at the SDK boundary, and `ProtocolErrorInfo` also recognizes it so auth-required detection keeps working. - Foreground dispatch helpers are generic over the protocol marker, and v2 request IDs come straight from the handles. - SDK 5 only matches transport-closed errors that the SDK itself produced, so peers can't forge them. A test that built one by hand now covers only the local disconnect case. The `agent_servers` tests still cover real transport loss. ## Spec changes - `session/close` and `session/delete`: a `-32002` response counts as success, on both v1 and v2. - Permission and elicitation requests that name an unknown session get `-32002` with `data.sessionId` instead of `-32603`. The v2 migration guide makes this required for Clients too. - v2 `usage_update.cost` is a patch field: omitted leaves it unchanged, `null` clears it. A v1 update without a cost still leaves it unchanged. - A `plan_update` whose type Zed can't render (`file`, `markdown`, or an unknown type) still replaces the plan with that `planId`, so the old entries stop showing. - A diff `patch` with an unrecognized `format` renders from `changes`, as if `patch` were omitted. - v2 never sends an empty prompt. - On v2, `session/cancel` no longer answers pending permission requests locally. They stay open until the agent cancels them with `$/cancel_request`, because requests for background work survive a cancel. v1, including v1 subagents, keeps answering them with `cancelled`. ## Follow-ups These are left for separate PRs: - Background-work UI: the `paused` status, the `background` list, and cancel buttons for tool calls and terminals. Until then, `paused` displays as in progress, which the spec allows. - Subagent `cwd` and `additionalDirectories`. - The new unstable methods: `auth/status`, `session/set_title`, `session/status`, `session/archive`. Release Notes: - Improved compatibility with the latest Agent Client Protocol, including handling of missing sessions, plan updates, diff formats, and cancellation of permission requests.
eval-cli
Headless Rust binary for running Zed's agent in evaluation and benchmark environments. It is designed for containerized harnesses such as Harbor and Pier, where the repository is already checked out and model API keys are provided via environment variables.
eval-cli uses the same NativeAgent + AcpThread pipeline as the production
Zed editor: a full agentic loop with tool calls, subagents, and retries, without
a GUI.
This directory also contains zed_eval/, the Python zed-eval package used to
build this binary, launch remote benchmark runs on Modal/Harbor/Pier, and fetch
results. For normal benchmark orchestration, start with
zed_eval/README.md.
Building
Native, for local testing on the same OS
cargo build --release -p eval_cli
Linux x86_64, for Harbor/Pier sandboxes
Harbor and Pier containers run Linux x86_64. From the repository root, use the Docker-based build script:
crates/eval_cli/script/build-linux
This produces target/eval-cli, an x86_64 Linux ELF binary. You can also
specify a custom output path:
crates/eval_cli/script/build-linux --output ~/bin/eval-cli-linux
Standalone usage
eval-cli \
--workdir /testbed \
--model anthropic/claude-sonnet-4-6 \
--instruction "Fix the bug described in..." \
--timeout 600 \
--output-dir /logs/agent
eval-cli reads provider API keys from environment variables such as
ANTHROPIC_API_KEY and OPENAI_API_KEY. It writes result.json, thread.md,
and thread.json to the output directory.
Exit codes
| Code | Meaning |
|---|---|
| 0 | Agent finished |
| 1 | Error, such as model/auth/runtime failure |
| 2 | Timeout |
| 3 | Interrupted by SIGTERM or SIGINT |
Running benchmarks
Most benchmark runs should use the Python zed-eval CLI instead of invoking
eval-cli directly. From the repository root:
crates/eval_cli/script/install-zed-eval
zed-eval doctor --create-volume
zed-eval run rf --from local --n-tasks 2
For one-off source runs without installing the tool globally, use
crates/eval_cli/script/zed-eval <args>.
See zed_eval/README.md for supported benchmarks, remote
builds, Modal setup, reporting, rejudging, baselines, and Harbor/Pier installed
agent usage.