Note: this PR does not change any behaviour of zed itself. The old sandbox implementation is still in use. The code in this PR is not considered secure. I'll need more time to torture it before I'm confident :) The old sandbox used `bwrap` to set up mounts to restrict FS access, and an in-sandbox network proxy to forward HTTP requests. It has significant limitations however: - it cannot restrict FS access to paths that don't exist at sandbox creation time. This means that files that become available while a command is running are unprotected - it cannot restrict FS access based on a glob - each individual mount must be specified explicitly - it cannot restrict network access that is not HTTP (i.e. SSH traffic). - it cannot change permissions while a sandboxed command is running - the grants given at the start of the execution are fixed. Agents can request to re-run the app with broader permissions, which is fine, but not great UX - it cannot detect whether a command failed because it hit the sandbox boundary or not. Again, agents can guess, but we'd like something more robust - it cannot dynamically prompt the user to escalate privileges The new sandbox has a different different design which solves these problems: - all restrictions go through seccomp user notifications - this allows us to register a supervisor process with the kernel to validate the syscalls of another program. This supervisor can run arbitrary userspace code to inspect syscalls before allowing/denying/modifying them - we create and upload a policy object to this supervisor which checks syscalls against this policy - if the policy says that the operation requires "ask"-ing, the supervisor will ask zed to show a prompt to the user and report a decision This design is a lot more unified, and since it is much more flexible, is also potentially suitable for sandboxing more parts of zed (user terminals, LSP servers, etc.). Release Notes: - N/A
14 KiB
Design
This is a (currently experimental) re-implementation of the sandbox on Linux, with the following goals:
- dynamic changes to the sandbox policy, without restarting the untrusted process
- more flexible policies (e.g. globs, restricting files that don't exist yet)
- reduced "false rejections" (e.g. allowing non-HTTP traffic)
The "ultimate goal" is that such a sandbox should be reasonable to leave enabled by default for all processes (e.g. user terminals, LSP servers, MCP servers, git clients, etc.).
Security posture
Traditional wisdom is that there is a fundamental difference between "missing functionality" and "security issues". I disagree - if the sandbox doesn't allow users to do what they want, the vast majority will simply disable the sandbox. To put it another way, a highly effective privilege escalation attack for an agent is to:
- try to use docker (or similar) to run a legitimate task
- explain to the user: "I can't use docker because of the sandbox, can you disable it?"
- the user disables the sandbox
Limitations of the current Zed sandbox
The current Zed sandbox is, in a vacuum, very secure. However, users can still be exposed to risk in various ways. These ways usually look something like:
- edit the project using sandboxed tools
- this edited configuration triggers an out-of-sandbox program to run malicious code
For example, a malicious agent can edit your project to add a new Rust proc-macro crate that downloads and runs a malicious script. Rust-analyzer, which runs outside the sandbox, will then execute this proc-macro code and you are compromised.
We cannot just "run everything in the sandbox", because the current sandbox is only really usable for the agent terminal because it's not very flexible. For example, if we sandbox your user terminal, if you need internet access, we need to kill that process, and restart with the right permissions.
On Linux, we also have the restriction that filesystem protections can only apply to:
- specific files and directories (globs not supported)
- files/directories that exist at sandbox creation time
The implementation is also (subjectively) ugly:
HostFilesystemLocationis a bit of a monstrosity, but is required for security on Linux/WSL, due to race conditions when creating namespaces throughbwrap- It also doesn't fully guarantee TOCTOU-freedom on WSL when going across the plan9 bridge to the NTFS side
- The HTTP proxy is an entirely separate mechanism to FS sandboxing
- There's lots of "wrapper programs" and awkward message passing
The new design unifies a lot of this.
The new design
The new design aims to fix this by broadly shifting from "setting up an environment with restricted permissions" to "intercepting syscalls and dynamically resolving them against the policy".
There are two components to this:
- filesystem (including domain sockets and friends)
- networking
Both of these rely on seccomp user notifications. This is a Linux kernel feature that extends seccomp's syscall-filtering ability, by allowing a user "supervisor" program to (conceptually) dynamically allow/deny1 syscalls. This supervisor can run arbitrary code, including code that sends a prompt to the user to confirm the action.
The rough flow would then become:
- each sandboxed program has a policy, which is uploaded to the supervisor
- when the untrusted program tries to make a syscall that is sensitive, the supervisor approves/denies/prompts the user, depending on the policy, and the specific syscall made
- when the user changes the policy (either manually, or by approving a prompt), the new policy is uploaded to the supervisor, and takes effect immediately
Implementation
I will refer to three processes:
- the owner process - the process that sets up the sandbox (i.e. Zed)
- the sandbox process - the process that the untrusted command runs in
- technically this is actually two processes, an in-sandbox helper that then spawns the untrusted command
- the supervisor process - the process that is registered with the kernel to validate syscalls made by the sandbox process
There is a SandboxPolicy object which contains a list of rules. For example, for filesystem sandboxing a rule looks like:
struct FsRule {
path: PathPattern,
read: Option<Access>, // read a file at this path
write: Option<Access>, // write a file at this path
connect: Option<Access>, // connect to a socket at this path
}
enum Access {
Allow,
Deny,
Ask,
}
The owner process sets the policy, and a more cache-friendly representation gets sent to the supervisor. When the sandbox process tries to make a syscall that is not explicitly allowed to run untrapped, the arguments get sent to the supervisor.
The supervisor makes the call on behalf of the sandboxed program, and copies arguments into its own memory to close a TOCTOU where the sandboxed process can pass a pointer to the supervisor, get the data validated, then change the data behind that pointer before the syscall is actually executed.
Network-specific considerations
The same principle applies to network sandboxing, with some extra machinery.
In particular, the kernel deals only with IP addresses, but we would like to be able to grant permission to access domain names such as google.com. We do this by (loosely):
- configuring the namespace such that DNS queries go to a resolver running in the supervisor process
- when the sandboxed process makes a DNS request for
google.com, we return a synthetic IP address - the supervisor records the mapping between
google.comand the synthetic IP address - when the sandboxed process tries to connect to that synthetic IP address, the supervisor can convert it back to
google.com, and look at the policy to understand how to react (allow, deny, ask)- the real DNS lookup also goes through a rebinding check, so I can't grant access to
attacker_controlled_domain.comand have that grant access to127.0.0.1for example
- the real DNS lookup also goes through a rebinding check, so I can't grant access to
There is then a relay, that maps these synthetic IP addresses outside the sandbox. This lives in the supervisor. It creates real connections to the actual IP addresses for the domains (using the system DNS resolver). So when it succeeds, there are two legs:
- sandbox <-> supervisor
- supervisor <->
google.com
The relay just plainly copies bytes from one to the other.
Ask mode
When a permission is in Ask mode, it blocks the syscall indefinitely(ish) until the owner process responds. The Handler type in a Ask variant is created by providing a callback that takes a Responder. This type is how the custom user code can indicate to the supervisor what the result was. It must do this using the dedicated methods. Dropping it is considered an error, which:
- panics in debug mode
- fails closed and warns in release mode
When it responds, by default it responds only for the current syscall. Unlike the old sandbox implementation, there's no obvious way to say "allow this syscall again in the future", because it often refers to ephemeral state (e.g. file descriptors). For example, if you allow writes to /foo/bar/baz.rs, and you want that approval to persist for the rest of the sandbox, should that apply to:
- only
/foo/bar/baz.rs /foo/bar/foo/
The linux_sandbox crate doesn't address this issue, but provides the Decision type, which is what a Responder actually responds with. It contains a decision for this syscall, but also a list of rules to remember for the rest of this session. It is up zed to determine what options to show to the user.
It's almost always wrong to not supply any extra rules to remember.
Tradeoffs
The one major tradeoff is performance: this design imposes a round trip to the supervisor on every syscall that may be blocked (opening files and sockets, writing, connecting). On a 9950x3d, a trapped open costs ~10 microseconds compared to ~0.4 microseconds untrapped. Relaying a 1 gigabit stream cost about 2.5% of a CPU core.. stat and friends, directory listing, and execve are not restricted, so don't get the overhead.
Implementation ideas (ai generated)
Future improvements
Synthetic IPv6 addresses for domains
The sandbox's DNS only hands out synthetic IPv4 addresses (from 198.18.0.0/15). A connection to a domain has two legs, joined by the supervisor's relay, which copies bytes rather than translating packets:
- app to relay, inside the sandbox, using the synthetic address;
- supervisor to the real server, on the host network, using whatever the name resolves to there (IPv6 included).
So leg 1 is always IPv4, while the real server can still be IPv6-only. Queries for IPv6 addresses (AAAA) get an empty answer (NOERROR, no records) rather than NXDOMAIN, so resolvers fall back to the IPv4 answer cleanly.
Raw IPv6 addresses (IPv6 grants, curl http://[2606:4700::1]) still work: the relay listens at the exact address the app connects to, with IP_FREEBIND set on the socket. Without it, IPv6 bind() only accepts addresses assigned to an interface and ignores the namespace's local routes, unlike IPv4.
What this gives up, all uncommon:
- explicitly IPv6-only invocations (
curl -6,ssh -6) get no address for a domain; - tools in IPv6-only environments that query only
AAAArecords; - agents debugging IPv6 itself get a confusing result.
If one of these matters, add a synthetic IPv6 range (a ULA /64) alongside the IPv4 one, using the same IP_FREEBIND relay path.
Revoking network grants
Narrowing the policy currently leaves existing TCP connections and UDP flows alive, just as already-open files keep working. If revoking a grant should cut access off, update_policy could report the connections and flows the new policy wouldn't allow (like still_open does for files) and offer to close them.
Relaying DNS to real servers
Port 53 of every address in the sandbox is answered by the supervisor's DNS. That's how queries reach it whatever server resolv.conf names, but it also means dig @8.8.8.8 gets our answers: it's indistinguishable from the system resolver asking its configured server. Relaying instead has costs: real answers make later connections raw-IP requests, so domain rules no longer apply, and an allowed resolver is a channel for exfiltrating data in queries. A middle ground: relay port 53 only when an Ip rule names that exact address and port, and answer everything else ourselves (including under Any or LocalNetwork rules).
Async supervisor
Every relayed TCP connection takes two threads, and every UDP flow and DNS query one, so a busy npm install can mean hundreds of threads. An event loop in the supervisor (smol, which Zed already uses: async-io plus a small executor) would make these tasks instead; seccomp notifications would stay with their leader/follower receivers, since prompts block.
Measured (9950X3D; benchmark harness in the benchmarking thread): at 1 Gbit/s every relay model costs 2.5–3% of a core, so throughput alone doesn't motivate this. For bursts of short connections, async halves the relay's CPU per exchange (21 vs 48 µs) and replaces 626 threads at 1000 concurrent connections with 3. Not urgent; revisit if supervisor resource usage becomes a problem.
Exact errors for non-blocking connects
A connect on a non-blocking socket succeeds as soon as it's allowed, and the host connection is made in the background: holding the syscall instead would freeze the app's event loop for a round trip per connection. A host that refuses or can't be reached then shows up as a reset after a successful connect (curl: (56) Recv failure: Connection reset by peer instead of Connection refused). Most clients use non-blocking sockets (curl, Go, Node, Python sockets with a timeout), so this is the common case. A middle ground: wait briefly for the host connect (a few milliseconds, enough for local and fast failures) before returning early.
Sandbox differences skill
In-sandbox behaviour is subtly different from unsandboxed Linux (exact error codes on denied operations, resets instead of refusals after non-blocking connects, getsockname paths, SO_PEERCRED pids). Collect the differences into a documented "skill" that agents can be given, so tools don't misread sandbox artefacts as environment failures.
Datagram unix sockets
socket and socketpair refuse SOCK_DGRAM unix sockets: one could send to any socket by path with sendmsg, whose address is behind a pointer, and seccomp can't read the address to notice. The same trap would also let unconnected UDP sendto reach ports no rule names: today those datagrams go nowhere, because a relay socket only exists for ports the policy names. This breaks sd_notify, syslog clients and datagram pairs used for IPC.
Measured (9950X3D; benchmark harness in the benchmarking thread): trapping every send* costs +10.8 µs of CPU per send with the current notification path — the supervisor saturates a core at 1 Gbit/s of QUIC-sized datagrams. An optimized path (answer on the receiving thread, SYNC_WAKE_UP, cached pidfds) brings it to ~+2.2 µs per send: +28% of a core at 1 Gbit/s, and +1% or less with GSO or sendmmsg batches of 16. Whether that budget is acceptable is a product call.
Unix socket peer credentials
The supervisor makes brokered connect calls on its copy of the app's socket, so a server's SO_PEERCRED reports the supervisor's pid (the uid is the same). Servers that authorize by pid would see the wrong process.
-
Technically, the supervisor doesn't "allow or deny" the syscall, since this introduces a race condition, where a malicious program makes a syscall, then, in between approval and the kernel executing it, changes the contents of one of the arguments which is behind a pointer (e.g. file paths). Instead, the syscall arguments (including any data behind a pointer) are copied into the supervisor, and the supervisor makes the syscall on behalf of the sandboxed program, and returns the result. ↩︎