Michael Kühnel efed1f245a edit_prediction: Send only format-specific FIM stop sequences (#65360)
## Summary

The FIM edit prediction path (`open_ai_compatible_api` and `ollama`)
always sent a fixed list of 14 stop sequences, covering the control
tokens of every supported prompt format, regardless of the configured
`prompt_format`. OpenAI-compatible servers that enforce the OpenAI limit
of 4 `stop` entries (e.g. vLLM) reject every such request with a 400, so
no predictions appear and the error is only visible in the log.

This change replaces `get_fim_stop_tokens()` with
`fim_stop_tokens(prompt_format)`, which returns only the markers that
`format_fim_prompt` uses for the selected format, plus the format's
end-of-text token where applicable, and never more than 4 entries. The
prompt format is already resolved (including `Infer`) before
`fim::request_prediction` is called, so no settings or schema changes
are needed. `clean_fim_completion` is unchanged and still truncates the
completion at all known markers client-side as a safety net.

Closes #65359

## Testing

- Added `fim_stop_tokens_respect_openai_stop_limit`, which asserts that
every FIM prompt format sends between 1 and 4 stop sequences.
- Added `fim_stop_tokens_match_prompt_markers`, which asserts that every
non end-of-text stop token actually appears in the prompt built by
`format_fim_prompt` for that format, so the two can't drift apart.
- `cargo test -p edit_prediction --lib fim`

To reproduce manually: run vLLM with a Qwen coder model, configure
`edit_predictions.provider` as `open_ai_compatible_api` with
`prompt_format: "qwen"`, and type in a source file. Before this change
every request fails with a 400 about too many stop sequences; after it,
predictions appear and the request body contains exactly 4 `stop`
entries.

## Self-review

- [x] I've reviewed my diff for quality, security, reliability, and
performance.
- ~~[ ] UI changes follow the
[checklist](https://zed.dev/docs/development/ui-checklist).~~
- [x] Tests cover the new or changed behavior.

Release Notes:

- Fixed OpenAI-compatible edit predictions failing on servers that limit
stop sequences to 4.

---------

Co-authored-by: zed-zippy[bot] <234243425+zed-zippy[bot]@users.noreply.github.com>
2026-10-09 14:56:21 +00:00
…
2026-09-17 09:59:21 +00:00
…
…
2026-10-09 09:15:09 +00:00
…
…
…
…
…
…

Zed

Zed CI

Welcome to Zed, a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.


Installation

On macOS, Linux, and Windows you can download Zed directly or install Zed via your local package manager (macOS/Linux/Windows).

Other platforms are not yet available:

Developing Zed

Contributing

See CONTRIBUTING.md for ways you can contribute to Zed.

Also... we're hiring! Check out our jobs page for open roles.

Licensing

Zed source code is licensed primarily under GPL-3.0-or-later, with Apache-2.0 components where marked.

License information for third party dependencies must be correctly provided for CI to pass.

We use cargo-about to automatically comply with open source licenses. If CI is failing, check the following:

  • Is it showing a no license specified error for a crate you've created? If so, add publish = false under [package] in your crate's Cargo.toml.
  • Is the error failed to satisfy license requirements for a dependency? If so, first determine what license the project has and whether this system is sufficient to comply with this license's requirements. If you're unsure, ask a lawyer. Once you've verified that this system is acceptable add the license's SPDX identifier to the accepted array in script/licenses/zed-licenses.toml.
  • Is cargo-about unable to find the license for a dependency? If so, add a clarification field at the end of script/licenses/zed-licenses.toml, as specified in the cargo-about book.

Sponsorship

Zed is developed by Zed Industries, Inc., a for-profit company.

If you’d like to financially support the project, you can do so via GitHub Sponsors. Sponsorships go directly to Zed Industries and are used as general company revenue. There are no perks or entitlements associated with sponsorship.

S
Description
No description provided
Readme 1.6 GiB
Languages
Rust 95.4%
JSON-with-Comments 2.4%
Python 0.8%
Inno Setup 0.3%
Tree-sitter Query 0.3%
Other 0.4%