mirror of
https://github.com/zed-industries/zed.git
synced 2026-11-20 07:30:22 +01:00
790dcefb01
This adds support for running self-hosted Sweep Next Edit edit prediction models through Zed's OpenAI-compatible provider. The target use case is a local deployment based on [sweepai/sweep-next-edit-1.5B](https://huggingface.co/sweepai/sweep-next-edit-1.5B), using the rewrite-window prompting approach described in [OSS Next Edit](https://blog.sweep.dev/posts/oss-next-edit). Personal user testing shows impressive quality from this small edit prediction model. This is valuable because it allows more flexibility for Zed users to self-host state of the art next edit prediction models! This PR addresses the self-hosted workflow discussed in https://github.com/zed-industries/zed/discussions/50929. Note that the self-hosted model path needed more than just a new dropdown value. Zed has to build the rewrite-window prompt shape the model expects, map the rewritten window back into anchored edits, and make `prompt_format: "infer"` work with the filename-style model identifiers that llama-server local server reports. The main changes are: - add a `Sweep` prompt format for OpenAI-compatible edit prediction providers and route it to a dedicated `SweepPrompt` model path - build Sweep rewrite-window prompts from the active cursor window, recent change history, and related file excerpts - convert rewrite responses back into anchored edits and suppress no-op rewrites - add fixed cursor-window extraction used by the Sweep rewrite prompt path - recognize `sweep-next-edit` model names during `infer`, including filename-style identifiers such as `sweepai_sweep-next-edit-1.5B_sweep-next-edit-1.5b.q8_0.v2.gguf` - reject reserved Sweep prompt tokens before sending malformed requests to a self-hosted server - update the docs to describe the actual Sweep rewrite-window prompt format Local setup used for manual verification: ```sh llama-server \ --hf-repo sweepai/sweep-next-edit-1.5B \ --hf-file sweep-next-edit-1.5b.q8_0.v2.gguf \ --ctx-size 8192 \ --host 127.0.0.1 \ --port 8080 ``` ```json { "edit_predictions": { "provider": "open_ai_compatible_api", "open_ai_compatible_api": { "api_url": "http://127.0.0.1:8080/v1/completions", "model": "sweepai_sweep-next-edit-1.5B_sweep-next-edit-1.5b.q8_0.v2.gguf", "prompt_format": "infer", "max_output_tokens": 512 } } } ``` The request that produced this PR included screenshots of the OpenAI-compatible provider configuration and the prompt format dropdown with `Sweep` selected. Tests added in this branch: - `test_sweep_prompt_format_routes_to_sweep_prompt_model` - `test_fixed_line_window_around_cursor_start_middle_and_end` - `test_sweep_prompt_request_prediction_diffs_rewritten_window_into_anchored_edits` - `test_sweep_prompt_request_prediction_returns_none_for_identical_rewrite` - `test_original_window_for_current_window_uses_latest_pre_edit_snapshot` - `test_original_window_for_current_window_returns_none_without_matching_history` - `test_recent_change_block_from_event_formats_original_and_updated_sections` Verification run locally: - `cargo test -p zed sweep_prompt_format_routes` - `cargo test -p zed subscribe_uses_stale_provider_config` - `cargo test -p edit_prediction sweep_prompt` - `cargo test -p edit_prediction fixed_line_window_around_cursor_start_middle_and_end` - `cargo test -p edit_prediction test_sweep_prompt_request_prediction_diffs_rewritten_window_into_anchored_edits` - `cargo test -p edit_prediction test_sweep_prompt_request_prediction_returns_none_for_identical_rewrite` - `./script/clippy -p zed -p edit_prediction -p settings_content` Release Notes: - Added support for self-hosted Sweep Next Edit models in OpenAI-compatible edit predictions, including the `sweep` prompt format and `infer` detection for `sweep-next-edit` model names. --------- Co-authored-by: Ben Kunkle <ben@zed.dev>