agent_ui: Improve model and configuration option search (#64147)

# Objective
Fixes #64144. 

Improve search in agent model and configuration option pickers.

Previously, searches could fail when punctuation differed between the
query and model name, such as `GPT 5.6` versus `GPT-5.6`. Configuration
options also could not be found by provider/group, description, or value
identifier, and results were limited to 100 entries.

## Solution

Use the existing `fuzzy_nucleo` matcher for agent model and
configuration option searches.

Include relevant metadata in each search candidate:

- Model name and provider/group
- Configuration option name, group, description, and value identifier

Return all matching configuration options while preserving their
existing ordering when scores are equal.

Rank matches against option/model names independently of their metadata,
then append metadata-only matches without duplicates. This keeps
descriptions and identifiers searchable without letting their lengths
demote exact name matches or reorder equally relevant names.

Run the shared pure-data search pipeline on the background executor,
including candidate construction, ranking, and result cloning. This
prevents large catalogs and descriptions from monopolizing foreground
task polls without changing the ranking rule.

## Testing

- Added and ran automated tests with the following:
	- Searches with punctuation differences, such as `GPT 5.6`
	- Searches by provider/group
- Searches across configuration option names, descriptions, groups, and
identifiers
	- Searches returning more than 100 configuration options
	- Existing fuzzy-match ordering behavior
- Exact option-name matches ahead of partial-name and metadata-only
matches
	- Stable name ordering despite different metadata lengths
	- Combined queries spanning names and metadata

Follow-up validation on the ranking fix:

- `cargo test -p agent_ui --lib`: all 521 tests passed. Both new
configuration-option ranking regression tests failed before the fix and
passed afterward.
- `./script/clippy -p agent_ui`: passed, including the available
cargo-shear and typo checks.
- `rustfmt --check --edition 2024 crates/agent_ui/src/config_options.rs
crates/agent_ui/src/model_selector.rs`: passed.
- `git diff --check`: passed.

Additionally compiled the app with `cargo run`, launched OpenCode agent
in ACP mode, and searched `GPT 5.6`, which produces `OpenAI/GPT-5.6 Sol`
and `OpenRouter/GPT-5.6 Sol`. See video below.

## Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

## Background-search validation

- `cargo test -p agent_ui --lib`: all 522 tests passed. The new
background-preparation test failed on the foreground implementation and
now passes across 20 scheduler seeds.
- `./script/clippy -p agent_ui -p agent_ui_benchmarks`: passed,
including cargo-shear and typo checks.
- `cargo fmt -p agent_ui -p agent_ui_benchmarks --check`, `git diff
--check`, and `./script/check-licenses`: passed.
- `./script/bench-agent-model-search --test`: all 20 workloads passed
before and after the change. The runner rejects any `test-support`
feature in the benchmark package graph.
- Ran quick mode before measured runs. Recorded `--save-baseline
foreground-preparation` on `b6157587dbfe96`, then ran the identical
benchmark with `--baseline foreground-preparation` on `4617d47567401e`.

Apple Silicon, Rust 1.98.1, `release-fast`, 15 Criterion samples, 200 ms
warm-up, 500 ms measurement. Representative broad-query results:

| Synthetic catalog | Total completion before → after | Foreground
task-poll p95 before → after |
| --- | --- | --- |
| 10,000 options, 128-byte descriptions | 10.35 → 12.49 ms | 7.36 → 0.46
ms |
| 10,000 options, 1,024-byte descriptions | 42.16 → 43.95 ms | 30.26 →
0.66 ms |

For the larger-description workload, foreground p99/max fell from
33.10/33.10 ms to 1.35/1.35 ms. Median foreground instructions fell from
879.9 million to 11.8 million, while total instructions stayed
approximately 2.14 billion. Observed synthetic 120 FPS budget overruns
fell from 66 to zero for this workload.

This improves foreground responsiveness rather than reducing total work.
Other Rust builds were active on the machine, so completion-time
estimates are noisy and no throughput speedup is claimed. The benchmark
measures the production search module with synthetic ACP-like options,
not provider requests, picker rendering, or live keystrokes. Its
foreground report includes warm-up/calibration, and budget overruns are
not literal display-dropped frames.

## Showcase


https://github.com/user-attachments/assets/04c1a59e-8454-49f8-af6c-47e1107f9416

---

Release Notes:

- Improved fuzzy searching for agent model and configuration option
pickers across Zed Agent and third-party ACP-compatible agents,
preserving name relevance and keeping search preparation off the UI
thread.

---------

Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
This commit is contained in:
Jacob Wolf
2026-10-20 13:06:38 +00:00
committed by GitHub
parent b0f1e1cb2f
commit 065d7ae597
12 changed files with 597 additions and 59 deletions
Generated
+14
View File
@@ -484,6 +484,7 @@ dependencies = [
"fs",
"futures 0.3.34",
"fuzzy",
"fuzzy_nucleo",
"git",
"git_ui_core",
"gpui",
@@ -553,6 +554,19 @@ dependencies = [
"zed_actions",
]
[[package]]
name = "agent_ui_benchmarks"
version = "0.1.0"
dependencies = [
"collections",
"criterion",
"futures 0.3.34",
"fuzzy_nucleo",
"gpui",
"gpui_platform",
"ordered-float 5.5.0",
]
[[package]]
name = "ahash"
version = "0.8.12"
+1
View File
@@ -10,6 +10,7 @@ members = [
"crates/agent_settings",
"crates/agent_skills",
"crates/agent_ui",
"crates/agent_ui_benchmarks",
"crates/ai_onboarding",
"crates/anthropic",
"crates/askpass",
+1
View File
@@ -58,6 +58,7 @@ file_icons.workspace = true
fs.workspace = true
futures.workspace = true
fuzzy.workspace = true
fuzzy_nucleo.workspace = true
git.workspace = true
gpui.workspace = true
gpui_tokio.workspace = true
+1
View File
@@ -21,6 +21,7 @@ mod language_model_selector;
mod mention_set;
mod message_editor;
mod mode_selector;
mod model_search;
mod model_selector;
mod model_selector_popover;
mod profile_selector;
+132 -29
View File
@@ -1,4 +1,4 @@
use std::{cmp::Reverse, rc::Rc, sync::Arc};
use std::{rc::Rc, sync::Arc};
use acp_thread::AgentSessionConfigOptions;
use agent_client_protocol::schema::v2 as acp;
@@ -6,11 +6,10 @@ use agent_servers::AgentServer;
use collections::HashSet;
use fs::Fs;
use fuzzy::StringMatchCandidate;
use fuzzy_nucleo::StringMatchCandidate;
use gpui::{
App, BackgroundExecutor, Context, DismissEvent, Entity, Subscription, Task, Window, prelude::*,
};
use ordered_float::OrderedFloat;
use picker::popover_menu::PickerPopoverMenu;
use picker::{Picker, PickerDelegate};
use settings::{AgentConfigOptionValue, SettingsStore};
@@ -22,6 +21,7 @@ use unicode_segmentation::UnicodeSegmentation;
use util::ResultExt as _;
use zed_actions::agent::ToggleModelSelector;
use crate::model_search;
use crate::ui::documentation_aside_side;
use crate::{
CycleFavoriteModels, CycleModeSelector, CycleThinkingEffort, ToggleProfileSelector,
@@ -1074,32 +1074,22 @@ async fn fuzzy_search_options(
query: &str,
executor: BackgroundExecutor,
) -> Vec<ConfigOptionValue> {
let candidates = options
.iter()
.enumerate()
.map(|(ix, opt)| StringMatchCandidate::new(ix, &opt.name))
.collect::<Vec<_>>();
let mut matches = fuzzy::match_strings(
&candidates,
query,
false,
true,
100,
&Default::default(),
executor,
)
.await;
matches.sort_unstable_by_key(|mat| {
let candidate = &candidates[mat.candidate_id];
(Reverse(OrderedFloat(mat.score)), candidate.id)
});
matches
.into_iter()
.map(|mat| options[mat.candidate_id].clone())
.collect()
model_search::search(options, query, executor, |index, option| {
(
StringMatchCandidate::new(index, option.name.clone()),
StringMatchCandidate::new(
index,
format!(
"{} {} {} {}",
option.name,
option.group.as_deref().unwrap_or_default(),
option.description.as_deref().unwrap_or_default(),
option.value.0,
),
),
)
})
.await
}
fn find_option_name(
@@ -1145,6 +1135,119 @@ mod tests {
use project::{AgentId, Project};
use std::{any::Any, cell::RefCell};
fn option_names(options: Vec<ConfigOptionValue>) -> Vec<String> {
options.into_iter().map(|option| option.name).collect()
}
#[gpui::test]
async fn config_option_search_matches_all_searchable_fields(executor: BackgroundExecutor) {
let options = vec![
ConfigOptionValue {
value: acp::SessionConfigValueId::new("openai/gpt-5.6-sol"),
name: "GPT-5.6 Sol".to_string(),
description: Some("Flagship reasoning model".to_string()),
group: Some("OpenAI".to_string()),
},
ConfigOptionValue {
value: acp::SessionConfigValueId::new("anthropic/claude-sonnet-4"),
name: "Claude Sonnet 4".to_string(),
description: None,
group: Some("Anthropic".to_string()),
},
];
for query in [
"GPT 5.6",
"GPT-5.6",
"gpt5.6 sol",
"openai",
"openai/gpt-5.6-sol",
"flagship",
"openai reasoning",
] {
assert_eq!(
option_names(fuzzy_search_options(options.clone(), query, executor.clone()).await),
["GPT-5.6 Sol"],
"query {query:?} should match the GPT model",
);
}
}
#[gpui::test]
async fn config_option_search_ranks_names_before_metadata(executor: BackgroundExecutor) {
let options = vec![
ConfigOptionValue {
value: acp::SessionConfigValueId::new("reasoner"),
name: "Reasoner".to_string(),
description: Some("High reasoning effort".to_string()),
group: None,
},
ConfigOptionValue {
value: acp::SessionConfigValueId::new("xhigh"),
name: "Extra High".to_string(),
description: None,
group: None,
},
ConfigOptionValue {
value: acp::SessionConfigValueId::new("high"),
name: "High".to_string(),
description: Some(
"High effort with advanced reasoning for complex tasks.".to_string(),
),
group: None,
},
];
assert_eq!(
option_names(fuzzy_search_options(options, "high", executor).await),
["High", "Extra High", "Reasoner"],
);
}
#[gpui::test]
async fn config_option_search_preserves_name_order_with_different_metadata(
executor: BackgroundExecutor,
) {
let options = vec![
ConfigOptionValue {
value: acp::SessionConfigValueId::new("model-a"),
name: "Model A".to_string(),
description: Some("Advanced reasoning for complex tasks".to_string()),
group: Some("Provider A".to_string()),
},
ConfigOptionValue {
value: acp::SessionConfigValueId::new("b"),
name: "Model B".to_string(),
description: None,
group: None,
},
];
assert_eq!(
option_names(fuzzy_search_options(options, "model", executor).await),
["Model A", "Model B"],
);
}
#[gpui::test]
async fn config_option_search_returns_more_than_one_hundred_matches(
executor: BackgroundExecutor,
) {
let options = (0..101)
.map(|index| ConfigOptionValue {
value: acp::SessionConfigValueId::new(format!("model-{index}")),
name: format!("Model {index}"),
description: None,
group: None,
})
.collect::<Vec<_>>();
assert_eq!(
fuzzy_search_options(options, "model", executor).await.len(),
101
);
}
#[gpui::test]
fn config_defaults_wait_for_authoritative_response(cx: &mut TestAppContext) {
for boolean in [false, true] {
+86
View File
@@ -0,0 +1,86 @@
use std::cmp::Reverse;
use collections::HashSet;
use fuzzy_nucleo::StringMatchCandidate;
use gpui::BackgroundExecutor;
use ordered_float::OrderedFloat;
pub(crate) async fn search<T, BuildCandidates>(
values: Vec<T>,
query: &str,
executor: BackgroundExecutor,
build_candidates: BuildCandidates,
) -> Vec<T>
where
T: Clone + Send + 'static,
BuildCandidates: Fn(usize, &T) -> (StringMatchCandidate, StringMatchCandidate) + Send + 'static,
{
let query = query.to_owned();
// Preparing and cloning candidates can block input even though matching is already asynchronous.
executor
.clone()
.spawn(async move {
let (names, candidates): (Vec<_>, Vec<_>) = values
.iter()
.enumerate()
.map(|(index, value)| build_candidates(index, value))
.unzip();
match_names_with_metadata(&names, &candidates, &query, executor)
.await
.into_iter()
.map(|index| values[index].clone())
.collect()
})
.await
}
async fn match_names_with_metadata(
names: &[StringMatchCandidate],
candidates: &[StringMatchCandidate],
query: &str,
executor: BackgroundExecutor,
) -> Vec<usize> {
let cancel_flag = Default::default();
let (mut name_matches, mut metadata_matches) = futures::join!(
fuzzy_nucleo::match_strings_async(
names,
query,
fuzzy_nucleo::Case::Ignore,
fuzzy_nucleo::LengthPenalty::On,
names.len(),
&cancel_flag,
executor.clone(),
),
fuzzy_nucleo::match_strings_async(
candidates,
query,
fuzzy_nucleo::Case::Ignore,
fuzzy_nucleo::LengthPenalty::On,
candidates.len(),
&cancel_flag,
executor,
),
);
// Metadata should expand the results without changing the relevance of name matches.
let sorting_key = |matched: &fuzzy_nucleo::StringMatch| {
(Reverse(OrderedFloat(matched.score)), matched.candidate_id)
};
name_matches.sort_unstable_by_key(sorting_key);
metadata_matches.sort_unstable_by_key(sorting_key);
let matched_name_ids: HashSet<_> = name_matches
.iter()
.map(|matched| matched.candidate_id)
.collect();
name_matches
.into_iter()
.chain(
metadata_matches
.into_iter()
.filter(|matched| !matched_name_ids.contains(&matched.candidate_id)),
)
.map(|matched| matched.candidate_id)
.collect()
}
+67 -30
View File
@@ -1,4 +1,4 @@
use std::{cmp::Reverse, rc::Rc, sync::Arc};
use std::{rc::Rc, sync::Arc};
use acp_thread::{
AgentModelIcon, AgentModelId, AgentModelInfo, AgentModelList, AgentModelSelector,
@@ -7,19 +7,19 @@ use acp_thread::{
use anyhow::Result;
use collections::{HashSet, IndexMap};
use futures::FutureExt;
use fuzzy::{StringMatchCandidate, match_strings};
use fuzzy_nucleo::StringMatchCandidate;
use gpui::{
Action, AsyncWindowContext, BackgroundExecutor, DismissEvent, FocusHandle, Subscription, Task,
TaskExt, WeakEntity,
};
use itertools::Itertools;
use ordered_float::OrderedFloat;
use picker::{Picker, PickerDelegate};
use settings::SettingsStore;
use ui::{DocumentationAside, IntoElement, prelude::*};
use util::ResultExt;
use zed_actions::agent::OpenSettings;
use crate::model_search;
use crate::ui::{
ModelSelectorFooter, ModelSelectorHeader, ModelSelectorListItem, documentation_aside_side,
};
@@ -441,44 +441,34 @@ async fn fuzzy_search(
) -> AgentModelList {
async fn fuzzy_search_list(
model_list: Vec<AgentModelInfo>,
group_name: Option<SharedString>,
query: &str,
executor: BackgroundExecutor,
) -> Vec<AgentModelInfo> {
let candidates = model_list
.iter()
.enumerate()
.map(|(ix, model)| StringMatchCandidate::new(ix, model.name.as_ref()))
.collect::<Vec<_>>();
let mut matches = match_strings(
&candidates,
query,
false,
true,
100,
&Default::default(),
executor,
)
.await;
matches.sort_unstable_by_key(|mat| {
let candidate = &candidates[mat.candidate_id];
(Reverse(OrderedFloat(mat.score)), candidate.id)
});
matches
.into_iter()
.map(|mat| model_list[mat.candidate_id].clone())
.collect()
model_search::search(model_list, query, executor, move |index, model| {
(
StringMatchCandidate::new(index, model.name.clone()),
StringMatchCandidate::new(
index,
format!(
"{} {}",
model.name,
group_name.as_deref().unwrap_or_default()
),
),
)
})
.await
}
match model_list {
AgentModelList::Flat(model_list) => {
AgentModelList::Flat(fuzzy_search_list(model_list, &query, executor).await)
AgentModelList::Flat(fuzzy_search_list(model_list, None, &query, executor).await)
}
AgentModelList::Grouped(index_map) => {
let groups =
futures::future::join_all(index_map.into_iter().map(|(group_name, models)| {
fuzzy_search_list(models, &query, executor.clone())
fuzzy_search_list(models, Some(group_name.0.clone()), &query, executor.clone())
.map(|results| (group_name, results))
}))
.await;
@@ -698,6 +688,53 @@ mod tests {
// Fuzzy search - test with specific model name
let results = fuzzy_search(models.clone(), "mistral".into(), cx.executor()).await;
assert_models_eq(results, vec![("ollama", vec!["mistral"])]);
let models = create_model_list(vec![
("OpenAI", vec!["GPT-5.6 Sol"]),
("Anthropic", vec!["Claude Sonnet 4"]),
]);
let results = fuzzy_search(models.clone(), "GPT 5.6".into(), cx.executor()).await;
assert_models_eq(results, vec![("OpenAI", vec!["GPT-5.6 Sol"])]);
let results = fuzzy_search(models.clone(), "openai 5.6".into(), cx.executor()).await;
assert_models_eq(results, vec![("OpenAI", vec!["GPT-5.6 Sol"])]);
let results = fuzzy_search(models, "openai".into(), cx.executor()).await;
assert_models_eq(results, vec![("OpenAI", vec!["GPT-5.6 Sol"])]);
}
#[gpui::test]
async fn test_fuzzy_match_prioritizes_names_over_group(cx: &mut TestAppContext) {
let models = create_model_list(vec![(
"High Provider",
vec!["Reasoner", "Extra High", "High"],
)]);
let results = fuzzy_search(models, "high".into(), cx.executor()).await;
assert_models_eq(
results,
vec![("High Provider", vec!["High", "Extra High", "Reasoner"])],
);
}
#[gpui::test(iterations = 20)]
async fn test_model_search_builds_candidates_in_background(executor: BackgroundExecutor) {
let background_executor = executor.clone();
let results = model_search::search(
vec!["High".to_string()],
"high",
executor,
move |index, name| {
assert!(!background_executor.is_main_thread());
(
StringMatchCandidate::new(index, name.clone()),
StringMatchCandidate::new(index, name.clone()),
)
},
)
.await;
assert_eq!(results, ["High"]);
}
#[gpui::test]
+26
View File
@@ -0,0 +1,26 @@
[package]
name = "agent_ui_benchmarks"
version = "0.1.0"
edition.workspace = true
publish.workspace = true
license = "GPL-3.0-or-later"
[lints]
workspace = true
[package.metadata.cargo-shear]
# Used by the path-included production module and the gpui::bench expansion.
ignored = ["futures", "gpui_platform"]
[dev-dependencies]
collections.workspace = true
criterion.workspace = true
futures.workspace = true
fuzzy_nucleo.workspace = true
gpui = { workspace = true, features = ["bench-support"] }
gpui_platform = { workspace = true, features = ["bench-support"] }
ordered-float.workspace = true
[[bench]]
name = "model_search"
harness = false
+1
View File
@@ -0,0 +1 @@
../../LICENSE-GPL
+47
View File
@@ -0,0 +1,47 @@
# Agent model search benchmark
This benchmark measures the search pipeline shared by the agent model and ACP
configuration-option pickers, including candidate construction, matching, ranking,
and result cloning. It records both completion latency and foreground task-poll
durations, so work moved to a background task is not mistaken for eliminated work.
The benchmark is a separate package because `agent_ui`'s dev dependencies enable
`test-support`. It compiles the production `model_search` module directly rather
than maintaining a benchmark-only copy of the search algorithm. The runner checks
the complete package feature graph and refuses to run if `test-support` leaks in.
From the repository root:
```sh
# Smoke: exercise every workload once and validate the results.
./script/bench-agent-model-search --test
# Quick timing check before a measured run.
./script/bench-agent-model-search --quick '1000-options-128-byte-descriptions-selective'
# Record a baseline before changing the production search implementation.
./script/bench-agent-model-search --save-baseline before
# Run the identical benchmark after the change.
./script/bench-agent-model-search --baseline before
```
Run measurements sequentially on an otherwise idle machine. Each measured case
uses 15 samples, 200 ms warm-up, and 500 ms measurement time; filter to a single
workload for longer local investigation. Use `BENCH_MEASUREMENT=wall-time` to
disable counters, or leave it unset to report supported hardware and allocation
counters alongside wall time.
Fixtures contain 100, 1,000, or 10,000 synthetic ACP-like options, with 128-byte or
1,024-byte descriptions. They cover broad, selective, combined name/metadata,
missing, and exact-name ranking queries. Workers and matcher pools are warm.
Each iteration includes one foreground clone of the input snapshot and the full
production search; fixture generation and result validation are outside timing.
The synchronous reference checks all returned value identifiers and their order.
The benchmark measures the search pipeline, not the full picker: it does not
measure provider requests, layout, painting, presentation, or live keystrokes.
No window is rendered. Foreground frame-budget overruns use GPUI's synthetic
120 FPS budget (8.33 ms); they are not display-dropped frames. The foreground
report includes warm-up and calibration, whereas Criterion's latency estimate
uses its measurement phase.
@@ -0,0 +1,209 @@
use std::{cmp::Reverse, fmt, sync::Arc, time::Duration};
use collections::HashSet;
use fuzzy_nucleo::{Case, LengthPenalty, StringMatch, StringMatchCandidate};
use gpui::{BenchAppContext, SharedString};
use ordered_float::OrderedFloat;
// Compile the production search without pulling in the rest of agent_ui.
#[path = "../../agent_ui/src/model_search.rs"]
mod model_search;
#[derive(Clone)]
struct OptionValue {
value: SharedString,
name: String,
description: Option<String>,
group: Option<String>,
}
struct SearchInput {
options: usize,
description_bytes: usize,
query_name: &'static str,
query: &'static str,
}
impl fmt::Display for SearchInput {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
formatter,
"{}-options-{}-byte-descriptions-{}",
self.options, self.description_bytes, self.query_name,
)
}
}
fn inputs() -> Vec<SearchInput> {
[(100, 128), (1_000, 128), (10_000, 128), (10_000, 1_024)]
.into_iter()
.flat_map(|(options, description_bytes)| {
[
("broad", "gpt"),
("selective", "GPT 5.6 000042"),
("combined", "openai reasoning"),
("missing", "nonexistent-zzzzzz"),
("name-ranking", "high"),
]
.into_iter()
.map(move |(query_name, query)| SearchInput {
options,
description_bytes,
query_name,
query,
})
})
.collect()
}
fn options(input: &SearchInput) -> Vec<OptionValue> {
let description = "Flagship reasoning model for coding, analysis, and complex tasks. ";
let mut description = description.repeat(input.description_bytes.div_ceil(description.len()));
description.truncate(input.description_bytes);
(0..input.options)
.map(|index| {
let (name, group, identifier) = match index % 4 {
0 => ("Claude Sonnet 4", "Anthropic", "anthropic/claude-sonnet-4"),
1 => ("Gemini 2.5 Pro", "Google", "google/gemini-2.5-pro"),
2 => ("GPT-5.6 Sol", "OpenAI", "openai/gpt-5.6-sol"),
_ => ("DeepSeek V3", "OpenRouter", "openrouter/deepseek-v3"),
};
let mut option = OptionValue {
value: format!("{identifier}/{index:06}").into(),
name: format!("{name} {index:06}"),
description: Some(description.clone()),
group: Some(group.to_string()),
};
match index {
0 => {
option.name = "High".to_string();
option.description =
Some("High effort with advanced reasoning for complex tasks.".to_string());
}
1 => {
option.name = "Extra High".to_string();
option.description = None;
}
2 => {
option.name = "Reasoner".to_string();
option.description = Some("High reasoning effort".to_string());
}
_ => {}
}
option
})
.collect()
}
fn candidates(index: usize, option: &OptionValue) -> (StringMatchCandidate, StringMatchCandidate) {
(
StringMatchCandidate::new(index, option.name.clone()),
StringMatchCandidate::new(
index,
format!(
"{} {} {} {}",
option.name,
option.group.as_deref().unwrap_or_default(),
option.description.as_deref().unwrap_or_default(),
option.value,
),
),
)
}
fn expected_values(options: &[OptionValue], query: &str) -> Vec<SharedString> {
let (names, metadata): (Vec<_>, Vec<_>) = options
.iter()
.enumerate()
.map(|(index, option)| candidates(index, option))
.unzip();
let mut names = fuzzy_nucleo::match_strings(
&names,
query,
Case::Ignore,
LengthPenalty::On,
options.len(),
);
let mut metadata = fuzzy_nucleo::match_strings(
&metadata,
query,
Case::Ignore,
LengthPenalty::On,
options.len(),
);
let sorting_key =
|matched: &StringMatch| (Reverse(OrderedFloat(matched.score)), matched.candidate_id);
names.sort_unstable_by_key(sorting_key);
metadata.sort_unstable_by_key(sorting_key);
let name_ids: HashSet<_> = names.iter().map(|matched| matched.candidate_id).collect();
names
.into_iter()
.chain(
metadata
.into_iter()
.filter(|matched| !name_ids.contains(&matched.candidate_id)),
)
.map(|matched| options[matched.candidate_id].value.clone())
.collect()
}
struct ValidatedResults {
results: Vec<OptionValue>,
expected: Arc<Vec<SharedString>>,
}
impl Drop for ValidatedResults {
fn drop(&mut self) {
// bench_task drops its output after timing, so validation doesn't inflate latency.
assert!(
self.results
.iter()
.map(|option| &option.value)
.eq(self.expected.iter()),
"search dropped, duplicated, or reordered options",
);
}
}
#[gpui::bench(
inputs = inputs(),
input_name = "case",
group = "agent_model_search",
fps = 120,
sample_size = 15
)]
fn model_search(input: &SearchInput, cx: &mut BenchAppContext) {
let options = Arc::new(options(input));
let expected = Arc::new(expected_values(&options, input.query));
match input.query_name {
"missing" => assert!(expected.is_empty()),
"name-ranking" => {
assert_eq!(
expected.first(),
options.first().map(|option| &option.value)
);
assert_eq!(expected.get(1), options.get(1).map(|option| &option.value));
}
_ => assert!(!expected.is_empty()),
}
let query = input.query;
cx.bench_task(|cx| {
let options = options.clone();
let expected = expected.clone();
let executor = cx.background_executor().clone();
cx.foreground_executor().spawn(async move {
let results = model_search::search(options.to_vec(), query, executor, candidates).await;
ValidatedResults { results, expected }
})
});
}
gpui::bench_group! {
name = benches;
config = criterion::Criterion::default()
.warm_up_time(Duration::from_millis(200))
.measurement_time(Duration::from_millis(500));
targets = model_search
}
gpui::bench_main!(benches);
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
feature_tree="$(cargo tree --locked -p agent_ui_benchmarks -e normal,build,dev,features)"
if grep -F 'feature "test-support"' <<< "$feature_tree"; then
echo "The model search benchmark must not enable test-support." >&2
exit 1
fi
cargo bench --locked -p agent_ui_benchmarks --bench model_search --profile release-fast -- "$@"