feat: token line gains +N per-turn increments; whole-number token format
display.format_token_count now rounds half-up to whole numbers with tier rollover (999999 -> "1M", never "1000k"; 275860 -> "276k"). display.format_token_line renders '-- I:<s>+<t> O:<s>+<t> C:<s>+<t>': the +N suffixes come from new last_turn token fields in the report (CLI --json raw counts + token_display last_turn_* keys; Hermes plugin passes its in-memory turn bucket), render +0 on zero increments, and fail soft when absent - every display path (CLI, --display-lines, GUI, Vibe hooks, Hermes plugin) changes together per the display-parity rule. Docs and tests updated; 125 pass. MINOR bump to 0.8.0. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This commit is contained in:
@@ -208,6 +208,22 @@ And add the cost information to this file for the current session.
|
||||
parity rule); README usage examples show the two-line default output.
|
||||
PATCH bump to 0.6.2; 140 tests pass.
|
||||
|
||||
#### (pending) - feat: token line gains +N per-turn increments; whole-number token format everywhere
|
||||
- **Date**: 2026-11-10
|
||||
- **Model**: mistral-vibe:zai-glm-5-3
|
||||
- **Session Cost**: $3.62 (running total at commit time)
|
||||
- **Turn Cost**: $0.75
|
||||
- **Total for commit**: $3.62
|
||||
- **Notes**: display.py format_token_count now rounds half-up to
|
||||
whole numbers with tier rollover (999999 -> "1M", never "1000k");
|
||||
format_token_line renders "-- I:<s>+<t> O:<s>+<t> C:<s>+<t>",
|
||||
+0 for zero increments, fail-soft without turn fields;
|
||||
--json last_turn gains raw token counts and token_display gains
|
||||
last_turn_* keys; Hermes plugin report carries the turn bucket
|
||||
(parity across all display paths). 125 tests pass; live CLI and
|
||||
hook relay verified. MINOR bump to 0.8.0. Plan:
|
||||
wiki/plans/agent-cost-estimator-token-line-turn-usage-plan.md.
|
||||
|
||||
---
|
||||
|
||||
## Instructions for Contributors
|
||||
|
||||
@@ -33,12 +33,13 @@ agent-cost-estimator --integrate # install Vibe hooks automatically
|
||||
```
|
||||
|
||||
With no output flag the tool prints two lines: the cost line
|
||||
(provider:model, session and latest turn cost) and the session token
|
||||
totals; large counts get k/M/B/T suffixes:
|
||||
(provider:model, session and latest turn cost) and the token counts.
|
||||
Each token field is `<session total>+<latest turn>` at whole numbers
|
||||
with k/M/B/T suffixes (`+0` when the turn added nothing):
|
||||
|
||||
```text
|
||||
mistral:mistral-medium | session:$3.40 | turn:$0.92
|
||||
-- I:1.01k O:2.0k C:10.00M
|
||||
-- I:276k+1k O:2k+900 C:250k+25k
|
||||
```
|
||||
|
||||
`-d/--details` prints the full table including formatted token
|
||||
|
||||
@@ -1723,9 +1723,11 @@ self-contained).
|
||||
default rates, and the token line when the report has token fields
|
||||
- `utils/formatting.py` re-exports `format_token_count` for existing
|
||||
importers (GUI)
|
||||
- Values below 1000 are shown as-is (`999`); larger values use 2
|
||||
decimal places and a suffix: `k` (thousand), `M` (million),
|
||||
`B` (billion), `T` (trillion), e.g. `1.50k`, `42.12M`, `1.00B`
|
||||
- Values below 1000 are shown as-is (`999`); larger values are
|
||||
rounded half-up to a whole number with a suffix: `k` (thousand),
|
||||
`M` (million), `B` (billion), `T` (trillion), e.g. `276k`, `42M`,
|
||||
`1B`; a value whose rounded integer reaches the next boundary
|
||||
rolls its tier (`999_999` -> `1M`, never `1000k`)
|
||||
|
||||
**CLI (`cli.py`):**
|
||||
|
||||
@@ -1733,16 +1735,22 @@ self-contained).
|
||||
|
||||
```text
|
||||
provider:model | session:$X.XX | turn:$Y.YY
|
||||
-- I:<input> O:<output> C:<cached>
|
||||
-- I:<input>+<turn> O:<output>+<turn> C:<cached>+<turn>
|
||||
```
|
||||
|
||||
- Each token field is `<session total>+<last turn>`; the `+N`
|
||||
suffixes appear only when the report's `last_turn` carries token
|
||||
counts (raw `input_tokens`, `cached_input_tokens`,
|
||||
`output_tokens`), and a zero increment renders as `+0`
|
||||
- `--display-lines` is the pinned contract the hooks relays; it is
|
||||
byte-identical to the default output (test-pinned)
|
||||
- `-s` and `-t` print a bare cost number only; no token line
|
||||
- `-d` prints `input tokens (fmt):` and `output tokens (fmt):` lines in
|
||||
addition to the raw counts
|
||||
- `--json` adds a `token_display` object with `input`, `cached`,
|
||||
`output`, and `fresh`, all pre-formatted strings
|
||||
`output`, `fresh`, and the formatted turn values
|
||||
`last_turn_input`, `last_turn_cached`, `last_turn_output` (null
|
||||
last turn); `last_turn` itself carries the three raw token counts
|
||||
|
||||
**GUI (`gui.py`, Tkinter and PyQt5):**
|
||||
|
||||
@@ -1769,6 +1777,10 @@ self-contained).
|
||||
verbatim copy of the package module shipped beside `__init__.py`
|
||||
(byte-identity pinned by `tests/test_agent_cost_estimator.py`);
|
||||
`--integrate hermes` reships the copy
|
||||
- The plugin's report carries the session totals from state.db and
|
||||
this turn's tokens from the in-memory turn bucket
|
||||
(`_turn_bucket`), so it renders the `+N` suffixes like every
|
||||
other path
|
||||
- Cost math, pricing lookup, and the billing-route label stay
|
||||
plugin-side (it publishes to the session event stream mid-turn);
|
||||
only the line format is shared
|
||||
|
||||
@@ -825,11 +825,26 @@ def main(argv=None):
|
||||
"input": format_token_count(usage.input_tokens),
|
||||
"cached": format_token_count(usage.cached_input_tokens),
|
||||
"output": format_token_count(usage.output_tokens),
|
||||
"fresh": format_token_count(usage.fresh_input_tokens)
|
||||
"fresh": format_token_count(usage.fresh_input_tokens),
|
||||
"last_turn_input":
|
||||
format_token_count(last_turn.input_tokens)
|
||||
if last_turn else None,
|
||||
"last_turn_cached":
|
||||
format_token_count(last_turn.cached_input_tokens)
|
||||
if last_turn else None,
|
||||
"last_turn_output":
|
||||
format_token_count(last_turn.output_tokens)
|
||||
if last_turn else None,
|
||||
},
|
||||
"last_turn": {
|
||||
"label": last_turn.label if last_turn else None,
|
||||
"cost_usd": turn_cost,
|
||||
"input_tokens":
|
||||
last_turn.input_tokens if last_turn else None,
|
||||
"cached_input_tokens":
|
||||
last_turn.cached_input_tokens if last_turn else None,
|
||||
"output_tokens":
|
||||
last_turn.output_tokens if last_turn else None,
|
||||
} if last_turn else None,
|
||||
"turns": [
|
||||
{"label": t.label, "calls": t.calls,
|
||||
|
||||
@@ -21,6 +21,12 @@ Consumes the ``--json`` report shape:
|
||||
cached_input_tokens session cached input
|
||||
output_tokens session output
|
||||
model_price_unavailable true when priced at default rates
|
||||
last_turn.input_tokens / cached_input_tokens / output_tokens
|
||||
last turn's raw token counts
|
||||
(optional: present -> the token line
|
||||
renders a "+N" per-field turn
|
||||
increment, e.g. "I:276k+1k"; absent
|
||||
-> session totals only)
|
||||
|
||||
``model_label`` overrides the provider:model label (the Hermes
|
||||
plugin passes its billing-route label).
|
||||
@@ -31,12 +37,22 @@ from __future__ import annotations
|
||||
NOTE_MODEL_PRICE_UNAVAILABLE = " **model price not available**"
|
||||
|
||||
|
||||
# Magnitude units, largest first: a value whose rounded integer
|
||||
# reaches the next boundary rolls its tier (999_999 -> "1M").
|
||||
_UNITS = (
|
||||
(1_000_000_000_000, "T"),
|
||||
(1_000_000_000, "B"),
|
||||
(1_000_000, "M"),
|
||||
(1_000, "k"),
|
||||
)
|
||||
|
||||
|
||||
def format_token_count(count: int) -> str:
|
||||
"""Format a token count with magnitude suffixes (k, M, B, T).
|
||||
|
||||
Numbers below 1000 print as-is; larger magnitudes print with two
|
||||
decimals and the matching suffix (1.50k, 1.23M, 1.00B, 1.00T).
|
||||
None/invalid counts format as "0".
|
||||
Whole numbers only, rounded half-up to the unit: 275_860 ->
|
||||
"276k", 1700 -> "2k", 1234_567 -> "1M". Numbers below 1000 print
|
||||
as-is (500 -> "500"); None or invalid counts format as "0".
|
||||
"""
|
||||
try:
|
||||
count = int(count or 0)
|
||||
@@ -47,14 +63,15 @@ def format_token_count(count: int) -> str:
|
||||
return str(count)
|
||||
if count < 1000:
|
||||
return str(count)
|
||||
elif count < 1_000_000:
|
||||
return f"{count / 1000:.2f}k"
|
||||
elif count < 1_000_000_000:
|
||||
return f"{count / 1_000_000:.2f}M"
|
||||
elif count < 1_000_000_000_000:
|
||||
return f"{count / 1_000_000_000:.2f}B"
|
||||
else:
|
||||
return f"{count / 1_000_000_000_000:.2f}T"
|
||||
# half-up rounding in integers, exact for any 64-bit count
|
||||
for i, (unit, suffix) in enumerate(_UNITS):
|
||||
if count >= unit:
|
||||
value = (2 * count + unit) // (2 * unit)
|
||||
if value >= 1000 and i:
|
||||
unit, suffix = _UNITS[i - 1]
|
||||
value = (2 * count + unit) // (2 * unit)
|
||||
return f"{value}{suffix}"
|
||||
return str(count)
|
||||
|
||||
|
||||
def _model_label(report: dict, model_label: str | None) -> str:
|
||||
@@ -86,10 +103,29 @@ def format_cost_line(report: dict, model_label: str | None = None) -> str:
|
||||
)
|
||||
|
||||
|
||||
def _turn_counts(report: dict) -> tuple[str, str, str] | None:
|
||||
"""Formatted last-turn token counts, or None when the report has
|
||||
no per-turn token fields (older harness reports, pruned turns)."""
|
||||
last_turn = report.get("last_turn") or {}
|
||||
if not any(last_turn.get(field) is not None for field in (
|
||||
"input_tokens", "cached_input_tokens", "output_tokens")):
|
||||
return None
|
||||
return (
|
||||
format_token_count(last_turn.get("input_tokens")),
|
||||
format_token_count(last_turn.get("cached_input_tokens")),
|
||||
format_token_count(last_turn.get("output_tokens")),
|
||||
)
|
||||
|
||||
|
||||
def format_token_line(report: dict) -> str | None:
|
||||
"""Line 2: ``-- I:<input> O:<output> C:<cached>`` from SESSION
|
||||
totals, or None when the report carries no token fields (older
|
||||
harness reports) - the cost line never depends on them.
|
||||
"""Line 2: ``-- I:<in>+<turn> O:<out>+<turn> C:<cached>+<turn>``
|
||||
from SESSION totals plus the last turn's tokens, or None when the
|
||||
report carries no token fields (older harness reports) - the cost
|
||||
line never depends on them.
|
||||
|
||||
The ``+N`` turn suffixes appear only when the report carries
|
||||
per-turn token fields under ``last_turn``; a zero increment
|
||||
renders as ``+0``. Missing data omits the parts; nothing errors.
|
||||
"""
|
||||
has_tokens = any(
|
||||
report.get(field) is not None
|
||||
@@ -98,11 +134,18 @@ def format_token_line(report: dict) -> str | None:
|
||||
)
|
||||
if not has_tokens:
|
||||
return None
|
||||
turn = _turn_counts(report)
|
||||
if turn is None:
|
||||
return (
|
||||
f"-- I:{format_token_count(report.get('input_tokens'))} "
|
||||
f"O:{format_token_count(report.get('output_tokens'))} "
|
||||
f"C:{format_token_count(report.get('cached_input_tokens'))}"
|
||||
)
|
||||
return (
|
||||
f"-- I:{format_token_count(report.get('input_tokens'))}+{turn[0]} "
|
||||
f"O:{format_token_count(report.get('output_tokens'))}+{turn[2]} "
|
||||
f"C:{format_token_count(report.get('cached_input_tokens'))}+{turn[1]}"
|
||||
)
|
||||
|
||||
|
||||
def format_display_lines(
|
||||
@@ -113,8 +156,11 @@ def format_display_lines(
|
||||
provider:model | session:$X.XX | turn:$Y.YY
|
||||
**model price not available** (only when priced at
|
||||
default rates)
|
||||
-- I:<input> O:<output> C:<cached> (only when tokens
|
||||
are in the report)
|
||||
-- I:<in>+<turn> O:<out>+<turn> (only when tokens
|
||||
C:<cached>+<turn> are in the report;
|
||||
the +N suffixes only
|
||||
when the report has
|
||||
last-turn tokens)
|
||||
"""
|
||||
lines = [format_cost_line(report, model_label)]
|
||||
if report.get("model_price_unavailable"):
|
||||
|
||||
@@ -25,7 +25,8 @@ Each harness has a `post_turn.py` hook script that:
|
||||
`system_message` JSON (with the `-- ` prefix on the first line)
|
||||
|
||||
The hooks hold NO formatting of their own: the cost line, the
|
||||
model-price note, and the token line (`-- I:X O:Y C:Z`) are all
|
||||
model-price note, and the token line (`-- I:<session>+<turn>
|
||||
O:<session>+<turn> C:<session>+<turn>`) are all
|
||||
rendered by `agent_cost_estimator.display` in the package — the
|
||||
display-path parity rule (SPEC.md 5.9). A format change ships with
|
||||
the CLI; the hooks pick it up automatically, so no re-integration
|
||||
|
||||
@@ -27,7 +27,7 @@ What it does (idempotent):
|
||||
After that, lines like:
|
||||
|
||||
-- zai:zai-glm-5-3 | session:$0.42 | turn:$0.16
|
||||
-- I:44.56M O:102.39k C:43.79M
|
||||
-- I:45M+58k O:102k+3k C:44M+40k
|
||||
|
||||
print after every assistant turn, mirroring the Mistral Vibe CLI
|
||||
hook. The hermes harness adapter reads usage back via:
|
||||
|
||||
@@ -669,7 +669,14 @@ def _print_cost_line(session_id: str, turn_id: str, model: str) -> None:
|
||||
# agent_cost_estimator.display (the --json report's fields).
|
||||
report = {
|
||||
"cost_usd": {"total_usd": _cost(session, rates)},
|
||||
"last_turn": {"cost_usd": {"total_usd": _cost(turn, rates)}},
|
||||
"last_turn": {
|
||||
"cost_usd": {"total_usd": _cost(turn, rates)},
|
||||
# turn tokens for the display's +N suffixes (the bucket
|
||||
# keys are the fold's inclusive contract)
|
||||
"input_tokens": turn["input"],
|
||||
"cached_input_tokens": turn["cached"],
|
||||
"output_tokens": turn["output"],
|
||||
},
|
||||
"input_tokens": session["input"],
|
||||
"cached_input_tokens": session["cached"],
|
||||
"output_tokens": session["output"],
|
||||
|
||||
@@ -21,6 +21,12 @@ Consumes the ``--json`` report shape:
|
||||
cached_input_tokens session cached input
|
||||
output_tokens session output
|
||||
model_price_unavailable true when priced at default rates
|
||||
last_turn.input_tokens / cached_input_tokens / output_tokens
|
||||
last turn's raw token counts
|
||||
(optional: present -> the token line
|
||||
renders a "+N" per-field turn
|
||||
increment, e.g. "I:276k+1k"; absent
|
||||
-> session totals only)
|
||||
|
||||
``model_label`` overrides the provider:model label (the Hermes
|
||||
plugin passes its billing-route label).
|
||||
@@ -31,12 +37,22 @@ from __future__ import annotations
|
||||
NOTE_MODEL_PRICE_UNAVAILABLE = " **model price not available**"
|
||||
|
||||
|
||||
# Magnitude units, largest first: a value whose rounded integer
|
||||
# reaches the next boundary rolls its tier (999_999 -> "1M").
|
||||
_UNITS = (
|
||||
(1_000_000_000_000, "T"),
|
||||
(1_000_000_000, "B"),
|
||||
(1_000_000, "M"),
|
||||
(1_000, "k"),
|
||||
)
|
||||
|
||||
|
||||
def format_token_count(count: int) -> str:
|
||||
"""Format a token count with magnitude suffixes (k, M, B, T).
|
||||
|
||||
Numbers below 1000 print as-is; larger magnitudes print with two
|
||||
decimals and the matching suffix (1.50k, 1.23M, 1.00B, 1.00T).
|
||||
None/invalid counts format as "0".
|
||||
Whole numbers only, rounded half-up to the unit: 275_860 ->
|
||||
"276k", 1700 -> "2k", 1234_567 -> "1M". Numbers below 1000 print
|
||||
as-is (500 -> "500"); None or invalid counts format as "0".
|
||||
"""
|
||||
try:
|
||||
count = int(count or 0)
|
||||
@@ -47,14 +63,15 @@ def format_token_count(count: int) -> str:
|
||||
return str(count)
|
||||
if count < 1000:
|
||||
return str(count)
|
||||
elif count < 1_000_000:
|
||||
return f"{count / 1000:.2f}k"
|
||||
elif count < 1_000_000_000:
|
||||
return f"{count / 1_000_000:.2f}M"
|
||||
elif count < 1_000_000_000_000:
|
||||
return f"{count / 1_000_000_000:.2f}B"
|
||||
else:
|
||||
return f"{count / 1_000_000_000_000:.2f}T"
|
||||
# half-up rounding in integers, exact for any 64-bit count
|
||||
for i, (unit, suffix) in enumerate(_UNITS):
|
||||
if count >= unit:
|
||||
value = (2 * count + unit) // (2 * unit)
|
||||
if value >= 1000 and i:
|
||||
unit, suffix = _UNITS[i - 1]
|
||||
value = (2 * count + unit) // (2 * unit)
|
||||
return f"{value}{suffix}"
|
||||
return str(count)
|
||||
|
||||
|
||||
def _model_label(report: dict, model_label: str | None) -> str:
|
||||
@@ -86,10 +103,29 @@ def format_cost_line(report: dict, model_label: str | None = None) -> str:
|
||||
)
|
||||
|
||||
|
||||
def _turn_counts(report: dict) -> tuple[str, str, str] | None:
|
||||
"""Formatted last-turn token counts, or None when the report has
|
||||
no per-turn token fields (older harness reports, pruned turns)."""
|
||||
last_turn = report.get("last_turn") or {}
|
||||
if not any(last_turn.get(field) is not None for field in (
|
||||
"input_tokens", "cached_input_tokens", "output_tokens")):
|
||||
return None
|
||||
return (
|
||||
format_token_count(last_turn.get("input_tokens")),
|
||||
format_token_count(last_turn.get("cached_input_tokens")),
|
||||
format_token_count(last_turn.get("output_tokens")),
|
||||
)
|
||||
|
||||
|
||||
def format_token_line(report: dict) -> str | None:
|
||||
"""Line 2: ``-- I:<input> O:<output> C:<cached>`` from SESSION
|
||||
totals, or None when the report carries no token fields (older
|
||||
harness reports) - the cost line never depends on them.
|
||||
"""Line 2: ``-- I:<in>+<turn> O:<out>+<turn> C:<cached>+<turn>``
|
||||
from SESSION totals plus the last turn's tokens, or None when the
|
||||
report carries no token fields (older harness reports) - the cost
|
||||
line never depends on them.
|
||||
|
||||
The ``+N`` turn suffixes appear only when the report carries
|
||||
per-turn token fields under ``last_turn``; a zero increment
|
||||
renders as ``+0``. Missing data omits the parts; nothing errors.
|
||||
"""
|
||||
has_tokens = any(
|
||||
report.get(field) is not None
|
||||
@@ -98,11 +134,18 @@ def format_token_line(report: dict) -> str | None:
|
||||
)
|
||||
if not has_tokens:
|
||||
return None
|
||||
turn = _turn_counts(report)
|
||||
if turn is None:
|
||||
return (
|
||||
f"-- I:{format_token_count(report.get('input_tokens'))} "
|
||||
f"O:{format_token_count(report.get('output_tokens'))} "
|
||||
f"C:{format_token_count(report.get('cached_input_tokens'))}"
|
||||
)
|
||||
return (
|
||||
f"-- I:{format_token_count(report.get('input_tokens'))}+{turn[0]} "
|
||||
f"O:{format_token_count(report.get('output_tokens'))}+{turn[2]} "
|
||||
f"C:{format_token_count(report.get('cached_input_tokens'))}+{turn[1]}"
|
||||
)
|
||||
|
||||
|
||||
def format_display_lines(
|
||||
@@ -113,8 +156,11 @@ def format_display_lines(
|
||||
provider:model | session:$X.XX | turn:$Y.YY
|
||||
**model price not available** (only when priced at
|
||||
default rates)
|
||||
-- I:<input> O:<output> C:<cached> (only when tokens
|
||||
are in the report)
|
||||
-- I:<in>+<turn> O:<out>+<turn> (only when tokens
|
||||
C:<cached>+<turn> are in the report;
|
||||
the +N suffixes only
|
||||
when the report has
|
||||
last-turn tokens)
|
||||
"""
|
||||
lines = [format_cost_line(report, model_label)]
|
||||
if report.get("model_price_unavailable"):
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "agent-cost-estimator"
|
||||
version = "0.7.0"
|
||||
version = "0.8.0"
|
||||
description = "Report token usage and estimated USD cost of AI agent sessions from local harness logs"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.9"
|
||||
|
||||
@@ -552,12 +552,12 @@ def test_cli_default_compact_line(tmp_path, capsys):
|
||||
+ 500_000 * 0.15 + 100_000 * 7.5) / 1e6
|
||||
turn_cost = (500_000 * 1.5 + 500_000 * 0.15
|
||||
+ 100_000 * 7.5) / 1e6
|
||||
# Output format is now: provider:model | session:$X | turn:$Y
|
||||
# followed by: -- I:X O:Y C:Z
|
||||
# Output format is: provider:model | session:$X | turn:$Y
|
||||
# followed by note line and: -- I:<s>+<t> O:<s>+<t> C:<s>+<t>
|
||||
# Model 'm1' is not in pricing, so it falls back to default with warning
|
||||
expected = (f"mistral-vibe:default | session:${session_cost:.2f} | "
|
||||
f"turn:${turn_cost:.2f}\n **model price not available**\n"
|
||||
f"-- I:2.00M O:100.00k C:500.00k")
|
||||
f"-- I:2M+1M O:100k+100k C:500k+500k")
|
||||
assert out == expected
|
||||
|
||||
|
||||
@@ -575,7 +575,9 @@ def test_cli_display_lines_flag_matches_default(tmp_path, capsys):
|
||||
default = capsys.readouterr().out
|
||||
assert cli.main(args + ["--display-lines"]) == 0
|
||||
assert capsys.readouterr().out == default
|
||||
assert "-- I:2.00M O:100.00k C:500.00k" in default
|
||||
# session totals double as the turn's counts (single call), so
|
||||
# the token line shows the same value twice per field
|
||||
assert "-- I:2M+2M O:100k+100k C:500k+500k" in default
|
||||
|
||||
|
||||
def test_cli_details_shows_the_full_table(tmp_path, capsys):
|
||||
@@ -1267,7 +1269,7 @@ def test_plugin_display_renders_canonical_lines():
|
||||
}
|
||||
assert ace_display.format_display_lines(report) == [
|
||||
"zai:zai-glm-5-3 | session:$0.42 | turn:$0.00",
|
||||
"-- I:44.56M O:102.39k C:43.79M",
|
||||
"-- I:45M O:102k C:44M",
|
||||
]
|
||||
|
||||
|
||||
@@ -1284,9 +1286,11 @@ def test_plugin_prints_token_line(tmp_path, monkeypatch, capsys):
|
||||
hp._on_post_llm_call(session_id="s1", turn_id="t1",
|
||||
model="zai-glm-5-3", platform="cli")
|
||||
out = capsys.readouterr().out
|
||||
assert f"-- I:{format_token_count(1000 + 900000 + 5000)} " \
|
||||
f"O:{format_token_count(20000)} " \
|
||||
f"C:{format_token_count(900000)}" in out
|
||||
# session totals from state.db; the (empty) turn bucket renders
|
||||
# as +0 suffixes on every field
|
||||
assert f"-- I:{format_token_count(1000 + 900000 + 5000)}+0 " \
|
||||
f"O:{format_token_count(20000)}+0 " \
|
||||
f"C:{format_token_count(900000)}+0" in out
|
||||
assert "session:$" in out and "turn:$" in out
|
||||
|
||||
|
||||
|
||||
+47
-74
@@ -1,101 +1,74 @@
|
||||
"""Tests for formatting utilities."""
|
||||
"""Tests for the token-count formatter.
|
||||
|
||||
import pytest
|
||||
The formatter lives in agent_cost_estimator.display; this module
|
||||
exercises it through the utils.formatting re-export (the import path
|
||||
GUI and older call sites use). Whole-number format: half-up rounding,
|
||||
tier rollover at the boundary (999999 -> "1M").
|
||||
"""
|
||||
|
||||
from agent_cost_estimator.utils.formatting import format_token_count
|
||||
|
||||
|
||||
class TestFormatTokenCount:
|
||||
"""Test cases for format_token_count function."""
|
||||
|
||||
def test_zero(self):
|
||||
"""Test that zero is formatted correctly."""
|
||||
def test_zero_and_invalid(self):
|
||||
"""Zero and None format as "0"."""
|
||||
assert format_token_count(0) == "0"
|
||||
assert format_token_count(None) == "0"
|
||||
|
||||
def test_small_numbers(self):
|
||||
"""Test numbers less than 1000."""
|
||||
"""Numbers less than 1000 print as-is."""
|
||||
assert format_token_count(1) == "1"
|
||||
assert format_token_count(500) == "500"
|
||||
assert format_token_count(999) == "999"
|
||||
|
||||
def test_exactly_one_thousand(self):
|
||||
"""Test the threshold at 1000."""
|
||||
assert format_token_count(1000) == "1.00k"
|
||||
|
||||
def test_thousands(self):
|
||||
"""Test numbers in the thousands range."""
|
||||
assert format_token_count(1000) == "1.00k"
|
||||
assert format_token_count(1500) == "1.50k"
|
||||
assert format_token_count(2000) == "2.00k"
|
||||
# 9999 / 1000 = 9.999 -> rounds to 10.00 with 2 decimal places
|
||||
assert format_token_count(9999) == "10.00k"
|
||||
assert format_token_count(10000) == "10.00k"
|
||||
assert format_token_count(100000) == "100.00k"
|
||||
# 999999 / 1000 = 999.999 -> rounds to 1000.00
|
||||
# But 1000.00k = 1.00M, so we need to check this
|
||||
# Actually 999999 / 1000 = 999.999 -> "1000.00k" which is wrong
|
||||
# Let's use 999000 instead
|
||||
assert format_token_count(999000) == "999.00k"
|
||||
"""The k tier, whole numbers, half-up rounding."""
|
||||
assert format_token_count(1000) == "1k"
|
||||
assert format_token_count(1700) == "2k"
|
||||
# half-up, not banker's rounding: 2.5 -> 3
|
||||
assert format_token_count(2500) == "3k"
|
||||
# 9.999 rounds to 10
|
||||
assert format_token_count(9999) == "10k"
|
||||
assert format_token_count(10000) == "10k"
|
||||
assert format_token_count(100000) == "100k"
|
||||
assert format_token_count(999499) == "999k"
|
||||
# the plan's example values
|
||||
assert format_token_count(275860) == "276k"
|
||||
assert format_token_count(250180) == "250k"
|
||||
|
||||
def test_exactly_one_million(self):
|
||||
"""Test the threshold at 1 million."""
|
||||
assert format_token_count(1_000_000) == "1.00M"
|
||||
def test_tier_rollover(self):
|
||||
"""A value whose rounded integer reaches the next boundary
|
||||
rolls its tier; "1000k" never occurs."""
|
||||
assert format_token_count(999999) == "1M"
|
||||
assert format_token_count(999999999) == "1B"
|
||||
assert format_token_count(999_999_999_999) == "1T"
|
||||
|
||||
def test_millions(self):
|
||||
"""Test numbers in the millions range."""
|
||||
assert format_token_count(1_000_000) == "1.00M"
|
||||
assert format_token_count(1_234_567) == "1.23M"
|
||||
assert format_token_count(1_500_000) == "1.50M"
|
||||
# 9999999 / 1000000 = 9.999999 -> rounds to 10.00
|
||||
assert format_token_count(9_999_999) == "10.00M"
|
||||
assert format_token_count(10_000_000) == "10.00M"
|
||||
|
||||
def test_exactly_one_billion(self):
|
||||
"""Test the threshold at 1 billion."""
|
||||
assert format_token_count(1_000_000_000) == "1.00B"
|
||||
"""The M tier."""
|
||||
assert format_token_count(1_000_000) == "1M"
|
||||
assert format_token_count(1_234_567) == "1M"
|
||||
assert format_token_count(1_500_000) == "2M"
|
||||
assert format_token_count(9_999_999) == "10M"
|
||||
|
||||
def test_billions(self):
|
||||
"""Test numbers in the billions range."""
|
||||
assert format_token_count(1_000_000_000) == "1.00B"
|
||||
assert format_token_count(1_500_000_000) == "1.50B"
|
||||
# 9999999999 / 1000000000 = 9.999999999 -> rounds to 10.00
|
||||
assert format_token_count(9_999_999_999) == "10.00B"
|
||||
assert format_token_count(10_000_000_000) == "10.00B"
|
||||
|
||||
def test_exactly_one_trillion(self):
|
||||
"""Test the threshold at 1 trillion."""
|
||||
assert format_token_count(1_000_000_000_000) == "1.00T"
|
||||
"""The B tier."""
|
||||
assert format_token_count(1_000_000_000) == "1B"
|
||||
assert format_token_count(1_500_000_000) == "2B"
|
||||
assert format_token_count(9_999_999_999) == "10B"
|
||||
|
||||
def test_trillions(self):
|
||||
"""Test numbers in the trillions range."""
|
||||
assert format_token_count(1_000_000_000_000) == "1.00T"
|
||||
assert format_token_count(1_500_000_000_000) == "1.50T"
|
||||
# 9_999_000_000_000_000 / 1_000_000_000_000 = 9.999 -> rounds to 10.00
|
||||
# Use 9_990_000_000_000 instead to get 9.99
|
||||
assert format_token_count(9_990_000_000_000) == "9.99T"
|
||||
"""The T tier."""
|
||||
assert format_token_count(1_000_000_000_000) == "1T"
|
||||
assert format_token_count(1_500_000_000_000) == "2T"
|
||||
assert format_token_count(9_999_999_999_999) == "10T"
|
||||
|
||||
def test_negative_numbers(self):
|
||||
"""Test that negative numbers are handled gracefully."""
|
||||
# Negative numbers shouldn't occur in practice, but handle them
|
||||
def test_negative(self):
|
||||
"""Negative numbers are out of contract but print as-is."""
|
||||
assert format_token_count(-1) == "-1"
|
||||
assert format_token_count(-1000) == "-1000"
|
||||
|
||||
def test_very_large_numbers(self):
|
||||
"""Test very large numbers beyond trillion."""
|
||||
assert format_token_count(10_000_000_000_000) == "10.00T"
|
||||
assert format_token_count(100_000_000_000_000) == "100.00T"
|
||||
|
||||
def test_rounding(self):
|
||||
"""Test that rounding is done correctly."""
|
||||
# 1500 / 1000 = 1.5 -> "1.50k"
|
||||
assert format_token_count(1500) == "1.50k"
|
||||
# 1550 / 1000 = 1.55 -> "1.55k"
|
||||
assert format_token_count(1550) == "1.55k"
|
||||
# 1555 / 1000 = 1.555 -> rounded to "1.56k"? No, .2f truncates to 1.55
|
||||
# Actually .2f means 2 decimal places, so 1.555 becomes 1.55 (banker's rounding)
|
||||
assert format_token_count(1555) == "1.55k"
|
||||
|
||||
def test_type_handling(self):
|
||||
"""Test that the function handles integer types correctly."""
|
||||
assert format_token_count(int(1000)) == "1.00k"
|
||||
assert format_token_count(1000) == "1.00k"
|
||||
"""Integer types (and int-coercible values) are handled."""
|
||||
assert format_token_count(int(1000)) == "1k"
|
||||
assert format_token_count(1000) == "1k"
|
||||
|
||||
+44
-4
@@ -63,7 +63,7 @@ class TestHookRelay:
|
||||
with the system_message '-- ' prefix on the first line."""
|
||||
hook = _load_hook(hook_name)
|
||||
cli_out = ("zai:zai-glm-5-3 | session:$0.01 | turn:$0.00\n"
|
||||
"-- I:42.12M O:95.07k C:41.36M")
|
||||
"-- I:42M+1k O:95k+900 C:41M+25k")
|
||||
calls, message = self._run_hook(
|
||||
hook, monkeypatch, capsys, cli_out)
|
||||
assert "--display-lines" in calls["cmd"]
|
||||
@@ -77,7 +77,7 @@ class TestHookRelay:
|
||||
hook = _load_hook(hook_name)
|
||||
cli_out = ("default:m | session:$0.01 | turn:$0.00\n"
|
||||
" **model price not available**\n"
|
||||
"-- I:2.00M O:100.00k C:500.00k")
|
||||
"-- I:2M+1M O:100k+100k C:500k+500k")
|
||||
calls, message = self._run_hook(
|
||||
hook, monkeypatch, capsys, cli_out)
|
||||
assert message == {"system_message": f"-- {cli_out}"}
|
||||
@@ -132,9 +132,49 @@ class TestDisplayParity:
|
||||
}
|
||||
assert format_display_lines(report) == [
|
||||
"zai:zai-glm-5-3 | session:$0.01 | turn:$0.00",
|
||||
"-- I:42.12M O:95.07k C:41.36M",
|
||||
"-- I:42M O:95k C:41M",
|
||||
]
|
||||
|
||||
def test_token_line_turn_suffixes(self):
|
||||
"""last_turn token fields render "+N" per field: the token
|
||||
line shows session total plus last turn for each of I/O/C."""
|
||||
report = {
|
||||
"pricing": {"model_key": "zai:m"},
|
||||
"cost_usd": {"total_usd": 0.42},
|
||||
"last_turn": {
|
||||
"cost_usd": {"total_usd": 0.01},
|
||||
"input_tokens": 1000,
|
||||
"cached_input_tokens": 25_000,
|
||||
"output_tokens": 900,
|
||||
},
|
||||
"input_tokens": 275_860,
|
||||
"cached_input_tokens": 250_180,
|
||||
"output_tokens": 1700,
|
||||
"model_price_unavailable": False,
|
||||
}
|
||||
assert format_display_lines(report)[-1] == (
|
||||
"-- I:276k+1k O:2k+900 C:250k+25k")
|
||||
|
||||
def test_token_line_zero_turn_increment_renders_plus_zero(self):
|
||||
"""A turn that added nothing still gets its suffix (+0);
|
||||
only MISSING data omits the parts."""
|
||||
report = {
|
||||
"pricing": {"model_key": "zai:m"},
|
||||
"cost_usd": {"total_usd": 0.42},
|
||||
"last_turn": {
|
||||
"cost_usd": {"total_usd": 0.00},
|
||||
"input_tokens": 0,
|
||||
"cached_input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
},
|
||||
"input_tokens": 906_000,
|
||||
"cached_input_tokens": 900_000,
|
||||
"output_tokens": 20_000,
|
||||
"model_price_unavailable": False,
|
||||
}
|
||||
assert format_display_lines(report)[-1] == (
|
||||
"-- I:906k+0 O:20k+0 C:900k+0")
|
||||
|
||||
def test_token_line_omitted_without_token_fields(self):
|
||||
report = {
|
||||
"pricing": {"model_key": "zai:m"},
|
||||
@@ -182,5 +222,5 @@ class TestDisplayParity:
|
||||
assert format_display_lines(report) == [
|
||||
"d:m | session:$0.42 | turn:$0.00",
|
||||
" **model price not available**",
|
||||
"-- I:2.00M O:100.00k C:500.00k",
|
||||
"-- I:2M O:100k C:500k",
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user