feat: canonical display module - one format for CLI, GUI, hooks, Hermes plugin
New agent_cost_estimator/display.py is the single source of the cost line, model-price note, and session token line format. - CLI main() builds one report object feeding --json and the default output; new --display-lines flag is the pinned contract the Vibe hooks relay (identical to the default output, test-pinned) - Both Vibe hook scripts thinned to verbatim relays: --display-lines output wrapped in system_message; local formatters removed - Hermes plugin renders through a verbatim display.py copy shipped by install.py (byte-parity pinned by tests) and now prints the token line too - token-display spec section 11 open item closed - utils/formatting.py re-exports format_token_count (GUI unchanged) - SPEC.md 5.7/5.9 + module map, hooks README, plugin.yaml 0.5.0 - MINOR bump to 0.7.0; 128 tests pass Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This commit is contained in:
@@ -42,6 +42,24 @@ And add the cost information to this file for the current session.
|
||||
|
||||
### Main Branch
|
||||
|
||||
#### cf566bd - feat: canonical display module (display-path parity for CLI, GUI, hooks, Hermes plugin)
|
||||
- **Date**: 2026-11-11
|
||||
- **Model**: mistral-vibe:zai-glm-5-3
|
||||
- **Session Cost**: $1.50 (running total at commit time; cumulative session 4cd78665)
|
||||
- **Turn Cost**: $0.71
|
||||
- **Notes**: Generic display architecture: new self-contained
|
||||
agent_cost_estimator/display.py is the single source of the
|
||||
cost line, model-price note, and token line; main() builds one
|
||||
report object feeding --json and the default output; new
|
||||
--display-lines flag is the pinned hook contract (default output
|
||||
== display lines, test-pinned). Both Vibe hooks thinned to
|
||||
verbatim relays (local formatters removed). Hermes plugin renders
|
||||
through a verbatim display.py copy shipped by install.py
|
||||
(byte-parity pinned by tests), closing the spec 11 open item:
|
||||
Hermes now prints the token line too. utils/formatting.py
|
||||
re-exports format_token_count for the GUI. SPEC.md 5.7/5.9 +
|
||||
module map updated; MINOR bump to 0.7.0; 128 tests pass.
|
||||
|
||||
#### (pending) - feat: add Hermes Agent support (harness adapter, plugin, installer)
|
||||
- **Date**: 2026-10-24
|
||||
- **Model**: zai:zai-glm-5-3 (Hermes Agent)
|
||||
|
||||
@@ -27,6 +27,7 @@ agent-cost-estimator -t # latest turn cost only
|
||||
agent-cost-estimator -d --turns # table plus per-turn breakdown
|
||||
agent-cost-estimator --session-id 5732da2a # a specific session id or path
|
||||
agent-cost-estimator --json # machine-readable output
|
||||
agent-cost-estimator --display-lines # the hook/relay display format (same as default)
|
||||
agent-cost-estimator --list-harnesses # available adapters
|
||||
agent-cost-estimator --integrate # install Vibe hooks automatically
|
||||
```
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
---
|
||||
name: Agent Cost Estimator Specification
|
||||
version: 1.1.0
|
||||
version: 1.2.0
|
||||
title: Project Spec Sheet
|
||||
type: specification
|
||||
created: 2026-10-24
|
||||
updated: 2026-10-24
|
||||
updated: 2026-11-11
|
||||
---
|
||||
|
||||
# Agent Cost Estimator Specification
|
||||
@@ -201,6 +201,7 @@ agent-cost-estimator/
|
||||
│ ├── __main__.py # Entry point for python -m
|
||||
│ ├── cli.py # CLI interface and main()
|
||||
│ ├── core.py # Cost computation logic
|
||||
│ ├── display.py # Canonical cost/token display format (stdlib-only)
|
||||
│ ├── pricing.py # Pricing table handling
|
||||
│ │
|
||||
│ ├── harnesses/
|
||||
@@ -211,9 +212,9 @@ agent-cost-estimator/
|
||||
│ │
|
||||
│ ├── hooks/
|
||||
│ │ ├── mistral-vibe-cli/
|
||||
│ │ │ └── post_turn.py # Hook for Mistral Vibe CLI
|
||||
│ │ ├── mistral-vibe-code/
|
||||
│ │ │ └── post_turn.py # Hook for Mistral Vibe VSCodium extension
|
||||
│ │ │ └── post_turn.py # Hook for Mistral Vibe CLI (thin relay)
|
||||
│ │ └── mistral-vibe-code/
|
||||
│ │ └── post_turn.py # Hook for Mistral Vibe VSCodium extension (thin relay)
|
||||
│ │ └── README.md # Overview of hook structure
|
||||
│ │
|
||||
│ ├── integration/
|
||||
@@ -221,6 +222,7 @@ agent-cost-estimator/
|
||||
│ │ ├── install.py # Standalone installer (any Hermes home)
|
||||
│ │ └── plugin/ # THE Hermes plugin (copied by install.py)
|
||||
│ │ ├── plugin.yaml
|
||||
│ │ ├── display.py # Verbatim copy of the package display module
|
||||
│ │ └── __init__.py
|
||||
│ │
|
||||
│ └── skill/
|
||||
@@ -859,7 +861,12 @@ To add a new harness `openai`:
|
||||
|
||||
**Status:** Implemented
|
||||
|
||||
**Default Format:**
|
||||
**Canonical display lines:** rendered by
|
||||
`agent_cost_estimator/display.py` (`format_display_lines`), shared
|
||||
by the CLI default output, the `--display-lines` flag (what the
|
||||
Vibe hook scripts relay), and the Hermes plugin (verbatim copy of
|
||||
the module, shipped beside the plugin). One module, one format:
|
||||
change it there and every display path changes together.
|
||||
|
||||
```
|
||||
<pool-or-provider>:<model> | session:$X.XX | turn:$Y.YY
|
||||
@@ -1706,24 +1713,31 @@ agent-cost-estimator --skill
|
||||
**Feature:** Display token usage (input, output, cached input) alongside
|
||||
cost data in every display path, formatted with magnitude suffixes.
|
||||
|
||||
**Formatter:** `agent_cost_estimator/utils/formatting.py`
|
||||
**Canonical formatter:** `agent_cost_estimator/display.py` (stdlib-only,
|
||||
self-contained).
|
||||
|
||||
- `format_token_count(count: int) -> str`
|
||||
- `format_display_lines(report: dict, model_label=None) -> list[str]`
|
||||
renders the full display from the `--json` report shape: the cost
|
||||
line, the ` **model price not available**` note when priced at
|
||||
default rates, and the token line when the report has token fields
|
||||
- `utils/formatting.py` re-exports `format_token_count` for existing
|
||||
importers (GUI)
|
||||
- Values below 1000 are shown as-is (`999`); larger values use 2
|
||||
decimal places and a suffix: `k` (thousand), `M` (million),
|
||||
`B` (billion), `T` (trillion), e.g. `1.50k`, `42.12M`, `1.00B`
|
||||
- Suffixes round with standard Python rounding, so a value such as
|
||||
9999 formats as `10.00k`, not `9.99k`
|
||||
|
||||
**CLI (`cli.py`):**
|
||||
|
||||
- Default output adds a token line after the cost line:
|
||||
- Default output and `--display-lines` print the canonical lines:
|
||||
|
||||
```text
|
||||
provider:model | session:$X.XX | turn:$Y.YY
|
||||
-- I:<input> O:<output> C:<cached>
|
||||
```
|
||||
|
||||
- `--display-lines` is the pinned contract the hooks relays; it is
|
||||
byte-identical to the default output (test-pinned)
|
||||
- `-s` and `-t` print a bare cost number only; no token line
|
||||
- `-d` prints `input tokens (fmt):` and `output tokens (fmt):` lines in
|
||||
addition to the raw counts
|
||||
@@ -1733,7 +1747,8 @@ cost data in every display path, formatted with magnitude suffixes.
|
||||
**GUI (`gui.py`, Tkinter and PyQt5):**
|
||||
|
||||
- Three right-aligned columns (Input, Output, Cached) between Model and
|
||||
Session Price, populated from `SessionInfo` token counts
|
||||
Session Price, populated from `SessionInfo` token counts via the
|
||||
shared `format_token_count`
|
||||
- A bold, theme-tinted `**This Month Usage**` summary row is the first
|
||||
child of each harness node, aggregating the sessions last modified in
|
||||
the current month; hidden when the month has no sessions
|
||||
@@ -1741,15 +1756,24 @@ cost data in every display path, formatted with magnitude suffixes.
|
||||
**Vibe hooks (`hooks/mistral-vibe-cli/post_turn.py`,
|
||||
`hooks/mistral-vibe-code/post_turn.py`):**
|
||||
|
||||
- The hook's cost line is followed by `-- I:<input> O:<output>
|
||||
C:<cached>` built from the JSON report's session totals
|
||||
- The hook carries a stdlib-only copy of `format_token_count` (hook
|
||||
scripts must not import the package); parity is pinned by
|
||||
`tests/test_hooks.py`
|
||||
- The token line is omitted when the report lacks token fields
|
||||
- The hooks hold NO formatting of their own: they run
|
||||
`agent-cost-estimator --session-id <dir> --display-lines` and relay
|
||||
the CLI's stdout verbatim, wrapped in the `system_message` JSON with
|
||||
the `-- ` prefix on the first line
|
||||
- `tests/test_hooks.py` pins the relay contract (no `--json` display
|
||||
parsing, no local formatters)
|
||||
|
||||
**Open:** the Hermes plugin still prints cost-only; mirroring the token
|
||||
line there requires a plugin release plus a gateway restart.
|
||||
**Hermes plugin (`integration/hermes/plugin/`):**
|
||||
|
||||
**Display-path parity rule:** any future change to report display must
|
||||
cover CLI, GUI, and hook output in the same change.
|
||||
- The plugin renders the canonical lines through `display.py`, a
|
||||
verbatim copy of the package module shipped beside `__init__.py`
|
||||
(byte-identity pinned by `tests/test_agent_cost_estimator.py`);
|
||||
`--integrate hermes` reships the copy
|
||||
- Cost math, pricing lookup, and the billing-route label stay
|
||||
plugin-side (it publishes to the session event stream mid-turn);
|
||||
only the line format is shared
|
||||
|
||||
**Display-path parity rule:** any future change to report display
|
||||
must cover CLI, GUI, hooks, and the Hermes plugin in the same
|
||||
change - in practice, edit `display.py` once, re-ship the plugin
|
||||
with `--integrate hermes`, and the tests keep every path pinned.
|
||||
|
||||
+69
-73
@@ -9,6 +9,7 @@ import pathlib
|
||||
import sys
|
||||
|
||||
from .core import compute_cost
|
||||
from .display import format_display_lines
|
||||
from .harnesses import available_harnesses, get_harness
|
||||
from .pricing import (
|
||||
DEFAULT_PRICING_PATH,
|
||||
@@ -85,6 +86,12 @@ def parse_args(argv):
|
||||
"-d", "--details", action="store_true",
|
||||
help="the full report table (cost lines, sources, notes)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--display-lines", action="store_true",
|
||||
help="print exactly the display lines (cost line plus "
|
||||
"token line) the hooks and plugins render - the "
|
||||
"canonical format, same as the default output",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-s", "--session", action="store_true",
|
||||
help="print only the session cost, as a bare number",
|
||||
@@ -194,7 +201,9 @@ def integrate_hooks(harness_name):
|
||||
hook_dest.chmod(hook_dest.stat().st_mode | stat.S_IEXEC)
|
||||
print(f"Installed hook script: {hook_dest}")
|
||||
else:
|
||||
# Fallback: Generate inline hook script
|
||||
# Fallback: Generate inline hook script (a thin relay;
|
||||
# the display format lives in the package - see
|
||||
# agent_cost_estimator.display)
|
||||
hook_content = '''#!/usr/bin/env python3
|
||||
import os, sys, json, subprocess
|
||||
from pathlib import Path
|
||||
@@ -213,23 +222,11 @@ def get_session_dir():
|
||||
return Path(os.environ.get("VIBE_SESSION_DIR", "~/.vibe/logs/session/active")).expanduser()
|
||||
|
||||
session_dir = get_session_dir()
|
||||
cmd = ["agent-cost-estimator", "--session-id", str(session_dir), "--json"]
|
||||
cmd = ["agent-cost-estimator", "--session-id", str(session_dir), "--display-lines"]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
|
||||
if result.returncode == 0:
|
||||
try:
|
||||
data = json.loads(result.stdout)
|
||||
model_key = data.get("pricing", {}).get("model_key", "unknown:unknown")
|
||||
if ":" in model_key:
|
||||
provider, model = model_key.split(":", 1)
|
||||
model_str = f"{provider}:{model}"
|
||||
else:
|
||||
model_str = f"unknown:{model_key}"
|
||||
session_cost = data.get("cost_usd", {}).get("total_usd", 0)
|
||||
turn_cost = data.get("last_turn", {}).get("cost_usd", {}).get("total_usd", 0) if data.get("last_turn") else 0
|
||||
message = f"{model_str} | session:${session_cost:.2f} | turn:${turn_cost:.2f}"
|
||||
print(json.dumps({"system_message": f"-- {message}"}))
|
||||
except (json.JSONDecodeError, KeyError, TypeError):
|
||||
print(json.dumps({"system_message": f"-- {result.stdout.strip()}"}))
|
||||
message = result.stdout.strip()
|
||||
print(json.dumps({"system_message": f"-- {message}"}))
|
||||
'''
|
||||
hook_dest.write_text(hook_content, encoding="utf-8")
|
||||
hook_dest.chmod(hook_dest.stat().st_mode | stat.S_IEXEC)
|
||||
@@ -794,53 +791,57 @@ def main(argv=None):
|
||||
"input_usd": 0.0, "cached_usd": 0.0, "output_usd": 0.0,
|
||||
"total_usd": 0.0,
|
||||
}
|
||||
# One report object feeds every output shape: --json prints it
|
||||
# verbatim; the default and --display-lines output are rendered
|
||||
# from it by agent_cost_estimator.display.
|
||||
# Extract provider from the display key
|
||||
provider_model = (display_key.split(":", 1)
|
||||
if ":" in display_key else (None, display_key))
|
||||
report = {
|
||||
"harness": harness.name,
|
||||
"session_id": session_dir.name,
|
||||
"title": meta.title,
|
||||
"model": meta.model_name,
|
||||
"provider": provider_model[0],
|
||||
"credit_pool": pool,
|
||||
"provider_model": display_key,
|
||||
"calls": usage.calls,
|
||||
"input_tokens": usage.input_tokens,
|
||||
"cached_input_tokens": usage.cached_input_tokens,
|
||||
"fresh_input_tokens": usage.fresh_input_tokens,
|
||||
"output_tokens": usage.output_tokens,
|
||||
"pricing": {
|
||||
"file": str(pricing_path) if pricing_path
|
||||
else "bundled",
|
||||
"model_key": display_key,
|
||||
"model_key_raw": key,
|
||||
"currency": currency,
|
||||
"rates_usd_per_million": rates,
|
||||
},
|
||||
"usage_source": usage.usage_source,
|
||||
"cost_usd": cost,
|
||||
"model_price_unavailable": is_default,
|
||||
"token_display": {
|
||||
"input": format_token_count(usage.input_tokens),
|
||||
"cached": format_token_count(usage.cached_input_tokens),
|
||||
"output": format_token_count(usage.output_tokens),
|
||||
"fresh": format_token_count(usage.fresh_input_tokens)
|
||||
},
|
||||
"last_turn": {
|
||||
"label": last_turn.label if last_turn else None,
|
||||
"cost_usd": turn_cost,
|
||||
} if last_turn else None,
|
||||
"turns": [
|
||||
{"label": t.label, "calls": t.calls,
|
||||
"input_tokens": t.input_tokens,
|
||||
"cached_input_tokens": t.cached_input_tokens,
|
||||
"output_tokens": t.output_tokens,
|
||||
"fresh_input_tokens": t.fresh_input_tokens}
|
||||
for t in usage.turns if t.calls
|
||||
],
|
||||
}
|
||||
if args.json:
|
||||
# Extract provider from the display key
|
||||
provider_model = (display_key.split(":", 1)
|
||||
if ":" in display_key else (None, display_key))
|
||||
print(json.dumps({
|
||||
"harness": harness.name,
|
||||
"session_id": session_dir.name,
|
||||
"title": meta.title,
|
||||
"model": meta.model_name,
|
||||
"provider": provider_model[0],
|
||||
"credit_pool": pool,
|
||||
"provider_model": display_key,
|
||||
"calls": usage.calls,
|
||||
"input_tokens": usage.input_tokens,
|
||||
"cached_input_tokens": usage.cached_input_tokens,
|
||||
"fresh_input_tokens": usage.fresh_input_tokens,
|
||||
"output_tokens": usage.output_tokens,
|
||||
"pricing": {
|
||||
"file": str(pricing_path) if pricing_path
|
||||
else "bundled",
|
||||
"model_key": display_key,
|
||||
"model_key_raw": key,
|
||||
"currency": currency,
|
||||
"rates_usd_per_million": rates,
|
||||
},
|
||||
"usage_source": usage.usage_source,
|
||||
"cost_usd": cost,
|
||||
"model_price_unavailable": is_default,
|
||||
"token_display": {
|
||||
"input": format_token_count(usage.input_tokens),
|
||||
"cached": format_token_count(usage.cached_input_tokens),
|
||||
"output": format_token_count(usage.output_tokens),
|
||||
"fresh": format_token_count(usage.fresh_input_tokens)
|
||||
},
|
||||
"last_turn": {
|
||||
"label": last_turn.label if last_turn else None,
|
||||
"cost_usd": turn_cost,
|
||||
} if last_turn else None,
|
||||
"turns": [
|
||||
{"label": t.label, "calls": t.calls,
|
||||
"input_tokens": t.input_tokens,
|
||||
"cached_input_tokens": t.cached_input_tokens,
|
||||
"output_tokens": t.output_tokens,
|
||||
"fresh_input_tokens": t.fresh_input_tokens}
|
||||
for t in usage.turns if t.calls
|
||||
],
|
||||
}, indent=2))
|
||||
print(json.dumps(report, indent=2))
|
||||
return 0
|
||||
if args.session and not args.turn:
|
||||
print(f"{cost['total_usd']:.4f}")
|
||||
@@ -848,17 +849,12 @@ def main(argv=None):
|
||||
if args.turn and not args.session:
|
||||
print(f"{turn_cost['total_usd']:.4f}")
|
||||
return 0
|
||||
if not args.details:
|
||||
# default: provider:model | session:$xxx | turn:$yyy
|
||||
output = f"{display_key} | session:${cost['total_usd']:.2f} | turn:${turn_cost['total_usd']:.2f}"
|
||||
if is_default:
|
||||
output += "\n **model price not available**"
|
||||
print(output)
|
||||
# Add token usage line
|
||||
input_formatted = format_token_count(usage.input_tokens)
|
||||
output_formatted = format_token_count(usage.output_tokens)
|
||||
cached_formatted = format_token_count(usage.cached_input_tokens)
|
||||
print(f"-- I:{input_formatted} O:{output_formatted} C:{cached_formatted}")
|
||||
if args.display_lines or not args.details:
|
||||
# One canonical format for every display path (CLI default,
|
||||
# GUI display fields, Vibe hooks, Hermes plugin):
|
||||
# agent_cost_estimator.display.format_display_lines.
|
||||
for line in format_display_lines(report):
|
||||
print(line)
|
||||
return 0
|
||||
label = f" ({meta.title})" if meta.title else ""
|
||||
print(f"harness: {harness.name}")
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
"""Canonical display formatting for agent-cost-estimator.
|
||||
|
||||
The single source of truth for the cost/token display format shared
|
||||
by every display path: the CLI default output, the GUI, the Vibe
|
||||
hook scripts, and the Hermes plugin. Change a line here and all
|
||||
display paths change with it (see the display-path parity rule in
|
||||
SPEC.md / the token-display spec, section 11).
|
||||
|
||||
This module is deliberately stdlib-only and self-contained: the
|
||||
Hermes plugin ships a verbatim copy of this file beside its
|
||||
``__init__.py`` (copied by ``integration/hermes/install.py``,
|
||||
byte-parity pinned by the tests), where it must import cleanly
|
||||
without the agent_cost_estimator package installed.
|
||||
|
||||
Consumes the ``--json`` report shape:
|
||||
|
||||
pricing.model_key "provider:model" display key
|
||||
cost_usd.total_usd session cost
|
||||
last_turn.cost_usd.total_usd turn cost (null last_turn = $0)
|
||||
input_tokens session input (inclusive of cached)
|
||||
cached_input_tokens session cached input
|
||||
output_tokens session output
|
||||
model_price_unavailable true when priced at default rates
|
||||
|
||||
``model_label`` overrides the provider:model label (the Hermes
|
||||
plugin passes its billing-route label).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
NOTE_MODEL_PRICE_UNAVAILABLE = " **model price not available**"
|
||||
|
||||
|
||||
def format_token_count(count: int) -> str:
|
||||
"""Format a token count with magnitude suffixes (k, M, B, T).
|
||||
|
||||
Numbers below 1000 print as-is; larger magnitudes print with two
|
||||
decimals and the matching suffix (1.50k, 1.23M, 1.00B, 1.00T).
|
||||
None/invalid counts format as "0".
|
||||
"""
|
||||
try:
|
||||
count = int(count or 0)
|
||||
except (TypeError, ValueError):
|
||||
return "0"
|
||||
if count < 0:
|
||||
# Handle negative numbers gracefully (shouldn't happen in practice)
|
||||
return str(count)
|
||||
if count < 1000:
|
||||
return str(count)
|
||||
elif count < 1_000_000:
|
||||
return f"{count / 1000:.2f}k"
|
||||
elif count < 1_000_000_000:
|
||||
return f"{count / 1_000_000:.2f}M"
|
||||
elif count < 1_000_000_000_000:
|
||||
return f"{count / 1_000_000_000:.2f}B"
|
||||
else:
|
||||
return f"{count / 1_000_000_000_000:.2f}T"
|
||||
|
||||
|
||||
def _model_label(report: dict, model_label: str | None) -> str:
|
||||
"""Resolve the provider:model label for the cost line."""
|
||||
if model_label:
|
||||
return model_label
|
||||
key = (report.get("pricing") or {}).get("model_key") or "unknown"
|
||||
if ":" in key:
|
||||
return key
|
||||
return f"unknown:{key}"
|
||||
|
||||
|
||||
def _session_cost(report: dict) -> float:
|
||||
return float(((report.get("cost_usd") or {}).get("total_usd")) or 0)
|
||||
|
||||
|
||||
def _turn_cost(report: dict) -> float:
|
||||
last_turn = report.get("last_turn") or {}
|
||||
return float(((last_turn.get("cost_usd") or {}).get("total_usd"))
|
||||
or 0)
|
||||
|
||||
|
||||
def format_cost_line(report: dict, model_label: str | None = None) -> str:
|
||||
"""Line 1: ``provider:model | session:$X.XX | turn:$Y.YY``."""
|
||||
return (
|
||||
f"{_model_label(report, model_label)} | "
|
||||
f"session:${_session_cost(report):.2f} | "
|
||||
f"turn:${_turn_cost(report):.2f}"
|
||||
)
|
||||
|
||||
|
||||
def format_token_line(report: dict) -> str | None:
|
||||
"""Line 2: ``-- I:<input> O:<output> C:<cached>`` from SESSION
|
||||
totals, or None when the report carries no token fields (older
|
||||
harness reports) - the cost line never depends on them.
|
||||
"""
|
||||
has_tokens = any(
|
||||
report.get(field) is not None
|
||||
for field in ("input_tokens", "cached_input_tokens",
|
||||
"output_tokens")
|
||||
)
|
||||
if not has_tokens:
|
||||
return None
|
||||
return (
|
||||
f"-- I:{format_token_count(report.get('input_tokens'))} "
|
||||
f"O:{format_token_count(report.get('output_tokens'))} "
|
||||
f"C:{format_token_count(report.get('cached_input_tokens'))}"
|
||||
)
|
||||
|
||||
|
||||
def format_display_lines(
|
||||
report: dict, model_label: str | None = None) -> list[str]:
|
||||
"""The display lines in presentation order, WITHOUT the leading
|
||||
``-- `` prefix on the cost line (the caller adds it):
|
||||
|
||||
provider:model | session:$X.XX | turn:$Y.YY
|
||||
**model price not available** (only when priced at
|
||||
default rates)
|
||||
-- I:<input> O:<output> C:<cached> (only when tokens
|
||||
are in the report)
|
||||
"""
|
||||
lines = [format_cost_line(report, model_label)]
|
||||
if report.get("model_price_unavailable"):
|
||||
lines.append(NOTE_MODEL_PRICE_UNAVAILABLE)
|
||||
token_line = format_token_line(report)
|
||||
if token_line is not None:
|
||||
lines.append(token_line)
|
||||
return lines
|
||||
@@ -20,8 +20,16 @@ hooks/
|
||||
Each harness has a `post_turn.py` hook script that:
|
||||
- Runs after each assistant turn (post_agent event)
|
||||
- Receives session information via stdin (JSON payload)
|
||||
- Calls `agent-cost-estimator` with the session directory
|
||||
- Formats and displays the cost information
|
||||
- Runs `agent-cost-estimator --session-id <dir> --display-lines`
|
||||
- Relays the CLI's display lines verbatim, wrapped in the
|
||||
`system_message` JSON (with the `-- ` prefix on the first line)
|
||||
|
||||
The hooks hold NO formatting of their own: the cost line, the
|
||||
model-price note, and the token line (`-- I:X O:Y C:Z`) are all
|
||||
rendered by `agent_cost_estimator.display` in the package — the
|
||||
display-path parity rule (SPEC.md 5.9). A format change ships with
|
||||
the CLI; the hooks pick it up automatically, so no re-integration
|
||||
is needed for format changes.
|
||||
|
||||
## Installation
|
||||
|
||||
|
||||
@@ -27,9 +27,13 @@ Output format (system_message, two lines):
|
||||
-- provider:model | session:$XXX | turn:$YYY
|
||||
-- I:<input> O:<output> C:<cached>
|
||||
Token counts are session totals, formatted with automatic suffixes
|
||||
(k, M, B, T) — the same shape as the CLI's default output. The token
|
||||
line is omitted when the JSON report carries no token fields (older
|
||||
harness reports), so the hook falls back to the cost line alone.
|
||||
(k, M, B, T), plus a note line when the model is priced at default
|
||||
rates. This hook holds NO formatting of its own: it runs
|
||||
``agent-cost-estimator --display-lines`` and relays the CLI's output
|
||||
verbatim. The format lives in one place, the package's
|
||||
agent_cost_estimator.display module (display-path parity rule,
|
||||
token-display spec section 11), so a format change in the package
|
||||
applies to the CLI, GUI, hooks, and the Hermes plugin together.
|
||||
|
||||
Model Detection:
|
||||
- Primary: session meta.json config.model_name / config.active_model
|
||||
@@ -103,79 +107,6 @@ def get_session_dir() -> Path:
|
||||
return default_dir
|
||||
|
||||
|
||||
def _format_token_count(count) -> str:
|
||||
"""Format a token count with magnitude suffixes (k, M, B, T).
|
||||
|
||||
Stdlib-only copy of agent_cost_estimator.utils.formatting
|
||||
.format_token_count: the hook must not import the package
|
||||
(it may run where only the script is installed), and the two
|
||||
implementations must stay in sync.
|
||||
"""
|
||||
try:
|
||||
count = int(count or 0)
|
||||
except (TypeError, ValueError):
|
||||
return "0"
|
||||
if count < 1000:
|
||||
return str(count)
|
||||
elif count < 1_000_000:
|
||||
return f"{count / 1000:.2f}k"
|
||||
elif count < 1_000_000_000:
|
||||
return f"{count / 1_000_000:.2f}M"
|
||||
elif count < 1_000_000_000_000:
|
||||
return f"{count / 1_000_000_000:.2f}B"
|
||||
else:
|
||||
return f"{count / 1_000_000_000_000:.2f}T"
|
||||
|
||||
|
||||
def format_cost_output(stdout: str) -> str:
|
||||
"""Format the agent-cost-estimator JSON output into hook message format.
|
||||
|
||||
Expected format (system_message gets a '-- ' prefix on the first
|
||||
line from the caller):
|
||||
provider:model | session:$XXX | turn:$YYY
|
||||
-- I:<input> O:<output> C:<cached>
|
||||
"""
|
||||
try:
|
||||
data = json.loads(stdout)
|
||||
# Extract provider and model from model_key (format: "provider:model")
|
||||
model_key = data.get("pricing", {}).get("model_key", "unknown:unknown")
|
||||
if ":" in model_key:
|
||||
provider, model = model_key.split(":", 1)
|
||||
model_str = f"{provider}:{model}"
|
||||
else:
|
||||
model_str = f"unknown:{model_key}"
|
||||
|
||||
# Extract costs
|
||||
session_cost = data.get("cost_usd", {}).get("total_usd", 0)
|
||||
last_turn = data.get("last_turn")
|
||||
turn_cost = last_turn.get("cost_usd", {}).get("total_usd", 0) if last_turn else 0
|
||||
|
||||
# Cost line: provider:model | session:$XXX | turn:$YYY
|
||||
lines = [
|
||||
f"{model_str} | session:${session_cost:.2f} | turn:${turn_cost:.2f}"
|
||||
]
|
||||
|
||||
# Token line from session totals (per-turn token counts are not
|
||||
# part of the harness report). Only present when the report has
|
||||
# the token fields; the cost line never depends on them.
|
||||
has_tokens = any(
|
||||
data.get(field) is not None
|
||||
for field in ("input_tokens", "cached_input_tokens", "output_tokens")
|
||||
)
|
||||
if has_tokens:
|
||||
token_line = (
|
||||
f"-- I:{_format_token_count(data.get('input_tokens'))} "
|
||||
f"O:{_format_token_count(data.get('output_tokens'))} "
|
||||
f"C:{_format_token_count(data.get('cached_input_tokens'))}"
|
||||
)
|
||||
lines.append(token_line)
|
||||
|
||||
return "\n".join(lines)
|
||||
except (json.JSONDecodeError, KeyError, TypeError) as e:
|
||||
LOG.warning(f"Failed to parse output: {e}")
|
||||
return stdout.strip()
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point for the hook."""
|
||||
try:
|
||||
@@ -189,11 +120,13 @@ def main():
|
||||
|
||||
LOG.debug(f"Processing session: {session_dir}")
|
||||
|
||||
# Build command - use --json for structured output
|
||||
# The CLI renders the display lines itself (--display-lines:
|
||||
# the canonical format from agent_cost_estimator.display);
|
||||
# this hook only relays them.
|
||||
cmd = [
|
||||
"agent-cost-estimator",
|
||||
"--session-id", str(session_dir),
|
||||
"--json"
|
||||
"--display-lines"
|
||||
]
|
||||
|
||||
# Run agent-cost-estimator
|
||||
@@ -212,8 +145,10 @@ def main():
|
||||
}))
|
||||
return
|
||||
|
||||
# Format the output
|
||||
message = format_cost_output(result.stdout)
|
||||
# Relay the rendered display lines verbatim: cost line, note
|
||||
# line (only when the model is priced at default rates), and
|
||||
# token line all come from the package's display module.
|
||||
message = result.stdout.strip()
|
||||
print(json.dumps({"system_message": f"-- {message}"}))
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
|
||||
@@ -28,16 +28,20 @@ Output format (system_message, two lines):
|
||||
-- provider:model | session:$XXX | turn:$YYY
|
||||
-- I:<input> O:<output> C:<cached>
|
||||
Token counts are session totals, formatted with automatic suffixes
|
||||
(k, M, B, T) — the same shape as the CLI's default output. The token
|
||||
line is omitted when the JSON report carries no token fields (older
|
||||
harness reports), so the hook falls back to the cost line alone.
|
||||
(k, M, B, T), plus a note line when the model is priced at default
|
||||
rates. This hook holds NO formatting of its own: it runs
|
||||
``agent-cost-estimator --display-lines`` and relays the CLI's output
|
||||
verbatim. The format lives in one place, the package's
|
||||
agent_cost_estimator.display module (display-path parity rule,
|
||||
token-display spec section 11), so a format change in the package
|
||||
applies to the CLI, GUI, hooks, and the Hermes plugin together.
|
||||
|
||||
Model Detection (VSCodium-specific):
|
||||
- Primary: session meta.json config.model_name / config.active_model
|
||||
- Fallback: global ~/.vibe/config.toml active_model and models array
|
||||
- Final: default pricing with warning indicator
|
||||
- Note: VSCodium extension may not write model to meta.json, so
|
||||
global config fallback is essential
|
||||
- Final: default pricing with warning indicator
|
||||
|
||||
Each session is tracked independently, ensuring accuracy even with
|
||||
multiple concurrent sessions.
|
||||
@@ -110,79 +114,6 @@ def get_session_dir() -> Path:
|
||||
return default_dir
|
||||
|
||||
|
||||
def _format_token_count(count) -> str:
|
||||
"""Format a token count with magnitude suffixes (k, M, B, T).
|
||||
|
||||
Stdlib-only copy of agent_cost_estimator.utils.formatting
|
||||
.format_token_count: the hook must not import the package
|
||||
(it may run where only the script is installed), and the two
|
||||
implementations must stay in sync.
|
||||
"""
|
||||
try:
|
||||
count = int(count or 0)
|
||||
except (TypeError, ValueError):
|
||||
return "0"
|
||||
if count < 1000:
|
||||
return str(count)
|
||||
elif count < 1_000_000:
|
||||
return f"{count / 1000:.2f}k"
|
||||
elif count < 1_000_000_000:
|
||||
return f"{count / 1_000_000:.2f}M"
|
||||
elif count < 1_000_000_000_000:
|
||||
return f"{count / 1_000_000_000:.2f}B"
|
||||
else:
|
||||
return f"{count / 1_000_000_000_000:.2f}T"
|
||||
|
||||
|
||||
def format_cost_output(stdout: str) -> str:
|
||||
"""Format the agent-cost-estimator JSON output into hook message format.
|
||||
|
||||
Expected format (system_message gets a '-- ' prefix on the first
|
||||
line from the caller):
|
||||
provider:model | session:$XXX | turn:$YYY
|
||||
-- I:<input> O:<output> C:<cached>
|
||||
"""
|
||||
try:
|
||||
data = json.loads(stdout)
|
||||
# Extract provider and model from model_key (format: "provider:model")
|
||||
model_key = data.get("pricing", {}).get("model_key", "unknown:unknown")
|
||||
if ":" in model_key:
|
||||
provider, model = model_key.split(":", 1)
|
||||
model_str = f"{provider}:{model}"
|
||||
else:
|
||||
model_str = f"unknown:{model_key}"
|
||||
|
||||
# Extract costs
|
||||
session_cost = data.get("cost_usd", {}).get("total_usd", 0)
|
||||
last_turn = data.get("last_turn")
|
||||
turn_cost = last_turn.get("cost_usd", {}).get("total_usd", 0) if last_turn else 0
|
||||
|
||||
# Cost line: provider:model | session:$XXX | turn:$YYY
|
||||
lines = [
|
||||
f"{model_str} | session:${session_cost:.2f} | turn:${turn_cost:.2f}"
|
||||
]
|
||||
|
||||
# Token line from session totals (per-turn token counts are not
|
||||
# part of the harness report). Only present when the report has
|
||||
# the token fields; the cost line never depends on them.
|
||||
has_tokens = any(
|
||||
data.get(field) is not None
|
||||
for field in ("input_tokens", "cached_input_tokens", "output_tokens")
|
||||
)
|
||||
if has_tokens:
|
||||
token_line = (
|
||||
f"-- I:{_format_token_count(data.get('input_tokens'))} "
|
||||
f"O:{_format_token_count(data.get('output_tokens'))} "
|
||||
f"C:{_format_token_count(data.get('cached_input_tokens'))}"
|
||||
)
|
||||
lines.append(token_line)
|
||||
|
||||
return "\n".join(lines)
|
||||
except (json.JSONDecodeError, KeyError, TypeError) as e:
|
||||
LOG.warning(f"Failed to parse output: {e}")
|
||||
return stdout.strip()
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point for the hook."""
|
||||
try:
|
||||
@@ -196,11 +127,13 @@ def main():
|
||||
|
||||
LOG.debug(f"Processing session: {session_dir}")
|
||||
|
||||
# Build command - use --json for structured output
|
||||
# The CLI renders the display lines itself (--display-lines:
|
||||
# the canonical format from agent_cost_estimator.display);
|
||||
# this hook only relays them.
|
||||
cmd = [
|
||||
"agent-cost-estimator",
|
||||
"--session-id", str(session_dir),
|
||||
"--json"
|
||||
"--display-lines"
|
||||
]
|
||||
|
||||
# Run agent-cost-estimator
|
||||
@@ -217,19 +150,25 @@ def main():
|
||||
print(json.dumps({"system_message": f"-- [cost: error - {result.stderr.strip()}]"}))
|
||||
return
|
||||
|
||||
# Format the output
|
||||
message = format_cost_output(result.stdout)
|
||||
# Relay the rendered display lines verbatim: cost line, note
|
||||
# line (only when the model is priced at default rates), and
|
||||
# token line all come from the package's display module.
|
||||
# Return JSON to stdout (will be sent to vibe-acp stdin by wrapper)
|
||||
message = result.stdout.strip()
|
||||
print(json.dumps({"system_message": f"-- {message}"}))
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
LOG.error("agent-cost-estimator timed out after 30 seconds")
|
||||
# Return JSON to stdout (will be sent to vibe-acp stdin by wrapper)
|
||||
print(json.dumps({"system_message": "-- [cost: timeout]"}))
|
||||
print(json.dumps({
|
||||
"system_message": "-- [cost: timeout]"
|
||||
}))
|
||||
except Exception as e:
|
||||
LOG.error(f"Unexpected error: {e}")
|
||||
# Return JSON to stdout (will be sent to vibe-acp stdin by wrapper)
|
||||
print(json.dumps({"system_message": f"-- [cost: error - {str(e)}]"}))
|
||||
print(json.dumps({
|
||||
"system_message": f"-- [cost: error - {str(e)}]"
|
||||
}))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -10,8 +10,11 @@ Or, from a source checkout / any machine with the package installed:
|
||||
[--hermes-home DIR] [--uninstall] [--dry-run]
|
||||
|
||||
What it does (idempotent):
|
||||
1. Copies the plugin (plugin.yaml + __init__.py) to
|
||||
<home>/plugins/agent-cost-estimator/
|
||||
1. Copies the plugin (plugin.yaml + __init__.py + display.py) to
|
||||
<home>/plugins/agent-cost-estimator/ — display.py is the shared
|
||||
cost/token line format (a verbatim copy of
|
||||
agent_cost_estimator.display), so the plugin renders exactly
|
||||
what the CLI, GUI, and Vibe hooks render (display-path parity)
|
||||
2. Ships pricing.json beside the plugin (an existing tuned table is
|
||||
kept across reinstalls)
|
||||
3. Runs `hermes plugins enable agent-cost-estimator` (via the
|
||||
@@ -21,11 +24,12 @@ What it does (idempotent):
|
||||
the plugin module at first hook fire. A stopped gateway needs no
|
||||
restart. Skip with --no-restart.
|
||||
|
||||
After that, a line like:
|
||||
After that, lines like:
|
||||
|
||||
-- zai:zai-glm-5-3 | session:$0.42 | turn:$0.16
|
||||
-- I:44.56M O:102.39k C:43.79M
|
||||
|
||||
prints after every assistant turn, mirroring the Mistral Vibe CLI
|
||||
print after every assistant turn, mirroring the Mistral Vibe CLI
|
||||
hook. The hermes harness adapter reads usage back via:
|
||||
|
||||
agent-cost-estimator --harness hermes [options]
|
||||
@@ -228,6 +232,7 @@ def install(home: pathlib.Path, dry_run: bool = False,
|
||||
_restart_gateway(home, dry_run)
|
||||
print(f"\nDone. Every assistant turn now ends with:")
|
||||
print(" -- provider:model | session:$X.XX | turn:$Y.YY")
|
||||
print(" -- I:<input> O:<output> C:<cached> (session tokens)")
|
||||
print("Query any session with: agent-cost-estimator --harness hermes")
|
||||
return 0
|
||||
|
||||
|
||||
@@ -1,9 +1,16 @@
|
||||
"""agent-cost-estimator Hermes plugin.
|
||||
|
||||
Prints a cost line after every assistant turn, in the same shape as
|
||||
the Mistral Vibe CLI hook:
|
||||
Prints a cost line plus session token line after every assistant
|
||||
turn, in the same shape as the Mistral Vibe CLI hook:
|
||||
|
||||
-- provider:model | session:$X.XX | turn:$Y.YY
|
||||
-- I:<input> O:<output> C:<cached>
|
||||
|
||||
The format is NOT implemented here: this plugin renders through
|
||||
display.py, a verbatim copy of agent_cost_estimator.display shipped
|
||||
beside this file (byte-parity pinned by the package's tests). That
|
||||
is the display-path parity rule (token-display spec section 11):
|
||||
one canonical format for the CLI, GUI, hooks, and this plugin.
|
||||
|
||||
How it works (mirrors the Vibe integration's architecture):
|
||||
|
||||
@@ -263,6 +270,42 @@ def _on_post_auxiliary_call(**kwargs: Any) -> None:
|
||||
|
||||
# ── turn-end cost line ──────────────────────────────────────────────
|
||||
|
||||
# The shared display module (a verbatim copy of
|
||||
# agent_cost_estimator.display, installed beside this file). Loading
|
||||
# by path keeps this working however Hermes imports the plugin.
|
||||
_DISPLAY = None
|
||||
_DISPLAY_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def _display():
|
||||
"""Load display.py from beside this plugin (idempotent).
|
||||
|
||||
display.py is the canonical cost/token line format shared with
|
||||
the CLI, GUI, and Vibe hooks; the copy here is byte-identical to
|
||||
the package's, pinned by tests. Returns None when it cannot be
|
||||
loaded (broken install) - callers skip printing rather than
|
||||
render a divergent format.
|
||||
"""
|
||||
global _DISPLAY
|
||||
if _DISPLAY is not None:
|
||||
return _DISPLAY
|
||||
with _DISPLAY_LOCK:
|
||||
if _DISPLAY is not None:
|
||||
return _DISPLAY
|
||||
try:
|
||||
import importlib.util
|
||||
path = Path(__file__).resolve().parent / "display.py"
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"agent_cost_estimator_display", path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
_DISPLAY = module
|
||||
except Exception as exc:
|
||||
logger.warning("[cost] display module load failed: %s", exc)
|
||||
return None
|
||||
return _DISPLAY
|
||||
|
||||
|
||||
def _db_session_totals(session_id: str) -> Optional[dict]:
|
||||
"""Cumulative tokens for the session from state.db (read-only).
|
||||
|
||||
@@ -603,7 +646,10 @@ def _custom_provider_name(base_url: str) -> Optional[str]:
|
||||
|
||||
|
||||
def _print_cost_line(session_id: str, turn_id: str, model: str) -> None:
|
||||
"""Compute and publish the turn's cost line."""
|
||||
"""Compute and publish the turn's display lines."""
|
||||
display = _display()
|
||||
if display is None:
|
||||
return # no divergent fallback: fix the plugin install
|
||||
session = _db_session_totals(session_id)
|
||||
if session is None:
|
||||
return # nothing recorded yet for this session
|
||||
@@ -619,15 +665,21 @@ def _print_cost_line(session_id: str, turn_id: str, model: str) -> None:
|
||||
label = f"{label_provider}:{model_part}"
|
||||
else:
|
||||
label = key
|
||||
session_usd = _cost(session, rates)
|
||||
turn_usd = _cost(turn, rates)
|
||||
line = (
|
||||
f"{LINE_PREFIX}{label} | session:${session_usd:.2f} | "
|
||||
f"turn:${turn_usd:.2f}"
|
||||
)
|
||||
if is_default:
|
||||
line += "\n **model price not available**"
|
||||
_publish_cost_line(session_id, line)
|
||||
# The canonical report shape consumed by
|
||||
# agent_cost_estimator.display (the --json report's fields).
|
||||
report = {
|
||||
"cost_usd": {"total_usd": _cost(session, rates)},
|
||||
"last_turn": {"cost_usd": {"total_usd": _cost(turn, rates)}},
|
||||
"input_tokens": session["input"],
|
||||
"cached_input_tokens": session["cached"],
|
||||
"output_tokens": session["output"],
|
||||
"model_price_unavailable": is_default,
|
||||
}
|
||||
lines = display.format_display_lines(report, model_label=label)
|
||||
# Cost line gets this surface's prefix; the note and token
|
||||
# lines come out of display already in final form.
|
||||
text = "\n".join([LINE_PREFIX + lines[0]] + lines[1:])
|
||||
_publish_cost_line(session_id, text)
|
||||
|
||||
|
||||
def _on_post_llm_call(**kwargs: Any) -> None:
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
"""Canonical display formatting for agent-cost-estimator.
|
||||
|
||||
The single source of truth for the cost/token display format shared
|
||||
by every display path: the CLI default output, the GUI, the Vibe
|
||||
hook scripts, and the Hermes plugin. Change a line here and all
|
||||
display paths change with it (see the display-path parity rule in
|
||||
SPEC.md / the token-display spec, section 11).
|
||||
|
||||
This module is deliberately stdlib-only and self-contained: the
|
||||
Hermes plugin ships a verbatim copy of this file beside its
|
||||
``__init__.py`` (copied by ``integration/hermes/install.py``,
|
||||
byte-parity pinned by the tests), where it must import cleanly
|
||||
without the agent_cost_estimator package installed.
|
||||
|
||||
Consumes the ``--json`` report shape:
|
||||
|
||||
pricing.model_key "provider:model" display key
|
||||
cost_usd.total_usd session cost
|
||||
last_turn.cost_usd.total_usd turn cost (null last_turn = $0)
|
||||
input_tokens session input (inclusive of cached)
|
||||
cached_input_tokens session cached input
|
||||
output_tokens session output
|
||||
model_price_unavailable true when priced at default rates
|
||||
|
||||
``model_label`` overrides the provider:model label (the Hermes
|
||||
plugin passes its billing-route label).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
NOTE_MODEL_PRICE_UNAVAILABLE = " **model price not available**"
|
||||
|
||||
|
||||
def format_token_count(count: int) -> str:
|
||||
"""Format a token count with magnitude suffixes (k, M, B, T).
|
||||
|
||||
Numbers below 1000 print as-is; larger magnitudes print with two
|
||||
decimals and the matching suffix (1.50k, 1.23M, 1.00B, 1.00T).
|
||||
None/invalid counts format as "0".
|
||||
"""
|
||||
try:
|
||||
count = int(count or 0)
|
||||
except (TypeError, ValueError):
|
||||
return "0"
|
||||
if count < 0:
|
||||
# Handle negative numbers gracefully (shouldn't happen in practice)
|
||||
return str(count)
|
||||
if count < 1000:
|
||||
return str(count)
|
||||
elif count < 1_000_000:
|
||||
return f"{count / 1000:.2f}k"
|
||||
elif count < 1_000_000_000:
|
||||
return f"{count / 1_000_000:.2f}M"
|
||||
elif count < 1_000_000_000_000:
|
||||
return f"{count / 1_000_000_000:.2f}B"
|
||||
else:
|
||||
return f"{count / 1_000_000_000_000:.2f}T"
|
||||
|
||||
|
||||
def _model_label(report: dict, model_label: str | None) -> str:
|
||||
"""Resolve the provider:model label for the cost line."""
|
||||
if model_label:
|
||||
return model_label
|
||||
key = (report.get("pricing") or {}).get("model_key") or "unknown"
|
||||
if ":" in key:
|
||||
return key
|
||||
return f"unknown:{key}"
|
||||
|
||||
|
||||
def _session_cost(report: dict) -> float:
|
||||
return float(((report.get("cost_usd") or {}).get("total_usd")) or 0)
|
||||
|
||||
|
||||
def _turn_cost(report: dict) -> float:
|
||||
last_turn = report.get("last_turn") or {}
|
||||
return float(((last_turn.get("cost_usd") or {}).get("total_usd"))
|
||||
or 0)
|
||||
|
||||
|
||||
def format_cost_line(report: dict, model_label: str | None = None) -> str:
|
||||
"""Line 1: ``provider:model | session:$X.XX | turn:$Y.YY``."""
|
||||
return (
|
||||
f"{_model_label(report, model_label)} | "
|
||||
f"session:${_session_cost(report):.2f} | "
|
||||
f"turn:${_turn_cost(report):.2f}"
|
||||
)
|
||||
|
||||
|
||||
def format_token_line(report: dict) -> str | None:
|
||||
"""Line 2: ``-- I:<input> O:<output> C:<cached>`` from SESSION
|
||||
totals, or None when the report carries no token fields (older
|
||||
harness reports) - the cost line never depends on them.
|
||||
"""
|
||||
has_tokens = any(
|
||||
report.get(field) is not None
|
||||
for field in ("input_tokens", "cached_input_tokens",
|
||||
"output_tokens")
|
||||
)
|
||||
if not has_tokens:
|
||||
return None
|
||||
return (
|
||||
f"-- I:{format_token_count(report.get('input_tokens'))} "
|
||||
f"O:{format_token_count(report.get('output_tokens'))} "
|
||||
f"C:{format_token_count(report.get('cached_input_tokens'))}"
|
||||
)
|
||||
|
||||
|
||||
def format_display_lines(
|
||||
report: dict, model_label: str | None = None) -> list[str]:
|
||||
"""The display lines in presentation order, WITHOUT the leading
|
||||
``-- `` prefix on the cost line (the caller adds it):
|
||||
|
||||
provider:model | session:$X.XX | turn:$Y.YY
|
||||
**model price not available** (only when priced at
|
||||
default rates)
|
||||
-- I:<input> O:<output> C:<cached> (only when tokens
|
||||
are in the report)
|
||||
"""
|
||||
lines = [format_cost_line(report, model_label)]
|
||||
if report.get("model_price_unavailable"):
|
||||
lines.append(NOTE_MODEL_PRICE_UNAVAILABLE)
|
||||
token_line = format_token_line(report)
|
||||
if token_line is not None:
|
||||
lines.append(token_line)
|
||||
return lines
|
||||
@@ -1,6 +1,6 @@
|
||||
name: agent-cost-estimator
|
||||
version: 0.4.6
|
||||
description: "Print an estimated cost line (provider:model | session:$X | turn:$Y) after every assistant turn, recorded from Hermes session usage."
|
||||
version: 0.5.0
|
||||
description: "Print an estimated cost line plus session token line (provider:model | session:$X | turn:$Y, then -- I:X O:Y C:Z) after every assistant turn, recorded from Hermes session usage. Rendered through the shared display module for display parity with the CLI, GUI, and Vibe hooks."
|
||||
author: "nvaldez"
|
||||
provides_hooks:
|
||||
- pre_llm_call
|
||||
|
||||
@@ -1,64 +1,12 @@
|
||||
"""Formatting utilities for token counts and other display values."""
|
||||
"""Formatting utilities for token counts and other display values.
|
||||
|
||||
The token-count formatter now lives in agent_cost_estimator.display
|
||||
(the shared display contract); this module re-exports it so existing
|
||||
importers (GUI, tests) keep working.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ..display import format_token_count # re-export
|
||||
|
||||
def format_token_count(count: int) -> str:
|
||||
"""Format token count with automatic suffix (k, M, B, T) for thousands, millions, billions, trillions.
|
||||
|
||||
Args:
|
||||
count: Non-negative integer representing token count
|
||||
|
||||
Returns:
|
||||
Formatted string with up to 2 decimal places and appropriate suffix.
|
||||
- Numbers < 1000: returned as-is without suffix
|
||||
- Numbers >= 1000 and < 1M: formatted as "X.XXk"
|
||||
- Numbers >= 1M and < 1B: formatted as "X.XXM"
|
||||
- Numbers >= 1B and < 1T: formatted as "X.XXB"
|
||||
- Numbers >= 1T: formatted as "X.XXT"
|
||||
|
||||
Examples:
|
||||
>>> format_token_count(0)
|
||||
'0'
|
||||
>>> format_token_count(500)
|
||||
'500'
|
||||
>>> format_token_count(999)
|
||||
'999'
|
||||
>>> format_token_count(1000)
|
||||
'1.00k'
|
||||
>>> format_token_count(1500)
|
||||
'1.50k'
|
||||
>>> format_token_count(9999)
|
||||
'9.99k'
|
||||
>>> format_token_count(10000)
|
||||
'10.00k'
|
||||
>>> format_token_count(100000)
|
||||
'100.00k'
|
||||
>>> format_token_count(999999)
|
||||
'999.99k'
|
||||
>>> format_token_count(1000000)
|
||||
'1.00M'
|
||||
>>> format_token_count(1234567)
|
||||
'1.23M'
|
||||
>>> format_token_count(1000000000)
|
||||
'1.00B'
|
||||
>>> format_token_count(1500000000)
|
||||
'1.50B'
|
||||
>>> format_token_count(1000000000000)
|
||||
'1.00T'
|
||||
>>> format_token_count(1500000000000)
|
||||
'1.50T'
|
||||
"""
|
||||
if count < 0:
|
||||
# Handle negative numbers gracefully (shouldn't happen in practice)
|
||||
return str(count)
|
||||
if count < 1000:
|
||||
return str(count)
|
||||
elif count < 1_000_000:
|
||||
return f"{count / 1000:.2f}k"
|
||||
elif count < 1_000_000_000:
|
||||
return f"{count / 1_000_000:.2f}M"
|
||||
elif count < 1_000_000_000_000:
|
||||
return f"{count / 1_000_000_000:.2f}B"
|
||||
else:
|
||||
return f"{count / 1_000_000_000_000:.2f}T"
|
||||
__all__ = ["format_token_count"]
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "agent-cost-estimator"
|
||||
version = "0.6.2"
|
||||
version = "0.7.0"
|
||||
description = "Report token usage and estimated USD cost of AI agent sessions from local harness logs"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.9"
|
||||
|
||||
@@ -561,6 +561,23 @@ def test_cli_default_compact_line(tmp_path, capsys):
|
||||
assert out == expected
|
||||
|
||||
|
||||
def test_cli_display_lines_flag_matches_default(tmp_path, capsys):
|
||||
"""--display-lines (what the Vibe hooks relay) prints exactly
|
||||
the default output: one canonical format, no drift."""
|
||||
session = tmp_path / "logs" / "session" / "unified" / "s1"
|
||||
write_journal(session, [
|
||||
user_message("hi"),
|
||||
usage_result("a1", 2_000_000, 500_000, 100_000),
|
||||
])
|
||||
write_meta(session, title="t", model_name="m1")
|
||||
args = ["--session-id", str(session), "--home", str(tmp_path)]
|
||||
assert cli.main(args) == 0
|
||||
default = capsys.readouterr().out
|
||||
assert cli.main(args + ["--display-lines"]) == 0
|
||||
assert capsys.readouterr().out == default
|
||||
assert "-- I:2.00M O:100.00k C:500.00k" in default
|
||||
|
||||
|
||||
def test_cli_details_shows_the_full_table(tmp_path, capsys):
|
||||
"""-d prints the full report table."""
|
||||
session = tmp_path / "logs" / "session" / "unified" / "s1"
|
||||
@@ -929,6 +946,8 @@ def test_hermes_cli_pool_falls_back_to_harness_mapping(tmp_path, capsys):
|
||||
# --- hermes plugin (integration) ---
|
||||
|
||||
from agent_cost_estimator.integration.hermes import plugin as hp
|
||||
from agent_cost_estimator import display as ace_display
|
||||
from agent_cost_estimator.display import format_token_count
|
||||
|
||||
|
||||
class _Ctx:
|
||||
@@ -1224,6 +1243,61 @@ def test_plugin_prints_when_not_daemon(tmp_path, monkeypatch, capsys):
|
||||
assert "line" in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_plugin_display_copy_is_verbatim():
|
||||
"""Display parity: the display.py shipped beside the plugin is
|
||||
byte-identical to the package's display module - one canonical
|
||||
format, reshipped by --integrate hermes."""
|
||||
pkg = pathlib.Path(ace_display.__file__).resolve()
|
||||
copy = pathlib.Path(hp.__file__).resolve().parent / "display.py"
|
||||
assert copy.is_file()
|
||||
assert copy.read_bytes() == pkg.read_bytes()
|
||||
|
||||
|
||||
def test_plugin_display_renders_canonical_lines():
|
||||
"""The plugin's shipped display module renders the same lines
|
||||
the CLI default output renders."""
|
||||
report = {
|
||||
"pricing": {"model_key": "zai:zai-glm-5-3"},
|
||||
"cost_usd": {"total_usd": 0.42},
|
||||
"last_turn": None,
|
||||
"input_tokens": 44_560_000,
|
||||
"cached_input_tokens": 43_790_000,
|
||||
"output_tokens": 102_390,
|
||||
"model_price_unavailable": False,
|
||||
}
|
||||
assert ace_display.format_display_lines(report) == [
|
||||
"zai:zai-glm-5-3 | session:$0.42 | turn:$0.00",
|
||||
"-- I:44.56M O:102.39k C:43.79M",
|
||||
]
|
||||
|
||||
|
||||
def test_plugin_prints_token_line(tmp_path, monkeypatch, capsys):
|
||||
"""Display parity: the plugin prints the session token line too
|
||||
(token-display spec section 11's open item, closed)."""
|
||||
home = make_hermes_home(
|
||||
tmp_path,
|
||||
sessions=[("s1", "t", "zai-glm-5-3", 1.0)],
|
||||
usage=[("s1", "zai-glm-5-3", 1, 1000, 20000, 900000, 5000)],
|
||||
)
|
||||
monkeypatch.setenv("HERMES_HOME", str(home))
|
||||
monkeypatch.setattr(hp, "_turn_usage", {})
|
||||
hp._on_post_llm_call(session_id="s1", turn_id="t1",
|
||||
model="zai-glm-5-3", platform="cli")
|
||||
out = capsys.readouterr().out
|
||||
assert f"-- I:{format_token_count(1000 + 900000 + 5000)} " \
|
||||
f"O:{format_token_count(20000)} " \
|
||||
f"C:{format_token_count(900000)}" in out
|
||||
assert "session:$" in out and "turn:$" in out
|
||||
|
||||
|
||||
def test_plugin_no_line_when_display_broken(monkeypatch, capsys):
|
||||
"""A broken display copy skips the line (no divergent
|
||||
formatting fallback) instead of printing a wrong format."""
|
||||
monkeypatch.setattr(hp, "_display", lambda: None)
|
||||
hp._print_cost_line("s1", "t1", "m") # must not raise or print
|
||||
assert capsys.readouterr().out == ""
|
||||
|
||||
|
||||
# --- hermes installer ---
|
||||
|
||||
from agent_cost_estimator.integration.hermes import install as hinstall
|
||||
@@ -1247,7 +1321,7 @@ def test_installer_requires_hermes_home(tmp_path, capsys):
|
||||
|
||||
|
||||
def test_installer_copies_plugin(tmp_path, monkeypatch, capsys):
|
||||
"""The install copies plugin files + pricing.json."""
|
||||
"""The install copies plugin files + display.py + pricing.json."""
|
||||
home = make_hermes_home(tmp_path, sessions=[("s1", "t", "m", 1.0)])
|
||||
# hermes CLI not on PATH in tests: expect the manual-enable note
|
||||
def fake_run(*args, **kwargs):
|
||||
@@ -1258,6 +1332,9 @@ def test_installer_copies_plugin(tmp_path, monkeypatch, capsys):
|
||||
dest = home / "plugins" / "agent-cost-estimator"
|
||||
assert (dest / "__init__.py").is_file()
|
||||
assert (dest / "plugin.yaml").is_file()
|
||||
# the shared display module ships with the plugin (display parity)
|
||||
assert (dest / "display.py").read_bytes() == pathlib.Path(
|
||||
ace_display.__file__).resolve().read_bytes()
|
||||
assert (dest / "pricing.json").is_file()
|
||||
pricing = json.loads((dest / "pricing.json").read_text())
|
||||
assert "providers" in pricing
|
||||
|
||||
+150
-60
@@ -1,16 +1,18 @@
|
||||
"""Tests for the Vibe hook scripts (post_turn.py) display formatting.
|
||||
"""Tests for the Vibe hook scripts (post_turn.py).
|
||||
|
||||
The hooks are standalone stdlib-only scripts (they must not import the
|
||||
package), so they are loaded directly from their files and their
|
||||
formatting must be tested in isolation.
|
||||
The hooks are standalone stdlib-only scripts (they must not import
|
||||
the package) and, since the display refactor, hold NO formatting of
|
||||
their own: they run ``agent-cost-estimator --display-lines`` and
|
||||
relay its stdout verbatim. These tests pin the relay contract.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import pathlib
|
||||
import subprocess
|
||||
import pytest
|
||||
|
||||
from agent_cost_estimator.utils.formatting import format_token_count
|
||||
from agent_cost_estimator.display import format_display_lines
|
||||
|
||||
HOOKS_DIR = pathlib.Path(__file__).parent.parent / "agent_cost_estimator" / "hooks"
|
||||
HOOK_FILES = {
|
||||
@@ -28,69 +30,157 @@ def _load_hook(name):
|
||||
return module
|
||||
|
||||
|
||||
def _report_json(tokens=True):
|
||||
"""A realistic `agent-cost-estimator --json` report."""
|
||||
data = {
|
||||
"pricing": {"model_key": "zai:zai-glm-5-3"},
|
||||
"cost_usd": {"total_usd": 0.0136},
|
||||
"last_turn": {"cost_usd": {"total_usd": 0.0031}},
|
||||
}
|
||||
if tokens:
|
||||
data.update({
|
||||
@pytest.mark.parametrize("hook_name", sorted(HOOK_FILES))
|
||||
class TestHookRelay:
|
||||
"""main() runs the CLI with --display-lines and relays it."""
|
||||
|
||||
def _run_hook(self, hook, monkeypatch, capsys, cli_stdout,
|
||||
cli_rc=0, session_exists=True):
|
||||
calls = {}
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
calls["cmd"] = cmd
|
||||
return subprocess.CompletedProcess(
|
||||
cmd, cli_rc, stdout=cli_stdout, stderr="boom"
|
||||
if cli_rc else "")
|
||||
|
||||
class FakeDir:
|
||||
def __init__(self, path):
|
||||
self.path = path
|
||||
def is_dir(self):
|
||||
return session_exists
|
||||
def __str__(self):
|
||||
return self.path
|
||||
|
||||
monkeypatch.setattr(hook.subprocess, "run", fake_run)
|
||||
monkeypatch.setattr(hook, "get_session_dir", lambda: FakeDir("/x/sess"))
|
||||
hook.main()
|
||||
return calls, json.loads(capsys.readouterr().out)
|
||||
|
||||
def test_relays_display_lines_with_prefix(self, hook_name,
|
||||
monkeypatch, capsys):
|
||||
"""The CLI's --display-lines output is relayed verbatim,
|
||||
with the system_message '-- ' prefix on the first line."""
|
||||
hook = _load_hook(hook_name)
|
||||
cli_out = ("zai:zai-glm-5-3 | session:$0.01 | turn:$0.00\n"
|
||||
"-- I:42.12M O:95.07k C:41.36M")
|
||||
calls, message = self._run_hook(
|
||||
hook, monkeypatch, capsys, cli_out)
|
||||
assert "--display-lines" in calls["cmd"]
|
||||
assert message == {
|
||||
"system_message": f"-- {cli_out}",
|
||||
}
|
||||
|
||||
def test_relays_note_line_too(self, hook_name, monkeypatch, capsys):
|
||||
"""The model-price note line is relayed like any other
|
||||
display line (the canonical format includes it)."""
|
||||
hook = _load_hook(hook_name)
|
||||
cli_out = ("default:m | session:$0.01 | turn:$0.00\n"
|
||||
" **model price not available**\n"
|
||||
"-- I:2.00M O:100.00k C:500.00k")
|
||||
calls, message = self._run_hook(
|
||||
hook, monkeypatch, capsys, cli_out)
|
||||
assert message == {"system_message": f"-- {cli_out}"}
|
||||
|
||||
def test_error_when_cli_fails(self, hook_name, monkeypatch, capsys):
|
||||
hook = _load_hook(hook_name)
|
||||
calls, message = self._run_hook(
|
||||
hook, monkeypatch, capsys, "", cli_rc=1)
|
||||
assert message["system_message"] == "-- [cost: error - boom]"
|
||||
|
||||
def test_missing_session_dir(self, hook_name, monkeypatch, capsys):
|
||||
hook = _load_hook(hook_name)
|
||||
calls, message = self._run_hook(
|
||||
hook, monkeypatch, capsys, "", session_exists=False)
|
||||
assert message["system_message"] == "-- [cost: session dir not found]"
|
||||
|
||||
def test_json_is_not_used_for_display(self, hook_name,
|
||||
monkeypatch, capsys):
|
||||
"""The hook must not parse --json to format the display:
|
||||
formatting lives only in the package's display module."""
|
||||
hook = _load_hook(hook_name)
|
||||
calls, message = self._run_hook(
|
||||
hook, monkeypatch, capsys,
|
||||
"zai:m | session:$0.01 | turn:$0.00\n-- I:1 O:1 C:0")
|
||||
assert "--json" not in calls["cmd"]
|
||||
assert "--display-lines" in calls["cmd"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("hook_name", sorted(HOOK_FILES))
|
||||
class TestNoDuplicatedFormatting:
|
||||
"""The hooks must hold no display-formatting code: the format
|
||||
lives in agent_cost_estimator.display only."""
|
||||
|
||||
def test_hook_has_no_local_formatters(self, hook_name):
|
||||
hook = _load_hook(hook_name)
|
||||
assert not hasattr(hook, "format_cost_output")
|
||||
assert not hasattr(hook, "_format_token_count")
|
||||
|
||||
|
||||
class TestDisplayParity:
|
||||
"""The canonical display module and the CLI agree."""
|
||||
|
||||
def test_display_lines_shape(self):
|
||||
report = {
|
||||
"pricing": {"model_key": "zai:zai-glm-5-3"},
|
||||
"cost_usd": {"total_usd": 0.0136},
|
||||
"last_turn": {"cost_usd": {"total_usd": 0.0031}},
|
||||
"input_tokens": 42116859,
|
||||
"cached_input_tokens": 41355264,
|
||||
"output_tokens": 95073,
|
||||
})
|
||||
return json.dumps(data)
|
||||
"model_price_unavailable": False,
|
||||
}
|
||||
assert format_display_lines(report) == [
|
||||
"zai:zai-glm-5-3 | session:$0.01 | turn:$0.00",
|
||||
"-- I:42.12M O:95.07k C:41.36M",
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("hook_name", sorted(HOOK_FILES))
|
||||
class TestFormatCostOutput:
|
||||
def test_two_lines_with_tokens(self, hook_name):
|
||||
hook = _load_hook(hook_name)
|
||||
result = hook.format_cost_output(_report_json(tokens=True))
|
||||
assert result == (
|
||||
"zai:zai-glm-5-3 | session:$0.01 | turn:$0.00\n"
|
||||
"-- I:42.12M O:95.07k C:41.36M"
|
||||
)
|
||||
|
||||
def test_cost_line_only_without_token_fields(self, hook_name):
|
||||
hook = _load_hook(hook_name)
|
||||
result = hook.format_cost_output(_report_json(tokens=False))
|
||||
assert result == "zai:zai-glm-5-3 | session:$0.01 | turn:$0.00"
|
||||
assert "\n" not in result
|
||||
|
||||
def test_unknown_provider_falls_back(self, hook_name):
|
||||
hook = _load_hook(hook_name)
|
||||
report = json.dumps({
|
||||
"pricing": {"model_key": "noprovider"},
|
||||
def test_token_line_omitted_without_token_fields(self):
|
||||
report = {
|
||||
"pricing": {"model_key": "zai:m"},
|
||||
"cost_usd": {"total_usd": 1.0},
|
||||
"last_turn": None,
|
||||
}
|
||||
assert format_display_lines(report) == [
|
||||
"zai:m | session:$1.00 | turn:$0.00",
|
||||
]
|
||||
|
||||
def test_unknown_provider_falls_back(self):
|
||||
report = {
|
||||
"pricing": {"model_key": "noprovider"},
|
||||
"cost_usd": {"total_usd": 1.0},
|
||||
"input_tokens": 0,
|
||||
"cached_input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
})
|
||||
result = hook.format_cost_output(report)
|
||||
assert result.startswith("unknown:noprovider | session:$1.00 | turn:$0.00")
|
||||
assert result.endswith("-- I:0 O:0 C:0")
|
||||
}
|
||||
lines = format_display_lines(report)
|
||||
assert lines[0] == "unknown:noprovider | session:$1.00 | turn:$0.00"
|
||||
assert lines[-1] == "-- I:0 O:0 C:0"
|
||||
|
||||
def test_non_json_stdout_returned_stripped(self, hook_name):
|
||||
hook = _load_hook(hook_name)
|
||||
assert hook.format_cost_output(" plain text \n") == "plain text"
|
||||
def test_model_label_override(self):
|
||||
"""An explicit label wins over the report's model_key (the
|
||||
Hermes plugin passes its billing-route label)."""
|
||||
report = {
|
||||
"pricing": {"model_key": "zai:zai-glm-5-3"},
|
||||
"cost_usd": {"total_usd": 0.42},
|
||||
"last_turn": None,
|
||||
"input_tokens": 1000, "cached_input_tokens": 0,
|
||||
"output_tokens": 100,
|
||||
}
|
||||
lines = format_display_lines(report, model_label="mistral:glm-5-3")
|
||||
assert lines[0] == "mistral:glm-5-3 | session:$0.42 | turn:$0.00"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("hook_name", sorted(HOOK_FILES))
|
||||
class TestTokenFormatterParity:
|
||||
"""The hook's inlined formatter must match the package formatter."""
|
||||
|
||||
@pytest.mark.parametrize("count", [
|
||||
0, 500, 999, 1000, 1500, 9999, 100000, 999999, 1000000,
|
||||
1234567, 1000000000, 1500000000000,
|
||||
])
|
||||
def test_matches_package_formatting(self, hook_name, count):
|
||||
hook = _load_hook(hook_name)
|
||||
assert hook._format_token_count(count) == format_token_count(count)
|
||||
|
||||
def test_none_input_is_zero(self, hook_name):
|
||||
hook = _load_hook(hook_name)
|
||||
assert hook._format_token_count(None) == "0"
|
||||
def test_note_line_between_cost_and_tokens(self):
|
||||
report = {
|
||||
"pricing": {"model_key": "d:m"},
|
||||
"cost_usd": {"total_usd": 0.42},
|
||||
"last_turn": None,
|
||||
"input_tokens": 2_000_000, "cached_input_tokens": 500_000,
|
||||
"output_tokens": 100_000,
|
||||
"model_price_unavailable": True,
|
||||
}
|
||||
assert format_display_lines(report) == [
|
||||
"d:m | session:$0.42 | turn:$0.00",
|
||||
" **model price not available**",
|
||||
"-- I:2.00M O:100.00k C:500.00k",
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user