Add section 5.8 GUI Interface with: - Cross-platform Python GUI (Windows and Linux support) - Default harness from %APPDATA%/agent-cost-estimator/settings.json - Tree view of sessions in chronological order (latest on top) - Three columns: model, session price, current turn price - Status: Not Implemented Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
48 KiB
name, version, title, type, created, updated
| name | version | title | type | created | updated |
|---|---|---|---|---|---|
| Agent Cost Estimator Specification | 1.1.0 | Project Spec Sheet | specification | 2026-10-24 | 2026-10-24 |
Agent Cost Estimator Specification
- 1. Product Overview
- 2. Architecture
- 3. Core Contracts
- 4. Package and Runtime
- 5. Feature Requirements
- 6. Configuration Files
- 7. Harness-Specific Configurations
- 8. Error Handling
- 9. Testing
- 10. Deployment
- Appendix A: Glossary
- Appendix B: Example Usage
- Appendix C: File Index
This document is the build contract for Agent Cost Estimator. A reader with
this file, and no source tree, must be able to produce the same observable
software. Status values are Implemented and Planned.
1. Product Overview
1.1. Purpose
Agent Cost Estimator provides real-time visibility into AI agent session costs by reading usage data directly from harness log files and calculating costs based on configurable pricing models (USD per million tokens).
1.2. Scope
In Scope:
- Reading token usage from AI harness session logs
- Calculating costs based on configurable pricing
- Supporting multiple AI harnesses (extensible adapter system)
- Supporting multiple AI providers with different pricing models
- CLI interface for manual cost queries
- Hook-based integration for real-time cost tracking
Out of Scope:
- Actual token generation or AI model interaction
- Authentication with AI providers
- Billing system integration
- Historical cost analysis across multiple sessions (future enhancement)
1.3. Non-Goals
- Providing official pricing data (prices are user-configured)
- Modifying AI harness behavior
- Storing session data long-term
- User authentication or authorization
1.4. Document Conventions
- File paths are relative to the repository root unless stated otherwise
MUST/SHALL= mandatory requirementSHOULD= strong recommendationMAY= optional feature- Code examples use Python 3.9+ syntax
2. Architecture
2.1. Component Diagram
┌─────────────────────────────────────────────────────────────────┐
│ Agent Cost Estimator │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ CLI (cli.py) ││
│ └──────────────────────┬──────────────────────────────────────┘│
│ │ │
│ ┌────────────────┴────────────────┐ │
│ │ │ │
│ ┌──────▼──────┐ ┌────────▼──────┐ │
│ │ core.py │ │ pricing.py │ │
│ │ (calculations)│ │ (pricing data)│ │
│ └──────┬──────┘ └────────┬──────┘ │
│ │ │ │
│ └────────────────────┼────────────────┘ │
│ │ │
│ ┌────────────────────▼────────────────┐ │
│ │ Harness Adapters │ │
│ │ (extensible interface: base.py) │ │
│ └──────────────────┬──────────────────┘ │
│ │ │
│ ┌─────────────────┼─────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ vibe.py │ │ openai.py │ │ ... │ │
│ │ (Mistral) │ │ (Planned) │ │ (Future) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
│
▼
Session Data (read-only)
├── meta.json
├── journal/*.jsonl
└── generations/*/projection-state.json
2.2. Data Flow
Vibe Session (CLI or VSCodium)
│
▼ (hook trigger: post_agent)
┌─────────────────────┐
│ post_turn.py Hook │◄── Receives: session_id, hook_event_name
│ (modular per harness)│ (from Vibe CLI or mistral-vibe-code)
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ agent-cost-estimator │◄── Args: --session-id <path> --json
│ CLI (cli.py) │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ VibeHarness │◄── Reads: meta.json, journal/*.jsonl,
│ (vibe.py) │ projection-state.json,
│ │ config.toml (fallback for model detection)
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ collect_usage() │◄── Returns: SessionUsage (tokens, calls, turns)
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ load_pricing() │◄── Reads: pricing.json (provider-based)
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ get_model_prices() │◄── Returns: (rates, key, currency, is_default)
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ compute_cost() │◄── Returns: cost breakdown (input, cached, output)
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Output Formatter │◄── Formats: default, details, JSON
└─────────────────────┘
│
▼
Hook Output: "mistral:mistral-medium-3.5 | session:$X | turn:$Y"
2.3. Module Structure
agent-cost-estimator/
├── agent_cost_estimator/
│ ├── __init__.py # Public API exports
│ ├── __main__.py # Entry point for python -m
│ ├── cli.py # CLI interface and main()
│ ├── core.py # Cost computation logic
│ ├── pricing.py # Pricing table handling
│ │
│ ├── harnesses/
│ │ ├── __init__.py # Harness registry
│ │ ├── base.py # HarnessAdapter base class (ABC)
│ │ ├── vibe.py # Vibe-specific implementation (CLI + VSCodium)
│ │ └── hermes.py # Hermes Agent (state.db + plugin ledger)
│ │
│ ├── hooks/
│ │ ├── mistral-vibe-cli/
│ │ │ └── post_turn.py # Hook for Mistral Vibe CLI
│ │ ├── mistral-vibe-code/
│ │ │ └── post_turn.py # Hook for Mistral Vibe VSCodium extension
│ │ └── README.md # Overview of hook structure
│ │
│ ├── integration/
│ │ └── hermes/
│ │ ├── install.py # Standalone installer (any Hermes home)
│ │ └── plugin/ # THE Hermes plugin (copied by install.py)
│ │ ├── plugin.yaml
│ │ └── __init__.py
│ │
│ └── skill/
│ └── SKILL.md # LLM usage instructions
│
├── pyproject.toml # Package metadata and dependencies
├── pricing.json # Default pricing configuration
├── README.md # Human-readable documentation
└── tests/
└── test_agent_cost_estimator.py # Test suite
3. Core Contracts
3.1. Harness Adapter Interface
File: agent_cost_estimator/harnesses/base.py
Purpose: Define the interface that all AI harness adapters MUST implement.
Contract:
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol
@dataclass
class SessionMeta:
"""Session metadata extracted from harness logs."""
title: str | None
model_name: str | None
alias: str | None
@dataclass
class TurnUsage:
"""Token usage for a single turn/assistant response."""
label: str
calls: int
input_tokens: int
cached_input_tokens: int
output_tokens: int
fresh_input_tokens: int
@dataclass
class SessionUsage:
"""Aggregate token usage for a session."""
input_tokens: int
cached_input_tokens: int
output_tokens: int
fresh_input_tokens: int
calls: int
turns: list[TurnUsage]
usage_source: str # e.g., "projection-state", "journal"
class HarnessAdapter(Protocol):
"""Interface for AI harness adapters.
To add a new harness:
1. Implement this interface in a new file under harnesses/
2. Register via entry point in pyproject.toml
"""
name: str # e.g., "vibe", "openai"
home_env_var: str # e.g., "VIBE_HOME", "OPENAI_HOME"
home_dir_name: str # e.g., ".vibe", ".openai"
def default_home(self) -> Path:
"""Return the default home directory for this harness."""
...
def resolve_session_dir(self, session_arg: str | None, home: Path) -> Path:
"""Resolve session directory from argument or default discovery.
Args:
session_arg: User-provided session ID or path, or None for auto-discovery
home: Harness home directory
Returns:
Absolute path to the session directory
Raises:
FileNotFoundError: If session not found
"""
...
def read_meta(self, session_dir: Path) -> SessionMeta:
"""Read session metadata from harness logs.
Args:
session_dir: Session directory path
Returns:
SessionMeta with title, model_name, and alias
"""
...
def collect_usage(self, session_dir: Path) -> SessionUsage:
"""Collect token usage data from harness logs.
Args:
session_dir: Session directory path
Returns:
SessionUsage with aggregate and per-turn token counts
"""
...
Registration:
To register a new harness adapter, add to pyproject.toml:
[project.entry-points."agent_cost_estimator.harnesses"]
my-harness = "my_package.harnesses:MyHarnessAdapter"
3.2. Pricing Table Format
File: agent_cost_estimator/pricing.json (bundled default)
Purpose: Define pricing models for different AI providers and models.
Format: Provider-based structure (version 2.0+)
{
"providers": {
"<provider_name>": {
"currency": "<currency_code>",
"models": {
"<model_name>": {
"input_price": <number>,
"cached_input_price": <number>,
"output_price": <number>,
"note": "<optional description>",
"currency": "<optional override>"
},
"default": {
"input_price": <number>,
"cached_input_price": <number>,
"output_price": <number>
}
}
}
}
}
Semantics:
providers: REQUIRED. Dictionary of provider configurationsprovider_name: Provider identifier (e.g., "mistral", "openai", "anthropic")currency: Provider-level currency (default: "USD" if not specified)models: REQUIRED. Dictionary of model pricing for this providermodel_name: Model identifier (e.g., "mistral-medium", "gpt-4")input_price: USD per million input tokens (prompt tokens)cached_input_price: USD per million cached input tokens (context cache)output_price: USD per million output tokens (completion tokens)- Model-level
currency: Optional override for specific model
Lookup Order:
- Exact match on
model_name(passed as first argument to get_model_prices) - Exact match on
alias(passed as second argument) - Provider's
defaultmodel - First provider's
defaultmodel (fallback)
Example:
{
"providers": {
"default": {
"currency": "USD",
"models": {
"default": {
"input_price": 1.5,
"cached_input_price": 0.15,
"output_price": 7.5,
"note": "Global fallback rates"
}
}
},
"mistral": {
"currency": "USD",
"models": {
"default": {
"input_price": 1.5,
"cached_input_price": 0.15,
"output_price": 7.5
},
"mistral-small": {
"input_price": 1.0,
"cached_input_price": 0.1,
"output_price": 3.0
},
"mistral-medium": {
"input_price": 1.5,
"cached_input_price": 0.15,
"output_price": 7.5
},
"mistral-large": {
"input_price": 2.5,
"cached_input_price": 0.25,
"output_price": 12.5
}
}
}
}
}
Validation Rules:
- All price values MUST be non-negative numbers
- Each provider MUST have a
modelsdictionary - Each provider's
modelsMUST contain at least one model - The
defaultprovider MUST exist (for fallback)
3.3. Hook Integration
Purpose: Enable real-time cost tracking in AI harness sessions.
Hook Types:
post_agent: Fires after assistant turns (RECOMMENDED for cost tracking)
Hook Contract:
-
Hook receives JSON payload on stdin:
{ "session_id": "<uuid>", "hook_event_name": "post_agent" } -
Hook MUST output JSON with
system_messagefield:{ "system_message": "-- <formatted cost output>" } -
Hook MUST be executable and return exit code 0 on success
Hook Installation:
# Automatic (recommended)
agent-cost-estimator --integrate mistral
# Manual
mkdir -p ~/.vibe/hooks
cp agent_cost_estimator/hooks/post_turn.py ~/.vibe/hooks/
chmod +x ~/.vibe/hooks/post_turn.py
4. Package and Runtime
4.1. Dependencies
Runtime Dependencies:
- Python >= 3.9
- Standard library only (no external dependencies for core functionality)
Development Dependencies:
- pytest >= 7.0.0
- pytest-cov >= 4.0.0
Packaging:
# pyproject.toml
[build-system]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"
4.2. Entry Points
CLI:
[project.scripts]
agent-cost-estimator = "agent_cost_estimator.cli:main"
Harness Adapters:
[project.entry-points."agent_cost_estimator.harnesses"]
vibe = "agent_cost_estimator.harnesses.vibe:VibeHarness"
hermes = "agent_cost_estimator.harnesses.hermes:HermesHarness"
4.3. Installation
From Source:
cd agent-cost-estimator
pip install .
Verify:
agent-cost-estimator --list-harnesses # Should output: hermes, vibe
agent-cost-estimator --skill # Should output SKILL.md
5. Feature Requirements
5.1. CLI Interface
Status: Implemented
Command: agent-cost-estimator [OPTIONS]
Options:
| Option | Type | Required | Default | Description |
|---|---|---|---|---|
--harness |
str | No | vibe |
Harness to read (vibe, hermes) |
--session-id |
str | No | auto | Session ID or directory path |
--home |
str | No | harness default | Harness data root directory |
--pricing |
str | No | bundled | Path to pricing.json file |
--model |
str | No | from session | Force pricing lookup key |
--input-price |
float | No | from pricing | Override input price |
--cached-input-price |
float | No | from pricing | Override cached input price |
--output-price |
float | No | from pricing | Override output price |
-d, --details |
flag | No | False | Full report table |
-s, --session |
flag | No | False | Session cost only (bare number) |
-t, --turn |
flag | No | False | Latest turn cost only (bare number) |
--turns |
flag | No | False | Per-turn breakdown (with -d) |
--json |
flag | No | False | JSON output |
--list-harnesses |
flag | No | False | List available harnesses |
--skill |
flag | No | False | Print SKILL.md |
--integrate HARNESS |
str | No | None | Install hooks for harness |
--update-pricing |
flag | No | False | Persist pricing to file |
--dry-run |
flag | No | False | Dry run (with --update-pricing) |
Exit Codes:
- 0: Success
- 1: Error (invalid arguments, session not found, etc.)
5.2. Vibe Harness Adapter
Status: Implemented
Purpose: Read session data from Mistral Vibe CLI logs.
File: agent_cost_estimator/harnesses/vibe.py
Implementation:
default_home(): Returns~/.vibe(or$VIBE_HOME)resolve_session_dir(): Discovers latest session or uses provided IDread_meta(): Readsmeta.jsonfor title, model_name, active_modelcollect_usage(): Reads from projection-state.json (preferred) or journal files
Session Discovery:
- If
--session-idprovided: Use directly - Otherwise: Find newest directory in
~/.vibe/logs/session/unified/
Usage Source Priority:
projection-state.json(survives journal rotation, has cumulative totals)- Journal files (current window on disk)
Cached Token Handling:
- Cached tokens are clamped to input tokens (matching Vibe's behavior)
- Fresh tokens = input_tokens - cached_input_tokens
5.2.1. Hermes Harness Adapter
Status: Implemented
Purpose: Read session data from Hermes Agent's SQLite store.
File: agent_cost_estimator/harnesses/hermes.py
Implementation:
default_home(): Returns$HERMES_HOMEor~/.hermesresolve_session_dir(): Accepts a session id (e.g.local-<hash>), astate.dbpath, or a profile home directory; None picks the session with the newestlast_activity_at. Returns aHermesSession(db_path + session_id;.nameexposes the session id for the CLI's display).read_meta(): Readssessions.titleandsessions.modelcollect_usage(): session totals fromsession_model_usage(authoritative cumulative counter); per-turn breakdown from the plugin's usage ledger
Session Discovery:
- If
--session-idis a session id: validate againstsessions - If it is a file: treat as
state.db - If it is a directory:
<dir>/state.db - Otherwise: newest
last_activity_atinsessions
Usage Source Priority:
state.dbsession_model_usage(cumulative, written by Hermes after every API call - survives ledger loss)- The plugin's JSONL ledger
(
<home>/plugin-data/agent-cost-estimator/usage/<session_id>.jsonl, per-call records grouped byturn_id) - only present once the plugin is installed
Token Bucket Mapping (Hermes -> tool contract):
Hermes's CanonicalUsage buckets are disjoint; the tool's
SessionUsage is Vibe-style (input includes cached):
| Hermes column | SessionUsage |
|---|---|
input_tokens |
fresh input (excl. cache) |
cache_read_tokens |
cached input |
cache_write_tokens |
fresh input (billed) |
input_tokens= input + cache_read + cache_write (=prompt_tokens)cached_input_tokens= cache_readfresh_input_tokens= input + cache_writeoutput_tokens= output (includes reasoning tokens)
Ledger Format (one JSON object per line):
{"ts": 1792847431.9, "session_id": "local-...", "turn_id": "t1",
"model": "zai-glm-5-3", "provider": "zai",
"api_request_id": "r1", "aux_task": null,
"label": "user message head", "calls": 1,
"input_tokens": 453000, "cached_input_tokens": 400000,
"output_tokens": 9000}
Records are deduplicated by api_request_id when present;
cached_input_tokens is clamped to input_tokens per record and
per turn.
5.3. Pricing Configuration
Status: Implemented
Features:
- Provider-based pricing structure
- Model-level currency overrides
- Default fallback (provider default → global default)
- Warning when using default pricing (
**model price not available**)
Lookup Priority:
- Passed model name (now uses
active_modelfirst) - Passed alias (
model_name) - Provider's default model
- First provider's default model
CLI Overrides:
agent-cost-estimator --input-price 2.0 --output-price 10.0 -s
Persist Overrides:
agent-cost-estimator --model my-model --input-price 2.0 --update-pricing
5.4. Cost Calculation
Status: Implemented
File: agent_cost_estimator/core.py
Formula:
input_cost_usd = (fresh_input_tokens * input_price + cached_input_tokens * cached_input_price) / 1_000_000
output_cost_usd = (output_tokens * output_price) / 1_000_000
total_cost_usd = input_cost_usd + output_cost_usd
Function:
def compute_cost(tokens: dict[str, int], rates: dict[str, float]) -> dict[str, float]:
"""Compute cost from token counts and pricing rates.
Args:
tokens: {"fresh": int, "cached": int, "output": int}
rates: {"input_price": float, "cached_input_price": float, "output_price": float}
Returns:
{"input_usd": float, "cached_usd": float, "output_usd": float, "total_usd": float}
"""
5.5. Hook System
Status: Implemented
Purpose: Display costs after each assistant turn in Vibe CLI.
Hook Script: agent_cost_estimator/hooks/post_turn.py
Behavior:
- Receives session_id from hook payload
- Resolves session directory
- Calls
agent-cost-estimator --session-id <path> --json - Formats output as
provider:model | session:$X | turn:$Y - Returns JSON with
system_messagefield
Output Examples:
-- mistral:mistral-medium | session:$3.40 | turn:$0.92
-- mistral:mistral-medium-3.5 | session:$0.00 | turn:$0.00
Error Handling:
- Session not found:
-- [cost: session dir not found] - Timeout:
-- [cost: timeout] - Other errors:
-- [cost: error - <message>]
5.6. Integration Command
Status: Implemented
Command: agent-cost-estimator --integrate <harness>
Supported Harnesses:
mistral,mistral-vibe-cli,vibe,cli(Vibe CLI)mistral-vibe-code,vscodium,code(VSCodium extension)hermes,hermes-agent(Hermes Agent plugin)
Modular Design:
The --integrate command accepts a harness name argument; each
harness dispatches to its own installer (Vibe family: inline hook
install; Hermes: agent_cost_estimator.integration.hermes.install).
Actions (Vibe family):
- Creates harness-specific hook directory
- Installs hook script
- Creates hooks.toml configuration
- Updates bash allowlist in config.toml
Actions (Hermes):
- Validates the Hermes home (
state.dborconfig.yamlpresent) - Copies the plugin (
plugin.yaml,__init__.py) to<home>/plugins/agent-cost-estimator/ - Ships
pricing.jsonnext to the plugin (preserves an already-tuned installed table on reinstall) - Runs
hermes plugins enable agent-cost-estimator(prints the manual step when thehermesCLI is not on PATH)
Examples:
agent-cost-estimator --integrate mistral
agent-cost-estimator --integrate hermes
agent-cost-estimator --rm-integrate hermes
The Hermes installer is also runnable standalone on any machine:
python -m agent_cost_estimator.integration.hermes.install
[--hermes-home DIR] [--uninstall] [--dry-run]
Future Extension:
To add a new harness openai:
- Add to
supported_harnesseslist inintegrate_hooks() - Add harness-specific directory logic if needed
- Update documentation
5.7. Output Formatting
Status: Implemented
Default Format:
provider:model | session:$X.XX | turn:$Y.YY
Details Format (-d):
harness: vibe
session: <session-id>
model: provider:model
<warning if default pricing>
model calls: N
usage source: projection-state
input tokens: N,NNN
output tokens: N,NNN
pricing: provider:model from bundled pricing.json
currency: USD
prices (USD/M): input X cached Y output Z
input cost: $X.XXXX
cached input cost: $X.XXXX
output cost: $X.XXXX
TOTAL: $X.XXXX
JSON Format (--json):
{
"harness": "vibe",
"session_id": "<uuid>",
"title": null,
"model": "mistral-vibe-cli-latest",
"provider": "mistral",
"provider_model": "mistral:mistral-medium-3.5",
"model_price_unavailable": false,
"calls": N,
"input_tokens": N,
"cached_input_tokens": N,
"fresh_input_tokens": N,
"output_tokens": N,
"pricing": {
"file": "bundled",
"model_key": "mistral:mistral-medium-3.5",
"currency": "USD",
"rates_usd_per_million": {
"input_price": X,
"cached_input_price": Y,
"output_price": Z
}
},
"usage_source": "projection-state",
"cost_usd": {
"input_usd": X,
"cached_usd": Y,
"output_usd": Z,
"total_usd": T
},
"last_turn": {...},
"turns": [...]
}
Warning Indicator:
When a model falls back to default pricing, all output modes include:
**model price not available**
5.8. GUI Interface
Status: Not Implemented
Purpose: Provide a graphical user interface for visualizing session costs and interacting with the cost estimator in a user-friendly way.
Platform Support:
- Windows: Native support
- Linux: Native support
- Uses cross-platform Python GUI framework (e.g., Tkinter, PyQt, or similar)
Default Harness Configuration:
-
Settings File Location:
- Windows:
%APPDATA%\agent-cost-estimator\settings.json - Linux:
$XDG_CONFIG_HOME/agent-cost-estimator/settings.json(defaults to~/.config/agent-cost-estimator/settings.json)
- Windows:
-
settings.json Structure:
{ "default_harness": "vibe" } -
Default Harness: On first launch, if settings.json does not exist, default to
vibeharness -
Harness Selection: User can change the default harness in settings, which persists across application restarts
GUI Components:
-
Main Window:
- Title: "Agent Cost Estimator"
- Menu bar with File, View, Help menus
- Status bar showing current harness and version
-
Session Tree View:
- Display: Tree structure showing all discovered sessions
- Ordering: Chronological order, latest session on top
- Root Nodes: Each harness (vibe, hermes) as a root node
- Child Nodes: Individual sessions under each harness
- Session Discovery: Auto-discover sessions from harness home directories
-
Columns Display:
- Column 1 - Model:
- Format:
provider:model(e.g.,mistral:mistral-medium) - Source: Session metadata from meta.json
- Format:
- Column 2 - Session Price:
- Format:
$X.XX(USD currency) - Value: Cumulative cost for the entire session
- Calculation: Sum of all token costs (input + output + cached) * pricing
- Format:
- Column 3 - Current Turn Price:
- Format:
$Y.YY(USD currency) - Value: Cost of the most recent turn in the session
- Calculation: Token costs for the latest turn * pricing
- Format:
- Column 1 - Model:
-
Tree View Behavior:
- Single click on session: Selects the session
- Double click on session: Opens detailed view for that session
- Right click on session: Context menu with actions (refresh, copy, etc.)
- Auto-refresh: Optional periodic refresh of session data
Data Source:
- Uses existing harness adapters to read session data
- Reuses pricing configuration from existing pricing system
- Combines data from both bundled and custom pricing files
Future Enhancements:
- Add per-turn breakdown view
- Add filtering and sorting capabilities
- Add export functionality (CSV, JSON)
- Add historical cost charts
- Add real-time cost tracking with live updates
6. Configuration Files
6.1. pricing.json
Location: agent_cost_estimator/pricing.json (bundled)
Purpose: Default pricing configuration for all supported providers/models.
Structure: See 3.2. Pricing Table Format
Bundled Version: Contains Mistral, Zai, and default provider configurations.
Customization:
Users can:
- Provide custom pricing file via
--pricingflag - Update pricing for a model via
--update-pricing - Override prices via CLI flags (
--input-price, etc.)
6.2. User Configurable Pricing Files
Status: Not Implemented
Location:
- Windows:
%APPDATA%\agent-cost-estimator\price.jsonand%APPDATA%\agent-cost-estimator\custom-price.json - Linux/macOS:
$XDG_CONFIG_HOME/agent-cost-estimator/price.jsonand$XDG_CONFIG_HOME/agent-cost-estimator/custom-price.json- If
$XDG_CONFIG_HOMEis not set, defaults to~/.config/agent-cost-estimator/
- If
Purpose: Allow users to maintain custom pricing configurations that persist across application updates and are easily accessible.
Files:
-
price.json:- Initialized from the repository's bundled
pricing.jsonon first run - Contains the default pricing configuration for all supported providers/models
- Serves as the base pricing data
- Initialized from the repository's bundled
-
custom-price.json:- Created as a blank template on first run
- Intended for user editing to add custom models, providers, or override existing rates
- Empty file if user has not added any customizations
Behavior:
- On application startup, both files are read if they exist
- The application merges pricing data from both files
- Priority Rule: In case of duplicate entries (same provider/model), values from
custom-price.jsontake precedence overprice.json - This allows users to override specific pricing without losing the ability to receive updates to the base pricing data
Directory Creation:
- The application MUST create the user configuration directory if it does not exist
- Both files MUST be created on first run (price.json from bundled version, custom-price.json as empty template)
- File permissions SHOULD allow user read/write access
Format: Both files follow the same structure as defined in 3.2. Pricing Table Format
Example custom-price.json:
{
"providers": {
"custom-provider": {
"currency": "USD",
"models": {
"my-custom-model": {
"input_price": 2.0,
"cached_input_price": 0.2,
"output_price": 10.0,
"note": "My custom pricing"
}
}
}
}
}
7. Harness-Specific Configurations
This section contains configuration details that are specific to individual AI harnesses. Each harness may have its own hooks, configuration files, and integration requirements.
Modular Structure:
To support multiple Vibe-based harnesses (CLI, VSCodium extension, etc.), hooks are organized
in a modular directory structure under agent_cost_estimator/hooks/:
hooks/
├── mistral-vibe-cli/
│ └── post_turn.py # Hook for Mistral Vibe CLI
├── mistral-vibe-code/
│ └── post_turn.py # Hook for Mistral Vibe VSCodium extension
└── README.md # Overview of hook structure
This modular approach allows:
- Different implementations for different Vibe-based harnesses
- Harness-specific model detection logic
- Clean separation of concerns
7.1. Vibe CLI (Mistral) Configuration
Harness: Mistral Vibe CLI
Adapter: agent_cost_estimator.harnesses.vibe:VibeHarness
Hook Location: agent_cost_estimator/hooks/mistral-vibe-cli/post_turn.py
7.1.1. Hook Implementation
Location: agent_cost_estimator/hooks/mistral-vibe-cli/post_turn.py
Purpose: Vibe CLI hook for real-time cost tracking.
Features:
- Session directory resolution (session_id → path)
- Error handling with graceful degradation
- JSON output format for Vibe CLI compatibility
- Logging for debugging
- Reads model from
meta.jsonconfig field (Vibe CLI standard)
Installation:
Automatic via --integrate mistral-vibe-cli command, or manual copy to ~/.vibe/hooks/
7.1.2. hooks.toml Configuration
Location: Created at ~/.vibe/hooks.toml during integration
Purpose: Vibe CLI hook configuration
Content:
[[hooks]]
name = "cost-tracker"
type = "post_agent"
command = "python ~/.vibe/hooks/mistral-vibe-cli/post_turn.py"
timeout = 30
strict = false
description = "Display cost after each assistant turn"
7.2. Vibe VSCodium Extension (Mistral) Configuration
Harness: Mistral Vibe VSCodium Extension (mistralai.mistral-vibe-code)
Adapter: agent_cost_estimator.harnesses.vibe:VibeHarness (shared with CLI)
Hook Location: agent_cost_estimator/hooks/mistral-vibe-code/post_turn.py
7.2.1. Hook Implementation
Location: agent_cost_estimator/hooks/mistral-vibe-code/post_turn.py
Purpose: VSCodium extension hook for real-time cost tracking.
Features:
- Session directory resolution (session_id → path)
- Error handling with graceful degradation
- JSON output format for Vibe CLI compatibility
- Logging for debugging
- Enhanced model detection for VSCodium extension:
- Checks VSCodium-specific configuration files
- Falls back to Vibe CLI standard locations
- Maintains compatibility with both CLI and VSCodium sessions
Model Detection Strategy:
The VSCodium extension (mistral-vibe-code) may not write model information to the
standard meta.json location. The hook uses an enhanced detection strategy:
- Primary: Read from
meta.jsonconfig.model_name (Vibe CLI standard) - VSCodium-specific: Check VSCodium extension state/storage for active model
- Fallback: Use default pricing with warning indicator
- Override: Allow environment variable
VIBE_MODELOVERRIDEfor manual setting
Installation:
Automatic via --integrate mistral-vibe-code command, or manual copy to ~/.vibe/hooks/
7.2.2. hooks.toml Configuration
Location: Created at ~/.vibe/hooks.toml during integration
Purpose: VSCodium extension hook configuration
Content:
[[hooks]]
name = "cost-tracker-vscodium"
type = "post_agent"
command = "python ~/.vibe/hooks/mistral-vibe-code/post_turn.py"
timeout = 30
strict = false
description = "Display cost after each assistant turn (VSCodium)"
7.2.3. VSCodium-Specific Considerations
The VSCodium extension (mistralai.mistral-vibe-code) uses the same underlying Vibe
engine but may have differences in:
- Session Storage: Same location (
~/.vibe/logs/session/unified/) - Model Configuration: May store model settings differently than CLI
- Hook Triggering: Depends on extension version and configuration
The hook is designed to be resilient to these differences.
7.3. Hermes Agent (Nous Research) Configuration
Harness: Hermes Agent
Adapter: agent_cost_estimator.harnesses.hermes:HermesHarness
Plugin Location: agent_cost_estimator/integration/hermes/plugin/
Installer: agent_cost_estimator/integration/hermes/install.py
Hermes integration is a plugin, not a hook script: Hermes
loads plugins from <home>/plugins/<name>/ (manifest
plugin.yaml + __init__.py with register(ctx)), gated by
hermes plugins enable <name>.
7.3.1. Plugin Implementation
File: agent_cost_estimator/integration/hermes/plugin/__init__.py
Manifest:
name: agent-cost-estimator
version: <package version>
provides_hooks:
- pre_llm_call
- post_api_request
- post_auxiliary_call
- post_llm_call
Registered hooks and payloads:
| Hook | Payload fields used | Action |
|---|---|---|
pre_llm_call |
turn_id, user_message |
capture the turn label (message head, 60 chars) |
post_api_request |
session_id, turn_id, model, provider, api_request_id, usage (CanonicalUsage dict) |
queue one ledger record |
post_auxiliary_call |
same + aux_task |
queue one ledger record (aux spend folds into the parent turn) |
post_llm_call |
session_id, turn_id, model, platform |
print the cost line |
Data flow:
- Each API call appends a JSON line (via a bounded in-process
queue and a single daemon drainer thread - the hook callback
itself never does I/O) to
<home>/plugin-data/agent-cost-estimator/usage/<session_id>.jsonl - The per-turn cost also accumulates in memory, so the printed turn figure is exact even before the ledger flushes
- At turn end (
post_llm_call- which Hermes fires after flushing its token queue to state.db), the plugin reads the session's cumulative totals fromstate.db(session_model_usage, read-only) and prints:
-- provider:model | session:$X.XX | turn:$Y.YY
- Session totals come from state.db (authoritative, like Vibe's
projection-state); the ledger provides the per-turn breakdown
across restarts (the tool's
--harness hermesadapter reads both).
Print gating:
- Prints only for interactive terminal surfaces: the gateway's
local adapter (
platform == "local", how current Hermes builds run CLI chat) and the non-gateway fallback ("cli") - Never prints for subagents (
platform == "subagent"), cron runs, or one-shot runs (HERMES_SINGLE_QUERY_SESSION=1, i.e.hermes chat -q) - their stdout must stay machine-readable - Subagent and auxiliary calls still RECORD to the ledger (their spend is real and shows in session totals)
Pricing resolution (per print):
pricing.jsonnext to the installed plugin (stdlib-only reader; tune rates by editing this file)- the
agent_cost_estimatorpackage's bundledpricing.json(when importable in Hermes's interpreter) - hard-coded defaults (1.5 / 0.15 / 7.5 USD per M) with the
model price not availablewarning
Failure policy: every callback swallows its own errors (debug-logged); a cost line must never break a turn. Ledger writes are best-effort (bounded queue, drop on overflow - session totals are unaffected).
Known limitation: Hermes skips a timeout-bounded hook
callback that is still running for the same turn_id, so under
MoA advisor fan-out a concurrently-fired duplicate call may not be
ledgered. Session totals are unaffected; only that call's
contribution to the per-turn breakdown can be lost.
7.3.2. Installation
# automatic (from the package)
agent-cost-estimator --integrate hermes
# standalone (any machine, source checkout, or installed package)
python -m agent_cost_estimator.integration.hermes.install
python path/to/install.py --hermes-home /custom/home --dry-run
The installer:
- Validates the target home (
state.dborconfig.yaml) - Copies
plugin.yaml+__init__.pyto<home>/plugins/agent-cost-estimator/ - Copies the bundled
pricing.jsonnext to the plugin, preserving an already-installed (user-tuned) table on reinstall - Runs
hermes plugins enable agent-cost-estimator; on failure or missing CLI prints the manual command
Removal:
agent-cost-estimator --rm-integrate hermes
python -m agent_cost_estimator.integration.hermes.install --uninstall
7.3.3. Verification
# after install + a Hermes restart + one turn:
agent-cost-estimator --harness hermes # session + latest turn
agent-cost-estimator --harness hermes -d # full table
hermes plugins list --plain --no-bundled # shows the enabled plugin
8. Error Handling
Principle: Fail gracefully, never break the user's session.
CLI Errors:
- Invalid arguments: Exit 1 with error message to stderr
- Session not found: Exit 1 with error message
- Pricing lookup failure: Exit 1 with error message
Hook Errors:
- Session directory not found: Return error message in JSON format
- agent-cost-estimator fails: Return error message with stdout/stderr
- Timeout: Return timeout message
- Unexpected errors: Return error message with exception
Pricing Errors:
- Model not in pricing.json: Falls back to default, displays warning
- Pricing file invalid: Exit 1 with validation error
- Pricing file missing: Exit 1 with FileNotFoundError
9. Testing
Framework: pytest
Coverage: All core functionality tested
Test Files:
tests/test_agent_cost_estimator.py(52 tests)
Test Categories:
- Core cost calculation
- Pricing table loading/lookup/validation
- Vibe harness adapter (journal parsing, projection, dedupe, session discovery)
- Hermes harness adapter (state.db totals, ledger fallback, dedupe, clamp, session discovery, CLI --json end to end)
- Hermes plugin (hook registration, CanonicalUsage bucket folding, record + print, subagent/single-query gating, pricing sources)
- Hermes installer (dry-run, home validation, plugin + pricing copy, uninstall)
- Harness registry
- CLI (pricing flags, -t, -s, --json, --update-pricing, --integrate, error exits)
Running Tests:
cd agent-cost-estimator
python -m pytest tests/test_agent_cost_estimator.py -v
10. Deployment
Packaging:
# Build
python -m build
# Install
pip install dist/agent_cost_estimator-*.tar.gz
Distribution:
- PyPI (future)
- Direct installation from Git repository
Versioning:
- Follows semantic versioning (MAJOR.MINOR.PATCH)
- Bump version in pyproject.toml for each release
- Maintain CHANGELOG (future enhancement)
Appendix A: Glossary
| Term | Definition |
|---|---|
| Session | A single interaction with an AI agent, with its own UUID and log directory |
| Turn | A single assistant response within a session |
| Harness | An AI agent framework (e.g., Vibe CLI, OpenAI SDK) |
| Provider | An AI model provider (e.g., Mistral, OpenAI, Anthropic) |
| Model | A specific AI model (e.g., mistral-medium, gpt-4) |
| Token | A unit of text that AI models process (input/output) |
| Cached Tokens | Tokens that were already in the model's context cache |
| Fresh Tokens | Tokens that are new in the current request (input_tokens - cached_input_tokens) |
| Hook | A callback that runs at specific events in the harness lifecycle |
Appendix B: Example Usage
Basic Usage:
# Quick summary of current session
agent-cost-estimator
# Full details
agent-cost-estimator -d
# Session cost only
agent-cost-estimator -s
# Turn cost only
agent-cost-estimator -t
# JSON output
agent-cost-estimator --json
# Specific session
agent-cost-estimator --session-id ~/.vibe/logs/session/unified/<uuid>
# Custom pricing file
agent-cost-estimator --pricing /path/to/pricing.json
# Override pricing for this run
agent-cost-estimator --input-price 2.0 --output-price 10.0
# Update pricing file with new rates
agent-cost-estimator --model my-model --input-price 2.0 --update-pricing
Integration:
# Install hooks for Vibe CLI
agent-cost-estimator --integrate mistral
# After installation, start a Vibe session
vibe "Your prompt"
# Output after each turn: -- mistral:mistral-medium | session:$X.XX | turn:$Y.YY
Configuration:
# List available harnesses
agent-cost-estimator --list-harnesses
# Print usage instructions for LLMs
agent-cost-estimator --skill
Appendix C: File Index
Core Modules:
| File | Purpose |
|---|---|
agent_cost_estimator/__init__.py |
Public API exports |
agent_cost_estimator/__main__.py |
Entry point |
agent_cost_estimator/cli.py |
CLI interface |
agent_cost_estimator/core.py |
Cost calculation |
agent_cost_estimator/pricing.py |
Pricing data handling |
Harness Adapters:
| File | Harness | Status |
|---|---|---|
harnesses/__init__.py |
Registry | Implemented |
harnesses/base.py |
Base interface | Implemented |
harnesses/vibe.py |
Mistral Vibe CLI | Implemented |
harnesses/hermes.py |
Hermes Agent | Implemented |
Hooks:
| File | Purpose | Status |
|---|---|---|
hooks/post_turn.py |
Vibe post-agent hook | Implemented |
hooks/mistral-vibe-cli/post_turn.py |
Vibe CLI hook | Implemented |
hooks/mistral-vibe-code/post_turn.py |
VSCodium hook | Implemented |
Integrations:
| File | Purpose | Status |
|---|---|---|
integration/hermes/plugin/plugin.yaml |
Hermes plugin manifest | Implemented |
integration/hermes/plugin/__init__.py |
Hermes plugin (hooks + cost line) | Implemented |
integration/hermes/install.py |
Standalone Hermes installer | Implemented |
Configuration:
| File | Purpose |
|---|---|
pricing.json |
Default pricing |
pyproject.toml |
Package metadata |
Documentation:
| File | Purpose |
|---|---|
SPEC.md |
This specification |
README.md |
User documentation |
skill/SKILL.md |
LLM usage instructions |
Tests:
| File | Purpose |
|---|---|
tests/test_agent_cost_estimator.py |
Test suite |