Files
agent-cost-estimator/SPEC.md
T
nvaldez 2af0ff0ee4 SPEC: add GUI interface feature requirement
Add section 5.8 GUI Interface with:
- Cross-platform Python GUI (Windows and Linux support)
- Default harness from %APPDATA%/agent-cost-estimator/settings.json
- Tree view of sessions in chronological order (latest on top)
- Three columns: model, session price, current turn price
- Status: Not Implemented

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
2026-11-03 16:42:55 +01:00

48 KiB

name, version, title, type, created, updated
name version title type created updated
Agent Cost Estimator Specification 1.1.0 Project Spec Sheet specification 2026-10-24 2026-10-24

Agent Cost Estimator Specification

This document is the build contract for Agent Cost Estimator. A reader with this file, and no source tree, must be able to produce the same observable software. Status values are Implemented and Planned.

1. Product Overview

1.1. Purpose

Agent Cost Estimator provides real-time visibility into AI agent session costs by reading usage data directly from harness log files and calculating costs based on configurable pricing models (USD per million tokens).

1.2. Scope

In Scope:

  • Reading token usage from AI harness session logs
  • Calculating costs based on configurable pricing
  • Supporting multiple AI harnesses (extensible adapter system)
  • Supporting multiple AI providers with different pricing models
  • CLI interface for manual cost queries
  • Hook-based integration for real-time cost tracking

Out of Scope:

  • Actual token generation or AI model interaction
  • Authentication with AI providers
  • Billing system integration
  • Historical cost analysis across multiple sessions (future enhancement)

1.3. Non-Goals

  • Providing official pricing data (prices are user-configured)
  • Modifying AI harness behavior
  • Storing session data long-term
  • User authentication or authorization

1.4. Document Conventions

  • File paths are relative to the repository root unless stated otherwise
  • MUST / SHALL = mandatory requirement
  • SHOULD = strong recommendation
  • MAY = optional feature
  • Code examples use Python 3.9+ syntax

2. Architecture

2.1. Component Diagram

┌─────────────────────────────────────────────────────────────────┐
│                         Agent Cost Estimator                        │
├─────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │                         CLI (cli.py)                              ││
│  └──────────────────────┬──────────────────────────────────────┘│
│                          │                                          │
│         ┌────────────────┴────────────────┐                        │
│         │                                 │                        │
│  ┌──────▼──────┐              ┌────────▼──────┐                  │
│  │ core.py     │              │ pricing.py    │                  │
│  │ (calculations)│              │ (pricing data)│                  │
│  └──────┬──────┘              └────────┬──────┘                  │
│         │                                 │                        │
│         └────────────────────┼────────────────┘                        │
│                          │                                          │
│         ┌────────────────────▼────────────────┐                        │
│         │              Harness Adapters            │                        │
│         │   (extensible interface: base.py)       │                        │
│         └──────────────────┬──────────────────┘                        │
│                            │                                      │
│              ┌─────────────────┼─────────────────┐                  │
│              ▼                 ▼                 ▼                  │
│       ┌──────────────┐   ┌──────────────┐   ┌──────────────┐        │
│       │ vibe.py      │   │ openai.py    │   │ ...          │        │
│       │ (Mistral)    │   │ (Planned)    │   │ (Future)      │        │
│       └──────────────┘   └──────────────┘   └──────────────┘        │
│                                                                     │
└─────────────────────────────────────────────────────────────────┘
                          │
                          ▼
            Session Data (read-only)
            ├── meta.json
            ├── journal/*.jsonl
            └── generations/*/projection-state.json

2.2. Data Flow

Vibe Session (CLI or VSCodium)
    │
    ▼ (hook trigger: post_agent)
┌─────────────────────┐
│ post_turn.py Hook    │◄── Receives: session_id, hook_event_name
│ (modular per harness)│   (from Vibe CLI or mistral-vibe-code)
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│ agent-cost-estimator │◄── Args: --session-id <path> --json
│ CLI (cli.py)         │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│ VibeHarness         │◄── Reads: meta.json, journal/*.jsonl,
│ (vibe.py)           │    projection-state.json,
│                     │    config.toml (fallback for model detection)
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│ collect_usage()     │◄── Returns: SessionUsage (tokens, calls, turns)
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│ load_pricing()      │◄── Reads: pricing.json (provider-based)
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│ get_model_prices()  │◄── Returns: (rates, key, currency, is_default)
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│ compute_cost()      │◄── Returns: cost breakdown (input, cached, output)
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│ Output Formatter    │◄── Formats: default, details, JSON
└─────────────────────┘
           │
           ▼
    Hook Output: "mistral:mistral-medium-3.5 | session:$X | turn:$Y"

2.3. Module Structure

agent-cost-estimator/
├── agent_cost_estimator/
│   ├── __init__.py           # Public API exports
│   ├── __main__.py           # Entry point for python -m
│   ├── cli.py                # CLI interface and main()
│   ├── core.py               # Cost computation logic
│   ├── pricing.py            # Pricing table handling
│   │
│   ├── harnesses/
│   │   ├── __init__.py       # Harness registry
│   │   ├── base.py           # HarnessAdapter base class (ABC)
│   │   ├── vibe.py           # Vibe-specific implementation (CLI + VSCodium)
│   │   └── hermes.py          # Hermes Agent (state.db + plugin ledger)
│   │
│   ├── hooks/
│   │   ├── mistral-vibe-cli/
│   │   │   └── post_turn.py  # Hook for Mistral Vibe CLI
│   │   ├── mistral-vibe-code/
│   │   │   └── post_turn.py  # Hook for Mistral Vibe VSCodium extension
│   │   └── README.md         # Overview of hook structure
│   │
│   ├── integration/
│   │   └── hermes/
│   │       ├── install.py    # Standalone installer (any Hermes home)
│   │       └── plugin/       # THE Hermes plugin (copied by install.py)
│   │           ├── plugin.yaml
│   │           └── __init__.py
│   │
│   └── skill/
│       └── SKILL.md          # LLM usage instructions
│
├── pyproject.toml            # Package metadata and dependencies
├── pricing.json              # Default pricing configuration
├── README.md                 # Human-readable documentation
└── tests/
    └── test_agent_cost_estimator.py  # Test suite

3. Core Contracts

3.1. Harness Adapter Interface

File: agent_cost_estimator/harnesses/base.py

Purpose: Define the interface that all AI harness adapters MUST implement.

Contract:

from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol

@dataclass
class SessionMeta:
    """Session metadata extracted from harness logs."""
    title: str | None
    model_name: str | None
    alias: str | None

@dataclass
class TurnUsage:
    """Token usage for a single turn/assistant response."""
    label: str
    calls: int
    input_tokens: int
    cached_input_tokens: int
    output_tokens: int
    fresh_input_tokens: int

@dataclass
class SessionUsage:
    """Aggregate token usage for a session."""
    input_tokens: int
    cached_input_tokens: int
    output_tokens: int
    fresh_input_tokens: int
    calls: int
    turns: list[TurnUsage]
    usage_source: str  # e.g., "projection-state", "journal"

class HarnessAdapter(Protocol):
    """Interface for AI harness adapters.
    
    To add a new harness:
    1. Implement this interface in a new file under harnesses/
    2. Register via entry point in pyproject.toml
    """
    
    name: str  # e.g., "vibe", "openai"
    home_env_var: str  # e.g., "VIBE_HOME", "OPENAI_HOME"
    home_dir_name: str  # e.g., ".vibe", ".openai"
    
    def default_home(self) -> Path:
        """Return the default home directory for this harness."""
        ...
    
    def resolve_session_dir(self, session_arg: str | None, home: Path) -> Path:
        """Resolve session directory from argument or default discovery.
        
        Args:
            session_arg: User-provided session ID or path, or None for auto-discovery
            home: Harness home directory
            
        Returns:
            Absolute path to the session directory
            
        Raises:
            FileNotFoundError: If session not found
        """
        ...
    
    def read_meta(self, session_dir: Path) -> SessionMeta:
        """Read session metadata from harness logs.
        
        Args:
            session_dir: Session directory path
            
        Returns:
            SessionMeta with title, model_name, and alias
        """
        ...
    
    def collect_usage(self, session_dir: Path) -> SessionUsage:
        """Collect token usage data from harness logs.
        
        Args:
            session_dir: Session directory path
            
        Returns:
            SessionUsage with aggregate and per-turn token counts
        """
        ...

Registration:

To register a new harness adapter, add to pyproject.toml:

[project.entry-points."agent_cost_estimator.harnesses"]
my-harness = "my_package.harnesses:MyHarnessAdapter"

3.2. Pricing Table Format

File: agent_cost_estimator/pricing.json (bundled default)

Purpose: Define pricing models for different AI providers and models.

Format: Provider-based structure (version 2.0+)

{
  "providers": {
    "<provider_name>": {
      "currency": "<currency_code>",
      "models": {
        "<model_name>": {
          "input_price": <number>,
          "cached_input_price": <number>,
          "output_price": <number>,
          "note": "<optional description>",
          "currency": "<optional override>"
        },
        "default": {
          "input_price": <number>,
          "cached_input_price": <number>,
          "output_price": <number>
        }
      }
    }
  }
}

Semantics:

  • providers: REQUIRED. Dictionary of provider configurations
  • provider_name: Provider identifier (e.g., "mistral", "openai", "anthropic")
  • currency: Provider-level currency (default: "USD" if not specified)
  • models: REQUIRED. Dictionary of model pricing for this provider
  • model_name: Model identifier (e.g., "mistral-medium", "gpt-4")
  • input_price: USD per million input tokens (prompt tokens)
  • cached_input_price: USD per million cached input tokens (context cache)
  • output_price: USD per million output tokens (completion tokens)
  • Model-level currency: Optional override for specific model

Lookup Order:

  1. Exact match on model_name (passed as first argument to get_model_prices)
  2. Exact match on alias (passed as second argument)
  3. Provider's default model
  4. First provider's default model (fallback)

Example:

{
  "providers": {
    "default": {
      "currency": "USD",
      "models": {
        "default": {
          "input_price": 1.5,
          "cached_input_price": 0.15,
          "output_price": 7.5,
          "note": "Global fallback rates"
        }
      }
    },
    "mistral": {
      "currency": "USD",
      "models": {
        "default": {
          "input_price": 1.5,
          "cached_input_price": 0.15,
          "output_price": 7.5
        },
        "mistral-small": {
          "input_price": 1.0,
          "cached_input_price": 0.1,
          "output_price": 3.0
        },
        "mistral-medium": {
          "input_price": 1.5,
          "cached_input_price": 0.15,
          "output_price": 7.5
        },
        "mistral-large": {
          "input_price": 2.5,
          "cached_input_price": 0.25,
          "output_price": 12.5
        }
      }
    }
  }
}

Validation Rules:

  • All price values MUST be non-negative numbers
  • Each provider MUST have a models dictionary
  • Each provider's models MUST contain at least one model
  • The default provider MUST exist (for fallback)

3.3. Hook Integration

Purpose: Enable real-time cost tracking in AI harness sessions.

Hook Types:

  • post_agent: Fires after assistant turns (RECOMMENDED for cost tracking)

Hook Contract:

  1. Hook receives JSON payload on stdin:

    {
      "session_id": "<uuid>",
      "hook_event_name": "post_agent"
    }
    
  2. Hook MUST output JSON with system_message field:

    {
      "system_message": "-- <formatted cost output>"
    }
    
  3. Hook MUST be executable and return exit code 0 on success

Hook Installation:

# Automatic (recommended)
agent-cost-estimator --integrate mistral

# Manual
mkdir -p ~/.vibe/hooks
cp agent_cost_estimator/hooks/post_turn.py ~/.vibe/hooks/
chmod +x ~/.vibe/hooks/post_turn.py

4. Package and Runtime

4.1. Dependencies

Runtime Dependencies:

  • Python >= 3.9
  • Standard library only (no external dependencies for core functionality)

Development Dependencies:

  • pytest >= 7.0.0
  • pytest-cov >= 4.0.0

Packaging:

# pyproject.toml
[build-system]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"

4.2. Entry Points

CLI:

[project.scripts]
agent-cost-estimator = "agent_cost_estimator.cli:main"

Harness Adapters:

[project.entry-points."agent_cost_estimator.harnesses"]
vibe = "agent_cost_estimator.harnesses.vibe:VibeHarness"
hermes = "agent_cost_estimator.harnesses.hermes:HermesHarness"

4.3. Installation

From Source:

cd agent-cost-estimator
pip install .

Verify:

agent-cost-estimator --list-harnesses  # Should output: hermes, vibe
agent-cost-estimator --skill          # Should output SKILL.md

5. Feature Requirements

5.1. CLI Interface

Status: Implemented

Command: agent-cost-estimator [OPTIONS]

Options:

Option Type Required Default Description
--harness str No vibe Harness to read (vibe, hermes)
--session-id str No auto Session ID or directory path
--home str No harness default Harness data root directory
--pricing str No bundled Path to pricing.json file
--model str No from session Force pricing lookup key
--input-price float No from pricing Override input price
--cached-input-price float No from pricing Override cached input price
--output-price float No from pricing Override output price
-d, --details flag No False Full report table
-s, --session flag No False Session cost only (bare number)
-t, --turn flag No False Latest turn cost only (bare number)
--turns flag No False Per-turn breakdown (with -d)
--json flag No False JSON output
--list-harnesses flag No False List available harnesses
--skill flag No False Print SKILL.md
--integrate HARNESS str No None Install hooks for harness
--update-pricing flag No False Persist pricing to file
--dry-run flag No False Dry run (with --update-pricing)

Exit Codes:

  • 0: Success
  • 1: Error (invalid arguments, session not found, etc.)

5.2. Vibe Harness Adapter

Status: Implemented

Purpose: Read session data from Mistral Vibe CLI logs.

File: agent_cost_estimator/harnesses/vibe.py

Implementation:

  • default_home(): Returns ~/.vibe (or $VIBE_HOME)
  • resolve_session_dir(): Discovers latest session or uses provided ID
  • read_meta(): Reads meta.json for title, model_name, active_model
  • collect_usage(): Reads from projection-state.json (preferred) or journal files

Session Discovery:

  1. If --session-id provided: Use directly
  2. Otherwise: Find newest directory in ~/.vibe/logs/session/unified/

Usage Source Priority:

  1. projection-state.json (survives journal rotation, has cumulative totals)
  2. Journal files (current window on disk)

Cached Token Handling:

  • Cached tokens are clamped to input tokens (matching Vibe's behavior)
  • Fresh tokens = input_tokens - cached_input_tokens

5.2.1. Hermes Harness Adapter

Status: Implemented

Purpose: Read session data from Hermes Agent's SQLite store.

File: agent_cost_estimator/harnesses/hermes.py

Implementation:

  • default_home(): Returns $HERMES_HOME or ~/.hermes
  • resolve_session_dir(): Accepts a session id (e.g. local-<hash>), a state.db path, or a profile home directory; None picks the session with the newest last_activity_at. Returns a HermesSession (db_path + session_id; .name exposes the session id for the CLI's display).
  • read_meta(): Reads sessions.title and sessions.model
  • collect_usage(): session totals from session_model_usage (authoritative cumulative counter); per-turn breakdown from the plugin's usage ledger

Session Discovery:

  1. If --session-id is a session id: validate against sessions
  2. If it is a file: treat as state.db
  3. If it is a directory: <dir>/state.db
  4. Otherwise: newest last_activity_at in sessions

Usage Source Priority:

  1. state.db session_model_usage (cumulative, written by Hermes after every API call - survives ledger loss)
  2. The plugin's JSONL ledger (<home>/plugin-data/agent-cost-estimator/usage/<session_id>.jsonl, per-call records grouped by turn_id) - only present once the plugin is installed

Token Bucket Mapping (Hermes -> tool contract):

Hermes's CanonicalUsage buckets are disjoint; the tool's SessionUsage is Vibe-style (input includes cached):

Hermes column SessionUsage
input_tokens fresh input (excl. cache)
cache_read_tokens cached input
cache_write_tokens fresh input (billed)
  • input_tokens = input + cache_read + cache_write (= prompt_tokens)
  • cached_input_tokens = cache_read
  • fresh_input_tokens = input + cache_write
  • output_tokens = output (includes reasoning tokens)

Ledger Format (one JSON object per line):

{"ts": 1792847431.9, "session_id": "local-...", "turn_id": "t1",
 "model": "zai-glm-5-3", "provider": "zai",
 "api_request_id": "r1", "aux_task": null,
 "label": "user message head", "calls": 1,
 "input_tokens": 453000, "cached_input_tokens": 400000,
 "output_tokens": 9000}

Records are deduplicated by api_request_id when present; cached_input_tokens is clamped to input_tokens per record and per turn.

5.3. Pricing Configuration

Status: Implemented

Features:

  • Provider-based pricing structure
  • Model-level currency overrides
  • Default fallback (provider default → global default)
  • Warning when using default pricing ( **model price not available**)

Lookup Priority:

  1. Passed model name (now uses active_model first)
  2. Passed alias (model_name)
  3. Provider's default model
  4. First provider's default model

CLI Overrides:

agent-cost-estimator --input-price 2.0 --output-price 10.0 -s

Persist Overrides:

agent-cost-estimator --model my-model --input-price 2.0 --update-pricing

5.4. Cost Calculation

Status: Implemented

File: agent_cost_estimator/core.py

Formula:

input_cost_usd = (fresh_input_tokens * input_price + cached_input_tokens * cached_input_price) / 1_000_000
output_cost_usd = (output_tokens * output_price) / 1_000_000
total_cost_usd = input_cost_usd + output_cost_usd

Function:

def compute_cost(tokens: dict[str, int], rates: dict[str, float]) -> dict[str, float]:
    """Compute cost from token counts and pricing rates.
    
    Args:
        tokens: {"fresh": int, "cached": int, "output": int}
        rates: {"input_price": float, "cached_input_price": float, "output_price": float}
        
    Returns:
        {"input_usd": float, "cached_usd": float, "output_usd": float, "total_usd": float}
    """

5.5. Hook System

Status: Implemented

Purpose: Display costs after each assistant turn in Vibe CLI.

Hook Script: agent_cost_estimator/hooks/post_turn.py

Behavior:

  1. Receives session_id from hook payload
  2. Resolves session directory
  3. Calls agent-cost-estimator --session-id <path> --json
  4. Formats output as provider:model | session:$X | turn:$Y
  5. Returns JSON with system_message field

Output Examples:

-- mistral:mistral-medium | session:$3.40 | turn:$0.92
-- mistral:mistral-medium-3.5 | session:$0.00 | turn:$0.00

Error Handling:

  • Session not found: -- [cost: session dir not found]
  • Timeout: -- [cost: timeout]
  • Other errors: -- [cost: error - <message>]

5.6. Integration Command

Status: Implemented

Command: agent-cost-estimator --integrate <harness>

Supported Harnesses:

  • mistral, mistral-vibe-cli, vibe, cli (Vibe CLI)
  • mistral-vibe-code, vscodium, code (VSCodium extension)
  • hermes, hermes-agent (Hermes Agent plugin)

Modular Design:

The --integrate command accepts a harness name argument; each harness dispatches to its own installer (Vibe family: inline hook install; Hermes: agent_cost_estimator.integration.hermes.install).

Actions (Vibe family):

  1. Creates harness-specific hook directory
  2. Installs hook script
  3. Creates hooks.toml configuration
  4. Updates bash allowlist in config.toml

Actions (Hermes):

  1. Validates the Hermes home (state.db or config.yaml present)
  2. Copies the plugin (plugin.yaml, __init__.py) to <home>/plugins/agent-cost-estimator/
  3. Ships pricing.json next to the plugin (preserves an already-tuned installed table on reinstall)
  4. Runs hermes plugins enable agent-cost-estimator (prints the manual step when the hermes CLI is not on PATH)

Examples:

agent-cost-estimator --integrate mistral
agent-cost-estimator --integrate hermes
agent-cost-estimator --rm-integrate hermes

The Hermes installer is also runnable standalone on any machine:

python -m agent_cost_estimator.integration.hermes.install
    [--hermes-home DIR] [--uninstall] [--dry-run]

Future Extension:

To add a new harness openai:

  1. Add to supported_harnesses list in integrate_hooks()
  2. Add harness-specific directory logic if needed
  3. Update documentation

5.7. Output Formatting

Status: Implemented

Default Format:

provider:model | session:$X.XX | turn:$Y.YY

Details Format (-d):

harness:  vibe
session:  <session-id>
model:    provider:model
<warning if default pricing>
model calls:        N
usage source:       projection-state
input tokens:       N,NNN
output tokens:      N,NNN
pricing:           provider:model from bundled pricing.json
currency:          USD
prices (USD/M):  input X cached Y output Z
input cost:         $X.XXXX
cached input cost:  $X.XXXX
output cost:        $X.XXXX
TOTAL:              $X.XXXX

JSON Format (--json):

{
  "harness": "vibe",
  "session_id": "<uuid>",
  "title": null,
  "model": "mistral-vibe-cli-latest",
  "provider": "mistral",
  "provider_model": "mistral:mistral-medium-3.5",
  "model_price_unavailable": false,
  "calls": N,
  "input_tokens": N,
  "cached_input_tokens": N,
  "fresh_input_tokens": N,
  "output_tokens": N,
  "pricing": {
    "file": "bundled",
    "model_key": "mistral:mistral-medium-3.5",
    "currency": "USD",
    "rates_usd_per_million": {
      "input_price": X,
      "cached_input_price": Y,
      "output_price": Z
    }
  },
  "usage_source": "projection-state",
  "cost_usd": {
    "input_usd": X,
    "cached_usd": Y,
    "output_usd": Z,
    "total_usd": T
  },
  "last_turn": {...},
  "turns": [...]
}

Warning Indicator:

When a model falls back to default pricing, all output modes include:

 **model price not available**

5.8. GUI Interface

Status: Not Implemented

Purpose: Provide a graphical user interface for visualizing session costs and interacting with the cost estimator in a user-friendly way.

Platform Support:

  • Windows: Native support
  • Linux: Native support
  • Uses cross-platform Python GUI framework (e.g., Tkinter, PyQt, or similar)

Default Harness Configuration:

  • Settings File Location:

    • Windows: %APPDATA%\agent-cost-estimator\settings.json
    • Linux: $XDG_CONFIG_HOME/agent-cost-estimator/settings.json (defaults to ~/.config/agent-cost-estimator/settings.json)
  • settings.json Structure:

    {
      "default_harness": "vibe"
    }
    
  • Default Harness: On first launch, if settings.json does not exist, default to vibe harness

  • Harness Selection: User can change the default harness in settings, which persists across application restarts

GUI Components:

  • Main Window:

    • Title: "Agent Cost Estimator"
    • Menu bar with File, View, Help menus
    • Status bar showing current harness and version
  • Session Tree View:

    • Display: Tree structure showing all discovered sessions
    • Ordering: Chronological order, latest session on top
    • Root Nodes: Each harness (vibe, hermes) as a root node
    • Child Nodes: Individual sessions under each harness
    • Session Discovery: Auto-discover sessions from harness home directories
  • Columns Display:

    • Column 1 - Model:
      • Format: provider:model (e.g., mistral:mistral-medium)
      • Source: Session metadata from meta.json
    • Column 2 - Session Price:
      • Format: $X.XX (USD currency)
      • Value: Cumulative cost for the entire session
      • Calculation: Sum of all token costs (input + output + cached) * pricing
    • Column 3 - Current Turn Price:
      • Format: $Y.YY (USD currency)
      • Value: Cost of the most recent turn in the session
      • Calculation: Token costs for the latest turn * pricing
  • Tree View Behavior:

    • Single click on session: Selects the session
    • Double click on session: Opens detailed view for that session
    • Right click on session: Context menu with actions (refresh, copy, etc.)
    • Auto-refresh: Optional periodic refresh of session data

Data Source:

  • Uses existing harness adapters to read session data
  • Reuses pricing configuration from existing pricing system
  • Combines data from both bundled and custom pricing files

Future Enhancements:

  • Add per-turn breakdown view
  • Add filtering and sorting capabilities
  • Add export functionality (CSV, JSON)
  • Add historical cost charts
  • Add real-time cost tracking with live updates

6. Configuration Files

6.1. pricing.json

Location: agent_cost_estimator/pricing.json (bundled)

Purpose: Default pricing configuration for all supported providers/models.

Structure: See 3.2. Pricing Table Format

Bundled Version: Contains Mistral, Zai, and default provider configurations.

Customization:

Users can:

  • Provide custom pricing file via --pricing flag
  • Update pricing for a model via --update-pricing
  • Override prices via CLI flags (--input-price, etc.)

6.2. User Configurable Pricing Files

Status: Not Implemented

Location:

  • Windows: %APPDATA%\agent-cost-estimator\price.json and %APPDATA%\agent-cost-estimator\custom-price.json
  • Linux/macOS: $XDG_CONFIG_HOME/agent-cost-estimator/price.json and $XDG_CONFIG_HOME/agent-cost-estimator/custom-price.json
    • If $XDG_CONFIG_HOME is not set, defaults to ~/.config/agent-cost-estimator/

Purpose: Allow users to maintain custom pricing configurations that persist across application updates and are easily accessible.

Files:

  1. price.json:

    • Initialized from the repository's bundled pricing.json on first run
    • Contains the default pricing configuration for all supported providers/models
    • Serves as the base pricing data
  2. custom-price.json:

    • Created as a blank template on first run
    • Intended for user editing to add custom models, providers, or override existing rates
    • Empty file if user has not added any customizations

Behavior:

  • On application startup, both files are read if they exist
  • The application merges pricing data from both files
  • Priority Rule: In case of duplicate entries (same provider/model), values from custom-price.json take precedence over price.json
  • This allows users to override specific pricing without losing the ability to receive updates to the base pricing data

Directory Creation:

  • The application MUST create the user configuration directory if it does not exist
  • Both files MUST be created on first run (price.json from bundled version, custom-price.json as empty template)
  • File permissions SHOULD allow user read/write access

Format: Both files follow the same structure as defined in 3.2. Pricing Table Format

Example custom-price.json:

{
  "providers": {
    "custom-provider": {
      "currency": "USD",
      "models": {
        "my-custom-model": {
          "input_price": 2.0,
          "cached_input_price": 0.2,
          "output_price": 10.0,
          "note": "My custom pricing"
        }
      }
    }
  }
}

7. Harness-Specific Configurations

This section contains configuration details that are specific to individual AI harnesses. Each harness may have its own hooks, configuration files, and integration requirements.

Modular Structure:

To support multiple Vibe-based harnesses (CLI, VSCodium extension, etc.), hooks are organized in a modular directory structure under agent_cost_estimator/hooks/:

hooks/
├── mistral-vibe-cli/
│   └── post_turn.py          # Hook for Mistral Vibe CLI
├── mistral-vibe-code/
│   └── post_turn.py          # Hook for Mistral Vibe VSCodium extension
└── README.md                # Overview of hook structure

This modular approach allows:

  • Different implementations for different Vibe-based harnesses
  • Harness-specific model detection logic
  • Clean separation of concerns

7.1. Vibe CLI (Mistral) Configuration

Harness: Mistral Vibe CLI Adapter: agent_cost_estimator.harnesses.vibe:VibeHarness Hook Location: agent_cost_estimator/hooks/mistral-vibe-cli/post_turn.py

7.1.1. Hook Implementation

Location: agent_cost_estimator/hooks/mistral-vibe-cli/post_turn.py

Purpose: Vibe CLI hook for real-time cost tracking.

Features:

  • Session directory resolution (session_id → path)
  • Error handling with graceful degradation
  • JSON output format for Vibe CLI compatibility
  • Logging for debugging
  • Reads model from meta.json config field (Vibe CLI standard)

Installation:

Automatic via --integrate mistral-vibe-cli command, or manual copy to ~/.vibe/hooks/

7.1.2. hooks.toml Configuration

Location: Created at ~/.vibe/hooks.toml during integration

Purpose: Vibe CLI hook configuration

Content:

[[hooks]]
name = "cost-tracker"
type = "post_agent"
command = "python ~/.vibe/hooks/mistral-vibe-cli/post_turn.py"
timeout = 30
strict = false
description = "Display cost after each assistant turn"

7.2. Vibe VSCodium Extension (Mistral) Configuration

Harness: Mistral Vibe VSCodium Extension (mistralai.mistral-vibe-code) Adapter: agent_cost_estimator.harnesses.vibe:VibeHarness (shared with CLI) Hook Location: agent_cost_estimator/hooks/mistral-vibe-code/post_turn.py

7.2.1. Hook Implementation

Location: agent_cost_estimator/hooks/mistral-vibe-code/post_turn.py

Purpose: VSCodium extension hook for real-time cost tracking.

Features:

  • Session directory resolution (session_id → path)
  • Error handling with graceful degradation
  • JSON output format for Vibe CLI compatibility
  • Logging for debugging
  • Enhanced model detection for VSCodium extension:
    • Checks VSCodium-specific configuration files
    • Falls back to Vibe CLI standard locations
    • Maintains compatibility with both CLI and VSCodium sessions

Model Detection Strategy:

The VSCodium extension (mistral-vibe-code) may not write model information to the standard meta.json location. The hook uses an enhanced detection strategy:

  1. Primary: Read from meta.json config.model_name (Vibe CLI standard)
  2. VSCodium-specific: Check VSCodium extension state/storage for active model
  3. Fallback: Use default pricing with warning indicator
  4. Override: Allow environment variable VIBE_MODELOVERRIDE for manual setting

Installation:

Automatic via --integrate mistral-vibe-code command, or manual copy to ~/.vibe/hooks/

7.2.2. hooks.toml Configuration

Location: Created at ~/.vibe/hooks.toml during integration

Purpose: VSCodium extension hook configuration

Content:

[[hooks]]
name = "cost-tracker-vscodium"
type = "post_agent"
command = "python ~/.vibe/hooks/mistral-vibe-code/post_turn.py"
timeout = 30
strict = false
description = "Display cost after each assistant turn (VSCodium)"

7.2.3. VSCodium-Specific Considerations

The VSCodium extension (mistralai.mistral-vibe-code) uses the same underlying Vibe engine but may have differences in:

  • Session Storage: Same location (~/.vibe/logs/session/unified/)
  • Model Configuration: May store model settings differently than CLI
  • Hook Triggering: Depends on extension version and configuration

The hook is designed to be resilient to these differences.

7.3. Hermes Agent (Nous Research) Configuration

Harness: Hermes Agent Adapter: agent_cost_estimator.harnesses.hermes:HermesHarness Plugin Location: agent_cost_estimator/integration/hermes/plugin/ Installer: agent_cost_estimator/integration/hermes/install.py

Hermes integration is a plugin, not a hook script: Hermes loads plugins from <home>/plugins/<name>/ (manifest plugin.yaml + __init__.py with register(ctx)), gated by hermes plugins enable <name>.

7.3.1. Plugin Implementation

File: agent_cost_estimator/integration/hermes/plugin/__init__.py

Manifest:

name: agent-cost-estimator
version: <package version>
provides_hooks:
  - pre_llm_call
  - post_api_request
  - post_auxiliary_call
  - post_llm_call

Registered hooks and payloads:

Hook Payload fields used Action
pre_llm_call turn_id, user_message capture the turn label (message head, 60 chars)
post_api_request session_id, turn_id, model, provider, api_request_id, usage (CanonicalUsage dict) queue one ledger record
post_auxiliary_call same + aux_task queue one ledger record (aux spend folds into the parent turn)
post_llm_call session_id, turn_id, model, platform print the cost line

Data flow:

  1. Each API call appends a JSON line (via a bounded in-process queue and a single daemon drainer thread - the hook callback itself never does I/O) to <home>/plugin-data/agent-cost-estimator/usage/<session_id>.jsonl
  2. The per-turn cost also accumulates in memory, so the printed turn figure is exact even before the ledger flushes
  3. At turn end (post_llm_call - which Hermes fires after flushing its token queue to state.db), the plugin reads the session's cumulative totals from state.db (session_model_usage, read-only) and prints:
-- provider:model | session:$X.XX | turn:$Y.YY
  1. Session totals come from state.db (authoritative, like Vibe's projection-state); the ledger provides the per-turn breakdown across restarts (the tool's --harness hermes adapter reads both).

Print gating:

  • Prints only for interactive terminal surfaces: the gateway's local adapter (platform == "local", how current Hermes builds run CLI chat) and the non-gateway fallback ("cli")
  • Never prints for subagents (platform == "subagent"), cron runs, or one-shot runs (HERMES_SINGLE_QUERY_SESSION=1, i.e. hermes chat -q) - their stdout must stay machine-readable
  • Subagent and auxiliary calls still RECORD to the ledger (their spend is real and shows in session totals)

Pricing resolution (per print):

  1. pricing.json next to the installed plugin (stdlib-only reader; tune rates by editing this file)
  2. the agent_cost_estimator package's bundled pricing.json (when importable in Hermes's interpreter)
  3. hard-coded defaults (1.5 / 0.15 / 7.5 USD per M) with the model price not available warning

Failure policy: every callback swallows its own errors (debug-logged); a cost line must never break a turn. Ledger writes are best-effort (bounded queue, drop on overflow - session totals are unaffected).

Known limitation: Hermes skips a timeout-bounded hook callback that is still running for the same turn_id, so under MoA advisor fan-out a concurrently-fired duplicate call may not be ledgered. Session totals are unaffected; only that call's contribution to the per-turn breakdown can be lost.

7.3.2. Installation

# automatic (from the package)
agent-cost-estimator --integrate hermes

# standalone (any machine, source checkout, or installed package)
python -m agent_cost_estimator.integration.hermes.install
python path/to/install.py --hermes-home /custom/home --dry-run

The installer:

  1. Validates the target home (state.db or config.yaml)
  2. Copies plugin.yaml + __init__.py to <home>/plugins/agent-cost-estimator/
  3. Copies the bundled pricing.json next to the plugin, preserving an already-installed (user-tuned) table on reinstall
  4. Runs hermes plugins enable agent-cost-estimator; on failure or missing CLI prints the manual command

Removal:

agent-cost-estimator --rm-integrate hermes
python -m agent_cost_estimator.integration.hermes.install --uninstall

7.3.3. Verification

# after install + a Hermes restart + one turn:
agent-cost-estimator --harness hermes        # session + latest turn
agent-cost-estimator --harness hermes -d    # full table
hermes plugins list --plain --no-bundled    # shows the enabled plugin

8. Error Handling

Principle: Fail gracefully, never break the user's session.

CLI Errors:

  • Invalid arguments: Exit 1 with error message to stderr
  • Session not found: Exit 1 with error message
  • Pricing lookup failure: Exit 1 with error message

Hook Errors:

  • Session directory not found: Return error message in JSON format
  • agent-cost-estimator fails: Return error message with stdout/stderr
  • Timeout: Return timeout message
  • Unexpected errors: Return error message with exception

Pricing Errors:

  • Model not in pricing.json: Falls back to default, displays warning
  • Pricing file invalid: Exit 1 with validation error
  • Pricing file missing: Exit 1 with FileNotFoundError

9. Testing

Framework: pytest

Coverage: All core functionality tested

Test Files:

  • tests/test_agent_cost_estimator.py (52 tests)

Test Categories:

  • Core cost calculation
  • Pricing table loading/lookup/validation
  • Vibe harness adapter (journal parsing, projection, dedupe, session discovery)
  • Hermes harness adapter (state.db totals, ledger fallback, dedupe, clamp, session discovery, CLI --json end to end)
  • Hermes plugin (hook registration, CanonicalUsage bucket folding, record + print, subagent/single-query gating, pricing sources)
  • Hermes installer (dry-run, home validation, plugin + pricing copy, uninstall)
  • Harness registry
  • CLI (pricing flags, -t, -s, --json, --update-pricing, --integrate, error exits)

Running Tests:

cd agent-cost-estimator
python -m pytest tests/test_agent_cost_estimator.py -v

10. Deployment

Packaging:

# Build
python -m build

# Install
pip install dist/agent_cost_estimator-*.tar.gz

Distribution:

  • PyPI (future)
  • Direct installation from Git repository

Versioning:

  • Follows semantic versioning (MAJOR.MINOR.PATCH)
  • Bump version in pyproject.toml for each release
  • Maintain CHANGELOG (future enhancement)

Appendix A: Glossary

Term Definition
Session A single interaction with an AI agent, with its own UUID and log directory
Turn A single assistant response within a session
Harness An AI agent framework (e.g., Vibe CLI, OpenAI SDK)
Provider An AI model provider (e.g., Mistral, OpenAI, Anthropic)
Model A specific AI model (e.g., mistral-medium, gpt-4)
Token A unit of text that AI models process (input/output)
Cached Tokens Tokens that were already in the model's context cache
Fresh Tokens Tokens that are new in the current request (input_tokens - cached_input_tokens)
Hook A callback that runs at specific events in the harness lifecycle

Appendix B: Example Usage

Basic Usage:

# Quick summary of current session
agent-cost-estimator

# Full details
agent-cost-estimator -d

# Session cost only
agent-cost-estimator -s

# Turn cost only
agent-cost-estimator -t

# JSON output
agent-cost-estimator --json

# Specific session
agent-cost-estimator --session-id ~/.vibe/logs/session/unified/<uuid>

# Custom pricing file
agent-cost-estimator --pricing /path/to/pricing.json

# Override pricing for this run
agent-cost-estimator --input-price 2.0 --output-price 10.0

# Update pricing file with new rates
agent-cost-estimator --model my-model --input-price 2.0 --update-pricing

Integration:

# Install hooks for Vibe CLI
agent-cost-estimator --integrate mistral

# After installation, start a Vibe session
vibe "Your prompt"
# Output after each turn: -- mistral:mistral-medium | session:$X.XX | turn:$Y.YY

Configuration:

# List available harnesses
agent-cost-estimator --list-harnesses

# Print usage instructions for LLMs
agent-cost-estimator --skill

Appendix C: File Index

Core Modules:

File Purpose
agent_cost_estimator/__init__.py Public API exports
agent_cost_estimator/__main__.py Entry point
agent_cost_estimator/cli.py CLI interface
agent_cost_estimator/core.py Cost calculation
agent_cost_estimator/pricing.py Pricing data handling

Harness Adapters:

File Harness Status
harnesses/__init__.py Registry Implemented
harnesses/base.py Base interface Implemented
harnesses/vibe.py Mistral Vibe CLI Implemented
harnesses/hermes.py Hermes Agent Implemented

Hooks:

File Purpose Status
hooks/post_turn.py Vibe post-agent hook Implemented
hooks/mistral-vibe-cli/post_turn.py Vibe CLI hook Implemented
hooks/mistral-vibe-code/post_turn.py VSCodium hook Implemented

Integrations:

File Purpose Status
integration/hermes/plugin/plugin.yaml Hermes plugin manifest Implemented
integration/hermes/plugin/__init__.py Hermes plugin (hooks + cost line) Implemented
integration/hermes/install.py Standalone Hermes installer Implemented

Configuration:

File Purpose
pricing.json Default pricing
pyproject.toml Package metadata

Documentation:

File Purpose
SPEC.md This specification
README.md User documentation
skill/SKILL.md LLM usage instructions

Tests:

File Purpose
tests/test_agent_cost_estimator.py Test suite