Files
agent-cost-estimator/TOKEN_DISPLAY_SPEC.md
T
nvaldez f6ce192ece Update token display spec: add monthly usage summary row
- Add section 3.0 for Monthly Usage Summary Row
- Display **This Month Usage** as first row under each harness
- Calculate aggregated token usage for current calendar month
- Include caching strategy (5-minute TTL)
- Add implementation examples for Tkinter and PyQt5

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
2026-11-10 17:43:55 +01:00

7.8 KiB

Token Display Specification

Version: 1.1.0
Date: 2026-11-10
Status: Draft
Author: LLM Assistant

Overview

This specification adds token usage display to both GUI and CLI interfaces of agent-cost-estimator.

Summary of Changes

  1. GUI: Add 3 columns (Input/Output/Cached tokens) with automatic suffixes (k, M, B, T)
  2. GUI: Add monthly usage summary row as first row under each harness
  3. CLI: Add token usage line with format -- I:X O:Y C:Z

1. Formatting Function

Create a utility function to format token counts with automatic suffixes:

# File: agent_cost_estimator/utils/formatting.py

def format_token_count(count: int) -> str:
    """Format token count with automatic suffix for large numbers.
    
    Args:
        count: Non-negative integer token count
        
    Returns:
        String with suffix: 0-999 (no suffix), 1k+, 1M+, 1B+, 1T+
        Always uses 2 decimal places for suffixed values.
        
    Examples:
        0 -> "0"
        500 -> "500"  
        1000 -> "1.00k"
        1500 -> "1.50k"
        1000000 -> "1.00M"
        1000000000 -> "1.00B"
        1000000000000 -> "1.00T"
    """
    if count < 1000:
        return str(count)
    elif count < 1_000_000:
        return f"{count / 1000:.2f}k"
    elif count < 1_000_000_000:
        return f"{count / 1_000_000:.2f}M"
    elif count < 1_000_000_000_000:
        return f"{count / 1_000_000_000:.2f}B"
    else:
        return f"{count / 1_000_000_000_000:.2f}T"

2. CLI Changes

Default Output

Current:

-- provider:model | session:$X.XX | turn:$Y.YY

New:

-- provider:model | session:$X.XX | turn:$Y.YY
-- I:<input> O:<output> C:<cached>

Example:

-- mistral:mistral-medium | session:$3.40 | turn:$0.92
-- I:1.01k O:2.0k C:10M

Implementation in cli.py:

from .utils.formatting import format_token_count

# After printing cost line
input_formatted = format_token_count(usage.input_tokens)
output_formatted = format_token_count(usage.output_tokens)  
cached_formatted = format_token_count(usage.cached_input_tokens)
print(f"-- I:{input_formatted} O:{output_formatted} C:{cached_formatted}")

Note: Token line is NOT displayed with -s or -t flags (bare number output).

3. GUI Changes

3.0. Monthly Usage Summary Row

New Feature: Display a summary row as the first row under each harness showing aggregated token usage for the current calendar month.

Display:

┌─ Harness (e.g., vibe)
├─ **This Month Usage** │ - │ 100k │ 50k │ 20k │ $12.34 │ - │ - │ -
├─ Session 1 │ model │ 10k │ 5k │ 2k │ $1.23 │ $0.45 │ ... │ ...
├─ Session 2 │ model │ 90k │ 45k │ 18k │ $11.11 │ $3.21 │ ... │ ...
└─ ...

Requirements:

  • Position: First child row under each harness
  • Visual: Bold font, gray background, special tag "monthly_summary"
  • Label: "This Month Usage"
  • Data: Sum of all sessions from current month
  • Caching: Cache for 5 minutes, invalidate on month change or new session

Implementation:

  1. Add calculate_monthly_usage(harness_name, year, month) function
  2. Call it during session refresh
  3. Insert as first row under each harness node
  4. Style with bold font and different background

Data Structure:

@dataclass
class MonthlyUsage:
    input_tokens: int
    output_tokens: int
    cached_input_tokens: int
    session_count: int
    total_cost_usd: float

Display Values:

  • Input Tokens: format_token_count(monthly_usage.input_tokens)
  • Output Tokens: format_token_count(monthly_usage.output_tokens)
  • Cached Tokens: format_token_count(monthly_usage.cached_input_tokens)
  • Session Price: ${monthly_usage.total_cost_usd:.2f}
  • Turn Price, DateTime, Hash: Empty

Column Structure

Add 3 new columns after the Model column:

# Column Header Width Alignment Data Source
0 Session Name Session Name 200px Left session.title
1 Model Model 150px Left session.model
2 Input Tokens Input 80px Right format_token_count(session.input_tokens)
3 Output Tokens Output 80px Right format_token_count(session.output_tokens)
4 Cached Tokens Cached 80px Right format_token_count(session.cached_input_tokens)
5 Session Price Session Price 100px Right ${session.session_price:.2f}
6 Turn Price Turn Price 100px Right ${session.current_turn_price:.2f}
7 DateTime DateTime 150px Left datetime_str
8 Hash Hash 100px Left session_hash

Implementation Steps

  1. Update SessionInfo dataclass in gui.py:
@dataclass
class SessionInfo:
    session_id: str
    title: str
    model: str
    session_price: float
    current_turn_price: float
    last_modified: datetime
    # Add these fields:
    input_tokens: int = 0
    output_tokens: int = 0
    cached_input_tokens: int = 0
  1. Populate token data when creating SessionInfo:
session_info = SessionInfo(
    session_id=str(session_dir.name),
    title=meta.title or "Untitled",
    model=key,
    session_price=costs['total_usd'],
    current_turn_price=turn_costs['total_usd'],
    last_modified=last_modified,
    input_tokens=usage.input_tokens,
    output_tokens=usage.output_tokens,
    cached_input_tokens=usage.cached_input_tokens
)
  1. Update TreeView columns in Tkinter version:
self.tree = ttk.Treeview(main_frame, 
    columns=("model", "input_tokens", "output_tokens", "cached_tokens",
             "session_price", "turn_price", "datetime", "hash"))

# Add headings
self.tree.heading("input_tokens", text="Input")
self.tree.heading("output_tokens", text="Output")
self.tree.heading("cached_tokens", text="Cached")

# Add column configs (right-aligned)
self.tree.column("input_tokens", width=80, anchor=tk.E, stretch=False)
self.tree.column("output_tokens", width=80, anchor=tk.E, stretch=False)
self.tree.column("cached_tokens", width=80, anchor=tk.E, stretch=False)
  1. Update tree insertion to include token data:
item_id = self.tree.insert(
    harness_node,
    self.tk.END,
    text=session.title,
    values=(
        session.model,
        format_token_count(session.input_tokens),
        format_token_count(session.output_tokens),
        format_token_count(session.cached_input_tokens),
        f"${session.session_price:.2f}",
        f"${session.current_turn_price:.2f}",
        datetime_str,
        session_hash
    ),
    tags=("session",)
)
  1. Repeat for PyQt5 implementation in the same file.

4. Test Cases

# tests/test_formatting.py
from agent_cost_estimator.utils.formatting import format_token_count

def test_basic():
    assert format_token_count(0) == "0"
    assert format_token_count(500) == "500"
    assert format_token_count(999) == "999"

def test_thousands():
    assert format_token_count(1000) == "1.00k"
    assert format_token_count(1500) == "1.50k"
    assert format_token_count(9999) == "9.99k"

def test_millions():
    assert format_token_count(1000000) == "1.00M"
    assert format_token_count(1234567) == "1.23M"

def test_billions():
    assert format_token_count(1000000000) == "1.00B"

def test_trillions():
    assert format_token_count(1000000000000) == "1.00T"

5. Files to Modify

New Files:

  • agent_cost_estimator/utils/__init__.py
  • agent_cost_estimator/utils/formatting.py
  • tests/test_formatting.py

Modified Files:

  • agent_cost_estimator/cli.py
  • agent_cost_estimator/gui.py

6. Backward Compatibility

CLI: Adding a new output line is a breaking change for scripts that parse CLI output.
Consider making it opt-in with a --tokens flag initially.

GUI: Adding columns is non-breaking.

JSON: Adding fields is backward compatible.