Update token display spec: add monthly usage summary row
- Add section 3.0 for Monthly Usage Summary Row - Display **This Month Usage** as first row under each harness - Calculate aggregated token usage for current calendar month - Include caching strategy (5-minute TTL) - Add implementation examples for Tkinter and PyQt5 Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This commit is contained in:
@@ -0,0 +1,269 @@
|
||||
# Token Display Specification
|
||||
|
||||
**Version:** 1.1.0
|
||||
**Date:** 2026-11-10
|
||||
**Status:** Draft
|
||||
**Author:** LLM Assistant
|
||||
|
||||
## Overview
|
||||
|
||||
This specification adds token usage display to both GUI and CLI interfaces of agent-cost-estimator.
|
||||
|
||||
## Summary of Changes
|
||||
|
||||
1. **GUI:** Add 3 columns (Input/Output/Cached tokens) with automatic suffixes (k, M, B, T)
|
||||
2. **GUI:** Add monthly usage summary row as first row under each harness
|
||||
3. **CLI:** Add token usage line with format `-- I:X O:Y C:Z`
|
||||
|
||||
## 1. Formatting Function
|
||||
|
||||
Create a utility function to format token counts with automatic suffixes:
|
||||
|
||||
```python
|
||||
# File: agent_cost_estimator/utils/formatting.py
|
||||
|
||||
def format_token_count(count: int) -> str:
|
||||
"""Format token count with automatic suffix for large numbers.
|
||||
|
||||
Args:
|
||||
count: Non-negative integer token count
|
||||
|
||||
Returns:
|
||||
String with suffix: 0-999 (no suffix), 1k+, 1M+, 1B+, 1T+
|
||||
Always uses 2 decimal places for suffixed values.
|
||||
|
||||
Examples:
|
||||
0 -> "0"
|
||||
500 -> "500"
|
||||
1000 -> "1.00k"
|
||||
1500 -> "1.50k"
|
||||
1000000 -> "1.00M"
|
||||
1000000000 -> "1.00B"
|
||||
1000000000000 -> "1.00T"
|
||||
"""
|
||||
if count < 1000:
|
||||
return str(count)
|
||||
elif count < 1_000_000:
|
||||
return f"{count / 1000:.2f}k"
|
||||
elif count < 1_000_000_000:
|
||||
return f"{count / 1_000_000:.2f}M"
|
||||
elif count < 1_000_000_000_000:
|
||||
return f"{count / 1_000_000_000:.2f}B"
|
||||
else:
|
||||
return f"{count / 1_000_000_000_000:.2f}T"
|
||||
```
|
||||
|
||||
## 2. CLI Changes
|
||||
|
||||
### Default Output
|
||||
|
||||
**Current:**
|
||||
```
|
||||
-- provider:model | session:$X.XX | turn:$Y.YY
|
||||
```
|
||||
|
||||
**New:**
|
||||
```
|
||||
-- provider:model | session:$X.XX | turn:$Y.YY
|
||||
-- I:<input> O:<output> C:<cached>
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```
|
||||
-- mistral:mistral-medium | session:$3.40 | turn:$0.92
|
||||
-- I:1.01k O:2.0k C:10M
|
||||
```
|
||||
|
||||
**Implementation** in `cli.py`:
|
||||
```python
|
||||
from .utils.formatting import format_token_count
|
||||
|
||||
# After printing cost line
|
||||
input_formatted = format_token_count(usage.input_tokens)
|
||||
output_formatted = format_token_count(usage.output_tokens)
|
||||
cached_formatted = format_token_count(usage.cached_input_tokens)
|
||||
print(f"-- I:{input_formatted} O:{output_formatted} C:{cached_formatted}")
|
||||
```
|
||||
|
||||
**Note:** Token line is NOT displayed with `-s` or `-t` flags (bare number output).
|
||||
|
||||
## 3. GUI Changes
|
||||
|
||||
### 3.0. Monthly Usage Summary Row
|
||||
|
||||
**New Feature:** Display a summary row as the **first row** under each harness showing aggregated token usage for the current calendar month.
|
||||
|
||||
**Display:**
|
||||
```
|
||||
┌─ Harness (e.g., vibe)
|
||||
├─ **This Month Usage** │ - │ 100k │ 50k │ 20k │ $12.34 │ - │ - │ -
|
||||
├─ Session 1 │ model │ 10k │ 5k │ 2k │ $1.23 │ $0.45 │ ... │ ...
|
||||
├─ Session 2 │ model │ 90k │ 45k │ 18k │ $11.11 │ $3.21 │ ... │ ...
|
||||
└─ ...
|
||||
```
|
||||
|
||||
**Requirements:**
|
||||
- Position: First child row under each harness
|
||||
- Visual: Bold font, gray background, special tag "monthly_summary"
|
||||
- Label: "**This Month Usage**"
|
||||
- Data: Sum of all sessions from current month
|
||||
- Caching: Cache for 5 minutes, invalidate on month change or new session
|
||||
|
||||
**Implementation:**
|
||||
1. Add `calculate_monthly_usage(harness_name, year, month)` function
|
||||
2. Call it during session refresh
|
||||
3. Insert as first row under each harness node
|
||||
4. Style with bold font and different background
|
||||
|
||||
**Data Structure:**
|
||||
```python
|
||||
@dataclass
|
||||
class MonthlyUsage:
|
||||
input_tokens: int
|
||||
output_tokens: int
|
||||
cached_input_tokens: int
|
||||
session_count: int
|
||||
total_cost_usd: float
|
||||
```
|
||||
|
||||
**Display Values:**
|
||||
- Input Tokens: `format_token_count(monthly_usage.input_tokens)`
|
||||
- Output Tokens: `format_token_count(monthly_usage.output_tokens)`
|
||||
- Cached Tokens: `format_token_count(monthly_usage.cached_input_tokens)`
|
||||
- Session Price: `${monthly_usage.total_cost_usd:.2f}`
|
||||
- Turn Price, DateTime, Hash: Empty
|
||||
|
||||
### Column Structure
|
||||
|
||||
Add 3 new columns **after the Model column**:
|
||||
|
||||
| # | Column | Header | Width | Alignment | Data Source |
|
||||
|---|--------|--------|-------|-----------|-------------|
|
||||
| 0 | Session Name | Session Name | 200px | Left | session.title |
|
||||
| 1 | Model | Model | 150px | Left | session.model |
|
||||
| **2** | **Input Tokens** | **Input** | **80px** | **Right** | **format_token_count(session.input_tokens)** |
|
||||
| **3** | **Output Tokens** | **Output** | **80px** | **Right** | **format_token_count(session.output_tokens)** |
|
||||
| **4** | **Cached Tokens** | **Cached** | **80px** | **Right** | **format_token_count(session.cached_input_tokens)** |
|
||||
| 5 | Session Price | Session Price | 100px | Right | ${session.session_price:.2f} |
|
||||
| 6 | Turn Price | Turn Price | 100px | Right | ${session.current_turn_price:.2f} |
|
||||
| 7 | DateTime | DateTime | 150px | Left | datetime_str |
|
||||
| 8 | Hash | Hash | 100px | Left | session_hash |
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
1. **Update SessionInfo dataclass** in `gui.py`:
|
||||
```python
|
||||
@dataclass
|
||||
class SessionInfo:
|
||||
session_id: str
|
||||
title: str
|
||||
model: str
|
||||
session_price: float
|
||||
current_turn_price: float
|
||||
last_modified: datetime
|
||||
# Add these fields:
|
||||
input_tokens: int = 0
|
||||
output_tokens: int = 0
|
||||
cached_input_tokens: int = 0
|
||||
```
|
||||
|
||||
2. **Populate token data** when creating SessionInfo:
|
||||
```python
|
||||
session_info = SessionInfo(
|
||||
session_id=str(session_dir.name),
|
||||
title=meta.title or "Untitled",
|
||||
model=key,
|
||||
session_price=costs['total_usd'],
|
||||
current_turn_price=turn_costs['total_usd'],
|
||||
last_modified=last_modified,
|
||||
input_tokens=usage.input_tokens,
|
||||
output_tokens=usage.output_tokens,
|
||||
cached_input_tokens=usage.cached_input_tokens
|
||||
)
|
||||
```
|
||||
|
||||
3. **Update TreeView columns** in Tkinter version:
|
||||
```python
|
||||
self.tree = ttk.Treeview(main_frame,
|
||||
columns=("model", "input_tokens", "output_tokens", "cached_tokens",
|
||||
"session_price", "turn_price", "datetime", "hash"))
|
||||
|
||||
# Add headings
|
||||
self.tree.heading("input_tokens", text="Input")
|
||||
self.tree.heading("output_tokens", text="Output")
|
||||
self.tree.heading("cached_tokens", text="Cached")
|
||||
|
||||
# Add column configs (right-aligned)
|
||||
self.tree.column("input_tokens", width=80, anchor=tk.E, stretch=False)
|
||||
self.tree.column("output_tokens", width=80, anchor=tk.E, stretch=False)
|
||||
self.tree.column("cached_tokens", width=80, anchor=tk.E, stretch=False)
|
||||
```
|
||||
|
||||
4. **Update tree insertion** to include token data:
|
||||
```python
|
||||
item_id = self.tree.insert(
|
||||
harness_node,
|
||||
self.tk.END,
|
||||
text=session.title,
|
||||
values=(
|
||||
session.model,
|
||||
format_token_count(session.input_tokens),
|
||||
format_token_count(session.output_tokens),
|
||||
format_token_count(session.cached_input_tokens),
|
||||
f"${session.session_price:.2f}",
|
||||
f"${session.current_turn_price:.2f}",
|
||||
datetime_str,
|
||||
session_hash
|
||||
),
|
||||
tags=("session",)
|
||||
)
|
||||
```
|
||||
|
||||
5. **Repeat for PyQt5** implementation in the same file.
|
||||
|
||||
## 4. Test Cases
|
||||
|
||||
```python
|
||||
# tests/test_formatting.py
|
||||
from agent_cost_estimator.utils.formatting import format_token_count
|
||||
|
||||
def test_basic():
|
||||
assert format_token_count(0) == "0"
|
||||
assert format_token_count(500) == "500"
|
||||
assert format_token_count(999) == "999"
|
||||
|
||||
def test_thousands():
|
||||
assert format_token_count(1000) == "1.00k"
|
||||
assert format_token_count(1500) == "1.50k"
|
||||
assert format_token_count(9999) == "9.99k"
|
||||
|
||||
def test_millions():
|
||||
assert format_token_count(1000000) == "1.00M"
|
||||
assert format_token_count(1234567) == "1.23M"
|
||||
|
||||
def test_billions():
|
||||
assert format_token_count(1000000000) == "1.00B"
|
||||
|
||||
def test_trillions():
|
||||
assert format_token_count(1000000000000) == "1.00T"
|
||||
```
|
||||
|
||||
## 5. Files to Modify
|
||||
|
||||
### New Files:
|
||||
- `agent_cost_estimator/utils/__init__.py`
|
||||
- `agent_cost_estimator/utils/formatting.py`
|
||||
- `tests/test_formatting.py`
|
||||
|
||||
### Modified Files:
|
||||
- `agent_cost_estimator/cli.py`
|
||||
- `agent_cost_estimator/gui.py`
|
||||
|
||||
## 6. Backward Compatibility
|
||||
|
||||
**CLI:** Adding a new output line is a breaking change for scripts that parse CLI output.
|
||||
Consider making it opt-in with a `--tokens` flag initially.
|
||||
|
||||
**GUI:** Adding columns is non-breaking.
|
||||
|
||||
**JSON:** Adding fields is backward compatible.
|
||||
Reference in New Issue
Block a user