Usage Analytics

BobaMixer provides comprehensive analytics to help you understand your AI usage patterns, optimize costs, and make data-driven decisions about model selection.

Overview

Analytics features include:

Real-time tracking - Tokens, cost, and latency per request
Historical trends - 7/30/90-day analysis
Multi-dimensional views - By profile, project, time, estimate level
Session tracking - Group related requests
Export capabilities - JSON, CSV for external analysis
Visualization - TUI dashboard with charts

Key Metrics

Token Usage

Input Tokens: Tokens sent to the AI model (prompt, context) Output Tokens: Tokens generated by the AI model (response)

bash

# View token usage
boba stats --today

# Output:
# Input Tokens:  45,230
# Output Tokens: 12,450
# Total Tokens:  57,680

Cost

Total Cost: Combined input + output cost in USD

Based on:

Exact usage from API responses (most accurate)
Pricing configuration (if exact not available)
Heuristic estimation (fallback)

bash

# View costs
boba stats --7d

# Output:
# Total Cost: $23.45
# Avg Cost/Day: $3.35

Latency

Latency: Time from request to response completion (milliseconds)

Useful for:

Identifying slow models
Optimizing for speed
Comparing providers

bash

# View latency stats
boba stats --by-profile --7d

# Output includes P50, P95, P99 latencies per profile

Estimate Accuracy

BobaMixer tracks estimation level for each request:

Exact: From API response usage field (100% accurate)
Mapped: From pricing config (95-99% accurate)
Heuristic: Character-based estimation (70-90% accurate)

bash

# View by estimate level
boba stats --by-estimate

# Output:
# Exact:     234 requests (85%)
# Mapped:    32 requests (12%)
# Heuristic: 9 requests (3%)

Viewing Statistics

Time-Based Views

bash

# Today
boba stats --today

# Yesterday
boba stats --yesterday

# Last 7 days
boba stats --7d

# Last 30 days
boba stats --30d

# Custom range
boba stats --from 2024-01-01 --to 2024-01-31

Example output:

Usage Statistics (Last 7 Days)
──────────────────────────────────────────
Requests:       156
Input Tokens:   245,320
Output Tokens:  68,450
Total Cost:     $47.85
Avg Latency:    1,234 ms

Daily Average:
  Requests:     22.3
  Cost:         $6.84
  Tokens:       44,824

Profile-Based Views

bash

# By profile
boba stats --by-profile --7d

# Specific profile
boba stats --profile claude-sonnet --30d

# Compare profiles
boba stats --by-profile --compare --7d

Example output:

Usage by Profile (Last 7 Days)
──────────────────────────────────────────
Profile         Requests  Cost    Tokens   P95 Latency
claude-opus     23        $34.50  156,789  2,345 ms
claude-sonnet   89        $12.30  98,234   1,234 ms
gpt-4o-mini     44        $1.05   58,901   892 ms
──────────────────────────────────────────
Total           156       $47.85  313,924

Project-Based Views

bash

# By project
boba stats --by-project --7d

# Specific project
boba stats --project my-app --30d

# All projects
boba stats --by-project --all --30d

Example output:

Usage by Project (Last 7 Days)
──────────────────────────────────────────
Project         Requests  Cost    % of Total
my-app          89        $28.50  59.6%
website         45        $12.30  25.7%
experiments     22        $7.05   14.7%
──────────────────────────────────────────
Total           156       $47.85  100%

Estimate Level Views

bash

# By estimate accuracy
boba stats --by-estimate --7d

Example output:

Usage by Estimate Level (Last 7 Days)
──────────────────────────────────────────
Level       Requests  % of Total  Avg Error
Exact       134       85.9%       0%
Mapped      18        11.5%       ~2%
Heuristic   4         2.6%        ~15%
──────────────────────────────────────────
Total       156       100%

Recommendation: Good accuracy. 85.9% exact estimates.

Advanced Analytics

Session Tracking

Sessions group related requests together.

bash

# List recent sessions
boba session list --7d

# View specific session
boba session show <session-id>

# Session statistics
boba stats --by-session --7d

Example session view:

Session: work-session-2024-01-15-001
──────────────────────────────────────────
Duration:    2h 34m
Requests:    23
Total Cost:  $8.45
Project:     my-app
Branch:      feature/new-ui

Request Timeline:
1. 09:23:12  claude-sonnet  $0.45  "Implement user login"
2. 09:45:34  claude-sonnet  $0.32  "Add form validation"
...
23. 11:57:23 gpt-4o-mini   $0.05  "Format code"

Trend Analysis

bash

# Daily trend
boba stats --trend daily --30d

# Weekly trend
boba stats --trend weekly --90d

# By profile over time
boba stats --trend daily --by-profile --30d

Example trend output:

Daily Cost Trend (Last 30 Days)
──────────────────────────────────────────
Date        Cost    Requests  Trend
2024-01-15  $8.45   23        ↑ +12%
2024-01-14  $7.53   19        ↓ -5%
2024-01-13  $7.92   21        ↑ +8%
...

Average:     $6.84/day
Peak:        $12.30 (2024-01-08)
Lowest:      $2.10 (2024-01-06)

Projection: $210.60/month (based on 30-day avg)

Cost Breakdown

bash

# Detailed cost breakdown
boba stats --breakdown --7d

# By model
boba stats --by-model --30d

# Input vs output cost
boba stats --cost-breakdown --7d

Example breakdown:

Cost Breakdown (Last 7 Days)
──────────────────────────────────────────
Component        Cost    % of Total
Input Tokens     $28.50  59.6%
Output Tokens    $19.35  40.4%
──────────────────────────────────────────
Total            $47.85  100%

By Model:
claude-3-5-sonnet-20241022  $34.50  72.1%
gpt-4-turbo-preview         $12.30  25.7%
gpt-4o-mini                 $1.05   2.2%

Latency Analysis

bash

# Latency statistics
boba stats --latency --7d

# By profile
boba stats --latency --by-profile --7d

# Percentile distribution
boba stats --latency --percentiles --7d

Example latency output:

Latency Statistics (Last 7 Days)
──────────────────────────────────────────
Profile         P50      P95      P99      Max
claude-opus     1,234ms  2,345ms  3,456ms  4,567ms
claude-sonnet   892ms    1,456ms  2,123ms  2,987ms
gpt-4-turbo     1,045ms  1,892ms  2,456ms  3,234ms
gpt-4o-mini     567ms    892ms    1,234ms  1,678ms

Recommendation: gpt-4o-mini fastest for latency-sensitive tasks

Exporting Data

CSV Export

bash

# Export to CSV
boba report --format csv --output usage-report.csv

# Open in Excel/Numbers
open usage-report.csv

CSV columns:

timestamp,profile,model,project,branch,input_tokens,output_tokens,cost,latency_ms,estimate_level
2024-01-15T09:23:12Z,claude-sonnet,claude-3-5-sonnet-20241022,my-app,feature/ui,1234,456,0.45,1234,exact
...

JSON Export

bash

# Export to JSON
boba report --format json --output usage-report.json

# Pretty print
boba report --format json | jq '.'

JSON structure:

json

[
  {
    "timestamp": "2024-01-15T09:23:12Z",
    "profile": "claude-sonnet",
    "model": "claude-3-5-sonnet-20241022",
    "project": "my-app",
    "branch": "feature/ui",
    "input_tokens": 1234,
    "output_tokens": 456,
    "cost_usd": 0.45,
    "latency_ms": 1234,
    "estimate_level": "exact",
    "session_id": "work-session-001"
  }
]

Filtered Exports

bash

# Last 30 days only
boba report --format json --from $(date -d '30 days ago' +%Y-%m-%d) --output last-month.json

# Specific project
boba report --format csv --project my-app --output my-app-usage.csv

# Specific profile
boba report --format json --profile claude-opus --output opus-usage.json

# High-cost sessions only
boba report --format json | jq 'select(.cost_usd > 1.0)' > high-cost.json

Visualization

TUI Dashboard

bash

# Launch dashboard
boba

# Interactive features:
# - Real-time stats
# - Profile breakdown chart
# - Cost trend graph
# - Recent requests list
# - Budget status

Dashboard sections:

Header: Current profile, active budget, alerts
Stats Panel: Today's usage summary
Chart View: Cost/token trends over time
Profile Breakdown: Pie chart of usage by profile
Recent Activity: Last 10 requests
Notifications: Alerts and suggestions

External Visualization

Export data and visualize in your tool of choice:

Python with Pandas:

python

import pandas as pd
import matplotlib.pyplot as plt

# Load data
df = pd.read_csv('usage-report.csv')

# Plot daily cost trend
df.groupby('date')['cost'].sum().plot()
plt.title('Daily AI Cost')
plt.ylabel('Cost (USD)')
plt.show()

Excel:

bash

# Export and open
boba report --format csv --output usage.csv
open usage.csv

# Create pivot table, charts, etc.

Grafana/InfluxDB (advanced):

bash

# Export JSON
boba report --format json > /var/lib/influxdb/import/boba.json

# Import to InfluxDB
influx write -b boba -f /var/lib/influxdb/import/boba.json

Common Analysis Patterns

1. Find Cost Optimization Opportunities

bash

# Find most expensive profiles
boba stats --by-profile --30d --sort-by cost

# Find high-cost, low-complexity tasks
boba report --format json | jq 'select(.input_tokens < 1000 and .cost_usd > 0.5)'

# Identify routing improvements
boba stats --by-profile --breakdown

2. Compare Model Performance

bash

# Latency comparison
boba stats --latency --by-profile --7d

# Cost efficiency (cost per 1K tokens)
boba stats --by-profile --efficiency --7d

# Quality vs cost trade-off
# (requires manual quality assessment)

3. Track Project Costs

bash

# Monthly cost by project
boba stats --by-project --30d

# Export for billing
boba report --format csv --by-project --output project-costs-$(date +%Y-%m).csv

# Identify expensive projects
boba stats --by-project --sort-by cost --30d

4. Monitor Estimate Accuracy

bash

# Check estimate distribution
boba stats --by-estimate --30d

# Find profiles with poor estimation
boba report --format json | jq 'group_by(.profile) | map({profile: .[0].profile, heuristic_pct: (map(select(.estimate_level == "heuristic")) | length) / length * 100})'

# Add pricing config for mapped estimation
boba edit pricing

5. Identify Usage Patterns

bash

# Peak usage times
boba report --format json | jq 'group_by(.timestamp[0:13]) | map({hour: .[0].timestamp[11:13], count: length})' | jq 'sort_by(.count) | reverse'

# Most common branch patterns
boba report --format json | jq 'group_by(.branch) | map({branch: .[0].branch, count: length})' | jq 'sort_by(.count) | reverse'

# Task type distribution
# (based on routing rules matched)

6. Budget Compliance

bash

# Daily spending vs budget
boba budget --status --detailed

# Projection accuracy
boba budget --project monthly

# Overspend analysis
boba stats --by-project --30d | grep "over budget"

Best Practices

1. Regular Review

bash

# Daily check
boba stats --today

# Weekly review
boba stats --7d --by-profile

# Monthly deep dive
boba report --format csv --30d --output monthly-review.csv

2. Track Key Metrics

Focus on:

Cost per day/week/month - Budget compliance
Cost per profile - Optimization opportunities
Estimate accuracy - Data quality
P95 latency - User experience

3. Export for Long-Term Storage

bash

# Monthly export
boba report --format json --30d --output archive/usage-$(date +%Y-%m).json

# Compress old data
gzip archive/usage-*.json

4. Correlate with Value

Track:

Which profiles produce best results?
What's the ROI of premium models?
Are faster models worth the cost?

(Requires manual quality tracking)

5. Set Up Alerts

bash

# Use budgets for automatic alerts
boba budget --set daily 50 --alert-at 75

# Check alerts daily
boba action

bash

# Generate team report
boba report --format csv --30d --output team-usage-$(date +%Y-%m).csv

# Share with stakeholders
# Include: total cost, breakdown, trends, optimizations

Troubleshooting

Missing Data

bash

# Check database
sqlite3 ~/.boba/usage.db "SELECT COUNT(*) FROM usage_records;"

# Verify tracking is working
boba stats --today

# Check logs
tail -f ~/.boba/logs/boba.log

Inaccurate Costs

bash

# Check estimate levels
boba stats --by-estimate

# Add/update pricing
boba edit pricing

# Verify profile costs
boba ls --profiles --verbose

Slow Queries

bash

# Vacuum database
sqlite3 ~/.boba/usage.db "VACUUM;"

# Analyze
sqlite3 ~/.boba/usage.db "ANALYZE;"

# Check database size
du -h ~/.boba/usage.db

Export Issues

bash

# Test export
boba report --format json | jq '.' > /dev/null

# Check permissions
ls -l ~/.boba/usage.db

# Verify jq installed (for JSON processing)
which jq

API Reference

CLI Commands

bash

# Statistics
boba stats [--today|--yesterday|--7d|--30d|--from DATE --to DATE]
boba stats --by-profile [--compare]
boba stats --by-project
boba stats --by-estimate
boba stats --latency [--percentiles]
boba stats --trend [daily|weekly]
boba stats --breakdown

# Reports
boba report --format [json|csv]
boba report --output FILE
boba report --from DATE --to DATE
boba report --project NAME
boba report --profile NAME

# Sessions
boba session list [--7d|--30d]
boba session show SESSION_ID
boba stats --by-session

Database Schema

sql

-- Usage records table
CREATE TABLE usage_records (
  id INTEGER PRIMARY KEY,
  timestamp INTEGER NOT NULL,
  profile TEXT NOT NULL,
  model TEXT NOT NULL,
  project TEXT,
  branch TEXT,
  session_id TEXT,
  input_tokens INTEGER,
  output_tokens INTEGER,
  cost_usd REAL,
  latency_ms INTEGER,
  estimate_level TEXT,
  explore BOOLEAN
);

-- Indexes for performance
CREATE INDEX idx_timestamp ON usage_records(timestamp);
CREATE INDEX idx_profile ON usage_records(profile);
CREATE INDEX idx_project ON usage_records(project);

Direct Database Access

bash

# Query directly
sqlite3 ~/.boba/usage.db "SELECT * FROM usage_records WHERE timestamp > strftime('%s', 'now', '-7 days');"

# Export to CSV
sqlite3 -header -csv ~/.boba/usage.db "SELECT * FROM usage_records;" > export.csv

# Custom analytics
sqlite3 ~/.boba/usage.db <<EOF
SELECT
  profile,
  COUNT(*) as requests,
  SUM(cost_usd) as total_cost,
  AVG(latency_ms) as avg_latency
FROM usage_records
WHERE timestamp > strftime('%s', 'now', '-30 days')
GROUP BY profile
ORDER BY total_cost DESC;
EOF

Next Steps

Budgets - Set up budget tracking
Routing - Optimize with routing rules
CLI Reference - Command documentation
Operations - Production analytics setup

Usage Analytics ​

Overview ​

Key Metrics ​

Token Usage ​

Cost ​

Latency ​

Estimate Accuracy ​

Viewing Statistics ​

Time-Based Views ​

Profile-Based Views ​

Project-Based Views ​

Estimate Level Views ​

Advanced Analytics ​

Session Tracking ​

Trend Analysis ​

Cost Breakdown ​

Latency Analysis ​

Exporting Data ​

CSV Export ​

JSON Export ​

Filtered Exports ​

Visualization ​

TUI Dashboard ​

External Visualization ​

Common Analysis Patterns ​

1. Find Cost Optimization Opportunities ​

2. Compare Model Performance ​

3. Track Project Costs ​

4. Monitor Estimate Accuracy ​

5. Identify Usage Patterns ​

6. Budget Compliance ​

Best Practices ​

1. Regular Review ​

2. Track Key Metrics ​

3. Export for Long-Term Storage ​

4. Correlate with Value ​

5. Set Up Alerts ​

6. Share Insights ​

Troubleshooting ​

Missing Data ​

Inaccurate Costs ​

Slow Queries ​

Export Issues ​

API Reference ​

CLI Commands ​

Database Schema ​

Direct Database Access ​

Next Steps ​

Resources ​

Usage Analytics

Overview

Key Metrics

Token Usage

Cost

Latency

Estimate Accuracy

Viewing Statistics

Time-Based Views

Profile-Based Views

Project-Based Views

Estimate Level Views

Advanced Analytics

Session Tracking

Trend Analysis

Cost Breakdown

Latency Analysis

Exporting Data

CSV Export

JSON Export

Filtered Exports

Visualization

TUI Dashboard

External Visualization

Common Analysis Patterns

1. Find Cost Optimization Opportunities

2. Compare Model Performance

3. Track Project Costs

4. Monitor Estimate Accuracy

5. Identify Usage Patterns

6. Budget Compliance

Best Practices

1. Regular Review

2. Track Key Metrics

3. Export for Long-Term Storage

4. Correlate with Value

5. Set Up Alerts

6. Share Insights

Troubleshooting

Missing Data

Inaccurate Costs

Slow Queries

Export Issues

API Reference

CLI Commands

Database Schema

Direct Database Access

Next Steps

Resources