Token Measurement and Compression Metrics
Learning Headroom
Why Measure Compression?
**Core idea**
Compression is useful only when its effect on the input budget can be measured.
**Quick example**
If 10,000 tokens become 7,000, then 3,000 tokens are removed. Later metrics describe that change as a ratio and a percentage.
```mermaid
flowchart LR
A["Messages before"] --> B["Compression transforms"]
B --> C["Messages after"]
C --> D["More headroom"]
```
Three Metrics, One Baseline
**Three definitions**
\[\text{tokens\_saved}=\text{before}-\text{after}\]
\[\text{compression\_ratio}=\frac{\text{after}}{\text{before}}\]
\[\text{savings\_percent}=\frac{\text{saved}}{\text{before}}\times100\]
**Small numerical example**
Given before = 10 and after = 4:
\[10-4=6\text{ saved}\]
\[4/10=0.4\text{ remains}\]
\[(6/10)\times100=60\%\text{ saved}\]
```mermaid
flowchart TD
B["Before: baseline"] --> S["Subtract after"]
B --> R["Divide after by before"]
S --> P["Divide saved by before"]
R --> Q["Ratio: remaining"]
P --> U["Percentage: removed"]
```
Fully Worked API Example
**Complete calculation**
Given before = 15,000 and after = 3,500.
1. \[15{,}000-3{,}500=11{,}500\] saved.
2. \[3{,}500/15{,}000=0.233333\ldots\approx0.23\] ratio.
3. \[11{,}500/15{,}000=0.766666\ldots\]
4. \[0.766666\ldots\times100\approx76.7\%\] saved.
Check: \[(1-0.233333\ldots)\times100\approx76.7\%\].
Compare Results Correctly
**Two runs**
Run A: 10,000 → 7,000, so 3,000 saved and 30% saved.
Run B: 10,000 → 4,000, so 6,000 saved and 60% saved.
B has twice the raw savings, a lower ratio, and a higher savings percentage.
**Why percentages matter**
Different starting sizes:
\[50{,}000-45{,}000=5{,}000\]
\[5{,}000/50{,}000\times100=10\%\]
\[10{,}000-7{,}000=3{,}000\]
\[3{,}000/10{,}000\times100=30\%\]
The second run saves fewer tokens but removes a larger fraction.
Boundaries and Practical Checks
**Boundary cases**
No change: before = after = 10,000 gives 0 saved, ratio 1.0, and 0% saved.
Expansion: before = 4,000 and after = 5,000 gives -1,000 saved, ratio 1.25, and -25% saved.
**Audit rule**
Require before > 0. Then check saved = before − after, ratio = after ÷ before, and saved fraction + remaining fraction = 1.
Back to course