Headroom and Context-Window Economics
Learning Headroom
Why Headroom Exists
**Capacity rule**
\(I+B\le C\), with \(H=C-I-B\).
**Worked numbers**
\(16{,}000-13{,}500-2{,}000=500\) tokens of headroom.
```mermaid
flowchart LR
A["Conversation input"] --> B["Context window"]
R["Reserved output"] --> B
B --> D{"Fits?"}
D -->|Yes| E["Generate reply"]
D -->|No| F["Take headroom action"]
```
Tokens, Cost, and Budget
**Budget calculation**
Given \(C=8{,}000\), \(B=1{,}500\), and \(I=5{,}900\): \(I_{\max}=6{,}500\), and \(H=2{,}100-1{,}500=600\).
**Tradeoff**
Increasing \(B\) by 100 decreases \(I_{\max}\) by 100.
Repeated Prompts Consume Capacity
**Three-turn total**
\(I_1=4{,}500\), \(I_2=5{,}000\), \(I_3=5{,}500\); therefore \(P=4{,}500+5{,}000+5{,}500=15{,}000\).
```mermaid
flowchart TD
A["Stable prefix"] --> B["Request 1 prefill"]
A --> C["Request 2 prefill"]
A --> D["Request 3 prefill"]
B --> E["Repeated processing"]
C --> E
D --> E
E --> F["Cost and latency"]
```
How Headroom Acts
```mermaid
flowchart LR
A["System prompt"] --> Z["Final context"]
B["Tool definitions"] --> Z
C["Older turns"] --> Z
D["Latest user message"] --> E{"Too large?"}
F["Latest tool result"] --> E
E -->|No| Z
E -->|Yes| G["Compress eligible newest blocks"]
G --> Z
```
**Effect of compression**
If \(C=10{,}000\), \(I=8{,}200\), and \(B=1{,}500\), then \(H=10{,}000-8{,}200-1{,}500=300\). Compressing 400 input tokens gives \(I=7{,}800\), so \(H=10{,}000-7{,}800-1{,}500=700\).
Tuning Headroom Safely
```mermaid
flowchart TD
A["Measure input I"] --> B["Choose output buffer B"]
B --> C["Compute H = C - I - B"]
C --> D{"Is H sufficient?"}
D -->|Yes| E["Keep context"]
D -->|No| F["Compress newest live blocks"]
F --> G["Recompute I and H"]
G --> D
```
**Buffer comparison**
For \(C=10{,}000\) and \(I=7{,}800\), \(B=1{,}500\) gives \(H=700\), while \(B=2{,}500\) gives \(H=-300\). The second plan exceeds capacity by 300 tokens.
**Quick check**
If compression removes 600 input tokens while \(C\) and \(B\) stay fixed, what happens to \(H\)? Answer: \(H\) increases by 600, because \(H=C-I-B\).
بازگشت به دوره