Live-Zone-Only Compression
Learning Headroom
From Stable-Prefix Layout to a Compression Rule
**Builds on ← 2.3 Stable-Prefix and Live-Zone Layout**
Conversation is split into two zones:
- Stable prefix (cache hot zone): system prompt, tool defs, older turns
- Live zone: newest user message + latest tool result
**Builds on ← 3.1 Headroom Pipeline and Integration Boundaries**
Content detection/routing already tags blocks. Now: WHERE does compression get applied?
**The rule:** Headroom compresses the live zone ONLY. The stable prefix is forwarded byte-for-byte.
**Where this leads →** 3.4 (Cache Mode vs Token Mode), 5.3 (CCR architecture), 6.2 (budget tuning) all assume this boundary.
```mermaid
flowchart LR
A["Stable prefix: system + tools + older turns"] -->|byte-for-byte| A2["Forwarded unchanged"]
B["Live zone: newest user msg + latest tool result"] -->|type-aware| B2["Compressed"]
```
What Exactly Gets Compressed
**Compressible (live zone) — newest content blocks only:**
- Latest user message
- Latest tool result / tool output
**Never compressed (cache hot zone):**
- System prompt
- Tool definitions
- All older turns (everything before the last N)
\[
\text{context\_budget} = \text{model\_context\_limit} - \text{output\_buffer\_tokens}
\]
Why this shape: the output buffer is carved off FIRST — compression only ever targets the live zone, never the reserved response space itself.
```mermaid
flowchart TD
T1["Turn 1 - old"] --> T2["Turn 2 - old"] --> T3["..."] --> TN["Turn N-1"]
TN --> LZ["Live zone: newest user msg"]
LZ --> TR["Latest tool result"]
style T1 fill:#94a3b8
style T2 fill:#94a3b8
style TN fill:#94a3b8
style LZ fill:#f59e0b
style TR fill:#f59e0b
```
Why Older Turns Stay Frozen
**older_turn_protection + system_prompt_protection**
Why freeze the prefix?
- Provider prompt caching requires the prefix to be **byte-identical** across turns (see 2.3)
- Mutating ANY earlier token invalidates the cache from that point forward
- Cache hit rate stays stable turn-to-turn only if the prefix never changes
**Turn protection default:** last 2 turns always preserved, in addition to the live zone.
**System prompt protection:** never dropped — holds persona, instructions, tool descriptions the model needs for the whole conversation.
```mermaid
flowchart LR
subgraph Frozen ["Frozen - cache hit"]
S["System prompt"] --> D["Tool defs"] --> O["Older turns"]
end
O --> P["Protected: last N turns"]
P --> L["Live zone: compressed"]
style Frozen fill:#dbeafe
style L fill:#f59e0b
```
Worked Example: Routing a Real Conversation
**Setup:** 6-turn conversation, \(\text{headroom\_keep\_turns} = 2\), a 40{,}000-token search result just returned as the newest tool output.
| Block | Position | Action |
|---|---|---|
| System prompt | prefix | forwarded byte-for-byte |
| Tool definitions | prefix | forwarded byte-for-byte |
| Turns 1–2 | prefix | forwarded byte-for-byte |
| Turns 3–4 (protected, keep_turns=2) | prefix | forwarded byte-for-byte |
| Latest user message (turn 5) | live zone | type-aware compression |
| Latest tool result (40{,}000 tokens) | live zone | type-aware compression, cached in CCR (hash = def456) |
**Step-by-step token accounting for the tool result:**
1. Original tool result size: \(40{,}000\) tokens.
2. Headroom compresses it and stores the full original in CCR under hash \(\text{def456}\).
3. In its place, a marker is inserted: `"compressed, retrieve: def456"` — roughly \(10\)–\(20\) tokens.
4. Net savings for this turn: \(40{,}000 - 20 \approx 39{,}980\) tokens freed in the live zone, while the original stays fully recoverable.
**message_order_preservation:** the sequence Turn 1 → Turn 2 → ... → Turn 5 → Tool Result stays identical — only block CONTENTS shrink, block ORDER never changes.
Edge Cases and the Misconception to Avoid
**Misconception to kill:** "Headroom drops old messages to save space."
**Reality:** Headroom never deletes conversation history. It only *compresses* the newest blocks; older content is retained, just uncompressed and unchanged.
> "IntelligentContext / RollingWindow (score-based history dropping) were retired. Headroom compresses fresh tool output and new turns only — it does not drop conversation history."
**Edge cases:**
- No large tool result this turn → live zone may compress little or nothing; that's expected, not a bug
- Small user message → compression may be skipped if already below threshold (type-aware, not forced)
- \(\text{headroom\_keep\_turns}\) raised → protected zone grows, live zone boundary shifts, but the rule is unchanged
**live_zone_identification check:** given any turn, ask — is this the newest user message or newest tool result? If no → protected, forward as-is. If yes → eligible for compression.
Why This Design Pays Off — and What's Next
**The trade this design makes:**
- Compress only 1–2 blocks per turn → smaller savings per turn, but...
- Prefix stays byte-identical → cache hit rate stays high across the WHOLE conversation
- Net effect: fewer tokens billed as "cache miss" cost, even though fewer bytes are compressed
**Recap of protections enforced:**
1. System prompt — never touched
2. Older turns (before protected window) — never touched
3. Protected last-N turns — never touched
4. Live zone (newest user msg + newest tool result) — only compressible target
5. Block order — always preserved
**Where this leads →**
- 3.4: Cache Mode vs Token Mode — trades off how hard you lean on this boundary
- 5.3: CCR Reversible Compression — how the compressed live-zone content gets stored & retrieved
- 6.2: Context Budget & Output Buffer Tuning — tuning `keep_turns` / `output_buffer_tokens` around this exact boundary
Back to course