Compression Strategy Metadata and Routing Authority
Learning Headroom
Why 'Where Did This Come From?' Matters
Builds on ← 5.3 CCR Reversible Compression Architecture, 5.4 CCR Lifetime and Retrieval Failure Handling
**Recall:** CCR stores an original alongside a compressed marker, keyed by a hash, so it can be retrieved later with `headroom_retrieve`.
**New problem:** Two different components produce CCR entries:
- SmartCrusher → compresses JSON arrays (tool outputs)
- ContentRouter → compresses code, logs, search results, text
When you're inspecting a stored CCR entry (debugging, auditing, tuning), you need to know: *which compression path produced this entry?*
Where this leads → 7.2 End-to-End Headroom Tuning Playbook: tuning decisions depend on knowing which strategy compressed what.
The Authoritative Field: compression_strategy
Every stored CCR entry carries metadata. Three fields matter for routing:
| Field | Role |
|---|---|
| `headroom_retrieve` | tool call that recovers the original |
| `HEADROOM_CCR_TTL_SECONDS` | sizes the local store lifetime |
| `compression_strategy` | **authoritative discriminator** of the producing path |
**The rule:**
\[
\text{producing path} = \text{value of } \texttt{compression\_strategy}
\]
- `compression_strategy` names the component/path that created the entry (e.g. a SmartCrusher array strategy vs. a ContentRouter code/log/text strategy).
- This field is written **at compression time**, by the component doing the compressing — so it's a direct, first-hand record, not a guess.
**Why this shape:** retrieval, lifetime, and provenance are three separate concerns — one field per concern, so each stays simple and unambiguous.
The Trap: Guessing From Payload Shape
**Tempting but WRONG approach:** infer the producing path by *looking at what the compressed payload looks like*.
> "This looks like a list of 20 summarized items, so it must have come from SmartCrusher."
**Why this fails — worked example:**
- ContentRouter compresses a **search results** payload (a list of ranked hits).
- The compressed output is *also* a short array of item-like entries.
- Visually, this is indistinguishable from SmartCrusher's compressed JSON-array output.
```mermaid
flowchart LR
A["Stored CCR entry: compact array shape"] -->|shape-based guess| B["Assume: SmartCrusher"]
A -->|metadata lookup| C["Read compression_strategy field"]
C --> D["Actual: ContentRouter search-results strategy"]
B -. wrong .-> E["Misrouted debugging / wrong retrieval assumptions"]
style B fill:#7f1d1d,color:#fff
style E fill:#7f1d1d,color:#fff
style D fill:#166534,color:#fff
```
**Rule:** payload shape is *not* routing evidence. Only `compression_strategy` is.
Worked Example: Validating a Retrieval Path
**Scenario:** A tool call returned 1000 log lines. Headroom compressed it. Later, an engineer needs to validate the stored CCR entry before trusting a retrieval.
**Given (stored CCR entry):**
```json
{
"hash": "log9f3a",
"payload": ["...20 summarized lines..."],
"compression_strategy": "content_router.log_compressor",
"created_at": "T0",
"ttl_seconds": 1800
}
```
**Step 1 — Resist shape inference.**
Payload is a short array (20 entries) → *looks* like SmartCrusher's typical output. Do **not** conclude that.
**Step 2 — Read `compression_strategy`.**
Value: `"content_router.log_compressor"` → producing path is **ContentRouter**, not SmartCrusher.
**Step 3 — Check freshness against TTL.**
Suppose the validation happens at time \(\text{now} = T0 + 1200\text{s}\).
\[
\text{elapsed} = \text{now} - \text{created\_at} = 1200\text{s}
\]
\[
\text{elapsed} \le \texttt{ttl\_seconds}? \quad 1200 \le 1800 \;\Rightarrow\; \text{Yes}
\]
Since 1200 seconds is less than the 1800-second TTL, the entry has **not** expired → original still retrievable via `headroom_retrieve(hash="log9f3a")`.
*(Contrast: if `now = T0 + 2000s`, then elapsed = 2000s > 1800s → expired. Per 5.4: regenerate the digest or re-read the source; still do **not** re-infer strategy from shape.)*
**Step 4 — Conclusion.**
Routing authority = `compression_strategy` field = ContentRouter's log-compression path. Retrieval path validated.
Edge Cases and the General Principle
**General principle:**
\[
\text{Provenance of a stored CCR entry} \;=\; \texttt{compression\_strategy}
\]
never payload shape, byte size, or structural resemblance.
**Edge cases:**
- **Ambiguous shapes across strategies:** both SmartCrusher and ContentRouter can emit short-array-looking payloads → shape *always* underdetermines origin.
- **Missing/malformed `compression_strategy`:** treat as a data-integrity problem to investigate, *not* as license to infer from shape.
- **Expired entry (5.4 link):** provenance question and retrievability question are independent — you can still read `compression_strategy` on metadata even if the payload's original is gone, but you cannot retrieve the original past TTL.
**Why this discipline matters for tuning (→ 7.2):**
Correct per-path attribution lets you measure SmartCrusher vs. ContentRouter savings/behavior separately — essential for the end-to-end tuning playbook.
Back to course