CacheAligner as an Observability Detector
Learning Headroom
From Byte-Identical Prefixes to Watching Them Drift
**Recap — Builds on ← 2.2 Byte-Identical Prefix Requirements**
Provider prompt caches only pay off when the prefix is byte-for-byte identical across calls. One stray byte early in the prompt invalidates the cached run.
**The problem**
Message assembly code is easy to get wrong: a timestamp, a reordered field, an extra newline slipped into the "stable" part of the prompt — and the cache silently stops working. Nobody gets an error. You just pay full price again.
**The question CacheAligner answers**
"Is my stable prefix actually staying stable, call after call?"
```mermaid
flowchart LR
A["Caller assembles prompt"] --> B["CacheAligner inspects prefix"]
B --> C{"Prefix identical to last wake?"}
C -->|Yes| D["warnings empty, prefix_changed = false"]
C -->|No| E["warnings + prefix_changed = true"]
E --> F["Caller fixes assembly logic"]
F --> A
```
**Where this leads →** 2.5 Cache-Safe Multi-Turn Forwarding needs a stable prefix to forward turns byte-faithfully, and 6.4 Observability Metrics and Tuning Decisions will use exactly these metrics to drive tuning calls.
Detector-Only: What CacheAligner Will Never Do
**Definition — detector-only behavior**
CacheAligner:
- inspects the prefix
- emits `warnings` for volatile content
- records observability data (`cache_metrics`)
CacheAligner does **not**:
- rewrite messages
- reorder content
- repair or stabilize the prefix
> "If CacheAligner warns about drift, keep the prefix stable in the caller. The transform is a detector, not a repair pass."
**Why it's built this way**
Only the caller knows *why* a field changed (new turn? bug? intentional?). A detector that silently "fixed" prefixes could hide real bugs or, worse, rewrite something that was supposed to change — so the fix authority stays with the caller.
**Also opt-in, not automatic**
- Disabled by default
- Hard-disabled inside the proxy (proxy mode uses cache mode instead, forwarding prior turns byte-faithfully)
```mermaid
flowchart TD
A[CacheAligner] -->|can| B["Read the prefix"]
A -->|can| C["Emit warnings"]
A -->|can| D["Record metrics"]
A -.->|cannot| E["Edit the prefix"]
A -.->|cannot| F["Reorder content"]
A -.->|cannot| G["Auto-repair drift"]
```
Reading the Detector's Report
**The report fields**
| Field | What it tells you |
|---|---|
| `warnings` | Which parts of the prefix look unstable |
| `cache_metrics.stable_prefix_bytes` | Size of the stable prefix, in bytes |
| `cache_metrics.stable_prefix_tokens_est` | Estimated token size of the stable prefix |
| `cache_metrics.stable_prefix_hash` | Hash of the stable prefix (for comparing across wakes) |
| `cache_metrics.prefix_changed` | Did the stable prefix drift since the last wake? |
| `cache_metrics.previous_hash` | Hash recorded on the previous wake |
| `markers` | Emitted `stable_prefix_hash` marker, for downstream tooling |
**Why a hash, not a byte diff?**
Comparing two hashes is \( O(1) \) to check and cheap to store between wakes; comparing full prefixes byte-by-byte every wake would mean keeping the whole previous prefix around. A hash is a short fingerprint: same bytes in → same hash out, any byte different → (almost certainly) a different hash.
\[
\texttt{prefix\_changed} = \big(\texttt{stable\_prefix\_hash} \neq \texttt{previous\_hash}\big)
\]
**stable_prefix_bytes vs stable_prefix_tokens_est**
- `stable_prefix_bytes`: exact, measured in raw bytes
- `stable_prefix_tokens_est`: an *estimate* — tokenization isn't 1:1 with bytes, so treat this as approximate sizing for budget/headroom math, not an exact count
Worked Example: Reading a Warm-Wake Report
**Setup**
An agent wakes twice in the same session. Its stable prefix is meant to be: system instructions + tool schema (this never changes turn to turn).
**Wake 1 report:**
```
warnings: []
stable_prefix_bytes: 4820
stable_prefix_tokens_est: 1190
stable_prefix_hash: "a1f9c3..."
prefix_changed: false
previous_hash: null
```
*First wake — no previous hash to compare against, so `prefix_changed` is `false` by default.*
**Wake 2 report:**
```
warnings: ["volatile content detected near byte offset 4790"]
stable_prefix_bytes: 4838
stable_prefix_tokens_est: 1195
stable_prefix_hash: "7be201..."
prefix_changed: true
previous_hash: "a1f9c3..."
```
**Step-by-step interpretation**
1. `previous_hash` = `"a1f9c3..."` — this matches Wake 1's `stable_prefix_hash` exactly. Good: CacheAligner correctly carried the last fingerprint forward.
2. `stable_prefix_hash` = `"7be201..."` — different from `previous_hash`. So `prefix_changed = true`.
3. Byte delta: \( 4838 - 4820 = 18 \) — **18 bytes were added** inside what was supposed to be the stable region.
4. `warnings` names the culprit: volatile content near byte offset 4790 — right at the tail of the stable region (since the stable region ends at byte 4838, offset 4790 is only 48 bytes from the end).
5. **Conclusion:** something like a timestamp or run ID got assembled *inside* the stable prefix instead of after it. The provider cache for this session just went cold on Wake 2.
The Fix Lives in the Caller, Not the Detector
**Diagnosis from the report:** volatile content (an 18-byte run ID / timestamp) landed inside the stable prefix, at its tail.
**Builds on ← 2.3 Stable-Prefix and Live-Zone Layout**
Correct layout: `[ stable prefix ][ live wake digest ][ run-specific context ]`
**The assembly bug**
```
[ system instructions ][ tool schema ][ run_id: xyz123 ] ← stable prefix (WRONG)
[ wake digest ]
```
The run ID was concatenated onto the end of the stable block before the wake digest — so every run mutates the "stable" prefix.
**The caller-side fix**
```
[ system instructions ][ tool schema ] ← stable prefix (fixed)
[ wake digest ][ run_id: xyz123 ] ← live zone, moved here
```
Move the run ID out of the stable block and into the live/run-specific zone, *after* the wake digest.
```mermaid
flowchart LR
subgraph Before["Before fix"]
A1["stable prefix + run_id"] --> A2["wake digest"]
end
subgraph After["After fix"]
B1["stable prefix only"] --> B2["wake digest + run_id"]
end
```
**General rule — warning-driven assembly fix**
CacheAligner warning → locate the byte offset it names → identify which caller-assembled field lives there → move that field out of the stable-prefix block, into the live wake digest or run-specific tail → re-run and confirm `prefix_changed = false` next wake.
Edge Cases and the Boundary With Cold-Prefix Recompaction
**When `prefix_changed` is expected, not a bug**
- First wake of a session: `previous_hash` is `null`, so `prefix_changed` is `false` by convention — there's nothing to compare against yet.
- A deliberate change to system instructions (a real deploy) will correctly show `prefix_changed = true` once. That's not drift — that's an intended new baseline.
**Repeated warnings across many wakes**
One `prefix_changed = true` reading = investigate. The *same* field flagged wake after wake = a systemic assembly bug (e.g., a timestamp generated fresh every call).
**Don't confuse detection with repair — twice**
1. CacheAligner itself never repairs the prefix (this lesson).
2. There *is* a separate mechanism that rewrites prefixes — the **cold-prefix hook** — but it only fires when the cache has already gone cold past the provider's TTL, since at that point byte-identical forwarding buys nothing anyway. It recompacts (dedupe, drop superseded reads, lossless folds) and re-caches. It never touches a *warm* cache — firing on a warm one would be the exact mistake CacheAligner is designed to help you catch and avoid.
```mermaid
flowchart TD
A["prefix_changed = true"] --> B{"Cache still warm?"}
B -->|Yes, unexpected drift| C["Bug: fix caller assembly ("this lesson")"]
B -->|No, TTL lapsed| D["Expected: cold-prefix hook may safely recompact"]
```
Wrap-Up: What You Can Now Do
**The full loop you now own**
```mermaid
flowchart LR
A["Caller assembles prefix"] --> B["CacheAligner: inspect"]
B --> C["warnings + cache_metrics"]
C --> D["You interpret: prefix_changed? which bytes? which field?"]
D --> E["You fix: move volatile field out of stable block"]
E --> F["Re-wake, confirm prefix_changed = false"]
F -.-> A
```
**Checklist for reading any CacheAligner report**
1. Is `warnings` empty? If not, read the byte offset it names.
2. Compare `stable_prefix_hash` to `previous_hash` — did the fingerprint change?
3. Did `stable_prefix_bytes` grow or shrink — by how much, and does that match a field you'd expect (timestamp, run ID, tool result)?
4. Locate the caller-side field at that offset; move it into the live wake digest / run-specific zone.
5. Re-wake and confirm `prefix_changed = false`.
**Where this leads →**
- **2.5 Cache-Safe Multi-Turn Forwarding** needs the stable prefix you just learned to protect, so it can forward prior turns byte-faithfully.
- **6.4 Observability Metrics and Tuning Decisions** will read these same `cache_metrics` fields as inputs to real tuning decisions.
Back to course