Stable-Prefix and Live-Zone Layout
Learning Headroom
From Byte-Identical Prefix to a Real Layout
**Recall (2.2):** the provider cache only hits when the prefix is byte-identical to a previous wake.
**New problem:** a real wake also carries a wake digest (memory summary) and run-specific stuff (tool output, loop state) that change *every single wake*.
If you interleave those with the stable instructions, the whole prefix becomes non-identical — cache miss, every time.
**The fix — a three-zone layout:**
```mermaid
flowchart LR
A["Stable Prefix"] --> B["Live Wake Digest"] --> C["Run-Specific Tail"]
style A fill:#166534,color:#fff
style B fill:#b45309,color:#fff
style C fill:#7f1d1d,color:#fff
```
- Zone 1: byte-identical every wake
- Zone 2: changes each wake, but sits *after* the cache boundary
- Zone 3: changes every run, sits last
**Builds on ←** 2.2 Byte-Identical Prefix Requirements
**Where this leads →** 2.5 Cache-Safe Multi-Turn Forwarding builds turn-by-turn forwarding on top of this exact ordering.
Why Order Matters: How Prefix Caching Reads Bytes
Provider caches match **left to right, from byte 0**. The match stops at the first differing byte.
```tikz
\begin{tikzpicture}[scale=1.0]
\draw[fill=green!30] (0,0) rectangle (4,1) node[midway]{stable prefix};
\draw[fill=orange!40] (4,0) rectangle (6.5,1) node[midway]{digest};
\draw[fill=red!30] (6.5,0) rectangle (9,1) node[midway]{tail};
\draw[->, thick, blue] (0,-0.6) -- (4,-0.6) node[midway, below]{cache hit region};
\draw[dashed] (4,-1) -- (4,1.3);
\node at (4,1.5) {\small match boundary};
\end{tikzpicture}
```
**Why the shape works:**
- Put volatile bytes *anywhere before* the end of the stable prefix → the match boundary collapses to wherever the first change is, even if that's byte 10.
- Put volatile bytes *after* the stable prefix ends → the whole stable region still matches, and only the tail is re-processed fresh.
**Formula for what you keep cached:**
\[
\text{cached\_bytes} = \min(\text{len}(P_{\text{now}}), \text{len}(P_{\text{prev}})) \text{ up to first mismatch}
\]
Every byte before that mismatch is free (served from cache); every byte from there on is recomputed.
**Worked check:** previous wake's prefix was 500 bytes of stable content followed by a digest. This wake, the stable content is still the same first 500 bytes, byte-identical. Byte 501 is where the (new) digest starts, and it differs from last time's digest. So cached_bytes = 500: everything from byte 0 to byte 500 is served from cache, and byte 501 onward is recomputed — exactly the stable-prefix span, no more, no less.
Worked Example: Laying Out a Real Wake
**Given** these prompt sections for one agent wake:
| Section | Content | Changes? |
|---|---|---|
| A | System prompt + tool contracts | Never |
| B | Project rules ('always cite sources') | Never |
| C | Current task instructions | Per wake (instruction-bearing!) |
| D | Memory digest ('last 5 turns summarized') | Per wake |
| E | Latest tool output (API result, 3KB) | Per run |
| F | Child-agent scratch context | Per run |
**Step 1 — identify stable-vs-volatile:** A, B never change → stable prefix. C changes but *carries instructions* → still goes right after the stable prefix, before the digest. D changes per wake → live digest zone. E, F change per run → tail.
**Step 2 — assemble in order:**
\[
\text{Prompt} = \underbrace{A \,\|\, B}_{\text{stable prefix}} \,\|\, \underbrace{C \,\|\, D}_{\text{live digest zone}} \,\|\, \underbrace{E \,\|\, F}_{\text{run-specific tail}}
\]
**Step 3 — check byte-identity across two wakes:**
| Section | Wake 1 bytes | Wake 2 bytes | Same? |
|---|---|---|---|
| A | 1200 | 1200 | Yes, byte-identical |
| B | 300 | 300 | Yes, byte-identical |
| C | "Summarize doc X" | "Summarize doc Y" | No, differs |
| D | digest v1 | digest v2 | No, differs |
Match boundary = end of B = byte 1500. So on wake 2, bytes 0–1500 (all of A‖B) are served from cache; from byte 1501 onward (C‖D‖E‖F) is recomputed. That is the entire point of this layout: 1500 bytes saved, every single wake.
Protecting Instruction-Bearing Content
**Rule:** instruction-bearing content must survive layout AND any later compression — it doesn't matter which zone it's placed in.
What counts as instruction-bearing:
- system messages (persona, tool contracts)
- current task / current directive
- explicit constraints ('never delete files without confirmation')
**Two different protections, don't conflate them:**
| Protection | What it guards | Mechanism |
|---|---|---|
| Layout position | Cache hit rate | Put volatile bytes after stable prefix |
| Content protection | Instruction survival | Never drop/compress system messages; task instructions stay intact even in the live zone |
**Common trap:** "live zone = safe to compress freely." False — the *live wake digest* is volatile for caching purposes, but if it contains an instruction ('the user asked to always respond in bullet points'), that instruction still must not be silently dropped.
**Worked example of the trap:** a summarizer compresses the live zone from 800 bytes to 150 bytes to save tokens. If it strips out the line "user asked to always respond in bullet points" along with the routine chit-chat it was trimming, the agent silently stops following that constraint next wake — a real behavior bug, not just a cache statistic.
**Where this leads →** 5.2 Instruction-Bearing Content Routing builds the full routing rules for exactly this distinction.
Finding the Cache Hot Zone in a Real Prompt
**Recall (1.4):** repeated wakes rebuild the same expensive prompt shape — that repetition is exactly what makes caching worth doing.
**Cache hot zone** = system prompt + tool definitions + older turns. It never mutates; that's what keeps hit rates stable.
**Identification checklist** — for each section, ask:
1. Is this byte-identical across the last N wakes? → hot zone (stable prefix)
2. Does it change but still carry directives? → front of live zone
3. Does it change and carry no directives (pure memory summary)? → live zone
4. Is it fresh output from *this* run only? → tail
**Worked check:** system prompt (unchanged 20 wakes) + tool defs (unchanged) + task C + digest D + tool result E:
```mermaid
flowchart TD
subgraph HotZone["Hot Zone - cached"]
A2["System prompt"]
A3["Tool definitions"]
end
subgraph LiveZone["Live Zone - not cached"]
C2["Task C - instruction"]
D2["Digest D"]
end
subgraph Tail["Tail - not cached"]
E2["Tool result E"]
end
A2-->A3-->C2-->D2-->E2
```
Only the hot zone contributes to `cache_metrics.stable_prefix_bytes` and gets hashed into `stable_prefix_hash`.
Edge Cases and What Comes Next
**Edge cases:**
- **Growing digest:** if the wake digest keeps growing every wake, it *never* moves into the stable prefix — it stays in the live zone forever, even though parts of it are old.
- **A 'stable' section that quietly drifts:** e.g. a tool definition list that gets a new tool added — this breaks byte-identity silently. This is a real failure mode, not a hypothetical.
- **Empty tail:** if there's no run-specific content this wake, the tail is simply empty — the layout order stays stable-prefix → live-digest, no gap needed.
- **Instruction inside the tail:** avoid this by design — route instruction-bearing content to the front of the live zone (slide 4), never let it drift into the tail.
**This layout is a rule the caller enforces — nothing checks it for you automatically.** That's exactly the gap the next node fills:
**Where this leads →** 2.4 CacheAligner as an Observability Detector: it inspects your assembled prefix and *warns* when it drifts, using `cache_metrics.prefix_changed` and `stable_prefix_hash` — but it never rewrites your layout. You still own getting the order right.
**Also feeds into →** 3.3 Live-Zone-Only Compression (compress only zones 2–3) and 7.1 Designing a Repeated-Wake Digest Strategy (structuring zone 2 itself).
بازگشت به دوره