Headroom Pipeline and Integration Boundaries
Learning Headroom
From Compressing Messages to Wiring a Pipeline
**Recall:** In 1.3 you measured tokens and compression ratios on a single message.
Now the question is: *how does a real request actually reach the compressor, and what happens after?*
Every Headroom request follows the same shape:
```mermaid
flowchart LR
A["Your Agent / App"] -->|tool outputs, logs, DB reads, RAG results| B["Headroom"]
B -->|compressed messages| C["LLM Provider"]
C -->|response| A
```
- **Headroom** = the compression pipeline you've been learning
- It sits **between** your app and the model provider (OpenAI, Anthropic, Gemini, Bedrock, 100+ via LiteLLM)
- Three ways to plug your app into that middle box: **proxy, SDK, integrations (incl. MCP)**
**Where this leads →** 3.2 (content routing) and 6.1 (config scopes) both assume you know *where in this pipeline* a decision gets made.
Three Entry Points, One Pipeline
All three feed the **same** compression pipeline underneath.
| Entry point | How it works | Code changes |
|---|---|---|
| **Proxy mode** | Run `headroom proxy`, point client's base URL at it | Zero — just change the URL |
| **SDK mode** | Call `compress()` (Python/TS) on messages before sending | Minimal — one function call |
| **Integrations (incl. MCP)** | LangChain, Vercel AI SDK, Agno, Strands, LiteLLM, MCP adapters | Framework-specific setup |
**Why three modes exist (not one):**
- Proxy mode: you don't control the client code (e.g. a closed tool) → intercept at the network layer
- SDK mode: you *do* control the code and want explicit, inline control
- MCP/integration mode: your agent already talks through a framework or protocol (Model Context Protocol) that expects a plug-in point, not a URL swap or manual call
Each mode is just a different **door into the same house** — the compression logic behind the door never changes.
Tracing a Request: Proxy Mode, Step by Step
**Setup:** An agent calls a coding tool that returns a 4,000-token build log. Client is configured with base URL `http://localhost:8787` (Headroom proxy) instead of `https://api.anthropic.com`.
```mermaid
sequenceDiagram
participant App as Agent/App
participant HP as Headroom Proxy (FastAPI)
participant LLM as Anthropic API
App->>HP: POST /v1/messages ("4000-token build log")
HP->>HP: per-provider handler (Anthropic)
HP->>HP: run compression pipeline
Note right of HP: build log -> keep failures/errors, drop passing noise ("80-95% typical")
HP->>LLM: forward compressed request
LLM-->>HP: model response
HP-->>App: response passed back
```
**Step-by-step:**
1. App sends request to what it *thinks* is the provider — really it's the proxy
2. Proxy's Anthropic handler receives it
3. Handler runs the **same** compression pipeline every other mode uses
4. Build log (4,000 tokens) → compressed to 300 tokens
5. Compression ratio: \( \dfrac{4000 - 300}{4000} = \dfrac{3700}{4000} = 0.925 \), i.e. a **92.5% reduction** — inside the documented 80–95% range for build logs
6. Compressed request forwarded **upstream** to the real Anthropic API
7. Response flows back through the proxy to the app, unchanged
**Builds on ← 1.3:** that 92.5% savings figure is computed with the exact same compression-ratio formula, \( \text{ratio} = \dfrac{\text{original} - \text{compressed}}{\text{original}} \), you practiced in lesson 1.3 — now you're seeing *where* in the request lifecycle it actually fires.
Tracing SDK Mode and MCP Mode
**SDK mode — explicit call, same pipeline:**
```python
from headroom import compress
messages = agent.get_context() # includes 4000-token build log
compressed = compress(messages) # you call this yourself
response = anthropic_client.send(compressed) # then forward upstream yourself
```
- No proxy process, no URL change
- **You** own the forwarding step — Headroom only compresses, it does not send the request onward
- Same pipeline as proxy mode: JSON arrays, logs, code, text all handled by the same rules
**MCP / integration mode — protocol-level plug-in:**
```mermaid
flowchart LR
A["Agent via MCP client"] -->|tool call/result| M["MCP Adapter"]
M -->|routes through| B["Headroom pipeline"]
B -->|compressed| C["LLM Provider"]
```
- MCP = Model Context Protocol — a standard way tools/agents exchange context
- Headroom ships an **MCP adapter** so tool results crossing the protocol get compressed automatically, without you writing a `compress()` call anywhere
- Also covers LangChain, Vercel AI SDK, Agno, Strands, LiteLLM — each needs framework-specific setup, but lands in the same pipeline
**Key contrast:** proxy = network interception, SDK = your code calls in, MCP/integration = framework calls in for you.
The Non-Negotiable Order: Compress THEN Forward
**Transform pipeline order (same in every mode):**
```mermaid
flowchart LR
R["Raw messages from app"] --> D{"Detect content type: JSON / logs / code / text / diff / image"}
D --> COMP["Compress per-type rules"]
COMP --> FWD["Forward upstream to provider"]
FWD --> RESP["Provider response"]
```
- Compression **always** happens before the request crosses the **upstream boundary** — the point where Headroom hands off to the real provider
- Nothing forwarded is uncompressed; nothing compressed is forwarded twice
- The provider (OpenAI/Anthropic/Gemini/Bedrock) **never sees** Headroom — it just receives a normal, smaller request
**Why this order matters, with numbers:** Suppose you flipped the order and forwarded first. The provider would bill and process the full 4,000-token build log from Slide 3 before any compression touched it — you'd pay for all 4,000 tokens upstream. Only *after* that would compression run, on a response that's already been paid for and processed at full size. The 92.5% saving from Slide 3 would never reach the provider call — it would be pointless. The pipeline exists specifically to sit *before* that boundary, on every entry point.
**Where this leads →** 3.2 (Content Detection and Content Routing) zooms into that first "Detect content type" box — how Headroom decides *which* compression rule applies before this ordering even starts.
Edge Cases and What Doesn't Change
**Common misconception:** "Proxy mode compresses differently than SDK mode."
**Reality:** the pipeline is identical. Only the *forwarding responsibility* differs.
| | Proxy | SDK | MCP/Integration |
|---|---|---|---|
| Who compresses | Headroom | Headroom | Headroom |
| Who forwards upstream | Headroom | **You** | Framework adapter |
| Client code changes | None | One call | Framework setup |
| Per-provider handling | Built-in (Anthropic/OpenAI/Gemini/Bedrock) | You choose the client | Adapter-specific |
**Extensibility note:** Headroom emits lifecycle events at every pipeline stage. Third-party code can register a **pipeline extension** (via the `headroom.pipeline_extension` entry point) to mutate `messages`, `tools`, `headers`, or `metadata` *before* upstream forwarding — without forking Headroom.
- Both the SDK client and the proxy dispatch the **same events** → one extension covers both deployments
- This only works *because* compress-then-forward order is fixed — extensions know exactly where they sit
**Builds on ← 1.2 (Agent Message Anatomy):** these are the same `messages`, `tools` structures you dissected there — now being mutated mid-pipeline.
**Where this leads →** 3.3 (Live-Zone-Only Compression) will show a specific case of the pipeline compressing only part of a message — built on this same fixed detect-compress-forward order.
بازگشت به دوره