Block-addressed editing: a 10x output-token cost reduction for document LLM agents
~10× less output, ~3× faster, measured. Plus the second lever — history compaction — that cuts multi-turn input ~60%.
This works perfectly when your LLM agent reads a document and updates it — the user asks "in the third paragraph, change 'retried three times' to 'retried five times'", and the agent edits the page.
Here is the fast and costly implementation — the one almost everyone ships first:
the model read the whole document
the model re-typed the whole document
we saved the whole document
Re-typed, literally: the edit tool's output is the complete new version of the page.
The document I used in the measurements below — is ~7.4 KB of markdown, which is roughly 1,550 tokens. So changing one sentence means the model generates ~1,560 tokens: the one sentence you asked for, plus every paragraph you didn't.
One changed sentence. ~1,560 generated tokens. And if you add new content, those output tokens start growing fast.
To see why that's the worst possible shape, look at how LLM pricing is split. Every call has two lanes:
input — the tokens you send: the conversation so far, the document
output — the tokens the model generates
The lanes are not priced alike. On a typical frontier model today, input runs on the order of $1–3 per million tokens while output runs $10–15 per million — roughly 8× more expensive. And the real gap is wider than the list price, because the two lanes cache differently: providers discount repeated input heavily — a prompt prefix the model has recently seen costs around a tenth of the normal input price — so re-sending the same document every turn is surprisingly cheap. Output gets no such discount. There is no cache for generation; every output token is full price, every time.
So: input is the cheap, cached lane. Output is the expensive, uncached one. And the implementation above answers a one-sentence request by pushing the entire document through the expensive lane.
Where the idea came from. Not from the LLM world at all — from the internals of Slate, the framework many block-based rich-text editors are built on. When you type into a Slate document, the page does not re-render. Slate keeps WeakMaps tracking which paths of the document actually changed — the dirty paths — and only the components for those blocks re-render. The clean blocks aren't re-rendered cheaply. They aren't re-rendered at all.
Staring at that, the question asked itself: my UI refuses to re-render blocks that didn't change — so why is my agent re-emitting them? Output tokens are the agent's render budget, and it was re-rendering the entire page on every keystroke.
That analogy is what led to the ~10× output cut below. (One honest note: the mechanism I ended up shipping looks less like React and more like git — content fingerprints compared at write time, rather than live dirty-tracking — but the question that started everything came straight from the renderer.)
This post is the method, with A/B numbers from a production agent: give every block of the document a stable id, let the agent address blocks instead of re-emitting documents, and verify every edit against a content fingerprint taken when the agent read the page.
Across a real five-edit session, output dropped from 8,128 tokens to 775 — about 10× — and edits came back ~3× faster. A second, independent technique (history compaction) cut multi-turn input by ~60% on top.
None of the individual ingredients here is my invention — I'll point at the neighbours later. What I can give you is the full composition for structured documents, traced end to end, with measurements.
First, how a writing agent actually works
If you already build agents, skip ahead. But the two facts in this section are load-bearing for everything after.
An agent is a model in a loop. You send a prompt; the model either answers or asks to call a tool — a function you exposed, like read_document or search. Your server runs the tool, appends the result to the conversation, and calls the model again. Repeat until the model answers without asking for a tool. This loop (reason → act → observe) is usually called ReAct.
user prompt
↓
model → "call edit tool with these args"
↓
server runs the tool, appends the result
↓
model → "done, here's my summary"
Fact one: the model has no hands. It never touches your database. It only describes operations; your server performs them. Like SQL — the client sends UPDATE … WHERE id = …, the database executes it.
Fact two: the model has no memory. Every turn, you re-send the entire conversation so far — the transcript. A follow-up tomorrow is a new HTTP request; your server loads the stored history (for example a JSON on a per-session database row), replays it to the model, and appends the new message. Anything you want the model to "remember" is something you are paying to re-send, every single turn.
Those two facts define the two costs this post attacks:
Output — everything the model generates, including tool arguments. Expensive, never cached.
Input — the replayed transcript. Cheaper, cache-discounted, but re-sent every turn, so it compounds.
The prerequisite: a stable id on every block
Here is the entire architectural bet, and it's one sentence:
A block of text (data) must have an identity that survives its content changing.
Most block-based editors already assign every block a short random id at creation time — for deep links, for drag-and-drop, for anchoring comments. If yours does, the prerequisite is free. That id is the handle the agent will use to say which block it means.
Why must it be an id? Why not "the third paragraph", or a hash of the block's content? Because those are the three possible addressing schemes, and two of them die on contact with a live document:
Position ("paragraph 3", "line 42") dies when anyone inserts or deletes above the target. Every address below the change shifts.
Content (a hash of the block's text) dies when anyone edits the block — the address was the content. Worse, it can't distinguish "this block moved" from "this block was deleted and a similar one created".
Identity (an id stored on the block) survives both. The block can move, the text can change; the id rides along.
The system needs to express one specific sentence: "the same block, changed." Only identity-addressing can say it — position and content addressing each collapse "same block" into the thing that changed. Hold that sentence; the fingerprints in the next section exist to finish it.
(If some blocks lack ids — legacy content — you can fall back to content+position handles. It works, degraded: addresses die on any edit, so every conflict looks like "block not found" instead of "block changed", and the follow-up optimization below gives up more often. Our system assigns real ids to everything the agent writes, so documents heal toward fully-addressed as they're edited.)
Markdown is the language of the agent
The document lives in the database as a JSON tree. The model never sees that.
Everything the model reads, and everything it writes, is markdown. On read, the server renders each block to markdown; on write, the model sends markdown back and the server parses just that fragment into blocks.
Two reasons, one obvious and one that took me longer:
The obvious one: models are dramatically better at markdown than at your internal JSON. They've read a lifetime of it. Emitting a paragraph of markdown is cheap and reliable; emitting your nested block schema, with every property right, is neither — and it spends tokens on syntax instead of content.
The less obvious one is about the fingerprints. The system needs to answer "has this block changed since the agent read it?" — which basically means:
"Would the model see different text at response time than it saw at read time?"
So we don't fingerprint the stored JSON. We fingerprint the rendered markdown — a SHA-1 of exactly the text the model was shown, produced by the same renderer at read time and at check time. That buys precision in both directions:
Internal properties can churn (metadata, layout props, id reassignments) without triggering a false conflict — if the rendered text is identical, the block is not stale for the model's purposes.
Any change the model would see, however small, flips the hash completely.
The fingerprint is a statement about the model's view, not about storage bytes. Hash the view, because the view is what the model's decisions were based on.
One more property matters: the fingerprints are cheap. ~50 bytes per block to store, microseconds to compute. A 200-block document snapshots to ~10 KB regardless of how much text the blocks hold.
The read: addresses to the model, fingerprints to the server
Take a five-block document — a webhooks guide for example:
## Webhooks
Deliveries are sent whenever a document is published.
Every request carries a signature header.
Failed deliveries are retried three times.
You can replay any delivery from the dashboard.
When the agent reads it, one walk over the blocks produces two artifacts with two destinations:
stored blocks (JSON tree, database)
↓ render each block to markdown, hash each rendering
┌──────────────────────────────┬──────────────────────────────┐
│ anchored markdown │ snapshot │
│ → goes TO the model │ → stays on the SERVER │
└──────────────────────────────┴──────────────────────────────┘
The model receives the full document — every block, complete — with one anchor line naming each block's id:
<!-- block:h1K2p9 -->
## Webhooks
<!-- block:aB3xK9 -->
Deliveries are sent whenever a document is published.
<!-- block:pQ7mN2 -->
Every request carries a signature header.
<!-- block:zT5rW8 -->
Failed deliveries are retried three times.
<!-- block:qF4dR7 -->
You can replay any delivery from the dashboard.
The server keeps the snapshot: for each block, the handle and the hash of its rendered markdown (without the anchor lines, just the actual text from the document).
{ "docId": "doc_web1",
"blocks": [
{ "handle": "h1K2p9", "hash": "6a5362c0…" },
{ "handle": "aB3xK9", "hash": "80c7da16…" },
{ "handle": "pQ7mN2", "hash": "0db39487…" },
{ "handle": "zT5rW8", "hash": "01e5960c…" },
{ "handle": "qF4dR7", "hash": "27788d86…" }
] }
The snapshot is the server's record of what the model saw. During a turn it's an in-memory map; between turns it's persisted as a JSON column on the same database row that stores the session's conversation history. The model never sees a hash. It doesn't know hashes exist.
anchors → the model ("which block do you mean?")
fingerprints → the server ("is that block still what you read?")
The edit: change the third paragraph from X to Y
Now the running example. You tell the agent:
In the third paragraph, change "retried three times" to "retried five times".
What the model does. It finds the third paragraph by reading — the full document is in its input, so "third paragraph" is a comprehension task, not a lookup API. That's block zT5rW8. It emits one tool call:
{ "tool": "edit_blocks", "docId": "doc_web1",
"operations": [
{ "op": "replace", "blockId": "zT5rW8",
"markdown": "Failed deliveries are retried five times." }
] }
That is the model's entire output for this edit!!! An address, and the new text for one block. In our logs: 80–170 output tokens per edit, against ~1,550 for the full rewrite. The other four blocks were read, understood, and never retyped.
(The tool speaks five operations: replace, insert_before, insert_after, delete, append. "Add a note under the heading" is an insert_after; "remove that paragraph" is a delete. Same machine throughout.)
What the server does. Before touching anything, it asks one question — is the block the model wants to change still the same as when the model read it?
The server does not hash the markdown the model just sent. Of course that hash would differ — new content hashing differently is the point of new content.
Instead, three versions of the target block are on the table:
the PAST — the snapshot hash: what the model READ
the PRESENT — the block as stored RIGHT NOW, re-rendered, re-hashed
the FUTURE — the op's markdown: what the model WANTS
The check compares past against present. The future is never hashed — it's the payload, not the evidence. In words:
"Since the model read this block, has anyone changed it?
No → the edit was decided while looking at current text. Apply it.
Yes → the edit was decided on a stale picture. Refuse — and show the model the present."
For our edit: the present renders to Failed deliveries are retried three times., which hashes to 01e5960c… — equal to the snapshot. Not stale. Proceed.
The apply is array surgery, not document reassembly. The server parses the model's one line of markdown into one new block (assigning it a fresh id), finds the target's position, and splices:
[ h1K2p9, aB3xK9, pQ7mN2, zT5rW8, qF4dR7 ]
↑ replace
[ h1K2p9, aB3xK9, pQ7mN2, NEW, qF4dR7 ]
Look at the four other blocks. They are the same stored objects. Not re-rendered, not re-parsed, not re-created — spliced around. This is the property no text-file edit format can give you, because a text file has no richer layer to protect: in a structured store, an untouched block is not a faithful reproduction of itself. It's the same bytes. It cannot drift, because it never moved.
Then one save, and the server refreshes its snapshot so the model's own edit counts as "seen" — otherwise the agent's next operation would collide with its own work.
And if someone else edited that paragraph first? Say a teammate changed it to Failed deliveries are retried with exponential backoff. while the model was thinking. The present now hashes to d80a9820…; the snapshot still says 01e5960c…. Mismatch → the whole call is rejected atomically (every operation validated before any is applied — a failed call means a byte-identical document), and the error is a targeted re-read:
Error: block "zT5rW8" was modified after you read it.
Current content of this block:
Failed deliveries are retried with exponential backoff.
Re-issue the operation against this content.
~100 tokens, and the model retries against reality instead of its recollection. The same style of self-correcting error covers a hallucinated block id (the error returns the real outline: every current handle plus the first 60 characters of its block). The failure mode is always "no edit — try again, informed", never "wrong edit". This is per-block optimistic concurrency, the same idea as HTTP's ETag/If-Match, applied per paragraph: a teammate editing block 12 never blocks the agent's edit of block 3.
The second turn: what the agent receives tomorrow
Next day, same session, you ask: "actually, make that paragraph mention the backoff delay too."
Look at what the model's input actually is tomorrow:
[replayed history]
turn 1: the FULL anchored document ← still here, re-sent verbatim
turn 1: the model's own edit_blocks call ← its record of what it changed
turn 1: "Edited: replaced block zT5rW8"
[fresh]
[Document unchanged since your last read] ← the delta line your new message
The model re-reads yesterday's full document every turn — that's what replaying history means. The one-line marker isn't the document. It's a certificate: "that copy of the document sitting earlier in your transcript, plus your own edits — still accurate. Trust it."
That reframing answers the cost question too, which is the part that makes the design clever rather than just correct. The full document is being paid for every turn — but in the input lane, as part of a byte-stable replayed prefix, which is exactly the lane that's cheap and cache-discounted.
This is a fresh HTTP request. The server replays the stored conversation — and here the doc snapshot earns its keep a second time. Instead of stapling a fresh full copy of the document onto the transcript (what we used to do, every turn), the server re-renders the current blocks, diffs their hashes against the snapshot, and injects one of three things:
nothing changed → one line:
[Document unchanged since your last read]
a few blocks changed → only those blocks, with their anchors:
--- Changed blocks ---
<!-- block:zT5rW8 -->
Failed deliveries are retried five times.
--- End of changed blocks ---
a lot changed (>30%) → give up, re-inject the whole document
The model's picture of the document is now three layers, all cheap: the original full injection (still in the replayed history), its own edit operations (also in history — the record of what it changed), and this delta. Full context, delta price. In our runs a typical delta injection was ~220 tokens where a full re-read was thousands.
If you know rsync, this is rsync — the snapshot plays the role of the receiver's checksum list, except the "receiver" being synchronized is the model's context window.
The other lever: compact the replayed history
Block editing cuts what the model writes. The transcript it reads every turn has its own disease, and a different cure.
By turn five of a session, the replayed history was carrying: the turn-1 document injection, a fresh full injection stapled to every user message, every full-document rewrite the model ever emitted as tool arguments, and every raw search-result dump. The same document, over and over, in slightly different wrappers — we measured 71,812 input tokens re-sent at the start of a single turn.
Compaction is a pure function that runs when history is replayed. No summariser model, no AI call. It walks the stored messages and swaps known bulky payloads for one-line placeholders:
| In history | Becomes |
|---|---|
| Old document injections | (content elided — a fresh copy is in the latest message) |
| Old full-rewrite tool arguments | [document content elided — see the current document or read it again] |
| Old read-tool results | [document content elided — call read_document again if needed] |
| Old search-result dumps | [search results elided] |
Read the placeholders again — they aren't apologies, they're directions. Each one tells the model where the authoritative version now lives: the fresh injection, or a tool it can call again. That's the rule the whole elision list was built from: remove bytes only when their authoritative version still reaches the model somewhere else. The user's words, the model's own replies and decisions, tool names, error messages — never touched.
The numbers
Everything below is from A/B runs of a production agent (GPT-5.1) against the same ~7.4 KB document with the same five small, surgical edit prompts — one variable toggled per experiment, token counts read straight from per-iteration logs.
Output — the headline. The edit call itself, per turn:
| Edit | block editing | whole-doc rewrite | ratio |
|---|---|---|---|
| #1 | 162 | 1,644 | 10.1× |
| #2 | 168 | 1,529 | 9.1× |
| #3 | 83 | 1,540 | 18.6× |
| #4 | 145 | 1,544 | 10.6× |
| #5 | 80 | 1,556 | 19.5× |
Session totals: 775 vs 8,128 output tokens — ~10× less. And the saving is flat: the rewrite pays full document price for a one-word change on turn five exactly as on turn one. There is no session length at which it catches up.
Speed. Median edit latency 5.3 s vs 16.9 s (~3.2×); averages 6.8 s vs 16.7 s. Generating ~1,550 tokens takes real wall-clock time; generating ~130 doesn't. This is the difference users feel on every single edit.
Input, as a side effect. Because block editing never deposits a full-document rewrite into the history, the replayed transcript grows ~294 tokens/turn instead of ~3,176 — 11× slower. Over five edits: 78,034 vs 180,032 input tokens (−57%) — without compaction even enabled.
Compaction, measured separately (block editing off in both runs, so the only variable is compaction):
| Follow-up | compacted input | uncompacted input |
|---|---|---|
| #1 | 14,790 | 15,063 |
| #3 | 17,488 | 43,716 |
| #5 | 20,880 | 71,812 |
Turn one is a tie — nothing to elide yet. By turn five, −71%; across the session, −60%, and the gap widens every turn. With compaction on, the transcript grows roughly as fast as the document itself; without it, by about a whole extra document copy per turn.
Correctness — the result that surprised me. Both modes applied 5/5 edits correctly, including two deliberate traps: a decoy (the target phrase appeared twice; only the asked-for instance was to change) and a two-mention update ("change every place it says 24 hours"). So I won't tell you block editing is more accurate — on well-specified edits it measurably isn't. The honest claim is different: it's equally correct while re-generating ~60× less text per edit, which means a far smaller surface for silent drift, and a ~3× shorter window in which a dropped connection can kill a half-finished rewrite. Same outcome, smaller blast radius.
How the two levers stack. They overlap on input (both attack the same history bloat from different ends), so the savings don't add — but they own different lanes, and the lanes are clean:
block editing owns OUTPUT (~10× less, ~3× faster — compaction can't touch output)
compaction owns INPUT (~60% less replayed history, compounding with session length)
One honest asterisk on compaction: rewriting the transcript prefix disturbs prompt-cache locality, so its effect on provider-side cost depends on your traffic pattern (with warm caches it's roughly neutral; with cold caches it wins outright). Its effect on raw token counts — which is also what usage quotas typically meter — is the clean ~60%.
Prior art, honestly
I arrived at this independently, and then discovered — as you should assume about any good idea under economic pressure — that the field was converging on the pieces. The pricing asymmetry (expensive uncached output, cheap cached input) pushes everyone toward the same shape:
Aider's search/replace edit formats attack the same retyping problem for code: emit only the diff, never the who le file (their unified-diffs write-up is the classic argument against whole-file rewrites).
antirez sketched line edits carrying a small content checksum, checked before apply — verify-before-write at line granularity, pos itional addressing.
OMP's "hashline" edits give every line a content-hash the model references instead of r etyping — content addressing, with the hashes travelling through the model's context.
Anthropic's context editing clears stale tool results from the transcript and leaves placeholders — the same family as our compaction.
Notion's API exposes block-id-targeted updates — stable ids on blocks, commercially at scale.
What I haven't seen assembled anywhere public is the composition this post describes, and each piece of the composition does specific work: identity addressing rather than position or content (addresses that survive both movement and editing); fingerprints of the rendered view rather than the storage bytes (staleness defined as "would the model see different text", eliminating false conflicts); fingerprints held server-side rather than routed through the model (no per-line token tax, nothing for the model to mangle); the persisted snapshot doing double duty — guarding writes and powering the delta re-injection that keeps follow-up turns cheap; and, available only because the document store is ours, untouched blocks that are never serialized at all. The deepest ancestry is older than all of us: per-block ETag/If-Match, git's content hashing, rsync's checksum lists — pointed at a context window instead of a cache or a network link.
If you're building an agent over any block-structured store — a CMS, a notes product, a site builder, an email composer — that composition transfers whole.
The honest limits
Input still scales with document size. The model reads everything, every session. Deliberate: input is the cheap, cacheable lane, and full context is what keeps edits grounded. This method saves where it's expensive and spends where it's safe.
One giant block is still one giant block. replace re-emits the whole target block, so nothing is saved inside a 20k-token table. If your users build monster blocks, you'll eventually want sub-block addressing.
Cross-block renames are the wrong job for it. An 18-occurrence rename cost 855 output tokens in our runs, because the model re-emitted every affected block. A plain server-side find-and-replace tool does it for ~40.
The concurrency is optimistic and per-block, not a global transaction. It catches the overwhelmingly common conflicts — and its failure mode is refusal, not corruption — but if you need multi-writer guarantees across blocks, that's CRDT territory, a different (and far heavier) machine.
Blocks without ids degrade the experience, not the safety. Content+position fallback handles keep edits sound but turn "block changed" conflicts into blunt "block not found" errors and make the delta optimization give up more often.
If you want to build this
The checklist is short:
You own a structured document store — a tree of typed blocks, not opaque text files.
Every block carries a stable id. If your editor already assigns ids for deep links or drag-and-drop, this is free. This is the prerequisite everything else stands on.
One deterministic per-block markdown renderer, used at read time and check time — the hashes are only comparable because both come from the same function.
Somewhere to persist the per-session snapshot — a JSON column next to your conversation history is plenty; ~50 bytes per block.
Errors that teach. Every rejection should hand the model the current truth it was missing — the block's present content, the real outline. The error channel is your sync channel.
Then the flow is the one you've just read:
The read hands the model addresses and keeps the fingerprints.
The edit names an address, and the fingerprints judge it — past against present — before anything moves.
The follow-up diffs the fingerprints, so the model's picture stays current at delta price.
The model reads everything, addresses one block, and retypes nothing else. That single change moved our bill's most expensive line item by 10× — and made the agent faster and less destructive at the same time. The rare alignment of cheaper, faster, and safer is usually a sign the previous design was paying for something nobody needed. Re-typed text nobody asked for was exactly that.
