Chapter 04 · Section I · 9 min read
4.1 What state needs to persist between cycles
Not everything a cycle produces needs to survive to the next one. The question is which slice of state carries forward, and which is thrown away when the cycle ends.
Without memory, every run of a loop starts from zero. It reads the same reusable knowledge, does the work, produces output, and forgets. That’s fine for cycles that are genuinely independent — extract fields from this invoice, translate this string. It’s a disaster for anything that has to build on yesterday’s decisions, avoid repeating yesterday’s mistakes, or roll up a week of activity.
Three tiers of state per cycle
Every cycle produces state in three tiers:
Ephemeral (throw away). Scratch work, intermediate reasoning, draft output that got replaced by a better draft. Nobody ever needs this again. When the cycle ends, delete.
Cycle output (usually consumed). The finished product of this cycle — the summary you emailed, the ticket you tagged, the file you wrote. Sometimes consumed immediately (the email got sent) and sometimes archived (kept for audit).
Cross-cycle memory (persist). The state that tomorrow’s cycle needs to see: what got processed today (so tomorrow doesn’t re-process it), what was decided (so tomorrow doesn’t contradict it), what needs follow-up (so tomorrow picks it up).
The third tier is memory. The first two aren’t.
What genuinely belongs in memory
Ask this question: if this cycle vanished from my logs, what would tomorrow’s cycle do wrong?
Concrete answers:
- Which items are already processed. Otherwise tomorrow re-does today’s work, or worse, sends yesterday’s email again.
- Running totals and rollups. Weekly metrics need to know what Monday’s cycle counted before Tuesday adds to it.
- Decisions with follow-up. “Escalated to human — waiting on response by Friday” needs to survive to Friday.
- Learned rules that turned out to be wrong. “Never auto-tag messages containing ‘urgent’ as low-priority — this failed on Tuesday.” Otherwise the loop keeps making the same mistake.
- A rolling summary of context. For long-running loops, a short “state of the world” summary that captures what’s happening this week, updated each cycle.
What emphatically does not belong in memory
- The full transcript of every prior cycle. Memory should be summaries and facts, not a raw log. A month of full transcripts is unaffordable to load into every future cycle.
- The model’s chain-of-thought. Reasoning traces from previous cycles are noise for the current cycle. Store the conclusion, not the deliberation.
- Copies of the reusable knowledge. That’s already in reusable knowledge (Chapter 3.2). Don’t duplicate.
- Sensitive data that doesn’t need to persist. Passwords, PII, one-time tokens. Even if the cycle needed to see them, don’t archive them.
The append-only pattern
The simplest useful memory is append-only. Each cycle writes a short entry:
{
"cycle_id": "triage-2026-07-22",
"timestamp": "2026-07-22T06:00:00Z",
"processed_ticket_ids": [1201, 1204, 1207, 1209],
"tickets_escalated": [1204],
"notes": "Confidence was low on 1204 due to unusual phrasing; verifier rejected auto-tag; escalated."
}Tomorrow’s cycle reads yesterday’s entries, sees which tickets were already handled, sees what escalated, and picks up from there. Old entries eventually get archived or garbage-collected, but the append-only nature makes it easy to reconstruct history when something goes wrong.
The problem with letting the model manage memory
Some loops give the model a “write to memory” tool and let it decide what to store. Sometimes this works. More often, the model writes a paragraph of narrative each cycle, and by cycle 200 the memory is a garbled, redundant essay nobody can use.
Better: your code decides what fields memory has. The model can populate those fields; it can’t invent new ones. Structured memory (JSON with a known schema) beats free-form memory in almost every long-running loop.
Where memory lives
Three common shapes:
- Files. For small, simple loops. A JSON file, a JSONL log, a Markdown doc updated per cycle. Cheap; works everywhere; hard to query if the loop runs for a year.
- A key-value store or small database. For loops with real query needs. SQLite is often plenty. Postgres if you have infrastructure. The schema forces you to think about what to store.
- A dedicated memory service. For loops embedded in larger systems, or when multiple loops share state. Overkill for most; right for a few.
Most loops start with a JSON file, graduate to SQLite when queries get annoying, and reach for anything bigger only when the volume demands it.
The morning question
The morning-after question — “what did the loop do last night?” — should have a single answer that reads out of memory in under a minute. If your memory design can’t answer that quickly, redesign it. Memory that can’t be inspected on demand isn’t memory; it’s a black box.
Check your understanding
Quick check
—