How often to rebuild the summary

Two systems use the same summarisation approach and behave completely differently at turn three hundred. One rebuilds its summary every few turns; the other rebuilds when the recent window fills up. The first has a character that has forgotten week one entirely. The second has a character that occasionally answers as though the last forty turns did not happen.

Cadence is not a tuning parameter. It selects which of two failures you get.

Rebuild frequency is generation count

A summary built from a previous summary loses information that the previous summary had already thinned. The degradation is generational, and the number of generations an old fact has passed through is determined entirely by how often you rebuild.

Rebuild every few turns and early history has been re-compressed dozens of times by mid-conversation. What survives is whatever kept getting selected — usually the gist, usually the things that were mentioned repeatedly, rarely the specific detail that was stated once and mattered.

Rebuild rarely and early history has passed through few generations, so what survives of it survives in better shape. The cost is on the other end: between rebuilds, the material that has not yet been summarised has to be carried in full or dropped. Carrying it means the input grows until the next rebuild. Dropping it means a live hole between the end of the summary and the start of the recent window, which produces the strangest failure in this whole area — a character with good recall of last month and none of yesterday.

The choice is between degrading old history smoothly and risking gaps in recent history. Frequent rebuilds smear the past; rare rebuilds are sharp until they are not.

Rebuilding from originals is a different operation

The generational problem exists because the input to the summariser is the previous summary. If the input is the original turns instead, the loss is one generation deep no matter how many times you rebuild.

This is the strongest available position and it has two real costs. You must retain the original turns, which is storage rather than per-turn cost and is therefore the cheap part. And each rebuild summarises the whole conversation rather than an increment, so rebuild cost rises with conversation length — a rebuild at turn eight hundred processes eight hundred turns. Take an illustrative round case: if you rebuild every fifty turns, the work you do over a thousand turns is far more than a rolling summary would have done, because each rebuild reprocesses everything before it.

The middle position is a layered one. Summarise blocks of original turns into block summaries, then combine block summaries into the carried summary, and rebuild only the top layer routinely. Any given fact passes through two generations rather than n, block summaries are reusable across rebuilds, and the arithmetic stays bounded. What it costs is a real data structure where you previously had one text field, plus the awkward question of what to do when a block summary turns out to have dropped something you now need.

There is no arrangement that reprocesses less and loses less. Reprocessing is what buys fidelity.

Trigger on state size, not on turn count

Turn-count triggers are the obvious implementation and the wrong one, because turns vary enormously in length. A hundred short exchanges and twenty long ones are not the same amount of material, and a fixed turn threshold makes rebuild timing depend on how talkative a user is.

Triggering on the size of the carried state gives you predictable input sizes, which is what the per-turn bill actually responds to. It also aligns the rebuild with the moment the state is about to become expensive, rather than at an arbitrary conversational point.

The tradeoff is legibility: a size trigger fires at unpredictable places in the conversation, sometimes mid-exchange, and a rebuild that lands between two turns of one exchange can drop context the next response needed. Keeping a fixed recent window intact regardless of rebuild timing is what makes that safe, and it is why the recent window and the summary should be sized independently.

The turn

THE TURN — rebuild cadence

  · Rebuild often, from the last summary
                    → carried state stays small and predictable;
                      cheapest per rebuild.

  · Generations accumulate
                    → early history is re-compressed dozens of
                      times and is gone by mid-conversation.

  · Rebuild rarely
                    → old history stays sharp, and unsummarised
                      recent material either inflates the input
                      or leaves a hole.

  · Rebuild from originals, in layers
                    → bounds loss to one or two generations;
                      each rebuild reprocesses more as the
                      conversation grows.

  · Either way, the summary
                    → PAID EVERY TURN, in every input, plus the
                      rebuild itself. Cadence is a cost curve,
                      not a setting.

  · No cadence recovers a dropped specific
                    → once a detail is out of every generation,
                      rebuilding cannot bring it back.

Detection

Instrument generation depth. For each summary, record how many summarisation passes the oldest material in it has been through. That single number explains most complaints about a character forgetting early history, and almost nobody records it.

Run fact-survival probes across rebuilds rather than at one moment. Plant a small set of specific facts at known points in a synthetic conversation, then check after each rebuild whether each fact is still retrievable. The curve of survival against generation depth is the measurement that tells you whether your cadence is viable, and it will show specifics dropping long before the gist does.

Watch for the recent hole directly. Compare failure rates on questions about material from the last few turns against material from the middle of the conversation. Recent failures that spike immediately before a rebuild are a trigger-timing problem, not a summarisation quality problem, and the two get confused constantly.

Track rebuild cost as its own line item, separately from turn cost. Rebuilds are bursty and easy to overlook in an average, and a from-originals strategy will make them the dominant term at long conversation lengths. If your cost model only counts turns, it is missing the part that grows fastest.

What this costs and what it doesn’t fix

Every rebuild strategy pays twice: once for the summary’s presence in every input from then on, and once for the rebuild work itself. Frequent rebuilds from the previous summary minimise both and maximise information loss. From-originals rebuilds minimise loss and grow superlinearly in total work. Layering is the compromise that keeps both bounded and costs you a data structure and some judgement about block boundaries.

Cadence does not fix the summary’s loss profile — it only controls how fast that profile is realised. A summary keeps gist and loses specifics at any cadence, and the facts a long relationship depends on are mostly specifics. If particular details must survive indefinitely, they need to be held as facts rather than left to compression, and that is a different mechanism with its own failures rather than a better version of this one.