Record delivery · LongHealth · November 2026

Memorilla and Cartridges on clinical records.

Two ways to give a frozen model a long record without pasting it in: a shared reader that turns passage vectors into 16 memory tokens, and a key/value prefix trained for each record. On LongHealth the trained prefix wins clearly, and plain retrieval is the method both have to beat.

Memorilla (arXiv 2610.21375) trains one reader module that reads any document collection without further training. Cartridges (arXiv 2506.06266) train a key/value (KV) cache prefix per corpus with self-study. Neither paper runs the other. This page compares both, and retrieval, with the same frozen Qwen3-8B, the same prompts and seeds, and one vLLM engine. The context artifacts page covers how knlp stores and serves Memorilla collections; this page covers answer quality.

Same decoder, engine, prompts and seeds in every armMemorilla fine-tuned with the authors’ trainerRetrieval matched to cartridge memoryFour patients, 80 questions

Result: Memorilla does not read these records usefully. The released reader scores like no record (0.425 against 0.413). Fine-tuned on the cartridges’ own training conversations, it reaches 0.446 without thinking and 0.421 with thinking. A native cartridge scores 0.571 and 0.700. Four times more Memorilla training changes nothing, and giving the reader only the five right passages changes nothing. At the cartridge’s KV memory, retrieval matches the cartridge without thinking and trails it by 0.08 to 0.13 with thinking. See the results.

01 / The two methods

What each method gives the decoder.

Memorilla

A 194M-parameter reader module (two cross-attention layers, 16 memory slots) sits beside a frozen decoder. Each passage of the record is stored as one Qwen3-Embedding-4B vector of 2,560 numbers. For each question, the reader attends from 16 learned, question-conditioned queries to those vectors and writes 16 soft tokens into the system turn. The decoder never sees passage text. One trained module serves every collection, so a new record costs only its embeddings.

Cartridge

A trained KV prefix for one record: one key and one value per layer and head for each of 506 to 644 positions per patient here (75 to 95 MB in BF16). It is trained with self-study: a model writes questions about the record, the decoder answers them with the full record in context, and the prefix learns to reproduce those answers without the record. Training costs about 112 H100-minutes per patient. The prefix is computed once and shared by every question about that record.

Retrieval is the third arm. It pastes the passages most similar to the question as text, using the same Qwen3-Embedding-4B vectors, the question alone as the query, cosine similarity, and most similar first. This is the RAG-k baseline from the Memorilla paper. Retrieval needs no training, but every question pays its own prefill.

02 / Setup

Keep everything equal except the record’s path.

LongHealth is a benchmark of 20 fictional patient records, each with 20 five-option questions. Each record is about 10,000 to 13,000 tokens. The panel uses patients 01, 03, 05 and 06, three sampling runs per question, and one fixed sampling seed per request, shared by every arm. Prefix caching is off. All arms run on one Qwen3-8B in one vLLM v0.33 build with a cartridge KV connector, on an RTX 4090.

There are two answer protocols. The panel protocol has thinking on and a 4,096-token answer cap. Every cartridge number in knlp uses it. The no-thinking protocol has a 1,024-token cap and matches the chat template Memorilla was trained with. Scoring is letter-aware: a bare option letter counts as that option. This matters for Memorilla, which often answers with a bare letter, and moves no other arm by more than 0.016. One panel’s sampling error is about 0.03 for a four-patient mean and 0.06 for one patient.

Memorilla collections split each record into passages of at most 256 reader tokens (about 50 per patient), embedded with the authors’ builder. Re-embedding PubMedQA with this builder reproduces the authors’ vectors (cosine at least 0.99979). The Memorilla prompt matches the authors’ build_messages token for token.

03 / Results

A trained prefix carries the record; 16 memory tokens do not.

ArmKV memory per questionNo thinkingThinking
No record00.4130.408
Memorilla, released reader16 tokens0.4250.200
Memorilla, fine-tuned on another patient16 tokens0.4330.400
Memorilla, fine-tuned on these patients, 2 epochs16 tokens0.4460.438
Memorilla, fine-tuned on these patients, 8 epochs16 tokens0.4460.421
Native cartridge506 to 644 slots0.5710.700
Retrieval, 5 passages of 256 tokens1,162 tokens0.662–0.6750.725–0.729
Full recordwhole record0.8120.879

Means over patients 01, 03, 05 and 06. The retrieval row shows two runs of the same arm. The released reader’s thinking score is off-template, because it was trained with thinking disabled; read its no-thinking score. The authors’ own evaluator and choice scoring, on all 20 patients, give the released reader 0.258 to 0.280 (chance is 0.20).

LongHealth accuracy against KV memory per question The fine-tuned Memorilla reader at 16 memory tokens scores 0.446 without and 0.421 with thinking. Retrieval rises from 0.59 at 233 tokens to 0.78 at 1,351 tokens. The native cartridge at 506 to 644 slots scores 0.571 without thinking and 0.700 with thinking. No record scores 0.41. 0.4 0.5 0.6 0.7 0.8 10 30 100 300 1,000 2,000 KV memory per question, in tokens (log scale; one token = one cartridge slot = 144 KiB) Panel accuracy No record, 0.41 Memorilla Cartridge Retrieval

Accuracy against the KV memory each method adds per question. For Qwen3-8B one token of context and one cartridge slot both cost 36 layers × 8 KV heads × 128 dimensions × 2 (key and value) × 2 bytes = 147,456 bytes. The full record (about 10,000 to 13,000 tokens) is off the right edge. The released reader (0.425 / 0.200) is omitted; it would sit on top of the fine-tuned points.

04 / Fair to Memorilla?

Fine-tuning, longer training and more compute do not close the gap.

The first comparison used the released reader, which never saw a clinical record. Each cartridge, by contrast, was trained on its patient’s self-study conversations. Every Memorilla score in its paper also comes from a reader trained on that benchmark’s training split. To remove this difference, the released stage-3 reader was fine-tuned on the same self-study conversations the cartridges used. The run used the authors’ train.py and recipe unchanged: AdamW at 1e-4, weight decay 0.01, 10% warmup then cosine decay to 10% of peak, batch 1 with gradient accumulation 16, loss on answer tokens only, and thinking disabled. The data are about 13,800 training rows for the four patients. A transfer control was trained only on patient 02 and scored on the panel patients.

Fine-tuning adds 0.02 to 0.03 over the released reader, and the transfer control gets most of that gain without seeing the panel patients. So most of what fine-tuning teaches is answer format, not these patients’ facts. On held-out patients 02, 04, 07 and 08, every fine-tuned checkpoint scores 0.358 to 0.383, against 0.400 with no record and 0.404 for the released reader.

Training length

A falling training loss cannot show whether the reader learns record facts or only answer style. A swap test separates the two. It scores 752 held-out self-study conversations twice per checkpoint: once with each conversation’s own patient collection, and once with another patient’s collection. The gap between the two losses is the part the right record explains.

CheckpointOwn recordWrong recordGapPanel, no thinking / thinking
Released1.0971.1170.0200.425 / 0.200
Epoch 1 of 80.7200.8760.1560.421 / 0.433
Epoch 20.6750.8810.2060.442 / 0.354
Epoch 40.6110.9360.3250.450 / 0.421
Epoch 60.5870.9560.3690.454 / 0.421
Epoch 80.5850.9650.3790.446 / 0.421

Record use nearly doubles from epoch 2 to epoch 6, and the wrong-record loss rises, so the reader depends on the record more and more. The panel does not follow. The rule was set before the run: training length is a limit if the best epoch beats the transfer control by 0.06 or more, and it is ruled out if the best epoch stays within 0.03 of the 2-epoch reader. The best epoch beats the control by 0.021 without thinking and 0.033 with thinking, and stays within 0.008 of the 2-epoch reader. Training length is ruled out.

Compute

Both methods keep the decoder frozen and backpropagate only to the decoder inputs, so a processed token costs about the same, roughly 4 × 8.2e9 FLOPs. The GPU type changes only wall time. Compare FLOPs and passes over the data, not GPU models.

Cartridges, four patientsMemorilla, 2 epochsMemorilla, 8 epochs
Trained parameters37.3M to 47.5M per patient194M, shared194M, shared
Passes over the data12.8 to 18.828
Training FLOPs8.3e184.0e171.6e18
Measured wall time7.5 H100-hours3.5 W7900-hours14 W7900-hours
Stored result75 to 95 MB per patient371 MB for all records371 MB
Panel, no thinking / thinking0.571 / 0.7000.446 / 0.4380.446 / 0.421

The 8-epoch run uses about a fifth of the cartridges’ FLOPs. Matching them would take about 43 epochs. The data argue against that run: four times more Memorilla training did not raise the panel score, and both the swap gap and the loss stop moving after epoch 6. Cartridges also saturate early: the patient-06 native cartridge scores 0.717 at step 120 and at step 240. Two costs are not counted: the authors’ pretraining of the released module, a fixed cost shared by every collection, and the shared self-study data generation.

Is the test fair?

The released reader on new records is exactly Memorilla’s design point, so that test is fair. Training on another patient adds in-domain data, which is generous. Training on the scored patients adds per-record training that Memorilla was built not to need, so it tilts the test toward Memorilla. Memorilla trails in all three settings. The failure falls inside a limit its paper states in Section 7: Memorilla “can lose fine-grained factual details,” likely “because the document encoder was never trained for memory compression.” Its paper evaluates personalization, long-document and knowledge question answering, and agent memory. It names health records as a possible collection but evaluates no clinical benchmark. Memorilla is about cost: at zero training per record, its competitor is retrieval, which its own paper runs and which wins here by about 0.25.

05 / Retrieval at equal memory

Retrieval matches a cartridge without thinking and trails it with thinking.

Neither paper compares retrieval with cartridges. The Cartridges paper compares methods as accuracy against KV memory, with no retrieval baseline. The Memorilla paper runs retrieval at a fixed passage count, mainly 50 passages, without matching memory. This test combines the two. It uses Memorilla’s retriever and places each arm on the Cartridges memory axis, using the retrieved token count measured per question. A stronger retriever could only raise retrieval’s score, so these numbers are a lower bound for retrieval.

ArmTokens, mean (max)No thinkingThinking
1 passage of 256233 (254)0.5920.600
1 passage of 512451 (506)0.6000.567
2 passages of 256457 (505)0.6370.604
5 passages of 128574 (616)0.6330.621
Native cartridge506 to 644 slots0.5710.700
10 passages of 1281,139 (1,211)0.7040.725
5 passages of 2561,162 (1,239)0.6620.729
3 passages of 5121,351 (1,503)0.7380.779

The decision rule was set before the run. If retrieval at a cartridge’s memory scores at or above the native cartridge, it matches trained cartridges with no training. If it scores more than 0.06 below, the cartridge earns its training cost. Without thinking, the three memory-matched arms score 0.03 to 0.07 above the cartridge. This is one to two sampling errors, so the claim is a match, not a win, and the margin comes mostly from patient 06 (cartridge 0.300, retrieval 0.68 to 0.82). With thinking, the matched arms score 0.08 to 0.13 below, and the cartridge leads on patients 01, 03 and 05. To reach the cartridge with thinking, retrieval needs about twice its memory.

The two methods do not cost the same, and the table does not measure total cost. A cartridge costs training once per record, then one stored prefix serves every question about it. Retrieval needs no training, but it prefills about 450 to 600 new tokens per question.

Memorilla on the right five passages

If Memorilla failed because it reads about 50 passages at once, giving it only the five passages retrieval pastes as text should help. It does not. On the passages that score 0.662 and 0.729 as text, the released reader scores 0.417 and 0.179, and the 8-epoch reader scores 0.433 and 0.429. Each is within 0.02 of the same reader on the whole collection. Five passages into 16 slots means the slots were not full, so adding slots would not help either. The loss is in what a retrieval embedding keeps or in how the module reads it. This test cannot separate those two. Doing so would need a reader that sees passage tokens, which is a different method.

06 / Why thinking helps a cartridge

Not longer answers: a short chain of reasoning finds the right fact.

Thinking raises the native cartridge by 0.13 and does not raise retrieval at the same memory. The cause is not answer length. With thinking, the cartridge’s median answer is 507 tokens and 32% of answers exceed 1,024 tokens. Memory-matched retrieval has a median of 1,083 to 1,349 tokens, and 52% to 57% exceed 1,024. For retrieval, long answers are mostly wrong: they score 0.34 to 0.38, against 0.77 to 0.92 for answers of 1,024 tokens or fewer. A long retrieval answer usually means the passages lack the fact.

The cartridge’s gain sits in a few questions. Thinking turns 14 of 80 questions from wrong to right for the cartridge, and 4 from right to wrong. For retrieval it gains 4 and loses 7 to 9. On those 14 questions:

ArmNo thinkingThinking
Native cartridge0.020.86
No record0.290.45
Retrieval, 5 passages of 1280.740.67
Full record0.74—

Without thinking, the cartridge does worse than no record on these questions. It gives the same wrong option in all three runs, including three questions the bare model answers correctly. With thinking it answers them. Seven of the 14 are patient-06 questions, which is why patient 06 moves from 0.300 to 0.683. The bare model also answers 4 of the 14 when it thinks, so those may come from general medical knowledge. The other 10 need the record. In one inspected case, the wrong option was a nearby fact from the same record: the original tumor’s size instead of the metastasis’s. One reading is that a cartridge stores related facts without sharp boundaries, and reasoning separates them. It rests on a few examples and is not tested. The practical rule is to score cartridges with thinking on: without it they understate what they hold, and they can give confident wrong answers on questions the bare model gets right.

07 / Limits

What these results do not show.

Four patients and 20 questions each, with one decoder. Each cartridge is one training seed. Earlier panels moved by up to about 0.17 per patient across seeds, and the cartridge side of the memory curve has one size per patient. Memorilla was fine-tuned with one seed and one learning rate. The authors’ cosine schedule was stretched over 8 epochs, because their trainer has no constant-rate option and their code was not changed. The retrieval arms use one retriever and the question alone as the query. The swap test measures record use on self-study conversations, not on panel questions. None of this says Memorilla fails in general: its paper reports wins on FactKG, TriviaQA, NarrativeQA and personalization, where joining weak clues across many documents matters more than recalling one exact value.

08 / Run it

Scripts and inputs.

All scoring goes through cas_compare_vllm.py. It runs one arm per invocation (ARM=none|full|rag|cart|memorilla), uses the panel prompts and scoring from cas_eval_table15.py, and writes per-patient summaries plus every answer. TOPK_DOCS sets the retrieval passage count, MEM_CONFIG the passage size, MEM_TOPK restricts Memorilla to the top passages, and THINK and MAX_COMPLETION select the protocol. Each run records the retrieved token counts.

# Retrieval: two passages of 256 tokens, no thinking
ARM=rag THINK=0 MAX_COMPLETION=1024 MEM_CONFIG=longhealth_c256 TOPK_DOCS=2 \
  MEM_SRC=/path/to/memorilla MEM_ROOT=/path/to/longhealth-collections \
  RUNS=3 MAX_Q=20 SEED=0 OUT_JSON=rag_c256_k2.json \
  python research/cartridges_cas/scripts/cas_compare_vllm.py

# Memorilla reading only the top five passages, thinking on
ARM=memorilla THINK=1 MAX_COMPLETION=4096 MEM_CONFIG=longhealth_c256 MEM_TOPK=5 \
  MEM_CKPT=/path/to/reader MEM_SRC=/path/to/memorilla MEM_ROOT=/path/to/longhealth-collections \
  RUNS=3 MAX_Q=20 SEED=0 OUT_JSON=mem_top5.json \
  python research/cartridges_cas/scripts/cas_compare_vllm.py

cas_memorilla_build.py builds the LongHealth collections and converts self-study conversations into Memorilla training data. cas_memorilla_swap_loss.py runs the own-against-wrong record test. Cartridges are trained with cas_train_isolated.py from conversations made by the cas_synth_* scripts. The Memorilla reader is the authors’ released checkpoint, fine-tuned with their training code.

Not yet public: the cartridge arm needs a vLLM v0.33 branch with the cartridge KV connector, which is not published yet. The other arms use only text prompts and prompt embeddings. The self-study conversations and trained cartridges are not published.

09 / Primary sources

Read the methods.

  1. Memorilla: Latent Semantic Memory for LLMs, v1. Section 2 covers the reader and frozen-decoder interface, Appendix A the related work that cites Cartridges, Appendix E the memory-slot ablations, and Section 7 the limits. Author code.
  2. Cartridges: Lightweight and general-purpose long context representations via self-study, v3. Section 5.1 and Figure 4 report LongHealth as accuracy against KV-cache memory.
  3. LongHealth benchmark: 20 fictional patient records, 400 multiple-choice questions.
  4. Cartridges in knlp and context artifacts and Memorilla serving in knlp.