What do the baseline and serving tests establish?

6. What is the TokenDance-style baseline?

The controlled workload has five synthetic 512-token prompts containing the same content at different absolute positions. One request primes LMCache; four siblings then recover 416 tokens each. That is 81.25% realized sharing after segment rounding from the nominal 80% generator setting. There are no matching prompt-aligned full blocks, so ordinary prefix reuse cannot explain the recovery.

This is our TokenDance-style implementation built on LMCache's CacheBlend feature. Our fork extends LMCBlender.blend() with explicit shared segments, position adjustment, request-owned execution state and required recalculation of missing positions. CacheBlend supplies selective recalculation of reused state. The serial comparison already includes our extensions; it uses our reproduction rather than the TokenDance authors' code.

The vLLM and LMCache branches execute collective position-independent recovery: they adjust retrieved keys for each sibling's positions and batch rotary-position and difference-analysis calls across compatible requests. Selection masks, attention and destination slots remain request-local. A separate Master–Mirror codec stores one full cache and exact differences for the other family members. Its encoded-byte measurement runs offline on captured tensors.

The independent verifier reloads the raw tensors and checks their hashes, K/V bits, logits, masks, operation ledgers and cleanup. A successful launcher exit or a self-reported driver flag does not establish a result.

7. How can the baseline be reproduced?

The current CUDA profile in the reproducer README runs the eight-case recovery experiment on one H100 with BF16, eager execution and Triton attention. The vLLM and LMCache branches include the current integration work; this particular control uses LocalCPU storage to isolate recovery correctness. It does not exercise DMA-BUF transfers.

Both branches are named 2026-10-16-feature-kmesh-current-validation. The profile pins source trees, the H100 runtime and model revision. Follow the README's image and compiler preparation before running:

bash tools/reproduce/tokendance/prepare_cuda.sh /workspace/kmesh-runtime
. /workspace/kmesh-runtime/run.env
make defconfig-tokendance-cuda
make tokendance-plan
TOKENDANCE_OUTPUT=/workspace/tokendance-h100 make tokendance-gpu

The model is Qwen/Qwen2.5-0.5B-Instruct at revision 7ae557604adf67be50417f59c2c2f167def9a775. Set source and model paths explicitly and use a fresh output directory. The launcher checks contracts and resolved configuration, runs each case in a fresh process, and writes raw tensors, semantic ledgers, an independent verification and offline codec results. Instrumentation makes these capture times unsuitable for speed comparisons.

Validate the workload or recheck a capture on CPU

make defconfig-tokendance-cpu
make tokendance-plan
make
make tokendance-test

This CPU profile validates the synthetic workload and reports executed_kv_reuse: false; it executes no model. To independently verify a current GPU capture, use Python with PyTorch and safetensors:

TOKENDANCE_BUNDLE=/path/to/eight-case-capture \
TOKENDANCE_SOURCE_PAIR=current \
TOKENDANCE_OUTPUT=/path/to/fresh-verification \
make tokendance-verify TOKENDANCE_PY=/path/to/torch-env/bin/python

Inspect verified.json for recovery checks and codec/codec.json for actual dense and encoded bytes. The GPU run's offline codec writes and reloads the family, requires bitwise restoration, and rejects a deliberately corrupted payload. These are correctness and storage-size measurements, not online GPU capacity or transfer results.

8. What do the baseline measurements establish?

H100 recovery experiment, October 16, 2026: all eight cases and the independent verifier pass on the component pins above. Serial and collective K/V match bit for bit, with zero maximum logit difference. The offline codec restores all four members exactly and rejects corruption. Full recomputation and recovery are not numerically equivalent. The H100 result record binds these findings to the exact sources and verifier digest.

RTX 4090 reference, October 23, 2026: the same eight model and control arms and independent verifier also pass on LMCache daa0b4ddc8551d29d21ff4ac51f913eb29dfed9a and vLLM 9df1493cae0b98380a3c3ecb9b2d41776a6d8735. Serial and collective K/V remain bitwise identical, with zero maximum logit difference. Full recomputation versus serial recovery differs by up to 0.4921875; its numerical diagnostic still fails. The offline codec again produces 17,506,917 bytes from 25,165,824 dense bytes, restores every bit and rejects corruption. This is an in-process LocalCPU model reference for the current branches. It does not exercise the new multiprocess span service or DMA-BUF model recovery. Use defconfig-tokendance-rtx4090 and the pinned RTX instructions; the earlier H100 profile and its measurements remain separate.

The dense byte count is four siblings × two K/V planes × 512 tokens × 24 layers × 128 features × two BF16 bytes. Shared spans align by content before encoding. The current CUDA run includes offline codec measurement on the captured tensors. It does not measure how many GPU pages an online engine can share.

The collective ledger records 24 cached-key recovery operations, 24 model rotary operations and one difference-analysis operation per participating member. Those are participation records for shared calls, not separate calls to sum across members. They demonstrate batching; they do not demonstrate constant work or a latency improvement.

9. Do reasoning forks need a new memory manager?

The next question after the recovery baseline is simpler: could explicit tree ownership save KV-cache blocks during test-time scaling? Here a child is its parent's exact token sequence plus an independent tail. A common suffix after different private histories is a different causal context and cannot establish exact sharing.

Measured on 2026-10-16: stock vLLM automatic prefix caching already reaches the fork-oracle block count in the tested trees while the ancestor remains cached. The experiment drives the real KVCacheManager on CPU and records actual block IDs. It treats allocation as completed prefill, without a model, scheduler, KV tensors or GPU. The matrix covers five tree shapes, 25 scenarios and six eviction points. The serving replay below checks the real model and scheduler separately.

The primary tree has a 4,096-token ancestor and five 512-token tails, using 16-token blocks. Sharing needs 256 ancestor blocks + 5 × 32 tail blocks = 416. Independent root-to-leaf copies write 256 + 5 × 288 = 1,696 blocks over the tree lifetime. Written blocks accumulate simulated prefill work; peak pool usage measures allocator metadata at one instant, including the reserved null block. The parent is freed before its children arrive. Four failed admissions explain the smaller write count in the uncached pressure control.

Non-aligned forks also reach their block oracle. A 4,101-token ancestor with five 509-token tails writes 422 blocks but recomputes five ancestor boundary tokens per child, 25 total. An explicit fork could clone that partial block; the tested capacity is the same. Tests check that full ancestor block IDs alias while sibling writable tails remain private.

Eviction exposes a separate cost. After unrelated traffic churns a 1,024-block pool, churn fractions 0, 0.25 and 0.50 preserve all 4,096 ancestor tokens for all five children. At 0.75, the first child hits 4,080 tokens and the next four hit 4,096. At 1.00 and 1.50, the first child misses the ancestor entirely and the next four hit fully. In this metadata admission sequence, the first child's simulated prefill repopulates the lost blocks. This does not establish a universal bound under interleaved traffic or preemption.

The current capture uses the serving branch and CUDA software environment with GPU visibility disabled. All 25 scenarios pass, and all 174 prefix-cache tests pass without skips. The allocator capture records the observed block IDs and accounting.

Reproduce this allocator check

Follow the current allocator instructions. Use vLLM 1b76bf9dd9d52efbd75fb89bf2619898d8f01233 from the serving branch. This adds the benchmark and tests while using the production memory manager. LMCache is not involved in the allocator check.

make defconfig-kmesh-fork-accounting-current
make kmesh-fork-plan
make kmesh-fork-test

export KMESH_FORK_VLLM_SOURCE=/absolute/path/to/pinned-vllm
export KMESH_FORK_PYTHON=/absolute/path/to/prepared-venv/bin/python
export KMESH_FORK_WHEEL_PACKAGE=/absolute/path/to/site-packages/vllm
export KMESH_FORK_OUTPUT=/path/to/fresh-fork-results
make
KMESH_FORK_OUTPUT=/path/to/fresh-contract-results make kmesh-fork-contracts

The pinned environment uses Python 3.13.15, PyTorch 2.13.0+cu130 and matching vLLM native extensions. The runner checks source identity, imports and runtime versions, hides GPUs and verifies the output independently. The workload uses seed 20260805 and two OpenMP threads.

10. Does sharing survive real serving?

Yes, for this exact-prefix fork tree. The H100 serving experiment runs seven cases on the current vLLM branch. All cases and the independent block-table verifier pass. A warm 4,096-token ancestor is physically shared by all five children; their writable tails remain separate. This tests real model execution and scheduler admission, complementing the CPU allocator check.

What creates the shared ancestor?

vLLM's automatic prefix caching (APC) creates the physical sharing. The reproducer submits five ordinary requests. Each contains the same 4,096 token IDs, followed by a different 512-token tail. There is no special request-fork operation or family declaration in this replay.

Within one engine, vLLM hashes each complete token block together with the preceding prefix's hash and relevant cache identity. Once its KV data has been computed, the block enters the prefix-cache index. A request with the same prefix finds that block and acquires a reference to the same physical block ID. Reference counts protect blocks while requests use them. Each divergent suffix receives separate writable blocks.

Child A block table: [shared ancestor blocks] + [private A blocks]
Child B block table: [shared ancestor blocks] + [private B blocks]
Child C block table: [shared ancestor blocks] + [private C blocks]

The harness supplies matching token prefixes; the production vLLM allocator supplies the sharing. See the pinned source for block hashing, prefix lookup and reference acquisition. Only complete cached blocks can be reused. Different earlier tokens, incompatible cache identities or eviction can prevent a hit.

This result establishes exact-prefix capacity sharing inside one engine. It does not establish cross-engine sharing or reuse of a span after a different private history. Each child can still read the shared KV data again during attention. Cascade attention addresses repeated attention reads; DMA-BUF storage restoration addresses a separate transfer path.

The warm case observes shared prefix IDs in all ten live sibling pairs. Its peak includes the null block and 20 child decode blocks beyond the CPU allocator's 417-block peak. With caching off, sibling block tables are disjoint. Both controls produce the same generated token IDs for the parent and all five children in this run.

A cold burst or full-pool churn makes one child compute the ancestor before the other four reuse it. The decode fork uses the parent's actual 128 generated tokens: the first child fills the last incomplete 16-token block, and the other two hit the complete 4,224-token prefix. These outcomes are specific to the tested workload and scheduling; they do not establish a bound under repeated interleaved eviction or preemption.

Reproduce the serving replay

Use the current CUDA preparation instructions in the baseline README, then run these commands in a GPU reservation or on an explicitly owned pod:

. /workspace/kmesh-runtime/run.env
make defconfig-kmesh-serving-cuda
make kmesh-serving-plan
KMESH_SERVING_OUTPUT=/workspace/serving-forks make
make kmesh-fork-test

The serving instructions describe the seven fresh processes and independent CPU recheck. The sanitized result binds the source, script and raw-result digests. The model is Qwen/Qwen2.5-0.5B-Instruct at the baseline's pinned revision. Runtime is Python 3.13.15, PyTorch 2.13.0+cu130, CUDA 13.0 and vLLM 0.30.1rc1.dev624+g84bcbc626, BF16 and eager execution with two OpenMP threads. Stock defaults resolve to FlashAttention, 16,384 maximum batched tokens, 1,024 maximum sequences, chunked prefill and asynchronous scheduling. The recorder checks allocation/free boundaries and also retains five-millisecond pool samples.