Prefill / decode through storage

The handoff is an ownership protocol

A prefill HTTP response alone does not prove storage is ready. A successful decoder response alone does not prove restoration: an ordinary cache miss may silently recompute. Native completions, cold-reader evidence, exact continuation and a deliberately failed-load control are required.

A failed or uncertain claimed read retains its lease. Lease expiry, client cancellation and a waiting-thread timeout do not prove that a device or reader stopped. Unclaimed publications have a distinct exact unread-release operation.

Why put the storage support in LMCache?

This is a division of responsibility, not a claim that vLLM should have no P/D support. LMCache integrates through vLLM's connector interface. The separate NIXL storage connector legitimately lives in vLLM because it supplies a direct NIXL path without LMCache. Its GPU qualification remains separate.

See vLLM's connector architecture and LMCache's P/D documentation . The latter legacy page redirects readers toward MP deployment; do not assume the two integration modes have identical controls.

Two LMCache integration modes

1. writer POST /storage-pd/prepare   # before model stores
2. run prefill; retain the first output token
3. writer POST /storage-pd/publish   # receive completed READY receipt
4. writer POST /storage-pd/claim     # exact reader/session identity
5. reader POST /storage-pd/adopt     # namespace, epoch, manifest, layout
6. run decode with producer token appended
7. writer POST /storage-pd/ack       # after successful restore/consumption

For a strict MP consumer, set lmcache.mp.strict_prefix=true , kv_role=kv_consumer and kv_load_failure_policy=fail . The fail policy alone is insufficient when prefetch fails before an external prefix is advertised. Strict mode requires all complete prompt chunks before the last prompt token and routes a miss through vLLM's real load-error path.

Move from a working NIXL deployment in stages

1. Preserve the current connector, backend and model-output baseline. Distinguish stock NixlConnector network P/D from LMCache using NIXL and from the separate experimental NIXL storage connector.
2. Choose which ownership layer you want. Existing NIXL applications can evaluate the new IO_URING plugin; LMCache applications can evaluate its direct backend. Neither choice requires deleting the existing baseline.
3. Use a private initialized scratch target and staged host I/O first. Verify independent payload readback and checkpoint/manifest identity; do not share LMCache/NIXL metadata formats.
4. Add classic registration, then DMA-BUF with a qualified kernel/exporter. A GPU payload route must demonstrate native DMA-BUF operations without a hidden host fallback.
5. Run a no-cache model oracle, monolithic/split controls, fresh cold restoration, exact continuation and chosen-token likelihood checks with identical inputs and decode settings.
6. Inject failed reads or corrupt owned test data and require strict rejection. Test mismatched receipts, duplicate ACK, unread release, pressure and cancellation before considering rollout.
7. Qualify the actual deployment topology. Same-GPU success is useful contract evidence; it is not two-GPU, cross-host shared-NVMe or crash-recovery qualification.

For compatible native NIXL users, the parameter sequence is host → fixed → DMA-BUF. For LMCache users, changing transfer_channel alone does not supply the MP dispatcher or convert a network transfer into durable publication.

Did we test NIXL P/D?

Yes. Native IO_URING now passes a same-GPU host-staged P/D profile with the CUDA staging fix below. Historical two-GPU results remain separate. Results must follow the exact backend and connector revisions below.

The native NIXL qualification uses the vLLM validation follow-up with the matching native NIXL branch. This qualification uses Llama-3.2-1B-Instruct, seed 1234, greedy output budgets 48/32/24 for the three case groups and concurrency 1/4/8 in the pressure group. All 57 continuations match; the staging correction and its regression are described below.

Final LMCache model profile: Llama-3.2-1B-Instruct 9213176726f574b556790deb65791e0c5aa438b6 , BF16, eager, seed 123, batches 1/4/8; five-token copy continuation. The 52 token sequences match; maximum chosen-token logprob delta is 0.004954 at the unchanged 0.02 tolerance. An earlier staged recall case failed at 0.026329; its cause remains unresolved. This is not a broad model-quality result.

These LMCache measurements were recorded on 15 October 2026 (PDT) using the published candidate. A separate deterministic in-process profile passes 27 cases each for its no-cache oracle, cold restore and warm repeat, along with stop handling, cancellation, receipt and causal timeline checks at the unchanged 0.02 bound. The earlier no-cache repeatability failure without batch invariance remains recorded. CPU guest tests separately cover host exporters on emulated NVMe with ext4, XFS and raw targets; they do not qualify GPU raw-device I/O or SSD performance.

Corrections found during validation

Both changes passed their configured source checks and review. The LMCache model and 32-profile MP measurements above use the published feature tip 10a941b54; the close correction is separately regression-tested. The native NIXL model replays use the corrected vLLM tip 2697fd5ead. Initial failures remain retained, and the original full and cumulative feature refs are unchanged.

Open work before a broader deployment claim

The latest MP profile is one writer, one reader, compatible BF16 full-attention layout and TP=PP=1. Automatic routing, independent multiwriter ownership, writer restart with live old readers, crash-safe leases, cross-host shared NVMe, GPU-direct raw-device qualification and performance are open. The AMD result is staged. Lossy FP8 model quality is not qualified by encoded-buffer ownership tests.

Storage removes a mandatory direct GPU-to-GPU KV transfer; it does not eliminate control messages or make arbitrary local disks a shared namespace. A network storage topology needs its own capability, visibility and durability proof.

Failure cleanup is case-specific. The missing-key rejection after a claim deliberately retains one writer extent rather than risk reusing bytes a consumer may still access. That control passes its fail-closed retention contract; it is not a normal zero-ownership result or automatic crash recovery.

Make targets for storage P/D

The current test profile pins the public LMCache and vLLM candidates. It exposes CPU contracts separately from GPU transport and model-serving evidence. P/D means prefill/decode; MP means multiprocess.

make defconfig-pd-storage
make pd-test-plan
make dmabuf-fetch
# Prepare each pinned application and its test environment first.
make pd-lmcache-test PYTHON=/path/to/lmcache-venv/bin/python
make pd-mp-test PYTHON=/path/to/lmcache-venv/bin/python
make pd-vllm-test PYTHON=/path/to/vllm-venv/bin/python

pd-cpu-test runs all three selections in order if one environment can import both pinned projects. Unexpected skips fail the target. Use separate targets and environments otherwise. The older pd-run runner remains a basic CPU/file smoke test; it does not implement the full acceptance ladder.

# Check an existing single-host in-process serving receipt:
make pd-timeline-check PD_TIMELINE=/path/to/serving.jsonl
# For a trace that actually used host fixed buffers:
make pd-timeline-check PD_TIMELINE=/path/to/serving.jsonl \
    PD_REQUIRED_PATH=host_fixed

The timeline target defaults to requiring dmabuf_fixed payload provenance. It checks an existing in-process causal receipt; it does not launch models, check model quality or validate MP HTTP/cross-host traces. Full writer/reader serving, recompute-oracle, failed-load and pressure runs remain separate procedures. See the kernel and hardware test targets and tools/reproduce/dmabuf/README.md for setup, dependencies and limits. No CPU-only result establishes GPU P/D.

Reviewable source

• LMCache full dated stack: 50 commits · MP configuration and HTTP contract
• NIXL native IO_URING backend: 2 commits · Backend parameters and limits
• vLLM NIXL storage connector: 3 commits
• Experimental kernel branch , pin 856b6583ab2f4581e81670845eadb69de7a68b30

Follow-up branches include their full feature stacks: LMCache b8845f7c, NIXL 711dcb4d and vLLM 2697fd5ead. NIXL's follow-up adds a GCC15 test-header fix and documents initialized read extents and locked-memory requirements; its production transfer code is unchanged.

Source branches are public snapshots and retain their pinned bases. These guides describe experimental support; no universal compatibility with current upstream versions is claimed.