Checkpoints, optimizer state, spilled objects, model weights and cached encodings already move substantial payloads through host DRAM. This page maps the implementations where that path is established and identifies the evidence for transfers larger than 2 MiB.
Here, CPU-initiated means host software submits the storage operation. The payload passes through host memory; NVMe DMA can move it without the CPU copying every byte. CPU-submitted GPU-direct I/O is a different path and is outside this page’s main workload set.
32–64 MBCheckpoint staging buffersBest configurations for FastPersist’s 512 MB transfer experiment; sizes use the paper’s units. See evidence
1–128 MiBRay Data block-size targetHost-memory objects that can spill to disk. This range includes objects below 2 MiB. See evidence
34.3 / 130.8 MBVideo encoder-cache entriesReported by LMCache’s in-process benchmark; entry sizes, not NVMe command sizes. See evidence
knlp connects workload research to reproducible experiments. ebpf-syscall and kvio provide starting points for capture and replay. This is a source-backed workload inventory; it does not report new knlp benchmarks or a measured benefit from a host folio pool.
Workloads and known I/O paths
All rows identify a host-memory storage path. Size evidence ranges from measured buffers and objects to configurable tensor sizes and read plans. None of these sources alone establishes a distribution of NVMe commands above 2 MiB.
Filters describe the named path, not every deployment. A row may appear in more than one filter. Linux buffered I/O includes the page cache; a library’s “unbuffered” option does not establish O_DIRECT.
64 MiB coalescing limitDefault combined-range limit in GetReadRanges(); planning only.
CPU readers decode, decompress and transform selected columns in host memory.
File reads / optional mmap Remote object-store paths need separate accounting.
Selected compressed bytes and actual local read sizes after filtering and coalescing.
Model loadingvLLM safetensors eager / lazy / prefetch
Weight tensors and file shardsA 4096 × 4096 BF16 matrix is 32 MiB; illustrative.
Eager loading reads files into CPU memory; lazy and prefetch paths use memory mapping / page cache.
Buffered / mmap This row excludes GPU-direct loaders.
Cold-start latency, resident cache hits, readahead and request splitting.
Keep the size evidence at the right layer
The same 64 MiB object can generate one library call, many syscalls or many NVMe commands. It can also be served entirely from RAM. A large application buffer is useful evidence for host-memory management, but is not a trace of device I/O.
Quantity
What it tells us
How to establish it
Object / tensor bytes
Useful payload and capacity demand.
Shape, dtype, serialization and compression.
Staging-buffer bytes
Working memory available to pipeline transfers.
Allocator configuration and measured buffer use.
Submitted I/O bytes
Work handed to the OS, including each AIO or io_uring operation.
Application instrumentation and submission traces.
Block / NVMe bytes
Requests after merging, splitting, mapping and cache effects.
Block and driver traces on the named target device.
Backing folio size
Memory allocation and representation granularity.
Inspect the actual backing; alignment alone is insufficient.
A useful existing baseline: DeepNVMe documents an AIO default of block_size=1048576 (1 MiB) and queue_depth=128. Large tensor transfers already work with smaller queued operations. A folio-pool evaluation must compare against that kind of reused-buffer implementation. [3]
Research notes and primary evidence
Published buffer-size experiment
Checkpointing: the clearest measured example
FastPersist’s §5.3.1 tests GPU-tensor writes through pinned CPU buffers to SSD. The table records the best tested buffer sizes; the authors use MB. For 512 MB transfers, 2 MB performed worst and the best double-buffer configuration was 2.87× faster. This measures pipeline buffering, not a folio-size or NVMe-command-size improvement. [1]
Transfer
Single buffer
Double buffer
16 MB
16 MB
8 MB
512 MB
64 MB
32 MB
Established host staging
Distributed checkpoint save and restore
PyTorch DCP’s asynchronous save copies model and optimizer state into CPU buffers before persistence. Its pinned-buffer option retains staging allocations across checkpoints. The standard filesystem writer is a buffered file path; another storage writer can change that behavior. [2]
Measure serialized writes and completion through the required durability boundary. A return from a buffered write is not evidence that the checkpoint is on stable storage. Compare identical checkpoint content and recovery semantics.
CPU computes on the payload
Optimizer state and repeated tensor offload
ZeRO-Infinity supports NVMe optimizer storage while keeping optimizer computation on the CPU. Its tiled pipeline can overlap the next read and the previous write with the current update. This makes host DRAM part of the computation itself. [4]
A GPU-owned allocation does not replace that CPU working set. The integration question is whether a reusable host pool can back the existing tensor buffers without inserting an extra copy. DeepNVMe’s AIO and GDS results should be treated as separate paths. [5]
Host object and library write verified
Ray spilling: a CPU-only starting point
Ray Data targets 1–128 MiB blocks, commonly Arrow tables or pandas dataframes. Its filesystem spill writer calls f.write(memoryview(buf)) for each object payload using ordinary file opens. That verifies a full-buffer library write, not one syscall. [6]
Ray restores spilled objects when needed. Preprocessing and large tool results give a natural test workload: force a controlled object-store budget, measure disk-backed spill and restore, and keep the task outputs identical. Object retention pins in Ray must not be confused with DMA page locking. [7]
Cache retention and staging differ
LMCache with host memory in the path
Use an explicit host-DRAM L1 with a filesystem L2 backend for the MP case. The native filesystem adapter offers use_odirect and alignment-aware slot transfers. The older in-process disk backend also allocates CPU read buffers; local_cpu: false disables retained CPU caching without proving that host staging disappears. [8]
Illustrative unsharded, all-layer BF16 KV: 2 × 32 layers × 256 tokens × 8 KV heads × 128 dimensions × 2 bytes = 32 MiB. Layerwise transfer, tensor parallelism, architecture and quantization change the actual object and request sizes. Record them from the selected implementation.
Measured non-KV cache entries
Video, image and audio encodings
LMCache’s in-process encoder cache stores a tensor per multimodal input hash, independently of attention KV. Its Qwen2.5-VL-7B BF16 video benchmark reports 34.3 MB entries for 32 sampled frames and 130.8 MB for 128 frames. CPU and disk backends are supported. [9]
Those are cache-entry measurements. The reported warm-request speedups do not establish disk-hit latency, and may include CPU-cache hits. This documentation is labeled deprecated in-process mode; verify the selected connector and version before adapting it to MP.
Read coalescing is implemented
Columnar data used by AI tools
Arrow’s GetReadRanges() plans selected Parquet column-chunk ranges, with a default 64 MiB combined-range limit. It performs no I/O itself. PyArrow’s prebuffering can coalesce reads before CPU deserialization; actual transfer sizes depend on columns, compression, filters and the filesystem. [10]
Choose files resident on the target NVMe filesystem for a local block-layer experiment. A 64 MiB S3 request alone is network evidence, not evidence of a local 64 MiB disk request.
Established model-loading paths
Weights, cold starts and model changes
vLLM documents eager safetensors loading into CPU memory, lazy memory mapping and prefetching into the OS page cache. These establish host-memory paths with large tensor and shard payloads. [11]
Cold model provisioning and repeatedly streaming nonresident weights are different workloads. DeepNVMe also evaluates ZeRO-Inference using host bounce buffers. Keep model-load frequency and actual cache misses explicit; a large model file does not imply continuing disk traffic during serving. [5]
Where host folios and a reusable pool fit
CPU producers and consumers need usable host memory: optimizer updates, dataframes and column decoding are examples. GPU checkpointing and cache offload instead use host memory as a staging or retention tier. GPU-direct paths can remove that tier in some deployments; they do not cover the CPU-native cases.
Three device properties, different meanings
MDTS describes the controller’s maximum transfer limit; NPWG describes recommended write granularity; NOWS describes optimal write size. Decode the fields and validity rules for the device. A value above PAGE_SIZE is not itself proof of a workload above 2 MiB or a requirement for an equally large folio. [12]
Also record max_hw_sectors_kb, max_sectors_kb, max_segments and max_segment_size, plus filesystem, DMA/IOMMU and stacked-device constraints. A large application request may be split before reaching the controller. [13]
The hypothesis to test
A device-associated, preallocated folio set could reduce allocation, page bookkeeping, scatter/gather or mapping costs for reusable host buffers. Benefits depend on the implementation and mapping lifetime; larger folios do not automatically produce larger DMA mappings.
A large I/O can span several folios. Compare ordinary reused registered buffers, available large-page backing and the proposed pool at equal useful work. Evaluate backing size, submission size and persistent mapping separately. No result on this page establishes that any one of them wins.
Related Linux work: DMA-BUF + io_uring. Buffer ownership is the connection: an accelerator can supply buffers for its own payloads, while CPU-native workflows need a host-memory allocation path. The pool should become backing for the real working buffer where possible; copying into another pool changes the trade-off.
Agentic workflows: follow the payload
An agent’s transcript can be small while its tools scan datasets, create large intermediates, load models or checkpoint execution environments. The storage workload comes from those operations and their frequency.
Activity
Established path / useful candidate
Evidence still needed
Data-analysis tools
Ray spilling and Arrow scans above provide concrete CPU-memory paths for table and tensor batches.
Capture a named agent task, its data scale and the local disk traffic per completed task.
Multimodal investigation
Large media artifacts and reusable encoder outputs; LMCache supplies an existing encoder-cache implementation.
Separate original media reads, decoded data, encoding reuse and attention KV; measure disk hits.
Sandbox pause / resume
E2B documents dirty-memory and rootfs snapshots plus a local template cache. Export and cache population are bulk-transfer candidates. [14]
Large-request hypothesis. Resume uses lazy page faults and optional prefetch; snapshot size does not establish large reads.
Model experiments
Model loading, checkpoints and optimizer offload are established above.
Count jobs, saves, bytes and cache misses. Remote API use alone does not imply local weight I/O.
RAG, packages and build artifacts
Ingestion, index construction, compaction and artifact staging deserve capture.
Name an implementation and measure requests. Online ANN lookups, metadata and small files need their own analysis.
Capacity, traffic and request size need separate measurements. Retained bytes describe footprint. Reads and writes per completed task describe turnover. Request-size distributions describe how that traffic reaches the storage stack. Deduplication, compression, RAM residency and remote storage can change all three.
Reproduce the storage question
Start with host-staged LMCache, a FastPersist-style checkpoint writer and Ray spill/restore. These cover inference, published large-buffer evidence and CPU-only processing. The integrations below are proposed experiments, not newly completed knlp benchmarks.
Comparison
Hold fixed
Measure
Buffer backing
Reuse and registration lifecycle, payload, NUMA placement, memory budget, submitted size and durability.
Allocation/pinning cost, CPU cycles per GiB, mapping work, faults and application latency.
Submitted size
Backing, logical data, total bytes in flight and completion semantics. Sweep 1, 2, 4, 8, 16 and 32 MiB where supported.
Actual NVMe sizes, splitting, throughput and tail latency. Unsupported large commands are not large-command tests.
Direct vs buffered
Useful work, total RAM budget, file layout, concurrency and durability.
Cold, mixed and resident-cache behavior; device bytes and CPU cost, including cache hits.
Full application
Same checkpoint, recovered object, model output or data-analysis result.
Checkpoint pause, restore time, task completion or TTFT; report any extra memcpy introduced.
Read-only queue-limit inventory
Set dev to the target block-device name. These sysfs reads do not change device settings. Inspect every relevant leaf device when testing RAID or device mapper; passthrough paths need their own limits.
dev=nvme0n1
for field in logical_block_size physical_block_size minimum_io_size \
optimal_io_size max_hw_sectors_kb max_sectors_kb \
max_segments max_segment_size; do
printf '%s: ' "$field"
cat "/sys/class/block/$dev/queue/$field"
done
Capture both sides of the filesystem boundary. Buffered reads that hit RAM never appear in a device trace; mmap requires fault/residency observations. For asynchronous I/O, record each submitted operation’s length, not just the syscall that submits a batch. Keep device requests separate from application calls when replaying traces.
Publish source revisions, source data or a reproducible generator, hardware and queue limits, filesystem/mount options, I/O mode, folio backing, cache budget, bytes in flight, repeated-run variation and application correctness. A bandwidth gain without matching useful work and completion semantics is insufficient.
Primary papers, implementation code and project documentation. Moving documentation and branches describe the reviewed paths; record immutable revisions for experiments. MB is retained where the source uses MB; MiB denotes 2²⁰ bytes.