Skip to content

Concepts · 03 · Memory traffic

Fewer bytes. Different limits.

Explore how batch size, context length, GPU bandwidth and KV-cache precision change decode behavior. The charts below connect traffic and kernel choices to the page’s decode-scaling data.