They have different resource profiles and interfere when co-located: a big prefill stalls ongoing decodes (latency spikes). Splitting them lets you scale and tune each fleet independently — more compute for prefill TTFT, more memory-bandwidth/instances for decode throughput — at the cost of transferring the KV cache between them.

Shipping the KV cache over the network adds latency and bandwidth pressure; it only pays off when interference savings exceed transfer cost. It also adds routing/orchestration complexity and failure modes (a decode node waiting on remote KV).