Almost certainly CPU-launch-bound decode: many tiny kernels per step, CPU can't feed the GPU. Enable CUDA graphs, increase batch size (raise max_num_seqs / accept more concurrency), and check for Python overhead on the hot path. Confirm with a profile showing gaps between kernels.

max_num_batched_tokens trades prefill chunk size vs decode latency: bigger = better prefill throughput but can stall decodes; tune to your prompt/output mix. gpu_memory_utilization sets how much HBM the KV cache may use — raise it to fit more concurrent sequences, but leave headroom for activations/CUDA-graph buffers to avoid OOM.