🔥Cursor releases MoE inference optimization technology Warp Decode, achieving a 1.84x throughput increase on Blackwell GPU

AI programming tool Cursor publishes a technical blog introducing its self-developed MoE inference acceleration method, Warp Decode. This method targets small-batch token generation scenarios on NVIDIA Blackwell GPUs, transforming the traditional expert-centric parallel strategy into an output-centric approach: each warp in the GPU is responsible for computing a single output value, independently traversing all routed experts and performing accumulation in registers, without the need for cross-warp synchronization or intermediate buffers. The traditional MoE inference pipeline consists of 8 stages, of which 5 are solely used for data transfer. Warp Decode compresses the entire MoE computation layer into 2 CU…