Three teams in one month have pushed LLM decode forward with megakernels: Inferact on Google TPUs, AMD with TileRT and F | Hanami
Three teams in one month have pushed LLM decode forward with megakernels: Inferact on Google TPUs, AMD with TileRT and FlyDSL, and now NVIDIA on GB300, now live on InferenceX.
We're excited to see these gains and dig into the engineering behind faster inference.
Megakernels combine work that normally runs in separate programs. Here's why that matters and how each team's approach works. (1/8)🧵