As mentioned in the repo’s README.md, unfortunately, we only have the maintainer bandwidth to support popular inference | Hanami
As mentioned in the repo’s README.md, unfortunately, we only have the maintainer bandwidth to support popular inference engines upstream.
InferenceX’s primary goal is to compare accelerators against accelerators, not inference engines against one another. For anyone interested in engine-to-engine comparisons, they are more than welcome to fork the repo and run it using their own GitHub Actions runners.
Reply to @logotype: @dylan522p Hey Dylan. Your team asked us to prove LayerScale on our own InferenceX fork. We did, on our own B200s, with public logs. We beat your published SGLang run at every concurrency we ran: 11–20% lower energy/token, 6–8× lower TTFT.
They just CLOSED the PR anyway. How is that an open benchmark???!!!
https://t.co/OLvkpIsZPM