My mind is blown by how far ahead Apple is with local LLMs. The new Mac Studio M5 Ultra will have a 512 GB model for aro | Hanami
My mind is blown by how far ahead Apple is with local LLMs. The new Mac Studio M5 Ultra will have a 512 GB model for around 15-20k. You can chain 6 of them together using thunderbolt 5 and this have over 3 TB of ram and a massive amount of CPU and GPU cores. This means you can run a frontier model with over 2 trillion parameters locally for around 100kish. Nivida is around 500k for the same capability.