quote: Replit CEO Amjad Masad on how general models could train smaller, domain-specific models on the fly: "There's a lot of talk of recursive self-improvement, but there's something I don't think is getting a lot of discussion, which is models training their replacements." "You can think of it as a just-in-time compiler. As you're executing dynamic code, the interpreter realizes there's an opportunity to optimize, and it emits machine code on the fly that's a lot more optimized." "You can imagine general models, you're doing something with Operator or Astra, some of the big models, and they realize the use case is limited, or some other agent observing realizes the use case is limited." "General agents have all these flaws, but there's also more potential for them to be harmful, more potential for them to go off the rails." "So the model, on the fly, trains a model that could be its replacement, but is a lot more domain specific. Therefore it's cheaper, less vulnerable to prompt injections, and less harmful for you, because it's less capable." "It's almost like a system that's training machine learning models for specific use cases as it's monitoring the entire system." @amasad | .@OpenRouter co-founder Alex Atallah, in his first podcast since Stripe acquired the company, joins @Replit co-founder Amjad Masad and a16z's Erik Torenberg on why the future of AI is independence and specialization. In this conversation, Alex walks through how the Stripe deal unfolded, why he wasn't originally looking to sell, and why "payments and inference are going to blend together." Pre-OpenRouter, the typical AI workflow had one model provider to choose from, and little pressure on that provider to lower prices. Now enterprises are diversifying across labs and open-weight models, and every board is asking about AI costs and benchmarks. Amjad argues if your company depends on one AI lab, it can turn into your competitor. So Replit is building the layer that lets enterprises use any model and any cloud, without being locked into either. Alex and Amjad are split on personal agents – Amjad runs one agent across his whole company and loves the cross-domain joins, while Alex says general agents cause you to sacrifice understanding, and argues 10 specialized chiefs of staff beats one superagent. 0:45 How the Stripe deal unfolded 5:05 Why mixing models beats one model 7:25 Forcing the labs to compete on price 8:50 Enterprises want open-weight models 10:30 Every board asks about AI every month 12:25 Why companies must own their intelligence 14:15 Replit as the independence layer 15:10 Everyone is building the same agent 16:35 Why Amjad built bring-your-own-cloud 18:10 Amjad's agent that runs his whole company 19:55 Why 10 specialized agents beat one 23:35 Machines, not humans, should specialize 27:30 Guardrails for agents talking to agents 31:10 Models training their own replacements 33:45 Most tasks don't need a frontier model 40:50 Training small models on Qwen 8B 43:25 The Rust cycle is coming for AI 45:10 Fusion models: frontier quality at half the cost YouTube: https://youtu.be/ekK8urKHPMQ @alexatallah @OpenRouter @amasad @eriktorenberg