B-TIER inspiration
Qwen 27B at home — a dev ditching frontier labs for local models
100k ft — What is this?
Qwen 27B at home — a dev ditching frontier labs for local models
50k ft — Why care?
An ex-lab engineer argues the free Qwen 27B model, run locally or in the cloud, handles long complex tasks better than frontier labs. Also claims Blackwell chips, including their Jetson Thor boxes, have broken speculative decoding. Opinion-heavy with strong unverified claims.
10k ft — Key takeaways
• Open-weight Qwen models are viable for long agentic tasks
• Speculative decoding issues on new Blackwell hardware may hurt LLM speed
• Treat the anti-lab claims as opinion
5k ft — Tools & techniques
Anthropic, Jetson Thor, OpenAI
speculative decoding
1k ft — Deep-dive / try it
I'm not using the AI Labs at all anymore
Transcript excerpt
I'm not using the labs at all anymore. I don't use Anthropic, I don't use OpenAI, I don't use, I've never used Grok, I don't use any of them. There's something better, and it's free. I do sometimes use it in the cloud, which is not something that I am a hater about. And let me give you a couple of examples of why I would do that. This model is free. You can run it at home. It's better than anything that the labs have. The labs which slowed down years ago and stopped getting better because their thesis about how the future of AI would work was wrong. And many of us realized that years ago when we were working there and left. Dario quit at that point to start a competitor and try to IPO first so he could cash out before Sam could. Back to the point. This model can do anything. You can give it a really complicated task that will take an extremely long time and it will do it. I have three examples now of that happening, and you can look at at least one of them at this point. Behind me you will see, those are Jetson Thors, okay? Those are $8,000 AI computers each. They're broken. All Blackwell chips are broken in a very fundamental way which is very typical of NVIDIA with new releases. Their current generation of hardware is frequently fundamentally broken. There's this thing that happens with speculative decoding which is like extremely fundamental to the speed of LLMs today. Especially big ones. Especially a mixture of experts, but also this Quinn 27b model. They can do…