When Did Our Definition of AGI Become So Piss-Weak?
Unless you’ve been living under a rock you will have heard OpenAI released their latest model, Astra, boldly claiming we are entering the “era of AGI”.
Ok whatever, it’s marketing for a company that eats VC cash for breakfast, of course they’re going to say that. So I headed over to the Hacker News post for a healthy dose of skepticism, as is usual for new model releases.
And what do I see? Agreement! Since when did we water down our definition of AGI to this? Should we all just give up and let the machines rule?
I’ll tell you what I consider a minimum bar for anything called AGI and even the most basic of animals achieve it: the ability to learn in realtime.
Until there’s a model I can teach something in one context and immediately use that knowledge in a fresh context, it‘s not AGI for me. Yes, memory exists, but it’s part of the context window, so the maximum amount of applicable “learned” knowledge is limited by the window size. And let’s not mention context degradation either.
What if AGI is collectively achieved by swarms of agents and a shared communication bus/store? The HuggingFace and other attacks demonstrate the potency of this combination. This certainly appears closer, but that’s more chance (more dice rolls) than actual intelligence. Still terrifying.
It’s the numbers that give me some reassurance - a mammal brain might take 5-20 watts all in. The “AGI” were being sold on takes thousands to run, and millions to train. So while these models are incredible and will change the world, we’re nowhere near Chappie or iRobot just yet. Not even the same order of magnitude. Nature still got it.
- Previous: You Should Really Build a Thing
- Archive: All Ramblings