@ben_j_todd - Full episode

Nathan Labenz going hard in recent pod: When Sam Altman, when DeepMind leadership, when Dario, say that the Singularity is near, I think it's time to really start taking it seriously. It was all fun and games, when it was GPT-2, GPT-3, oh look you can do is some funny creative stuff, isn't scale amazing?... We are now at human PhD level on small to mid-sized tasks. And this stuff is still not that complicated. It's not that there's been tons of intricate techniques developed, simple stuff is working, multiple different paths are working. It kind of all works. Possibly we had to get over some threshold before the reinforcement learning paradigm could kick in on top of language models—but if so, it seems like we're there. It definitely seems like we should expect meaningfully superhuman performance, in a fast-growing range of domains. Math is for sure, programming is for sure, we've already seen o3 get up into the top 200 coders in the world. There's no shortage of the ability to reward models for getting the right answers on these coding problems. I think that's going to generalise to everything. They're showing already that there is some generalisation from reasoning tasks to other tasks—o1 is better at legal analysis than base GPT-4o...It's hard to imagine that you're not going to be able to get enough reward signal across a wide enough range of domains, to the point where we see meaningfully superhuman performance across a pretty wide range of tasks. For weakly superhuman intelligence, I think the path to that is increasingly quite clear. Doesn't have too many missing pieces left and I think it's very credible what the leaders are saying that we're looking at one to three years, where AI is better than any human in every task in 2027. It's so hard for me to imagine what the world is like when that happens, but it's increasingly not that hard to imagine what the AI is like that could satisfy that definition. I think Singularity is, in fact, near.