Computer scientist Richard Sutton, a recipient of the Turing Award, considers it unlikely that pure computing power alone will lead to genuine intelligence. In his assessment, fundamentally new learning methods are needed. Of all people, he, who emphasized the importance of general computing methods over human-designed knowledge in his essay on the so-called bitter lesson, is now issuing a clear warning to the advocates of scaling large language models.
Sutton receives support from Jerry Tworek, a former researcher at OpenAI. He too agrees that blindly scaling transformer models is not enough. Tworek goes further, saying that the fixation on scaling has led an entire generation of scientists astray. Both voices come from within research and touch on a core point of the current debate about the path toward artificial intelligence.
The criticism comes at a time when a growing number of startups are focusing on new approaches. These companies target the gap between what can be achieved through more computing power and what one might understand as intelligence. The basic idea is that progress will not come solely from larger models and more hardware, but from other methods of learning.
It remains open, however, which of the new learning methods will prevail. It is equally uncertain whether the startups' bets will pay off. The warnings from Sutton and Tworek thus mark above all a shift in the debate: away from the question of how much computing power is still needed, and toward the question of how machines should learn at all.