Reasoning in Language Models: Identification and Application
Much More Research Is Needed on Reasoning by Language Models
Language models, such as those used in popular chatbots, do indeed show signs of reasoning, but there’s always a “but.” This is according to research by linguist Stefan Schouten into whether language models truly reason and how robust that reasoning is.
For example, when presented with sentences that allow for different interpretations, GPT-4 chose the correct interpretation in over 83 percent of cases. That’s much better than guessing, but not flawless. Logical structures can also be seen in the internal connections between concepts and sentences, but models can, for example, be influenced by irrelevant information.
Millions of people use AI chatbots every day for advice, study, and work. New “reasoning models” reveal their thought processes, but text that resembles reasoning is not yet proof that actual reasoning is taking place. Furthermore, the reasons a model provides for an answer or decision do not necessarily correspond to what actually preceded it internally.
The research methods developed help scientists and AI companies look inside models and verify how reliably they reason. More research is needed to determine whether language models can also robustly solve problems they have never encountered before.