BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Vrije Universiteit Amsterdam//NONSGML v1.0//EN
NAME:PhD defence F.S. Schouten
METHOD:PUBLISH
BEGIN:VEVENT
DTSTART:20261102T134500
DTEND:20261102T151500
DTSTAMP:20261102T134500
UID:phd-defence-f-s-schouten@8F96275E-9F55-4B3F-A143-836282E12573
CREATED:20261105T020722
LOCATION:Main building VU, 1105, Auditorium, De Boelelaan, 1081 HV, Amsterdam
SUMMARY:PhD defence F.S. Schouten
X-ALT-DESC;FMTTYPE=text/html: Reasoning in Languag
e Models: Identification and Application
Much More Resear
ch Is Needed on Reasoning by Language Models
Language models,
such as those used in popular chatbots, do indeed show signs of reaso
ning, but there’s always a “but.” This is according to research
by linguist Stefan Schouten into whether language models truly reaso
n and how robust that reasoning is.
For example, when presented
with sentences that allow for different interpretations, GPT-4 chose
the correct interpretation in over 83 percent of cases. That’s muc
h better than guessing, but not flawless. Logical structures can also
be seen in the internal connections between concepts and sentences,
but models can, for example, be influenced by irrelevant information.
Millions of people use AI chatbots every day for advice, study
, and work. New “reasoning models” reveal their thought processes
, but text that resembles reasoning is not yet proof that actual reas
oning is taking place. Furthermore, the reasons a model provides for
an answer or decision do not necessarily correspond to what actually
preceded it internally.
The research methods developed help sci
entists and AI companies look inside models and verify how reliably t
hey reason. More research is needed to determine whether language mod
els can also robustly solve problems they have never encountered befo
re.
More information on the thesis.
DESCRIPTION: Reasoning in Language Models: Identification and Applicat
ion Much More Research Is Needed on Reasoning by Language Models<
/h3>Language models, such as those used in popular chatbots, do indee
d show signs of reasoning, but there’s always a “but.” This is
according to research by linguist Stefan Schouten into whether langua
ge models truly reason and how robust that reasoning is.For example,
when presented with sentences that allow for different interpretation
s, GPT-4 chose the correct interpretation in over 83 percent of cases
. That’s much better than guessing, but not flawless. Logical struc
tures can also be seen in the internal connections between concepts a
nd sentences, but models can, for example, be influenced by irrelevan
t information.Millions of people use AI chatbots every day for advice
, study, and work. New “reasoning models” reveal their thought pr
ocesses, but text that resembles reasoning is not yet proof that actu
al reasoning is taking place. Furthermore, the reasons a model provid
es for an answer or decision do not necessarily correspond to what ac
tually preceded it internally.The research methods developed help sci
entists and AI companies look inside models and verify how reliably t
hey reason. More research is needed to determine whether language mod
els can also robustly solve problems they have never encountered befo
re.More information on the thesis.
END:VEVENT
END:VCALENDAR