What Large Language Models can or can't do: Scientist explains

The Eliza Effect of AI ͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­͏   ­ Forwarded this email? [Subscribe here]() for more The V4 of NextBig-What delivers you big ideas on tech and culture from the world’s best books, podcasts and reports (and of course, great original commentaries). [Get 20% off for 1 year]( --------------------------------------------------------------- [What Large Language Models can or can't do: Scientist explains]( The Eliza Effect of AI Jul 16   [READ IN APP](   David Chiang, PhD University of Notre Dame, Assoc. Professor, Department of Computer Science and Engineering shares a very hands-on approach towards LLMs to explain what they can or can’t do. His recent talk helps any non-tech professional get a very good understanding of LLMs (Large Language Models) and importantly LLM’s limitations and capabilities. --------------------------------------------------------------- [Star Wars BB-8]( The Eliza Effect vs. The AI Effect Language models like ChatGPT have sparked debates about artificial intelligence. Two opposing perspectives often arise: - The Eliza Effect: Overestimating AI's intelligence based on superficial interactions. - The AI Effect: Dismissing AI capabilities once we understand how they work. [Upgrade to paid]( Balancing these views is crucial for accurately assessing AI systems. The Power of Next Word Prediction Language models operate by predicting the next word in a sequence. This seemingly simple task enables them to: • Write poetry • Solve complex math problems • Play chess at a high level "Large language models like ChatGPT are just predicting the next word, but that entails much more than just knowing that H comes after T. You have to be a grandmaster at chess to make the best possible completion." The Illusion of Conversation Despite their conversational abilities, language models lack true dialogue understanding: • They don't maintain separate internal states for participants. • They struggle with consistent role-playing in conversations. • Their responses are generated on the fly, not from stored information. The Limits of State Tracking Current language model architecture has limitations in tracking and manipulating information over long sequences: • They process input all at once, not sequentially like humans. • This leads to difficulties with tasks requiring long-term state tracking. • Demonstrated through cup-swapping games where models fail with complex sequences. Perception vs. Reality "The difference between the perception and the reality of the language model can be quite wide." Language models can convincingly simulate intelligent conversation, masking their fundamental limitations: • Lack of true internal state • Inability to process information sequentially • Difficulties with long-term context tracking The Future of AI Assessment While specific limitations may be addressed in future iterations, the challenge of accurately evaluating AI systems will persist: - Critical examination of AI technologies remains crucial. - Balancing appreciation for AI abilities with awareness of constraints is key. "Language models do such a good job at pretending to be what they are, these limitations can be serious while at the same time being very difficult to see on the surface." Understanding both the impressive capabilities and inherent limitations of language models is essential for navigating the evolving landscape of artificial intelligence. You’re currently a free subscriber. For the complete experience, upgrade your subscription. [Upgrade to paid]( India based subscribers can subscribe using the Razorpay link below (as Stripe doesn’t work with all Indian cards). [Subscribe with Razorpay]( Follow us on [Twitter]( | [LinkedIn]( | [YouTube](   [Like]( [Comment]( [Restack](   © 2024 Zakti Techmedia Private Limited 677, HSR Layout, Bangalore-560102 [Unsubscribe]() [Get the app]( writing]()