Emerging Technologies

This AI quizbot is helping students to learn

Students study in the Reading Room at Suzzallo Library on the University of Washington campus in Seattle, Washington, U.S., September 20, 2018. Picture taken on September 20, 2018. REUTERS/Lindsey Wasson - RC165172DA60

ChatBots can recognize near-miss answers and offer additional guidance and even encouragement to the student. Image: REUTERS/Lindsey Wasson

Andrew Myers
Assistant Professor of Civil and Environmental Engineering, Northeastern University
Our Impact
What's the World Economic Forum doing to accelerate action on Emerging Technologies?
The Big Picture
Explore and monitor how Artificial Intelligence is affecting economies, industries and global issues
A hand holding a looking glass by a lake
Crowdsource Innovation
Get involved with our crowdsourced digital platform to deliver impact at scale
Stay up to date:

Artificial Intelligence

In a study with 36 students learning with either a flashcard app or a QuizBot, the team found that students correctly recalled over 25 percent more right answers for content covered in QuizBot and spent more than 2.6 times longer studying with QuizBot versus flashcards.

Both the flashcard app and QuizBot taught information ranging from science and personal safety to advanced English vocabulary. They had identical sequencing algorithms for selecting which item to present next to students.

The longer study time is a key part of why QuizBot works so much better, according to Emma Brunskill, an assistant professor in computer science at Stanford University and coauthor of the paper that will appear in the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. The team presented its findings at CHI 2019 in Glasgow, Scotland.

“QuizBot is more conversational and more fun. Students felt like they had a true study partner,” says graduate student Sherry Ruan, who led the study.

ChatBots are an offshoot of artificial intelligence in which computer programs interact with humans via text message. They are far more common in customer service and ecommerce applications than in classrooms. If you’ve ever chatted with a customer service rep when contemplating an online purchase, there’s a chance you’ve interacted with a ChatBot.

QuizBot is a new spin on the chatbot formula. It posts factual questions via text, much like a teacher. The student types in answers, asks clarifying questions, and requests hints. QuizBot then comprehends and responds conversationally, as if another human is on the other side. Unlike flashcards, ChatBots can recognize near-miss answers and offer additional guidance and even encouragement to the student.

Have you read?

One hurdle in designing such an educational system is that the computer must be able to recognize correct answers in various forms. Not all students answer the same way, using the same words or syntax. And, of course, there is always the potential for typos. That’s where artificial intelligence comes in.

To test QuizBot’s accuracy in grading, the team randomly selected 11,000 conversational logs from their studies. They found that QuizBot was right 96.5 percent of the time—just five incorrect assessments out of 144 questions it asked. Of those, one was a typo (“ture” for “true”) and three occurred because the algorithm penalized answers that were too short, but still correct. Only one error resulted because QuizBot misinterpreted the language.

The researchers think QuizBot could be the dawn of a post-flashcard world for informal learning. “I think there’s a lot of excitement around chatbots in general, though they aren’t in widespread use in education, just yet,” says Brunskill, who also directs the Artificial Intelligence for Human Impact Lab. “But that should change.”

Additional researchers from Stanford, Colby College, and Tsinghua University in China also contributed. The TAL Education Group provided funding for this research.

Don't miss any update on this topic

Create a free account and access your personalized content collection with our latest publications and analyses.

Sign up for free

License and Republishing

World Economic Forum articles may be republished in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License, and in accordance with our Terms of Use.

The views expressed in this article are those of the author alone and not the World Economic Forum.

Related topics:
Emerging TechnologiesFourth Industrial RevolutionEducation and Skills
World Economic Forum logo
Global Agenda

The Agenda Weekly

A weekly update of the most important issues driving the global agenda

Subscribe today

You can unsubscribe at any time using the link in our emails. For more details, review our privacy policy.

Solar storms hit tech equipment, and other technology news you need to know

Sebastian Buckup

May 17, 2024

About Us



Partners & Members

  • Join Us

Language Editions

Privacy Policy & Terms of Service

© 2024 World Economic Forum