Collaborative Research: Data-Driven Interviewing to Study the Embodied Cognitive Shifts that Occur During Geometry Learning

People’s body movements can reveal a lot about students’ mathematical reasoning. Examples such as using one’s whole body to explore properties of geometric shapes and raising eyebrows during mathematical insight illustrate ways people’s movements are closely linked to how they think. Despite the value of these nonverbal indicators of students’ learning experiences, this body-mind connection remains understudied. In this project various data sources are gathered from learners playing a video game designed to improve mathematical reasoning through movement and speech. While learning, students occasionally use their bodies to express mathematical insights and trouble spots. These events are important enough that they can influence students’ learning and attitudes toward mathematics, but subtle enough that teachers may miss them in the buzzing dynamics of the classroom. AI will help the research team select when to interview students about these rare but significant moments in their learning, combining the speed and pattern recognition of computers with the depth and insight of humans' natural conversation. This approach creates a rich dataset for analysis and to develop design principles that support mathematics learning through movement. The findings will advance a deeper understanding of how people learn using nonverbal and verbal thought processes and ways to better support these thought processes. The broader impacts include improving mathematics learning for everyone, especially for those who rely on nonverbal thought processes that might be overlooked using current learning designs.

In this project, investigators target two common and critical moments in mathematics learning: 1) forming mathematical insights, and 2) encountering trouble spots that indicate a student is struggling to adapt their understanding to new information. Findings from the five phases of research provide the investigators with detailed findings that can be leveraged to develop new theories and improve classroom learning. In Phase 1 of the project, initial data are collected (including movement, speech and interaction) as students play a mathematics learning game that encourages them to use their bodies. Phase 2 involves creating automated AI detectors, driven by scientific hypotheses, that can recognize students' insights and trouble spots in real-time. In Phase 3, additional data collected with the detectors from phase 2 alert trained interviewers to critical events during student learning, prompting data-driven interviews that gather evidence on cognitive, metacognitive, and affective processes. Phase 4 uses learning analytics and data from these AI detection-driven interviews to build a theoretical model explaining the interplay of cognition, metacognition, affect, and embodiment. Finally, Phase 5 generates practical design principles for classroom instruction and embodied learning technologies, informed by the resultant model and empirical findings. The broader impacts include improving mathematics learning for everyone, especially for those who rely on nonverbal thought processes that might be overlooked using current learning designs.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Award Number: 2601422
Principal Investigator: Stephen Hutt
Funds Obligated: $158,654
State: MN