ClassMeta: Designing Interactive Virtual Classmate to Promote VR Classroom participation (CHI 2024🏅)
ClassMeta is a GPT-4-powered agent designed to promote classroom participation by simulating active students in virtual reality. Embodied as 3D avatars capable of speech and gestures, these agents interact with real instructors and peers to enrich engagement and collective learning. Our comparative study shows that AI-driven peer presence can foster more dynamic, participatory, and immersive learning experiences.
tHE pROBLEM
Instructors often seek students to be responsive in the class.However, speaking up in front of the whole class could be intimidating, which makes students reluctant to the instructor or their classmates.
Active Student
Active peers play a pivotal role in shaping the collective learning experience.
Their participation enhances question–answer exchanges, enriches group discussions, motivates note-taking, and fosters overall classroom discipline and engagement.
However, the presence of such active students is not guaranteed in a classroom.
However, the presence of such active students is not guaranteed in a classroom.
ClassMETA: Interaction Design
We model the behaviors of the ClassMeta agent through three interaction types that emulate active student participation:
1. Interaction with Instructor — the agent can respond to instructor’s request by either raising a question or answering a question.
2. Interaction with Students — the agent contributes to group discussions, shares insights, or moderates off-topic conversations to sustain focus.
3. Self Behavior — the agent autonomously takes notes on key lecture points and prompts the instructor to elaborate on unclear content.
1. Interaction with Instructor — the agent can respond to instructor’s request by either raising a question or answering a question.
2. Interaction with Students — the agent contributes to group discussions, shares insights, or moderates off-topic conversations to sustain focus.
3. Self Behavior — the agent autonomously takes notes on key lecture points and prompts the instructor to elaborate on unclear content.
sYSTEM sTRUCTURE
The multi-user VR classroom is powered by the Photon Unity Networking engine, enabling real-time interaction between instructors, students, and the ClassMeta agent. Speech from both students and instructors is streamed to Firebase, where it is transcribed by Microsoft Azure and processed by GPT-4 using a system prompt derived from the lecture script.
GPT-4 then generates two types of responses:
Non-verbal behaviors — triggering predefined gestures and movements of the ClassMeta agent.
Verbal responses — synthesized through ElevenLabs’ text-to-speech engine and delivered as spoken dialogue within the virtual classroom.
GPT-4 then generates two types of responses:
Non-verbal behaviors — triggering predefined gestures and movements of the ClassMeta agent.
Verbal responses — synthesized through ElevenLabs’ text-to-speech engine and delivered as spoken dialogue within the virtual classroom.
User Study
We conducted a between-group comparative study featuring two conditions: a baseline VR classroom with only a human instructor and students, and an experimental classroom enhanced with the ClassMeta virtual student agent. Both groups received identical lecture content.
Results indicate that participants in the ClassMeta condition paid significantly more attention to the virtual agent when the agent engaged in classroom participation, suggesting strong engagement effects. Note-taking quality was substantially higher, and interaction analysis revealed increased student-driven discussions with reduced instructor intervention. Post-lecture surveys showed significant improved comfort in initiating and following group discussions, and increased willingness to take notes. Pre–post tests demonstrated significantly greater learning gains in four of six key competencies.
Results indicate that participants in the ClassMeta condition paid significantly more attention to the virtual agent when the agent engaged in classroom participation, suggesting strong engagement effects. Note-taking quality was substantially higher, and interaction analysis revealed increased student-driven discussions with reduced instructor intervention. Post-lecture surveys showed significant improved comfort in initiating and following group discussions, and increased willingness to take notes. Pre–post tests demonstrated significantly greater learning gains in four of six key competencies.
Full Video
Personal Thought
ClassMeta was born out of a simple but important observation—active students can make a classroom come alive, yet they’re not always there. And when they’re missing, the gap only grows wider. It’s what researchers call the Matthew Effect: those who already engage and perform well tend to find themselves in even better learning environments, while quieter or less confident students can get left behind.
I’ve always found this cycle troubling. Active students usually come from families with more educational resources, and they often end up in schools filled with peers like themselves. Meanwhile, classrooms in less privileged areas may have fewer of these active voices, making it harder to create that vibrant, collaborative energy we associate with good learning.
Even teachers, often unintentionally, give more attention to the students who participate the most, which means quieter students get fewer opportunities to grow. With ClassMeta, I wanted to explore how technology could balance that. By introducing virtual “active” students, we can simulate participation, spark engagement, and give every classroom—regardless of its makeup—a fairer, more dynamic learning atmosphere.
I’ve always found this cycle troubling. Active students usually come from families with more educational resources, and they often end up in schools filled with peers like themselves. Meanwhile, classrooms in less privileged areas may have fewer of these active voices, making it harder to create that vibrant, collaborative energy we associate with good learning.
Even teachers, often unintentionally, give more attention to the students who participate the most, which means quieter students get fewer opportunities to grow. With ClassMeta, I wanted to explore how technology could balance that. By introducing virtual “active” students, we can simulate participation, spark engagement, and give every classroom—regardless of its makeup—a fairer, more dynamic learning atmosphere.