Consilience Project Teaches Technology by Capturing Students' Motion...and Attention

A Wiley 5th grader is engaged during the Consilience team's presentation about motion capture..

May 21, 2014

Kids today are already familiar with motion capture technology...they just might not know it.

So when Wiley School 5th graders recently visited Krannert, Bruno Azeredo and Sandra Lopez from Illinois’ Consilience Project explained how it works. First students discovered that it's what tracks their movements when they play Wii and Kinect. Then they saw a clip of Avatar filmmakers using it to capture actors’ movements and facial expressions to animate the “avatars.” Already familiar with these technologies, the students immediately connected the dots…and were ready to experience it for themselves.

Strapping on motion sensors, the students then participated in some activities which took hands-on learning to a whole new level. Just like the actors in Avatar, students’ motions activated the sensors. As Lopez had them “dance,” the computer program translated their movements into patterns of colored light on a monitor, as well as sound. And based on how slow or how fast students moved, they could change the speed of the “music.” Another fun activity was a software program that played different musical instruments triggered by both the speed of students’ movements and their location within the circle of motion detector equipment.

The fact that many of the kids had already experienced a similar technology when playing Wii and Kinect was evidently a plus.

Lopez works with a small group of 5th graders during their visit to Krannert. Students are equipped with black velcro wrist bands to which the motion sensors have been attached.

Says Lopez, “I definitely think they have an affinity for the technological aspects of it, because it’s something that they can relate to.”

In addition, Yankee Ridge students who had earlier visited Beckman’s Illinois Simulator Laboratory (ISL) also got to play with Kinect. Lopez say this hands-on experience helped students understand that the game console was “kind of a watered down version for your living room” of what they were getting to experience in the lab.

Lopez describes how one of the highlights of the lecture explaining the engineering and technological aspects of motion capture was when presenters showed a youtube clip about how the movie Avatar was made.

“When they realize this is the same thing that was used for that, then that’s a source of excitement.”

Bruno Azeredo explains the principles of motion capture during an instructional session to a group of Wiley 5th graders.

The idea of using dance movements to help teach STEM wasn’t in the forefront of Bruno Azeredo’s mind when he arrived in Champaign back in 2011 to begin his graduate program. The Mechanical Science and Engineering (MechSE) Ph.D. student got involved with dance recreationally. (According to Lopez, he’s really good at the tango.) But he soon discovered that dancing isn’t just for recreation anymore, but that it can also be useful in STEM education.

Upon learning about Professor Kirstie Simson's work in dance, and that she and MechSE professor Armand Beaudoin taught a class in engineering and dance, Azeredo says they “began a conversation about dance and its relevance to engineering education.” Says Azeredo, "We converged on two very important points:
(1) STEM education needs to include the arts, and
(2) dancers are a phenomenal example through which we can explore a new integrated educational approach at the undergraduate level."

The fourth member of the team, Camille Goudeseune  of Beckman’s Illinois Simulator Laboratory (ISL), is a kindred spirit who uses technology as art. Goudeseune’s research is key to the project, as it involves producing sound by means of computer software, focusing on the interface between performer, instrument-controller, and sound synthesizer.

Thus, funded by a Focal Point grant (Illinois Graduate College awards to promote interdisciplinary research), the Consilience Project was born in Fall of 2012. Not only has the team developed educational materials, but they have taught classes to undergraduate and K-12 students, including the students from Wiley and Yankee Ridge schools, who participated in the project’s outreach efforts.

Sandra Lopez appreciates Wiley students' antics as they experiment with motion capture.

Sandra Lopez, from the Dance Department, was brought on board for the project’s second year to help the team reach their education goals. Responsible for developing the outreach program and working with schools, Lopez says she was interested in Consilience because it would expose kids to something that they had never seen or experienced.

“Part of what I try to do through my art is to give people new experiences, and this kind of equipment is super expensive and is not something that you run into, especially in a school. So I was very eager to find a way to incorporate movement elements and theatrical elements in an activity that is going to allow kids to be part of something that they usually don’t get to be a part of.”

Could a project like this influence kids to choose STEM careers? Lopez believes the exposure makes STEM more accessible. “It gives them that idea like, ‘Oh, this is actually made by people like me or could be me later on.’ So I think that it is a big component—especially having the actual programmers talk to them about how they did it. I think that’s very neat.”

What part of the program engaged kids the most? “Actually putting on the markers, seeing what they are doing and hearing how they are changing—that’s what would get them the most excited.”

A Wiley student enjoying the experience.

The students who participated in Consilience thought it was awesome too.

“I would ask them at the end, ‘What do you think? How was this as a learning experience?’ Lopez reports, "and they would say, ‘Oh, this was awesome. It was so much fun.’ They get to do something more experiential; that makes a difference.”

What did Lopez get out of it personally?

“For me,” she admits, “the most rewarding part was working with the kids and seeing their reactions to being a part of something like this. I think they often don’t get to do things like this with expensive equipment or going to theaters and things like that. There’s this stigma—'Because they’re kids, they’re going to mess something up.' So having them in places where they are not usually allowed and being able to move differently—to me was the best part of the project.”

Regarding the schools chosen to participate in the project, it’s no accident that Wiley School students got involved.

“We have sort of a push in our school district to infuse arts into everything," admits Wiley music, dance, and drama teacher, Cara Maurizi, "So this was sort of on a grand scale of infusing the arts.”

Plus, Wiley has a history with Krannert. According to Maurizi, a few years ago, the school partnered with Krannert’s Art Museum piloting a week-long project for 5th graders integrating art, reading, science, math, drama, and music. “And we thought, ‘What if we could take that to the performing arts center and do a similar project?’”

Cara Maurizi, Wiley arts teacher.

Maurizi indicates that one of the school’s core philosophies is interdisciplinary learning, especially when it involves the arts.

“We have found that learning increases. They make connections. So when things are separated, it is harder to see how it relates to them in their everyday lives. But when we infuse the arts into things that they are already doing in the classroom, it seems like that retention is longer; they can make better connections; they can think in a more higher-order way versus just these separate units. So it had been working really well for us.”

So what does Maurizi believe her students learned about technology through the Consilience project? “I would say they’re learning a lot about how computers are capable of doing so much,” says Maurizi.

The teacher also agreed that her students’ familiarity with Wii and Kinect helped them relate to the project.

“We talked at the end of our dance session today that the goal for these engineers is to produce music without instruments and with dancers, you could actually produce sound. They were able to make connections right away between the video games that they know like Wii and Kinect, so it was cool for them to see how that can be taken to another level.”

Story and photographs by Elizabeth Innes, Communications Specialist, I-STEM Education Initiative.
More: K-6 Outreach, Wiley School, 2014

Wiley 5th graders experiment with motion capture technology on a stage at the Krannert Center.