Strategic Communications and Marketing News Bureau

Jazz-playing robot will provide insight into how computers communicate with humans

CHAMPAIGN, Ill. — When jazz musicians improvise alongside each other, they interact in ways that are as much about feeling the music as about thinking. But suppose one of the musicians is a robot. Can it improvise a jazz solo in response to what its human partner is playing?

A University of Illinois researcher is designing a robot – actually a computer system – that will communicate with humans through jazz improvisation and provide insight into artificial intelligence and human-computer interaction.

The project – called MUSICA (MUSical Improvising Collaborative Agent) – is part of the Defense Advanced Research Projects Agency’s new Communicating with Computers program, which aims to “enable symmetric communication between people and computers,” where computers are collaborators just like people are. The jazz robot is an unconventional approach to the problem.

“It’s approaching this communication from a different perspective than linguistically, and maybe we can learn something new about computers and humans,” said Ben Grosser, a professor of new media in the School of Art and Design. Grosser is also affiliated with the National Center for Supercomputing Applications, where he is helping develop a new initiative in critical technology studies.

“We proposed that music improvisation is a novel space for investigating communication because it’s not just cognitive interaction,” Grosser said of the MUSICA project. “It’s also an embodied experience. You feel the music as much as hear it and think about it. You react.”

To build a software system that might be able to improvise music, Grosser and his team will create a knowledge database of canonical jazz solos. Their approach is to computationally analyze these solos using “image schema,” a way that people understand their world using spatial concepts. For example, a musician might play “inside,” meaning their notes fit within what the song’s chord changes suggest, or “outside,” meaning they’re pushing the boundaries of what is appropriate.

“It’s a way of understanding what jazz musicians do and analyzing what happens between them,” said Grosser, who has a long history of performance and interest in jazz, has an MFA in new media and a master’s degree in music composition, and worked in the School of Music’s Computer Music Project during graduate school.

He’ll then add a performance system that will analyze what a human performer is playing in real time, including the beat, pitch, harmony and rhythm, and consult what it has learned about jazz solos to communicate musically in response. The system should “understand not just what should come next but when a response should happen in time,” according to the project proposal. Grosser hopes to have a system that will be able to perform a sophisticated “call and answer” musical response in a year.

“The ultimate goal is that a human performer should perceive what it hears from the system as musical communication,” he said. “I don’t expect our jazz robot to be a Miles Davis. Maybe it can be a high schooler, if we can really nail it.”

The project builds on previous work by Grosser, including an interactive robotic painting machine. The machine creates oil paintings while considering what it “hears” as input into the painting process.

Grosser said he is interested in not just artificial intelligence, but also the cultural, social and political effects of software, and the question of whether a computer system can truly make art.

The jazz robot project “is valuable for evaluating computational systems, but also in how we think about what art is. I like the questions it poses,” he said. “We’re increasingly interacting with intelligent systems. How do the designs of these systems change our interactions with them and with other humans?”

Working with Grosser on the five-year, $2.4 million project are Kelland Thomas and Clay Morrison, professors in the University of Arizona’s School of Information; and Colin Dawson, a math professor at Oberlin College.

 

Editor’s notes: To reach Ben Grosser, email grosser@illinois.edu.

Read Next

Life sciences Portrait of the research team posing together.

Minecraft players can now explore whole cells and their contents

CHAMPAIGN, Ill. — Scientists have translated nanoscale experimental and computational data into precise 3D representations of bacteria, yeast and human epithelial, breast and breast cancer cells in Minecraft, a video game that allows players to explore, build and manipulate structures in three dimensions. The innovation will allow researchers and students of all ages to navigate […]

Arts Photo of seven dancers onstage wearing blue tops and orange or yellow flowing skirts. The backdrop is a Persian design.

February Dance includes works experimenting with live music, technology and a ‘sneaker ballet’

CHAMPAIGN, Ill. — The dance department at the University of Illinois Urbana-Champaign will present February Dance 2025: Fast Forward this week at Krannert Center for the Performing Arts. February Dance will be one of the first performances in the newly renovated Colwell Playhouse Theatre since its reopening. The performances are Jan. 30-Feb. 1. Dance professor […]

Honors portraits of four Illinois researchers

Four Illinois researchers receive Presidential Early Career Award

CHAMPAIGN, Ill. — Four researchers at the University of Illinois Urbana-Champaign were named recipients of the Presidential Early Career Award for Scientists and Engineers, the highest honor bestowed by the U.S. government on young professionals at the outset of their independent research careers. The winners this year are health and kinesiology professor Marni Boppart, physics professor Barry Bradlyn, chemical and biomolecular engineering professor Ying […]

Strategic Communications and Marketing News Bureau

507 E. Green St
MC-426
Champaign, IL 61820

Email: stratcom@illinois.edu

Phone (217) 333-5010