Robots Taking Over Music, With Humans by Their Side

This Friday marks the start of the fifth annual Atlanta Science Festival, kicking off with Rise Up, Robots, a variety show featuring an assortment of robotic performers.

One of those performers will be Shimon, a marimba-playing robot that uses machine learning to develop new and inventive compositions. Shimon was created by Gil Weinberg, professor and founding director of Georgia Tech’s Center for Music Technology.

Many at Tech have heard of Shimon and its ability to improvise jazz melodies. This Friday, though, the musical robot will tread uncharted territory, showcasing a new rock composition composed by Zach Kondak, a graduate student in music technology, who will also play drums and guitar. Joining Weinberg and Kondak will be Richard Savery, also a graduate student in music technology, on saxophone.

“In the past, we have trained Shimon using jazz, classical, and pop, but rock is very new for us,” said Weinberg. “This will be our first time going for a prog-rock piece with a strong mathematical structure.”

Shimon could not always craft original compositions. It was originally programmed to improvise by predicting notes one at a time using probabilistic analysis. In the past few years, though, Weinberg and his team have implemented deep-learning techniques that allow Shimon to improvise longer structure, original melodies.

“Not only can Shimon create music like no human can, it can analyze and combine styles of different performers to create a whole new set of styles. I can tell Shimon to play 30 percent in the style of John Coltrane, 30 percent in the style of Thelonious Monk, and 40 percent in my own style, and Shimon will use all of that learned information to create something new.”

Shimon could have been designed more like a traditional computer – playing compositions via speaker to create a greater assortment of sounds – but Weinberg did not just want his device to play, he wanted it to perform acoustically. He and his team were very intentional about designing a machine that played a physical instrument and could connect with human bandmates.

“With Shimon, we wanted to create a machine that could perform in live settings. Shimon makes expressive gestures, allowing human performers to respond to visual cues that can facilitate more expressive improvisation, creativity, and connection,” said Weinberg.

Some may fear that new technologies like Shimon will make human musicians obsolete, but Weinberg does not believe an actual robot uprising is due anytime soon.

“The risks of artificial intelligence are not what most people think. The most common fear is that robots and artificial intelligence will destroy jobs, but I actually think that they will create more jobs,” said Weinberg. “Everyone used to think that the camera would destroy painting, but it never did. In fact, the camera not only created many new jobs in the new industry of photography, it also pushed realistic paint and sketch art to new styles and movements such as expressionism and impressionism.”

That creativity-spurring disruption, said Weinberg, is just what artificial intelligence can do for music. It’s collaboration with, not displacement by, machines that future human artists will experience.

“Creativity has never been about making something from nothing. The integration of ideas is where creativity happens,” said Weinberg. “Shimon reminds you of a style or an artist, all while being completely new. Our hope is that when you listen to it, something on an emotional level will connect.” That, said Weinberg, is where the creative power of Shimon realizes its full potential to inspire human musicians.

Weinberg hopes to demonstrate this creative collaboration on Friday, as well as in future projects that he and his team are currently developing.

“Now, we can use EEG technology to see how the brain responds when a person plays with humans versus when they play with artificial intelligence,” said Weinberg. “We’re also working on a rock-opera about artificial intelligence in which much of the music will be created by AI itself. We want this opera to be a metacommentary on how this technology is transforming the world.”

Rise Up, Robots will be held at the Ferst Center for the Performing Arts at 7 p.m. on March 9, and will also feature a robotic comedian, a bionic arm, and a robot petting zoo. You can find out more about the event here. Tickets are $15 and can be purchased on the Arts@Tech site. The Atlanta Science Festival will take place throughout the weekend at locations across Atlanta.