Skip to main content

Biography

I build musical instruments and open-source software that let people play music with AI in real time. My work shows how humans and AI systems can collaborate under the fast, high-stakes conditions of live performance, and turns those lessons into tools used by musicians, students and other researchers. I lead the Sound, Music and Creative Computing Lab at the ANU.

Charles Martin is an Australian computer scientist, percussionist and digital artist blending traditional instruments with cutting-edge technology. His innovative performances explore artificial intelligence and augmented realities, creating captivating sonic landscapes. Martin’s work has been featured across Australia, Europe, and the USA.

Charles develops musical apps such as MicroJam and PhaseRings, researches creative AI, and performs music with Ensemble Metatone and Andromeda is Coming. At the ANU, Charles teaches creative computing and leads research into intelligent musical instruments. His lab’s focus is on developing new intelligent instruments, performing new music with them, and bringing them to a broad audience of musicians and performers.

Prior to his present appointment as a Senior Lecturer in computer science at the ANU, Charles was a postdoctoral research fellow on the EPEC project (Engineering Predictability with Embodied Cognition) in the University of Oslo’s Robotics and Intelligent Systems research group (2016-2019). He now leads the Australian node of Norway’s MishMash Centre for AI and Creativity.

Why this work matters

The problem

Most AI systems are built for tasks where a person types a request and waits for an answer. Live music is different: a performer and a machine have to act together, continuously, in front of an audience, with no time to review the output. Understanding how people can trust, direct and share creative ownership with an AI system under these conditions is an open problem, that we are only beginning to meet in everyday computing.

Who benefits

Performing musicians and composers who want AI as a collaborator rather than a replacement. Students learning to build creative software. Robotics and embodied-AI researchers who need methods for machines that act and adapt in the physical world. Through the MishMash Centre for AI and Creativity, where I lead an ANU node, the work reaches the sectors that centre serves: the creative industries, cultural heritage institutions, schools and lifelong learning, and arts-based health and well-being. In Australia, the music technology and creative industries sector benefits through the events, courses and open tools this work produces.

What changes

The lab's platform, IMPSY, turns any musical interface into a predictive instrument that learns from its player. It can run on a Raspberry Pi so it can live inside an instrument, and it uses small-data models that artists can train, inspect and control themselves, an alternative to opaque large-scale generative AI. It is free and open source. My students have built on it: Sandy Ma's IMPSY-based commission was shown at Science Gallery Melbourne in 2025, and lab work on it has been published at CHI, Creativity and Cognition and NIME. The companion keras-mdn-layer library is used by machine learning researchers well outside music. IMPSY's approach comes from EPEC (Engineering Predictability with Embodied Cognition), a Research Council of Norway project at the University of Oslo where I was a postdoctoral fellow from 2016 to 2019. EPEC built systems that sense, learn and predict future events in two settings: musical smartphone apps, which became my predictive-instrument line, and adaptive robots, where the DyRET quadruped learned to change its body and gait on real terrain, published in Nature Machine Intelligence. The models differ, but the problem is the same: prediction and adaptation for real-time action in the physical world. That heritage is why I am now the ANU node of MishMash, Norway's 173M NOK (A$25M) national Centre for AI and Creativity, bringing more than 200 researchers and 30 cultural organisations into reach of my lab. At home, the Sound and Music Computing course I started at the ANU has grown from 7 students in 2019 to 180 in 2025, my students have interned at Dolby and Sony AI, and hosting NIME 2025 brought the field's flagship international conference to Canberra for the first time.

Evidence

  • Google Scholar: 1,246 citations, h-index 16, i10-index 31, with 83% of citations since 2021 (retrieved 2026-09-10).
  • Postdoctoral fellow on EPEC (Engineering Predictability with Embodied Cognition), Research Council of Norway grant 240862, University of Oslo, 2016 to 2019.
  • Nature Machine Intelligence (2021) and Evolutionary Computation (2021) papers on real-world robot morphology adaptation from EPEC, the most cited work on my Scholar profile.
  • IMPSY: open-source intelligent instrument platform, 12 releases since 2019, running on Raspberry Pi hardware; basis of Sandy Ma's 2025 commission for Science Gallery Melbourne and of lab papers at CHI 2025, Creativity and Cognition 2025 and NIME 2024 to 2026.
  • keras-mdn-layer: 171 GitHub stars, 43 forks and 10 citing papers from outside the lab (retrieved 2026-09-10).
  • Lead of the ANU node of MishMash, the Norwegian Centre for AI and Creativity (173M NOK, about A$25M, from 2026): 200+ researchers, 22 Norwegian universities and research institutes, 30+ cultural organisations, with ANU the only Australian partner.
  • General Chair, NIME 2025 at the ANU: 199 attendees (149 in person, 50 remote), 96 papers, 44 musical works, 9 workshops, 7 concerts; proceedings published before the conference; accessibility chair introduced.
  • Student industry engagement: three PhD internships at Dolby and one at Sony AI, with a joint Sony AI paper at AAAI 2026; PhD students funded by the CSIRO Next Generation Graduate Program and a CSIRO Cobotics top-up.
  • Sound and Music Computing course at the ANU grown from 7 students (2019) to 180 (2025); co-led the Human Centred and Creative Computing major, now the third-largest study focus in the School of Computing and its most gender-diverse (48% non-male, 2025).
  • Three PhD completions and one thesis under examination, plus 30+ Honours and Master students; graduates hold postdoctoral positions at Université de Lille and the University of Oslo.
  • 15+ performances with intelligent musical instruments in 2024 to 2026, including the curated SoundOut Festival 2026 and a peer-reviewed performance at NIME 2026, the largest NIME to date.
  • ERC Consolidator Grant reviewer (2025); Associate Chair for ACM CHI 2026, ACM MM 2024 and ACM DIS 2022; elected NIME Board member (2024); Senior Member of the ACM and the IEEE.

Images

More details

Charles is a co-founder of Ensemble Evolution, an international percussion trio that has been featured at the Percussive Arts Society International Convention, Lithuania’s Procesas music laboratory, and Canberra’s You Are Here festival. The group released two albums of their own compositions, Sounds from the Treetops and Solstormen Live. Charles also developed interactive soundtracks and media for his cross-artform group Last Man to Die that toured Australia with their performance/installation work in 2010.

In 2012, Charles completed a Master of Music Performance in Percussion at Luleå University of Technology’s School of Music and Media in Piteå, Sweden. The otherworldly environment of Northern Sweden became a rich source of inspiration for Charles, resulting in Nordlig Vinter, his debut album of works for percussion and computer, which was released in 2013.

From 2013–2016, Charles was a PhD candidate at the ANU Research School of Computer Science, where he created iPad apps for music-making and new networked software for mediating and extending ensemble improvisation through machine learning and gesture recognition. Charles’ research work was published in ACM CHI, the premier international conference on Human-Computer Interaction, and at NIME (New Interfaces for Musical Interaction).

Charles was a postdoctoral fellow with the Robotics and Intelligent Systems research group at the University of Oslo from 2016–2019, working on EPEC (Engineering Predictability with Embodied Cognition), a Research Council of Norway project to design systems that sense, learn and predict future events. EPEC ran two lines of work in parallel: adaptive robots, including the DyRET quadruped later published in Nature Machine Intelligence, and predictive musical systems, Charles’ line, which integrated machine learning into musical smartphone apps, bio-sensor music controllers, and self-contained musical instruments. The culmination of this work was IMPS (interactive music prediction system), a standalone deep learning model for predicting musical gestures on a variety of musical interfaces, which continues today as IMPSY.

In 2019, Charles returned to the ANU and is now a senior lecturer in computer science where he teaches creative computing, music technology and continues his research in interactive intelligent systems. Charles believes strongly in collaboration between computing and creative arts to build better creative systems and train more “human” computer scientists. He is active in music technology outreach into the community such as developing autonomous guitar installations in Oslo and the electronic carillon clavier in Canberra as well as publicly releasing electronic music apps. He performs as a percussionist and computer musician both around Australia and internationally.

More information about Charles’ music and research can be found on his website: https://charlesmartin.au