In September 2024 I travelled to Utrecht, Netherlands for NIME 2024, later stopping by Oslo to visit colleagues at the University of Oslo.
The trip to NIME involved presenting a workshop and seeing lab members Yichen Wang and Sandy Ma presenting their musical work at the conference.
Building NIMEs with Embedded AI Workshop
My workshop was called Building NIMEs with Embedded AI and was organised in collaboration with Teresa Pelinski from Queen Mary University of London.
Teresa’s research involves running AI models on the Bela, and my work involves running AI models on the Raspberry Pi — a difference that I frame as a creative battle of ideas. Together we were catering to whatever linux single-board computing platform NIMErs were interested in!
We split the day into two long sessions with the goal of getting everybody to get a basic AI model working on our respective systems. The workshop was (as expected) very busy with interest from NIMErs new and old.
In preparation for my half of the workshop, I had built some ultraportable AI NIME development kits. Each kit included:
- a Raspberry Pi Zero 2 W: the cheapest way to build an AI-powered NIME
- a 16GB SD card
- an OTG USB dongle to get a USB socket out of the Pi Zero
- 2x USB A to micro B cables, one for power and one for…
- a BBC Microbit v2
The Microbits were a bit of an unusual addition to the system. I wanted a very lightweight way to connect the Pi Zeros to something with sensor input and sound output. I had last used Microbits for teaching COMP2300 and Sound and Music Computing so I had lots around my lab. The Microbits have an on board speaker (sound output check) and an accelerometer (sensor input check) and the little LED matrix display (more feedback check). Programming them in their native micropython environment, I could get sounds, not necessarily anything interesting or very fun, but a noisy kind of sound effect related to the accelerometer. I frame these as the world’s worst NIME and I may not be too far wrong. The code is here if you are interested. In any case, they worked for demonstration purposes connected to the IMPSY Pi systems.
Performances and Demos
The conference featured a rich program of performances and demos alongside the paper and workshop sessions. Here’s a few images of NIMErs interacting with cool works.
Sandy Ma and I presented our work in as “work in progress” demos (accurate as our works are works in progress! I showed my IMPSY intelligent music platform running with a Roland S-1 synth and Sandy demonstrated her Touching Wires sensor blanket.
A highlight of this NIME was seeing my SMCClab students Yichen Wang and Sandy Ma performing their new work Unspoken as part of the conference.
Getting music works into NIME is no easy task. Space (and time) is tighter in the music program than for papers so fewer works are programmed and the competition can be really tough. I was so proud to contribute as a supervisor to this work, and happy to watch from the audience this time!
Yichen describes Unspoken as: “A mixed reality duet performance that explores the aesthetic possibilities of augmented reality technology as a medium for collaborative musical expression. Musicians with their tangible musical systems, communicate gestural and spatial musical intentions through a shared AR interface.”
This work combines my lab’s research directions of AR performance and collaboration to create an original work that couldn’t exist without the technologies involved. The staging and performance of Unspoken in Utrecht was just stunning. Well done Yichen and Sandy!
Announcing NIME2025
The end of NIME is a traditional time to announce the next NIME to the community. This was a big moment for me as we are hosting NIME2025 in Canberra! I introduced the rough plans for NIME at ANU, including our plan for hybrid and remote options for attendance, and played our teaser video publicly for the first time.
University of Oslo Meetings
Next stop was Oslo to catch up with colleagues at the University of Oslo and around town. This was a chance to catch up with Dr Benedikte Wallace to hear what has been happening since completing her PhD and starting as a postdoc, and to connect with the rest of my network at ROBIN and RITMO.
Going back to Oslo always feels like heading to another home, even more so now that Oslo’s new trams are the same as Canberra’s.
Among many meetings with colleagues and new students I was able to repeat my AI NIME workshop with students from the Musicology Department (and give a little concert for them) as well as present my recent research on Intelligent Musical Instruments to the ROBIN group.
Thanks to colleagues and friends Jim, Kirsten, Kyrre, Alexander, Benedikte, Tejaswinee, and Bernt-Isak for the good conversations and productive meetings!