Interactive Machine Learning for Music

If you are interested in using IML yourself, here are some tools you might find helpful:

• The Wekinator website contains free downloads, sample code to connect Wekinator to many sensors and music environments, and tutorials to get you started.
• The Flucoma project includes a number of ML tools in Max/MSP, SuperCollider, and PureData.
• Teachable Machine by Google uses an approach to interactive machine learning based on Wekinator to allow anyone to build simple image, audio, and pose classifier models in the browser.
• ml5.js is a set of JavaScript machine learning libraries that is compatible with the very popular P5.js environment for programming animations and interactive visuals. It also allows you to load and interact with models you’ve made using Teachable Machine.
• MIMIC is a web-based creative coding platform that I worked on, with colleagues from the Creative Computing Institute, University of Sussex, and Durham University. It contains high-level libraries and many examples for using IML and generative ML in the browser.
• InteractML is an IML platform for the Unity and Unreal game engines, created in partnership with Phoenix Perry and other collaborators.

If you want to learn more about IML for musical instrument building or other creative pursuits, these are the resources I recommend:

• Machine Learning for Musicians and Artists is an online course I designed and taught, which focuses on IML and builds hands-on skills using Wekinator.
• Apply Creative Machine Learning is another online course, which explores creative ML, including IML, using JavaScript.
• Research on applications of ML in music and other creative fields is usually available to read without a paywall, including in the proceedings of conferences such as:
• New Interfaces for Musical Expression
• the International Society for Music Information Retrieval
• AI Music Creativity
• the International Conference on Computational Creativity
• and the NeurIPS workshop on ML for Creativity and Design.