Interactive Machine Learning for Music

For instance, classification is a type of supervised learning technique in which the goal is to build a function or model that is capable of outputting a label or category in response to some input values. Outside of music, classification has been very important in, for instance, e-mail spam classification: an email service typically computes some numbers related to the types of words present in an email, then it predicts whether the email is spam or not. In music performance, we might build a classifier that uses some numbers computed from a webcam to determine in real-time whether a DMI performer is waving her hand or not, or which of five hand gestures a performer is making. The output of this classifier—the current “category” of hand gesture—can then be used as a simple trigger, for instance turning samples on or off.

Regression is another type of supervised learning. Its goal is to build a model that is capable of outputting a number in response to some set of input values. In principle, this number could take on any real value (e.g., integers, fractional values like 1.5, even negative values like -22.1). For instance, a very simple regression model might be the function “output = 5 x input”, in which case the model would output -10 for the input value -2, and it would output 5.5 for the input 1.1. In music, a regression model could be used to set any number of synthesis or processing values, for instance setting the length of a virtual string in a physical model of a violin, or setting the EQ of a filter, or the amplitude of an oscillator. (In practice, of course, an instrument builder would typically build in some safeguards, ensuring that the values used to control the music are reasonable, such as ensuring gain remains between 0 and 1). In the common case that an instrument requires many such parameters to be controlled simultaneously, one simple approach is to use several regression models in parallel, each one controlling one musical parameter.

It is worth noting that today, machine learning has become synonymous with “neural networks”—a type of learning algorithm which is loosely inspired by the way that neurons in the brain work. Neural networks can be used for either classification or regression (among other things); likewise, historically, a great variety of learning algorithms have been proposed and used, and sometimes, other algorithms may be preferable.