An overview of AI for Music and Audio Generation

I’ll discuss recent advances in AI for music creation, focusing on Machine Learning (ML) and Human-Computer Interaction (HCI) coming from our Magenta project (g.co/magenta). I’ll argue that generative ML models by themselves are of limited creative value because they are hard to use in our current music creation workflows. This motivates research in HCI and especially good user interface design. I’ll talk about a promising audio-generation project called Differentiable Digital Signal Processing (DDSP; Jesse Engel et al.) and about recent progress in modeling musical scores using Music Transformer (Anna Huang et al.). I’ll also talk about work done in designing experimental interfaces for composers and musicians. Time permitting I’ll relate this to similar work in the domain of creative writing. Overall my message will be one of restrained enthusiasm: Recent research in ML has offered some amazing advances in tools for music creation, but aside from a few outlier examples, we’ve yet to bring these models successfully into creative practice.