Teaching the Arts to Machines

Doug will give an overview talk about Magenta, a project investigating music and art generation using deep learning and reinforcement learning. He will discuss some of the goals of Magenta and how it fits into the general trend of AI moving into our daily lives. He'll talk about two specific recent projects. First he wil discuss his research on Teaching Machines to Draw with SketchRNN, a LSTM recurrent neural network able to construct stroke-based drawings of common objects. SketchRNN is trained on thousands of crude human-drawn images representing hundreds of classes. Second he will talk about NSynth, a deep neural network that learns to make new musical instruments via a WaveNet-style temporal autoencoder. Trained on hundreds of thousands of musical notes, the model learns to generalize in the space of musical timbres, allowing musicians to explore new sonic spaces such as sounds that exist somewhere between a bass guitar and a flute. This will be a high-level overview talk with no need for prior knowledge of machine learning models such as LSTM or WaveNet.

 

 

Short bio

 

Doug is a Research Scientist at Google leading Magenta, a Google Brain project working to generate music, video, image and text using deep learning and reinforcement learning. A main goal of Magenta is to better understanding how AI can enable artists and musicians to express themselves in innovative new ways. Before Magenta, Doug led the Google Play Music search and recommendation team. From 2003 to 2010 Doug was an Associate Professor in Computer Science at the University of Montreal's MILA Machine Learning lab, where he worked on expressive music performance and automatic tagging of music audio.