The 300-Year-Old Machine Learning Model
Google’s Doodle in honor of composer J.S. Bach highlights the crossovers between music and computer science. The Doodle uses machine learning to harmonize a melody in the style of Bach. It productizes a machine learning model to incite creativity yet reveals challenges with computer-generated music that continue to puzzle scientists and composers.
Google’s Doodle brings attention to the intersections of music and computer science. Artificial intelligence as a compositional aide is one such example. Many products automate music composition including Coucou (it uses Coconet, the same model that powers the Google Doodle). Coconet is unique for being trained on J.S. Bach’s compositions; it is more common to use popular and film music as material. Coconet uses Bach’s chorales to predict harmonizations in the Classical music style.
Music Harmony
Music emerges through harmonization, an iterative process where added lines of music complement a melody. Without it, composers would quickly exhaust combinations of one-line melodies. Bach was pioneering in developing an approach to harmony that has been in use since the 1600s. The harmonic framework proposed by Bach can be summarized in a series of rules and applied in over four hundred of his chorales. Today, musicians who pursue formal training write four-part chorales in Bach’s style as part of their studies. Passing a harmony exam is a milestone while pursuing any conservatory or university degree in music; it is a metric for musical understanding.
Although Bach’s music was composed three hundred years ago, it is still used today as the foundation for machine learning models at the frontier of AI-music creation. Coconet writes music from fragments of 306 Bach chorales. The result is a model that takes incomplete melodies as an input (a user provides two measures of notes) and outputs a harmonized score.
Democratizing Creativity
The Google Doodle democratizes creativity by productizing machine learning. Coconet’s first iteration required programming proficiency making it inaccessible to creators with little-to-no programming knowledge. The second edition (Coucou) allowed users to draw, rather than code, melodies. New challenges emerged; Coucou introduced a music notation system unknown to computer scientists and composers!
Google’s ‘Big Picture’ team designed the Doodle by taking Western music notation and adding interactive elements to increase engagement. Pre-written melodies and Easter Eggs help non-musicians participate, as well. While the same model powers all of these products, the Google Doodle was the first to receive global attention because of its compelling and user-friendly design.
Would Google Pass a Harmony Exam?
The Google Doodle is successful in harmonization, although a few noteworthy mistakes point to misunderstandings. Coconet’s approach to harmonization is different from a composer’s. Coconet uses orderless modeling, where harmony and melody are two variables in constant conversation with one another. For each user-inputted note, Coconet prioritizes one of the harmony (vertical organization of sound) or melody (horizontal organization of sound). The model neglects a third variable: the larger structure of a work that is fundamental to creating a cohesive piece of music.
An underlying rule in harmony is that chorales must start and finish on the tonic or home chord (the most stable harmonization in a key signature). In between, music transits through several passages of instability. Google’s AI-generated harmonization often ends with a non-tonic chord because Coconet is trained on fragments, causing the model to lose sight of the overall musical structure. As a result, Google’s music lacks direction.
Coconet also neglects the socio-cultural context in which music exists. Bach wrote primarily for Church choirs, and his music reflects this. Word painting is a technique used by Bach to evoke the literal meaning of a song’s lyrics. Below is an example: his melody reaches its highest point when referencing the Heavens (Himmel in German). A similar effect exists in popular music; mickey mousing refers to the mirroring of film music and actions on a screen. Allowing users to input extra-musical parameters (mood, audience) is the next step in building useful AI tools for composition.
Bridging the Past and Future
Bach’s approach to harmony has governed music composition for centuries. While Coconet mistakes some of these rules, artificial intelligence has made impressive progress in musical understanding. The successful productization of Coconet in Google’s Doodle offers a glimpse into the future possibilities afforded by technology to the arts. New improvements will continue to emerge: the machine learning model will improve based on feedback provided through the Doodle’s popular interface.