Flow machines is a research project funded by the European Research Council (ERC) and coordinated by François Pachet. Flow machines addresses the issue of enhancing individual creativity by looking at it through the concept of “reflexive interactions”. Reflexive Interactions are human-machine interactions with a system that attempts to imitate the user’s style. The vision behind flow machines is that manipulating images of oneself creates novel and very effective ways to boost creativity. This was demonstrated in particular with the Continuator system, and with interactions with jazz professionals, as well as children (see also the ongoing Miror project). The flow machines project will push these ideas further by proposing a radically new way of looking at content creation tools.
Creativity and innovation are the new motto of the modern world. We are constantly exhorted to be creative, to think out of the box, to push our limits. Not only individuals, but also companies constantly show that they are on the verge of themselves. A quick look at today’s big brand logos is revealing: Sony’s “like.no.other” and “go create”, NEC “empowered by innovation”, Hummer “like nothing else”, Mercury cars “New doors opened”, Dodge “Different”, Mercedes-Benz “Unlike any other”, Toshiba “Power of innovation”, HP “think beyond”, Intel’s “leap ahead”, Samsung “Be creative” or Apple “Think different”. Everything around us reminds us to always be innovative, create new things and be singular. Of course, we are used to such vacuous advertisement and we learned not to even notice. However, they are a sign that our civilization is obsessed by innovation to a point that defies rationality: If everyone was innovative and different, well, there would not be any difference at all. We are trying to take, however, all these exhortations seriously.
The key idea of the Flow Machines project is to relate the notion of creativity to the notion of “style”. Style is what makes an author (composer, writer, painter, etc.) recognizable, “different”. The assumption we make is that the key to being different is to invent a singular “style”, a unique way to do things. Of course, inventing a style is a difficult, life-long process, and so far only few individuals managed to invent styles that stand out in their respective fields: Picasso, Shakespeare, Mozart, Paul McCartney to name a few. Studying how these people came to invent their style suggests that this is basically a process of manipulating the styles of other creators to create new objects until something interesting emerges: Picasso has drawn many conventional bulls before he invented his abstract style which is so prevalent today.
The Flow Machines project takes a computer science perspective on style: how can a machine understand style and turn it into a computational object? An object that users can manipulate to create new objects with their own constraints. Technically we develop new technologies based on Markov models. Markov processes are well-known tools used to model statistical properties of temporal sequences. They are used everywhere, from economics to Google ranking algorithms. They are very good for capturing local properties of temporal sequences and abstract them into well-known Mathematical concepts (e.g., a transition matrix). However, they are also notoriously difficult to control. For instance, a Markov process can easily be estimated to represent information about how musical notes in a given style succeed to each other. This model can then be used to produce new sequences that will sound more or less similar to the initial sequences. The difficulty arises when one wants to impose simple properties that cannot be expressed as local transition probabilities to these sequences. For instance, imposing that the last note equals the first one, or that the notes follow some metric pattern, or that the total duration be fixed in advance. Or, even more difficult, that the melody is nicely “balanced”, for instance exhibiting a well-known 1/f property, characteristic of natural phenomena!
We are currently investigating style and creativity under several perspectives: Technically we have made substantial progress in developing efficient Markov Constraints algorithms that can apply many types of constraints to arbitrary Markov models. Conceptually we are starting to build authoring tools in musical composition and text writing that enable people to generate content by manipulating the style of an existing author, possibly themselves.
Suppose you want to create a song; that is, a melody with an underlying harmony. You can imagine that you choose some notes or pattern you like, then you can ask the system to “fill in the blanks” with the style of your preferred composer, such as Paul McCartney or Tom Jobim. The system proposes a sequence in the style of the author you chose and that satisfies your constraints (preferred notes, imposed harmonies, etc.).
If you don’t like the proposed sequence the system will suggest another one, and so on until you are satisfied with the result. The same ideas hold for text. A system, called Perec, illustrate this idea by allowing users to create song lyrics in the style of a preferred author and various constraints such as rhymes, prosody or syntax.
This fascinating and challenging project is just starting and our ambition is high: Invent the next generation of authoring tools that will help people manipulate the styles of their preferred authors, in music and text. These applications should be fun to use, should create Flow states, should push users to experiment with new ideas, and eventually create their own style. Stay tuned.
Read the “First Year Overview“