Music and visual arts have not only developed along parallel tracks during the past several hundreds of years. Their paths have often crossed, leading into mergers and hybrids. Particularly since around 1900, many efforts have been made to create "visual music" involving new instruments such as color pianos. These developments are connected with media arts and contemporary music by multiple threads. This lecture presents a media archaeology of the connections between sounds and music, pointing out key motifs that have emerged between the eighteenth century and the present.
The Los Angeles Philharmonic’s multimedia presentation of Edgard Varèse’s Amériques launched the new in/SIGHT series at Walt Disney Concert Hall. The presentation of Amériques was led by conductor laureate Esa-Pekka Salonen and accompanied by Refik Anadol's site-specific architectural video installation, which was developed to illuminate and enhance Varèse's composition and to activate the architecture of Walt Disney Concert Hall. The dynamic visual program Anadol created used custom-built algorithmic sound analysis to listen and respond to the music in real time, using architecture as a canvas and light as a material. Additionally, Salonen’s movements, as he conducted, were captured by next-generation Microsoft Kinect hardware and 3-D depth camera analysis to inform the visuals displayed. The result was a powerful and immersive experience for the audience that engaged their visual and auditory senses.
I will speak about the status today of the interactions between music and image, through the developing forms of visual concerts and interactive installations. More specifically, I will concentrate on some of the pieces that will be presented during the two visual concerts with the music of Kaija Saariaho and mine on November 7th and 8th at Schoenberg Hall, and will also present the installations Nox Borealis and Garden of Dreams, and finally the project Distant Mirrorsm, which has musicians play together over the Internet with a visual and musical score produced in real time.
Because of the very different ways sound and light are perceived, and also because of some stark physical differences between the media, paradigms that "work" in the audio domain are often ill-suited to that of video, and vice versa. In this talk I'll describe the different approaches we've taken to the two problems and seek ways that insights from one domain might be applied to the other.
“How is it that rhythm and melodies, altho only sound, resemble states of the soul?” —Aristotle.
Music and animation are both arts that create something analogous to experience or consciousness. Imagine walking into a beautiful forest… you experience the light and the sound… the environment is revealed as you move through it. What do we mean by experiential art? Visuals and music both use a timeline, but their expression takes different forms. Multi-dimensional and multi-sensory works are changing what artists create and how the work is experienced. Patterson and Reckinger will give an overview of their work and process of making visual music in their careers.
Déserts is a film designed to accompany a live performance of the music of Edgard Varèse. Varèse had wanted to create a total image/sound work, but was unable to realize the visual component in his lifetime. He left only very general references to the images he envisioned for the music, preferring to leave the specifics to the filmmaker, as in the following statement:
“For me, déserts is a highly evocative word. It suggests space, solitude, detachment. To me it means not only deserts of sand, sea, mountains, and snow, of outer space, of deserted city streets, not only those stripped aspects of nature that suggest bareness and aloofness but also the remote inner space of the mind no telescope can reach, a world of mystery and essential loneliness.”
Premiered in 1954, Varèse’s Déserts is notable for its use of taped sound collages that interrupt the live music at three points in the score. The film uses this basic structure to describe the stark contrast between the interior space of a man alone in a windowless room, and diverse scenes of an external world devoid of people – shimmering desert vistas, undulating underwater landscapes, stark vacant streets at night, and the intense luminous heat of a raging fire. Here, images of desolate and violent destruction do not necessarily signify negation, loss, or final annihilation, but rather they become vehicles of transcendence and transformation, ultimately focused on a deepening knowledge of the self. In the end, the stark relief established between interior and exterior, solitude and space, breaks down in a crescendo of destruction and liberation, both musically and visually, as surface appearances are shattered and the two isolated worlds above and below the water merge into one.
NOCTURNE
(1994)Nocturne
was written in early 1994 and dedicated to the memory of the Polish composer, Witold Lutoslawski. The piece is an exploration of early ideas for Kaija Saariaho’s violin concerto,Graal Théâtre
, which was premiered in London at the 1995 BBC Promenade Concerts by Gidon Kremer. The first performance ofNocturne
was by John Storgårds on 16 February 1994 in Helsinki.BALLADE
(2005)Why Ballade? Manny asked specifically for a piece bearing this title, and I wrote it for him in July 2005.
FROM THE GRAMMAR OF DREAMS
(1988/2002)The Bell Jar
,Ariel
.The texts are strong, dealing with life and death, escaping into madness, self-destruction and the fight against it. Nevertheless, the piece includes an evolution: the painful nightmare ends in daylight and life. The emotional context of these texts, powerful in the extreme, led me to look for strict rules of musical organization, to contrast the emotional power.
However, these rules do not always proceed in a rational or combinatorial thinking, but rather in the manner of our dreams, where thoughts are transformed into visual images with their colors, juxtapositions, movements and directions. I have sought to operate in the same way by opening the text with two voices, and creating with them five different soundscapes.
Note about this performance: in the present version, one voice is pre-recorded by the same singer, and spatialized around the audience, together with the live voice but with different movements, reenforcing the ambiguity.
FRISES
(2011)Frises
was born of violinist Richard Schmoucler’s request who told me his idea of combining different works around Bach’s second partita for solo violin, particularly in relation to the last part, the Chaconne. He asked me to compose a piece to be performed after Bach’s Chaconne and start it with the note that ends this second partita movement, the D.My piece has four parts. I focused in each of them on the idea of one historical ostinato variation form, using as starting point carillon, passacaglia, ground bass and chaconne. There are four variations around a theme, a harmonic process or other musical parameter.
To expand the ideas and possibilities of the instrument, I added an electronic dimension to the work. According to its character, each part has a different processing. In general and in accordance with the score, prepared sound materials are set off by the musician during the piece. These materials are completed by real0time transformations of the violin sounds.
My aim in composing this piece was to create a rich work for violin with four very different and independent parts. The first part, “Frise jaune”, is a prelude, a flexible improvisation around a constant D, coloured by harmonics and the electronic part consisting of bell sounds. This part is also inspired by the idea of « carillon », a continuous melodic variation.
The second part, “Frise de fleurs”, is based on a harmony created on a ground bass. Sequences of successive chords are gradually enriched before opening to achieve a more free and lyrical development.
The third part, “Pavage” is inspired by transformations of a source material by a mathematical process where a frieze is a filling of a line or a band by a geometric figure without holes or overflow, like the paving . But I do not work in the sense of perfect symmetry - as with the cob - blestones of a patterned ground - rather to create continual metamorphosis, in the sprit of some MC Escher’s images, though less consistently.
The last part, “Frise grise”, is like a strange procession, solemn, fragile, but at the same time solved. The idea of passacaglia is here realized with slow triplets, the constant accompaniment of the left hand pizzicati on three strings, while the melody is evolving on the fourth which is not part of the accompaniment. The thematic material evolves descending slowly from E - the highest string - to G - the fourth string. The music finally reaches the initial D in double stop which take us back to the beginning of the piece.
The titles are inspired by the mathematical ideas mentioned above but also by Odilon Redon’s painted friezes which I saw recently in an exhibition dedicated to his work; especially the Yellow Frieze, Frieze of Flowers, and Gray Frieze.
Frises was composed for and dedicated to Richard Schmoucler, and commissioned by the Borusan Art Centre, Istanbul.
PRELUDE
(2005)NOANOA
(1992)NoaNoa
(‘Fragrant’ 1992) was born from the ideas I had for flute while writing my ballet music Maa. I wanted to write down, exaggerate, even abuse certain flute mannerisms that had been haunting me for some years, and thus force myself to move onto something new.Formally I experimented with an idea of developing several elements simultaneously, first sequentially, then superimposed on each other. The title refers to a wood cut by Paul Gauguin called NoaNoa. It also refers to a travel diary of the same name, written by Gauguin during his visit to Tahiti in 1891-93. The fragments of phrases selected for the voice part in the piece come from this book.
NoaNoa is also a team work. Many details in the flute part were worked out with Camilla Hoitenga. The electronic part was developed under the supervision of Jean Baptiste Barrière and programmed by Xavier Chabot.
SIX JAPANESE GARDENS
(1994)
As the title indicates, the piece is divided into six parts.
All these parts give specific look at a rhythmic material, starting from the simplistic first part, in which the main instrumentation is introduced, going to complex polyrhythmic or ostinato figures, or alternation of rhythmic and purely coloristic material.
The selection of instruments played by the percussionist is voluntarily reduced to give space for the perception of rhythmic evolutions. Also, the reduced colours are extended with the addition of an electronics part, in which we hear nature’s sounds, ritual singing, and percussion instruments recorded in the Kuntachi College of Music with Shinti Ueno. The ready-mixed sections are triggered by the percussionist during the piece, from a Macintosh computer.
All the work for processing and mixing the pre-recorded material was done with a Macintosh computer in my home studio. Some transformations are made with the resonant filters in the CHANT program, and with the SVP Phaser Vocoder. This work was made with Jean-Baptiste Barrière. The final mixing was made with the ProTools program with the assistance of Hanspeter Stubbe Teglbjaerg.
The piece was commissioned by the Kunitachi College of Music and written for Shinti Ueno.
LONH
(1996)Formally, the piece loosely follows the form of the poem, and is thus divided into nine sections. Some of the symmetrical and repeating aspects are found in the solo soprano part, which as such uses rather freely the elements for the original text, so that the resulting text is rather a collage based on Rudels song.
In the electronic part, one can hear the text in three languages: Occitan (Provençal), French and English. The texts in Occitan were read by the poet Jacques Roubaud - who has intensively studied this poem and also translated it into French - and also by Julie Parsillé, a young French girl. The French version was read by Jean-Baptiste Barrière, and the English text by Dawn Upshaw, whose recorded singing voice is also part of the sonic material heard from the electronic part.
This vocal material, as well as some concrete sounds of for example birds, wind and rain, were processed with a large variety of IRCAM transformation programs, such as resonant filters (CHANT programme), cross-synthesis and me stretching with the phase-vocoder (AudioSculpt programme), etc.
After a preliminary mixing with ProTools, the sound materials were projected into a virtual acoustic, moved through a three-dimensional space (Spatialisateur programme on the IRCAM Music Workstation).
Lonh was a prologue for my first opera project and dedicated to Dawn Upshaw.
In this project, Erik Satie’s Vexations (1893-4) was chosen as a concept, a radical effort to deconstruct the framework of his piece and combine it with the illusory space and transgress the normal boundaries of the listening and viewing experience, to set out to transform the conventional flat cinema projection screen into a three dimensional kinetic and architectonic space of visualisation of the various repetitions by using contemporary algorithms.
Light is the major element in the experiment, used to blur and interconnect the boundaries between the two realms actual/fictional and physical/virtual. It signifies the threshold between the simulacrum space created by the projection technology, and the physical space where the viewer stands. The experiment will discuss the inherent spatial qualities of immersive virtual environments and their effect on the embodied person. Through the presented framework, the experiment intends to question the relativity of perception and how it informs the apprehension of our surroundings. Rather than approaching the medium as a means of escape into some disembodied techno-utopian fantasy, the project sees itself as a means of return, i.e. facilitating a temporary release from our habitual perceptions and culturally biased assumptions about being in the world, to enable us, however momentarily, to perceive ourselves and the world around us freshly.
PARTITA NO.3 IN E MAJOR, BWV 1006: PRELUDIO
“The wonderful thing is that the music of Bach can appeal to the eye and intellect when you look at it on the page. But its fundamental appeal is to the ear and to the emotions and to the spirit when it’s performed. And that’s the huge joy of performance, that you can see it coming alive before your eyes.” —Sir John Eliot Gardiner
qsqsqsqsqqqqqqqqq
(2009)Inspired by Perich’s music, we created our moving images using systems and visual processes. We overlap layers of movement, color and texture to create a choreographic tapestry. This audio-visual counterpoint takes the simplest digital expressions – one-bit tones and moving color vector shapes - and mixes them on stage with the musicians. The jubilant and bright, bell-like quality of the sound inspired us to use lots of color. The moving shapes are like little performers expressing the energy and gestures in the music.
SUITE NO.1 IN G MAJOR, BWV 1007: SARABANDE
The sarabande was a dance dating from the Spanish New World colonies in the 1500s originally known as the “zarabanda”. The dance was considered indecent and actually banned in Spain in 1583. It’s popularity soared in the 16-17th centuries and crossed the ocean back to Spain, spreading to Italy and on to the French court in the 1700s. By then, the sarabande had become a slow stately dance. The 18th century Baroque composers typically included a Sarabande in their dance suites.
Bach’s cello with its focus on the solo voice, makes the listener aware of the silence. Restraint and negative space in our images reflect this solo voice. Atmospheric particles of light reflect the resonance and overtones from the cello. The dancers in our piece are improvising in solitude, each imagining a partner. The dynamic ebbs and flows as the solitary figure breaks from stillness to movement and back again.
AND SO THE WIND BLEW...
(2009)
“After I decided to compose this piece, I started thinking about the sound of wind chimes and how the wind is typically the performer of these instruments. I put even more thought into this concept and realized that the wind could perform a ‘piece’ on wind chimes with an infinite amount of variations, day in and day out. Perhaps when the wind is blowing and activating wind chimes hanging on a porch or on a tree in the backyard we do not pay enough attention to the wind’s performance.”
The Koshinski piece immediately suggested to us images of natural elements: wind, water and trees. The music had a randomness to it, yet there is a growing sense of pattern and order as the piece progresses and develops. We liked the idea of synthesizing randomness and order. Nature often appears random, but our focused attention brings awareness of the inherent order in nature’s movement. We liked the apparent contradiction of stillness and motion in the music. The moving image is a sequence still images united by time. There is a tension between stillness and motion. But, a sequence of still images also creates flow and unity. We explored this dialectic in And So The Wind Blew…
To allow new art to develop is to allow it to change us. What would be the use of art, if it was not to change the world, even just for a moment?
CHRÉODE
(1983)This research on musical processes, their fields of action and their limits, is a strategy of approaching the musical territory, as it has been renewed by the possibilities brought in by computers.
A very general purpose of this project, using the CHANT and FORMES programs developed at IRCAM, was to experiment with different types of organizations, and at a higher level to structure them in time and formally.
The piece won the Prix de la Musique Numérique at the Concours International de Musique Electro-acoustique in Bourges in 1983. It is dedicated to the CHANT/ FORMES project and to Kaija Saariaho.
The piece is a kind of evocation of Maurice Maeterlinck’s play Les Aveugles (The Blind), itself based loosely on the similarly titled painting by Pieter Bruegel the Elder, and is a revisiting and development of the flute materials I composed for the multimedia show Deux Songes de Maeterlinck d’après Bruegel (Two Dreams of Maeterlinck after Bruegel), first performed by Hoitenga at the Festival Les Musiques in Marseilles in May 2007.
In this new piece, the flutist is in a certain way playing the character of the blind people lost in the forest, the people of the Maeterlinck play. She is lost in an unknown world, and must heighten all her sensations and skills in order to try to survive the dangers all around her.
The virtuoso flute playing is challenged by sophisticated electronic transformations in an uncertain conflict, one whose outcome may be left open, undecided. Images, mixing cross-transformations of the live performance of the flutist with images of forests devastated by storm, are meant to represent and accompany this quest undertaken by means of the senses.
Violance proceeds from the search for a new scenic and musical form, merging instrumental writing, images, texts, and sounds transformed by computer. The piece belongs to my Reality Checks cycle, which includes interactive installations, stage pieces, and concert works. All are investigations, by means of the senses, into questions of identity and representation in the digital age, as explored and renewed by bringing about dynamic interactions between artistic disciplines in computer-assisted creative work.
This cycle includes, among other works, a piece for cello and electronics, Cellitude (a compound of “cello” and “solitude”), based on an old Japanese poem on the difficulty of distinguishing between dream and reality. Violance is its continuation in spirit, this time concerned with the idea of violence.
The piece starts out from the Massacre of the Innocents as described in Matthew’s gospel, painted again by Pieter Bruegel the Elder, and given a literary interpretation, after this painting by Maurice Maeterlinck, then a young poet. I have adapted Maeterlinck’s text to develop its universal dimension, outside of any religious and nationalistic context, and unfold its span for all times and places.
Materials are staged, assembled, and processed, together with other sources from various origins, to propose an enigmatic re-reading, a mise en abîme altogether of the myth, the painting, and the poem, an attempt at an extra-temporal reflection on the representation of violence and war.
A timeless African lullaby, computer-analyzed, produced melodic interpolations for the violin part and harmonic textures for the electronic. The “child’s voice” reciting the text is created from that of Raphaële Kennedy, and the visual aspect combines prepared imagery with live capture.
The work was commissioned by the French government and performed for the first time at the Théâtre de La Criée, Marseilles, in May 2003.
A relatively restricted set of instruments (bell plate, low cow bell, and tympani, Korean gong, Chinese cymbal, log drum, bongos, temple blocks, snare drum, crotales), was chosen to represent the different timbre families on a sort of conceptual map. Categories defined as such were then used to elaborate interpolations, formal developments that constitute paths through the sonic material represented. A similar approach was carried out for rhythm, starting from archetypes, rhythmic characters.
Music then proceeds from explorations of qualities of time, light, and color, and also of language games, these appearing progressively in the electronics.
Each of the percussion player’s gestures is prolonged by the computer, triggering bits of language, processing of the sound of the instruments, synthesis of musical fragments, and also prepared sequences and processing of live images of the percussionist, as well as of different natural sources that were pre-recorded.
Like Violance and Cellitude, Time Dusts belongs to the Reality Checks series, which stages interactive situations under the form of installations, as well as concert pieces under the form of performances, both based on the confrontation, in one case of the spectator, in the other of the musician, to his or her own reflection and its electronic becomings.
Sounds and images, captured and transformed in real time, are mixed and interpolated with pre-recorded sources coming from percussion instruments and other origins, mainly natural.
Thus the electronic involvement prolongs the instrumental writing, reveals a hidden becoming of the instruments (cf. Gérard Grisey: “Music is the becoming of sounds”). In this case, vowels and consonants prolong percussive attacks and resonances, quasi-obsessional pulsations and polyrhythms, to become figures in a musical dramaturgy that takes place at the borders of music, language, and image.
These women knew, in the one case, the noise and the fury of the Commune, in the other that of the Second World War. Witnesses of the social violence of their times, they held great hopes, smashed in great disappointments. They knew, too, the uprooting of forced exile: imprisonment for the one, resulting in the revealing discovery of another oppressed culture in New Caledonia, and for the other an emigration that led to ultimate disillusion, exhaustion, and death in England.
Ekstasis is built on two extremely strong and contrasted poems: “La Porte” (“The Gate”, 1941) by Simone Weil, and “Pensée dernière” (“Last Thought”, 1887) by Louise Michel. While Simone Weil evokes waiting at the threshold of the experience of the sacred, Louise Michel delivers the song of indignation and the exhortation to fight.
The two women are incarnated by a single soprano, varying between registers and colors, a voice with two faces, a character with two dimensions, closely intertwined. Meanwhile, other fragments of the two women’s writings, related to the chosen poems, are sung by a recorded “choir” of the same soprano multi-tracked, processed and spatialized around the audience by electronic means.
Ekstasis, a piece for soprano, electronics, and video display, takes the form of a double portrait, or better, a cross-portrait, which oscillates between the background noise of the violence of the world and the internal silence of reflection, hesitates between engagement in armed struggle and withdrawal into the self in search of the absolute. Here are two experiences that proceed from the ekstasis, the “stepping aside,” outside of normality, or outside of the world, two feminine singularities asserting themselves during the era of the masses (cf. Elias Canetti: Masses and Power) to encounter the utopia of a reconciled community.
The electronics include a studio-recorded part, triggered during the performance, and a real-time part with transformations and de-multiplication of the voice, but also with voice synthesis, using especially the techniques developed at IRCAM and other musical research centers with which I have collaborated, such as the Computer Music Center at Columbia.
The visual part extends the vocal and electronic sound in an abstract way, transporting in real time the face of the singer into virtual sets evoking the characters’ emotional evolution.