History of Spatially Distributed Performance

aka Milestones in Real-Time Networked Media

This page is an early attempt to document the key points in the history of real time networked media, specifically related to distributed human interaction as evidenced by the possibility of networked music applications. Some of the milestones listed are based on satellite communication of analog signals, which, while not strictly "network-based" are considered sufficiently relevant to warrant inclusion here.

Note that prior examples exist of network audio distribution (e.g., Elisha Gray's transmission of "familiar melodies" over telegraph wire in a public demonstration on December 29, 1874). No doubt inspired by this demonstration, Jules Verne wrote the following in "Une Ville Ideale" (1875). "Le célèbre broyeur d'ivoire, Pianowski, jouait à Paris, à la salle Hertz; mais au moyen de fils électriques, son instrument était mis en communication avec des pianos de Londres, de Vienne, de Rome, de Pétersbourg et de Pékin. Aussi, lorsqu'il frappait une note, la note identique résonnait-elle sur le clavier de ces pianos lointains, dont chaque touche était mue instantanément par le courant voltaïque!"

For the purpose of this history, we restrict the listing to those events that were actually implemented in practice, and allowed for some form of interactive performance between multiple performers or participants.

(Another excellent resource for more on networked performance.)

Public Supply

Goal: combine radio station with telephone network to create two-way public aural space

Summary:

• mixed calls arriving at 10 telephones of WBAI radio station (New York) and broadcast the resulting mix over the radio
• in 1977, expanded this concept with Radio Net, a nationwide network with 190 radio stations

The Performing Arts & the Future of Television

Goal: remote masters class of ballet; regisseur [dance master] in New York critiqued the dancers in Colorado and his comments sent back to them

Summary:

• curated event for the American Association for the Advancement of Science at the Library and Museum of the Performing Arts in NY
• used satellite transmission to relay video of the Colorado Concert Ballet to NY
• ballet master Michael Katcharoff, gave a master class from NY, with his voice sent to dancers via telephone line to reduce latency
• Katcharoff kept forgetting there was no return video link, so he would say things like, "No, you must do it like THIS!" The dancers then asked, "Like what?"

Satellite Arts Projects

Goal: several distributed performing artists appear and perform together in the same live image; able to see and talk with each other

Summary:

• use satellite transmission over long distance networks
• performed a number of telecollaborative dance and music scores to determine what genres could be supported and what new genres would emerge

The Last Nine Minutes

Goal: use satellite feeds to create multisite video art performances

Summary:

• artist interacted with audiences in other countries

Hole in Space

Goal: Experiment with satellite video link for a variety of artistic projects. Summary:

• life-sized video projection screens in Los Angeles and New York City
• connected by satellite, each screen displayed life-size images of unexpecting passers-by on the opposite coast

Canadian Coastlines: Canonic Fractals for Musicians and Computer Band

Goal: mixed live/pre-recorded distributed radio performance

Summary: 'Radiophonic' composition for synchronized, live radio broadcast performance on CBC Radio, from Halifax, Toronto, and Winnipeg. Four voices of an eight-voice canon are performed by eight musicians, the remaining four -- 'the computer band'--played as digital synthesizer sequences pre-recorded on tape, each voice entering in turn in exact melodic/rhythmic imitation. The click tracks are timed so that the eight voices come into melodic/rhythmic unison--phase--five times during the piece; i.e., the voices momentarily catch up with one another, only the next moment to continue the acceleration or deceleration, as the case may be.

Digicon 83: Night Satellite

Goal: multisite audio performance

Summary:

• three distributed composers play Fairlight CMI instruments from Vancouver, Tokyo, and Sydney, linked by satellite
• satellite delays of 300 ms
• performers must compensate: Syndey has to hold back its entrances for a beat after hearing a cue from Canada, which in turn must pause after getting a cue from Japan.

The HUB

Goal: network-based electroacoustic performance

Summary:

• musicians built their own instruments, comprising hardware and software
• "instruments" took advantage of recent technologies of MIDI synthesizer and microprocessor
• initial experimental concert connected two groups of three musicians at the Clocktower and the Experimental Intermedia Center in New York City, at each location over telephone lines via modem
• Lancaster notes that "Altough the group performed at separate locations a few times, it created its strongest and most interesting work with all the participants in the same room, interacting directly with each other and with the emergent algorithmic behavior of each new piece"

Satellite Symphony: Beethoven and One Woman's Dream

Goal: Could five hundred voices around the world be connected via satellite to an international all-star symphony orchestra in Montreal's Place des Arts, and sign Beethoven's "Ode to Joy" in global harmony and -- more or less -- at the same time?

Pieces played: Ode to Joy

Arrangement: Moscow, Geneva, San Francisco, World Philharmonic Orchestra (in Montreal), performed live before a television audience

Summary:

• satellite delay of 1 second

Distributed Music: A Foray into Networked Performance

Goal: demonstrate network "Flow Synchronization Protocol" (developed at BBN) to combine data for one-way streaming
Pieces played: Haydn Piano Trio, No. 1 in G, Finale
Arrangement: Piano as continuo (conductor) in Los Angeles, violin and cello in Boston
Summary:

• several performances, including 2nd ACM Conference on Multimedia Computing, SF, CA (Oct 1994)
• the group created and synchronized three real-time streams of music from different Internet hosts
• delays in the order of 200 ms
• audience scattered across DARTnet, funding sponsor listening in DC
• difficult for performers to be listeners
• network synchronization of streams produced good quality (audio clip)

Cyber Soirée

Goal: demonstrate ATM-based technology for audio and video streaming of a four-way jazz performance
Pieces played: Mike Murley's "New Dreams"
Arrangement: musicians at Bravo, Bamboo Club, and York University (Toronto), CITI (Montreal)
Summary:

• featured jazz musicians, dancers, and painters collaborating in real time
• symmetrical audio and video feeds
• performers suffered from more than a 0.5s delay but learned to compensate through extensive practice with the technology

Distributed Musical Rehearsal Environment

Goal: support distributed rehearsal tasks with conductor at different location from musicians
Pieces played: Handel's "Israel in Egypt" and Britten's "Abraham and Isaac"
Arrangement: one singer at GMD-Sankt Augustin, Germany; 2nd singer and pianist in Geneva (video clip of final system)
Summary:

• ATM based environment (25 Mbps) using M-JPEG encoding
• 80 ms one-way delay due to locking of audio with video
• later, 31 ms one-way delay was achieved
• after some test and trial, singers managed to synchronize relative to a third "listener" location
• 160 ms echo resulted in "extreme confusion" for the singers

Opening Ceremony 1998 Winter Olympics

Goal: conduct choruses on five continents
Pieces played: Beethoven's "Ode to Joy"
Arrangement: 200 singers each in Sydney, New York, Beijing, Berlin, False Bay
2000 singers at Olympic Stadium
conductor, 8 soloists, and orchestra in Nagano
Summary:

• time lag adjustor used to eliminate satellite delay
• all network events timed to the orchestra

World's First Remote Barbershop Quartet

Goal: multi-location barbershop quartet over Internet2 networks
Pieces played: "Beer Barrel Polka," "In The Good Old Summertime," "The Internet2 song"
Participants: Tenor: Brent Gerber, at North Dakota State University
Lead: Jo Knox, at the University of Alaska Fairbanks
Bari: Kent Bradshaw, at Syracuse University
Bass: Greg Economides, at Texas A&M University
Conductor: Bob Dixon, at Ohio State University
Summary:

• quartet rehearsed via web
• each of the 4 singers in different cities, conductor in 5th
• audience in 5th city along with mixer
• network delay variances prevented the singers from seeing or hearing each other and from seeing Dixon conducting
• technical means needed to deal with the network delays

Music Video Recording via Internet2

Goal: multi-location music video recording session using real-time streaming video over Internet2 networks
Participants: NYU, USC, U Alabama-Birmingham, U Miami and U Georgia School of Music
Summary:

• Optivision NACTM-3000 live streaming video servers located at each campus
• VS-ProTM playback system at the University of Georgia School of Music
• streaming broadcast MPEG-2 video and dual channel audio
• musicians were simultaneously connected for the performance via timing tracks to a mixing board
• signals were merged into a final, recorded song

QoS Enabled Audio Teleportation

Goal: streaming professional-quality audio to remote destinations using established internet pathways
Arrangement: conference site in Dallas connected to CCRMA (Stanford) for the SuperComputing 2000 conference
Summary:

• real-time Internet transmission of CD-quality sound at 750 kbps
• TCP/IP streaming with QoS bounds on latency and jitter
• two-way telephone-style communication, streaming audio without buffering from a remote tape deck
• two musicians, from separate booths in Dallas, played "together" in the same space on the Stanford campus but delay was severe

The Technophobe and the Madman

Goal: distributed musical, combining music, video, and interactivity
Arrangement: two musicians in Troy and two at NYU - keyboards and drums
Summary:

• due to latency issues, one set of musicians lead
• resulting sound is a "free, temporal presentation of material that is not based on pulse,"

Network Musical Performance

Goal: gestural coding (e.g. MIDI) used to manage data under low bandwidth conditions, suitable for networked transmission in distributed musical performance
Arrangement: one musician at Berkeley, another at CalTech, each playing on MIDI keyboard; local feedback only

SoundWIRE

Goal: streaming professional-quality audio from remote locations in such a way as to make musical collaboration possible
Piece played: Brahms' Sonata for Piano and Violoncello in E minor, Op. 38
Arrangement: cellist at various locations, pianist at CCRMA (Stanford)
Summary:

• UDP/IP streaming used instead of TCP with QoS
• good results obtained between Internet2 headquarters in Armonk, NY and Stanford (audio clip) in which round trip delay was estimated at 125ms, with poorer results between McGill and Stanford, reportedly due to dropouts (audio clip)
• Chris Chafe reports that the delays were noticeable but the musicians were able to "catch-up" during the pauses
• recent work at CCRMA has succeeded in bringing down the delay within a LAN environment to <10ms

Bronto

Goal: minimal latency bidirectional transmission to support the audio and video demands of a distributed musical duet
Piece played: Lidel's Duet 1 (First Movement) from Three Duets for Violin
Arrangement: one violinist at McGill, a second at the Haute Etudes Commerciales, several kilometres away, both in Montreal
Summary:

• transfer of audio samples from sound card to network buffer in 64-sample chunks for minimal latency
• video frames transferred at full-resolution (768x480) uncompressed YUV at 30Hz (approx. 220 Mbps)
• since the video acquisition is slower than audio, no attempt was made to synchronize the two; we felt it was more important to output the audio samples immediately
• audio latency was sufficiently low (approx. 20 ms) for real-time distributed duet with no extra effort by performers
• video latency was slightly noticeable, but quite tolerable
• MPEG videos of rehearsal (a Hayden duet, played on Nov. 2, 2001 with audio only) and the actual performance

Summary Update: June 13, 2002:

• (with assistance of Chris Chafe and Scott Wilson, CCRMA)
• cross-continental jazz jam between McGill and CCRMA (Stanford)
• video frames transferred at full resolution (768x480) uncompressed YUV at 30Hz (approx. 220 Mbps) from Stanford to McGill
• video frames transferred from McGill to Stanford at full resolution in a pseudo-run-length encoding manner in order to reduce demands on the bandwidth-starved PCI bus (33 MHz/32 bits) at the Stanford end
• again, no attempt to synchronize audio and video
• with video turned off, we were able to decrease the audio buffering with resulting latency sufficiently close to the one-way network delay (approximately 45 ms)¹ to support reasonable synchronization between our vocalist and the instruments at both ends of the connection

This document authored by Jeremy Cooperstock
Last update: January 27, 2013