History of Spatially Distributed Performance
aka Milestones in Real-Time Networked Media
This page is an early attempt to document the key points in the
history of real time networked media, specifically related to
distributed human interaction as evidenced by the possibility of
networked music applications. Some of the milestones listed are based
on satellite communication of analog signals, which, while not
strictly "network-based" are considered sufficiently relevant to
warrant inclusion here.
Note that prior examples exist of network audio distribution (e.g.,
Elisha Gray's
transmission
of "familiar melodies" over telegraph wire in a public
demonstration on December 29, 1874). No doubt inspired by this
demonstration, Jules Verne wrote the following in "Une Ville Ideale"
(1875). "Le célèbre broyeur d'ivoire, Pianowski, jouait
à Paris, à la salle Hertz; mais au moyen de fils
électriques, son instrument était mis en communication
avec des pianos de Londres, de Vienne, de Rome, de Pétersbourg
et de Pékin. Aussi, lorsqu'il frappait une note, la note
identique résonnait-elle sur le clavier de ces pianos
lointains, dont chaque touche était mue instantanément par le
courant voltaïque!"
For the purpose of this history, we restrict the listing to those
events that were actually implemented in practice, and allowed for
some form of interactive performance between
multiple performers or participants.
(Another excellent resource for more on
networked
performance.)
Max Neuhaus, 1966
Goal:
combine radio station with telephone network to create
two-way public aural space
Summary:
- mixed calls arriving at 10 telephones of WBAI radio station (New York) and broadcast the resulting mix over the radio
- in 1977, expanded this concept with Radio Net, a nationwide network with 190 radio stations
Mark Schubin, 1975
Goal:
remote masters class of ballet; regisseur [dance master] in New York critiqued the dancers in Colorado and his comments sent back to them
Summary:
- curated event for the American Association for the Advancement of Science at the Library and Museum of the Performing Arts in NY
- used satellite transmission to relay video of the Colorado Concert Ballet to NY
- ballet master Michael Katcharoff, gave a master class from NY, with his voice sent to dancers via telephone line to reduce latency
- Katcharoff kept forgetting there was no return video link, so he would say things like, "No, you must do it like THIS!" The dancers then asked, "Like what?"
Kit Galloway and Sherrie Rabinowitz, 1975-1977
Goal:
several distributed performing artists appear and perform together in the same live image; able to see and talk with each other
Summary:
- use satellite transmission over long distance networks
- performed a number of telecollaborative dance and music scores to determine what genres could be supported and what new genres would emerge
Douglas Davis, 1977
Goal:
use satellite feeds to create multisite video art performances
Summary:
- artist interacted with audiences in other countries
several similar projects include "Send/Receive" (Liza Bear, Willoughby Sharp, Sharon Grace, Carl Eugene Loeffler: 1977), "WorldPool" (Robert Adrian, Norman White: 1977), Audio Scene 79 (Modern Art Galerie, Vienna, 1979), "Artists Use of Telecommunications Conference" (Museum of Modern Art, San Francisco, 1980), "Hole in Space" (Galloway and Rabinowitz, New York - Los Angeles, 1980), "The World in 24 Hours" (Robert Adrian, Ars Electronica, 1982)
Kit Galloway and Sherrie Rabonowitz, November 11, 1980
Goal:
Experiment with satellite video link for a variety of artistic projects.
Summary:
- life-sized video projection screens in Los Angeles and New York City
- connected by satellite, each screen displayed life-size images of unexpecting passers-by on the opposite coast
Larry Austin, May 10, 1981
Goal:
mixed live/pre-recorded distributed radio performance
Summary: 'Radiophonic' composition for synchronized, live radio
broadcast performance on CBC Radio, from Halifax, Toronto, and
Winnipeg. Four voices of an eight-voice canon are performed by eight
musicians, the remaining four -- 'the computer band'--played as digital
synthesizer sequences pre-recorded on tape, each voice entering in
turn in exact melodic/rhythmic imitation. The click tracks are timed
so that the eight voices come into melodic/rhythmic
unison--phase--five times during the piece; i.e., the voices
momentarily catch up with one another, only the next moment to
continue the acceleration or deceleration, as the case may be.
Jean Piché, Osamu Shoji, Martin Wesley-Smith, August 1983
Goal:
multisite audio performance
Summary:
- three distributed composers play Fairlight CMI instruments
from Vancouver, Tokyo, and Sydney, linked by satellite
- satellite delays of 300 ms
- performers must compensate: Syndey has to hold back its entrances
for a beat after hearing a cue from Canada, which in turn must pause
after getting a cue from Japan.
Bischoff, Brown, Perkis, Stone, Trayle, Gresham-Lancaster, 1985
Goal:
network-based electroacoustic performance
Summary:
- musicians built their own instruments, comprising hardware and software
- "instruments" took advantage of recent technologies of MIDI synthesizer and microprocessor
- initial experimental concert connected two groups of three musicians at the Clocktower and the Experimental Intermedia Center in New York City, at each location over telephone lines via modem
- Lancaster notes that "Altough the group performed at separate locations a few times, it created its strongest and most interesting work with all the participants in the same room, interacting directly with each other and with the emergent algorithmic behavior of each new piece"
Francoise Legrand, 1988
Goal:
Could five hundred voices around the world be connected via satellite
to an international all-star symphony orchestra in Montreal's Place
des Arts, and sign Beethoven's "Ode to Joy" in global harmony and --
more or less -- at the same time?
Pieces played:
Ode to Joy
Arrangement:
Moscow, Geneva, San Francisco, World Philharmonic Orchestra (in
Montreal), performed live before a television audience
Summary:
- satellite delay of 1 second
Eve M. Schooler et al, USC/ISI and BBN, Sept. 21, 1993
Goal:
demonstrate network "Flow Synchronization Protocol" (developed at BBN)
to combine data for one-way streaming
Pieces played:
Haydn Piano Trio, No. 1 in G, Finale
Arrangement:
Piano as continuo (conductor) in Los Angeles, violin and cello in Boston
Summary:
- several performances, including 2nd ACM Conference on Multimedia
Computing, SF, CA (Oct 1994)
- the group created and synchronized three real-time streams of music
from different Internet hosts
- delays in the order of 200 ms
- audience scattered across DARTnet, funding sponsor listening in DC
- difficult for performers to be listeners
- network synchronization of streams produced good
quality (audio clip)
Paul Hoffert, February 18, 1996
Goal:
demonstrate ATM-based technology for audio and video streaming
of a four-way jazz performance
Pieces played:
Mike Murley's "New Dreams"
Arrangement:
musicians at Bravo, Bamboo Club, and York University (Toronto), CITI (Montreal)
Summary:
- featured jazz musicians, dancers, and painters collaborating in real time
- symmetrical audio and video feeds
- performers suffered from more than a 0.5s delay but
learned to compensate through extensive practice with the technology
Dimitri Konstantas, et al., May 30, 1996
Goal:
support distributed rehearsal tasks with conductor at different
location from musicians
Pieces played: Handel's "Israel in Egypt" and
Britten's "Abraham and Isaac"
Arrangement:
one singer at GMD-Sankt Augustin, Germany; 2nd singer and pianist in Geneva
(video clip of final system)
Summary:
- ATM based environment (25 Mbps) using M-JPEG encoding
- 80 ms one-way delay due to locking of audio with video
- later, 31 ms one-way delay was achieved
- after some test and trial, singers managed to synchronize
relative to a third "listener" location
- 160 ms echo resulted in "extreme confusion" for the singers
Seiji Ozawa, February 7, 1998
Goal:
conduct choruses on five continents
Pieces played:
Beethoven's "Ode to Joy"
Arrangement:
200 singers each in Sydney, New York, Beijing, Berlin, False Bay
2000 singers at Olympic Stadium
conductor, 8 soloists, and orchestra in Nagano
Summary:
- time lag adjustor used to eliminate satellite delay
- all network events timed to the orchestra
Internet2 Initiative, November 1, 2000
Goal:
multi-location barbershop quartet over Internet2 networks
Pieces played:
"Beer Barrel Polka," "In The Good Old Summertime," "The Internet2 song"
Participants:
Tenor: Brent Gerber, at North Dakota State University
Lead: Jo Knox, at the University of Alaska Fairbanks
Bari: Kent Bradshaw, at Syracuse University
Bass: Greg Economides, at Texas A&M University
Conductor: Bob Dixon, at Ohio State University
Summary:
- quartet rehearsed via web
- each of the 4 singers in different cities, conductor in 5th
- audience in 5th city along with mixer
- network delay variances prevented the singers from seeing or
hearing each other and from seeing Dixon conducting
- technical means needed to deal with the network delays
Internet2 Initiative, November 4, 2000
Goal:
multi-location music video recording session using real-time streaming
video over Internet2 networks
Participants:
NYU, USC, U Alabama-Birmingham, U Miami and U Georgia School of Music
Summary:
- Optivision NACTM-3000 live streaming video servers located at each campus
- VS-ProTM playback system at the University of Georgia School of Music
- streaming broadcast MPEG-2 video and dual channel audio
- musicians were simultaneously connected for the performance via
timing tracks to a mixing board
- signals were merged into a final, recorded song
Chris Chafe, CCRMA, November 6, 2000
Goal:
streaming professional-quality audio to remote destinations using
established internet pathways
Arrangement:
conference site in Dallas connected to CCRMA (Stanford) for the
SuperComputing 2000 conference
Summary:
- real-time Internet transmission of CD-quality sound at 750 kbps
- TCP/IP streaming with QoS bounds on latency and jitter
- two-way telephone-style communication, streaming audio without
buffering from a remote tape deck
- two musicians, from separate booths in Dallas, played "together"
in the same space on the Stanford campus but delay was severe
RPI and NYU, February 20, 2001
Goal:
distributed musical, combining music, video, and interactivity
Arrangement:
two musicians in Troy and two at NYU - keyboards and drums
Summary:
- due to latency issues, one set of musicians lead
- resulting sound is a "free, temporal presentation of material that
is not based on pulse,"
John Wawrzynek, Berkeley, May 9, 2001
Goal:
gestural coding (e.g. MIDI) used to manage data under low
bandwidth conditions, suitable for
networked transmission in distributed musical performance
Arrangement:
one musician at Berkeley, another at CalTech, each playing on MIDI
keyboard; local feedback only
Chris Chafe, CCRMA (Stanford), August 17, 2001
Goal:
streaming professional-quality audio from remote locations in such a
way as to make musical collaboration possible
Piece played:
Brahms' Sonata for Piano and Violoncello in E minor, Op. 38
Arrangement:
cellist at various locations, pianist at CCRMA (Stanford)
Summary:
- UDP/IP streaming used instead of TCP with QoS
- good results obtained between Internet2 headquarters
in Armonk, NY and Stanford (audio clip)
in which round trip delay was estimated at 125ms, with
poorer results between McGill and Stanford, reportedly
due to dropouts (audio clip)
- Chris Chafe reports that the delays were noticeable
but the musicians were able to "catch-up" during the
pauses
- recent work at CCRMA has succeeded in bringing down
the delay within a LAN environment to <10ms
Jeremy Cooperstock and Stephen Spackman, McGill University, November 9, 2001
Goal:
minimal latency bidirectional transmission to support the audio and
video demands of a distributed musical duet
Piece played:
Lidel's Duet 1 (First Movement) from Three Duets for Violin
Arrangement:
one violinist at McGill, a second at the Haute Etudes Commerciales, several
kilometres away, both in Montreal
Summary:
- transfer of audio samples from sound card to network
buffer in 64-sample chunks for minimal latency
- video frames transferred at full-resolution (768x480) uncompressed YUV
at 30Hz (approx. 220 Mbps)
- since the video acquisition is slower than audio, no attempt was
made to synchronize the two; we felt it was more important to output the
audio samples immediately
- audio latency was sufficiently low (approx. 20 ms)
for real-time distributed duet with no extra effort by performers
- video latency was slightly noticeable, but quite tolerable
- MPEG videos of rehearsal (a
Hayden duet, played on Nov. 2, 2001 with audio only) and
the actual performance
Summary Update: June 13, 2002:
- (with assistance of Chris Chafe and Scott Wilson, CCRMA)
- cross-continental jazz jam between McGill and CCRMA (Stanford)
- video frames transferred at full resolution (768x480) uncompressed YUV
at 30Hz (approx. 220 Mbps) from Stanford to McGill
- video frames transferred from McGill to Stanford at
full resolution in a pseudo-run-length encoding manner in order to reduce
demands on the bandwidth-starved PCI bus (33 MHz/32 bits) at the Stanford end
- again, no attempt to synchronize audio and video
- with video turned off, we were able to decrease the audio buffering
with resulting latency sufficiently close to the one-way
network delay (approximately 45 ms)1 to support reasonable
synchronization between our vocalist and the instruments at both ends of the connection
1This web page previously misreported the figure as 90 ms, but this was the measured RTT, not one-way network delay.
This document authored by Jeremy Cooperstock
Last update: January 27, 2013