Related Links

Overview

The Ultra-Videoconferencing project at McGill University has developed a flexible, low-latency IP transport system for audio, video, and vibrosensory data. We have used this system for a range of demanding applications including live concert streaming (1999), remote mixing (2000), collaborative performance (2001), distance masters classes over SDI (2002), and remote video interpreting of sign language using three simultaneous DV streams (2003). Our software supports high-resolution multichannel audio and video from either frame grabbers using v4l or v4l2 interfaces (e.g., Bt878 chipset), digital video (DV) cameras, DCAM (raw, uncompressed video) 1394 devices, and standard or high-definition Serial Digital Interface (SDI). We are also working on mosaicing of video displays with minimimal latency and moving toward super-resolution techniques to create video displays beyond the quality of high-definition.

Development began with a project to stream Dolby AC-3 audio (5.1 surround sound) over the Internet from Montreal to New York, as part of the 1999 Audio Engineering Society Convetion. The following year, the software was rewritten to support the streaming of 12 channels of 24bit/96kHz PCM audio between Montreal and Los Angeles.

Since then, the software was expanded to include the transport of analog, digital, and SDI video, and further updates reduced the transport latency to levels enabling effective interaction at a distance, as used for a real-time violin duet, a cross-continental jazz jam, remote sign-language interpreting, and distance Masters classes with Maestro Pinchas Zuckerman.

The development of this software was supported largely by research grants and contracts from Canarie Inc. and Valorisation Recherche Quebec.

Interactive Cross-Continental Jazz

On October 24, 2002, musicians from Stanford University and McGill joined in a highly successful evening of cross-continental jazz, playing together in surround sound and full-screen video, over the Internet, using our Ultra-Videoconferencing system in analog video mode. Despite the performers being at opposite sides of the continent, the system maintained a delay (during the good moments -- as problems with an intervening router caused periodic delays that were almost double) below 50 milliseconds, which feels to the musicians almost like being on the same stage.

Video Clips

  • Bobby Hebb's "Sunny" | full MP4 (49 MB) |
  • Herbie Hancock's "Chameleon" | full MP4 (49 MB) | QuickTime extract (7.7 MB) |

    Note that the "extract" videos are encoded with H.263 and are thus, highly compressed. The audio track, captured on DAT at the Stanford side, was later stiched into the video by best approximation. The audio clips frequently in the Sunny track because of excessively high microphone gains at the McGill end.

  • Discovery Channel segment on the event: | MP4 |
The actual first attempt at this experience took place on June 13, 2002. Running the entire McGill end from a single machine, the system latency did introduce a perceptible audio echo, which was removed at the end of the evening by transmitting audio-only. Nevertheless, the event was a major success in demonstrating the feasibility of Internet support for highly interactive events across the continent.
Daniel Levitin (foreground) jams with Alex Carot (bass) and Estabin Wilson (sax); photo credit: Peter Marshall, CANARIE Inc.)

Remote Masters Classes with Pinchas Zuckerman

Through early 2002, we experimented with Maestro Pinchas Zuckerman on distance teaching applications using the Ultravideoconferencing system in uncompressed SDI video mode and supported by multichannel audio.

Video Clips

  • sample of Zuckerman's Masters class | MP4 (5.6 MB) |

Véronique Mathieu (foreground) plays for Maestro Zuckerman (on screen)

Distributed Violin Duet

On November 9, 2001, a violin duet was performed over a wide area, high speed computer network in full-screen video as part of the RISQ 2001 Conference. To our knowledge, this marked the world's first demonstration of such a high quality and low-latency system, using internet protocol communications.

Although separated by several kilometres, the two performers were able to hear each other in "near-real-time," (approximately 20 ms delay) allowing them to synchronize their playing as if standing together on the same stage. A one-way video transmission from accompanied the audio -- problems with the second video card prevented this from being a bi-directional video demonstration.

Video Clips

  • Nov. 2, 2001 rehearsal of Hayden duet | MPEG (18.5 Mbytes) |
  • Nov. 9, 2001 performance of Lidl duet | MP4 (17 Mbytes) |

Nathan Picklyk (on stage) performs with Monica Guenter (on screen)