Health Services Virtual Organization
This project was funded by Canarie's Network Enabled Platforms (NEP) program. Participating organizations included Northern Ontario School of Medicine, Lakehead University, McGill Medical Simulation Centre,McGill Centre for Intelligent Machines, National Research Council Canada, Communications Research Centre Canada, Stanford University School of Medicine, iDeal Consulting, and Innovation in Learning, Inc.
The HSVO aims to create a sustainable research platform for experimental development of shared ICT-based health services. This includes support for patient treatment planning as well as team and individual preparedness in the operating room, emergency room, general practice clinics, and patients' bedsides. In the context of the Network-Enabled Platforms program, the project seeks to offer such support to distributed communities of learners and health-care practitioners. Achieving these goals entails the development of tools for simultaneous access to the following training and collaboration resources: remote viewing of surgical procedures (or cadaveric dissections), virtual patient simulation involving medical mannequins and software simulators, access to 3D anatomical visualization resources, and integration of these services with the SAVOIR middleware along with the Argia network resource management software.
Demos
Try the Multiple Angle Viewer to experience multiple viewing perspectives of a thorax procedure (note: no view interpolation on this version).
Real-time stereoscopic view interpolation from an arbitrary perspective.
Publications
- Real-time free viewpoint video from a range sensor and color cameras. Pelletier, S.; and Cooperstock, J. R. Machine Vision and Applications, 24(4):739-751. May 2013.
- End-User Viewpoint Control of Live Video from a Medical Camera Array. Blum, J.; Sun, H.; Olmos, A.; and Cooperstock, J. R. In International Conference on Distributed Smart Cameras, August 2011.
- Multiple Angle Viewer for Remote Medical Training. Olmos, A.; Lachapelle, K.; and Cooperstock, J. R. In International Workshop on Multimedia Technologies for Distance Leaning, pages 19-24, Firenze, Italy, October 2010. ACM
- Integrating Simulation Devices and Systems. Ellaway, R. H.; Topps, D.; Lachapelle, K.; and Cooperstock, J. R. Studies in Health Technology and Informatics, 142:88-90. January 2009.
Main activities at McGill
The project will deliver computer-based services that can be shared by
health organizations across Canada. An overview of the main use cases
of this project is illustrated here. Our lab is principally involved
in three main areas of the project:
- implementation of a video rendering architecture based on camera array for remote viewing of cadaveric dissections from arbitrary viewpoints
- user interface and integration of multiple edge devices (data and computational resources, including a Laerdal medical mannequin) for simulation training
- investigation of effective user interfaces for access to and interaction with multiple medical datasets
Chronological list of our activities
Deployment of the camera array at the MSC
August 2010 marked the end of the HSVO project at our lab. During this month, we were busy deploying the camera array at the Medical Simulation Centre (MSC) at McGill. With this final post we share with you some pictures or the deployment and an interesting surgical session where a surgeon at McGill teaches remotely some suturing techniques to a couple of students at NOSM.
View Point Interpolation using a
Time-Of-Flight (TOF) camera
The video presents a demonstration of our view point interpolation developed as part of the camera array. In these examples, a Time-Of-Flight (TOF) camera was employed to obtain a low-resolution approximation of the scene shape. Images from rgb cameras are then projected onto this model in order to perform view interpolation.
View Point Interpolation
This video presents a demonstration of our view point interpolation developed as part of the camera array. A link to our publication on this front will be listed shortly.
Simulation Challenge Series
During the months of April, May and June we will be hosting the Simulation Challenge series. These are a series of activities run with various medical schools in order to test our platform. The Universities involved in the series are McGill University, Ottawa University, NOSM and Cork University. Here we share with you some pictures showing the students interacting with their remote peers.
Work session at NRC
We got the opportunity to travel to Fredericton and meet the HSVO tech team. It was an interesting meeting about Savoir and messages specifications for our platform. Here we share some pictures from the work sessions.
Setting up a Medical Scenario
We had the opportunity to put together a demonstration involving a high fidelity mannequin, camera array and open Labyrinth. The video presents us setting up the lab in fast forward mode. Within the scenario a 4th year student, Ada, is supervised by a remote instructor, Dr. David Topps, who was the author of the scenario. This demonstration was put together with great help of the McGill Simulation Center; a big kind thank you to them. Background music in the video by Tycho, "Past Is Prologue".
Usability Study for the Camera Array Interface
We started the year completing our first usability study centred around the camera array interface, and documented in the report listed below. Thanks to all the second year medical students at McGill who came and tested the interface.
DOWNLOAD: Camera Array Interface Study, tasks and questionnaire forms - private access to HSVO.
Prototyping the physical structure of the camera array
We had the oportunity to put together a prototype around the physical structure of the camera array. We brought it to the McGill Simulation Centre in order to see how it fits within the actual dissection setting. We also mounted a camera and tested the field of view within this scenario.
Putting together the patient mannequin simulator
Through early next year we are hosting a patient mannequin simulator, the SimMan 3G from Laerdal. In these pictures you can see how the mannequin was assembled with instructions via a conference call with Dr. Rachel Ellaway, one of the Principal Investigators. We have named him Patricio for his stay with us.
Thorax dissection session: Camera array prototype
This is another iteration of our camera array prototype. Here we have added a footage of a dissection session of the thorax. We are finishing some of the missing functions and then use it for a series of user testing sessions with medical students.
User interface for the patient simulation session
We used the video footage of a previously recorded mannequin simulation session and put together a small flash prototype aiming to show how users could view from different viewing points this session. The footage belongs to the pediatric emergency simulation (from an earlier entry).
High fidelity prototype for the overall HSVO user interface
Continuing the various iterations around the user interface for the future HSVO system, another html (image based) prototype has been put together to discuss and test with the team.
Pediatric emergency: Patient simulator session
We had the opportunity to video record a session with a mannequin patient simulator. Students had to save the live of a teenager who suffered a severe accident while playing basketball. We then took this video footage and edited for the International Conference on Residency Education, 2009 at Victoria, BC. In the video three different parties get the opportunity to collaborate remotely: a group of students and the mannequin in Montreal, the mannequin operator and a tutor in Ottawa, and another tutor in Sudbury, Ontario. Kind thanks to all the staff at the McGill Simulation Center for all their help in making this possible.
Scripting the HSVO use cases and task scenarios
This document present the narrative scripts for the user case scenarios, and the various steps followed by the users in order to complete a task. This script can be read along with the sketch prototype presented in the previous post.
Sketching the overall user interface for HSVO
Thinking of how users would call and interact with all the tools and services deployed with the HSVO system, we started to sketch out the overall interface. The pictures presented here shown some of the line of thought. The sketches were scanned and used to create an low fidelity html prototype to discuss and test with the team.
Observing surgical video recordings
The McGill simulation center has already the capability to record dissection sessions and surgical procedures with a camera mounted in the ceiling. The camera can be operated remotely (position, zoom, pan and tilt). Given this, we took the job of watching hours of their recordings in order to learn how often they update their camera view point and zooming settings while recording a session.
The monkey prototype
We had the opportunity to put together a prototype in order to show how users could interact with the camera array. It is called the monkey prototype because the 36 viewing points are video shots of a plastic monkey moving his head.
Prototyping the physical placement of the cameras
The photographs presented here present a scale model of some of the ideas we have been brainstorming around building the physical array of cameras.
Camera array user interface proposal
This document present the persona, use case scenarios and another user interface proposal for the camera array.
Contributions to the HSVO Network Enabled Platform Specification
The reports listed bellow present the discussions and contributions around the definition of the specifications of the HSVO Network Enabled Platform, part of it was based on our previous field studies studies.
Knee pain in athletes: Demo
We have put together a script around a remote teaching scenario. Here the tutor explain to her local and remote students the various knee pains that cyclists could develop. The scenario makes use of an early prototype of a camera array, and two visualization tools Volseg and RSV.
DOWNLOAD: Script - private access to HSVO.
VIDEO: Knee pain in Athetes - private access to HSVO.
Communication diagram: Round trip scenario using a mannequin patient simulator
This communication diagram presents a round trip medical scenario using two tools (or edge devices): a mannequin patient simulator and OpenLabyrinth . This diagram depicts the earlier thoughts about the messages needed to be passed to and from the core of the HSVO system (known as SAVOIR).
Building medical scenarios: Paper prototype
Allowing tutors to build medical scenarios with various tools (or edge devices) is at the core of what the HSVO system. This document presents the first paper prototype of the user interface that aims at facilitating this task.
Early thoughts on the camera array workflow and user interface
This document present some early thoughts around the user interface for the camera array. This became our first web prototype in Jan, 2009.
Sketching the main overall scenarios around the HSVO project
The illustrations here represent aim at describing the overall scenarios that the HSVO project will cover.
Messages and data flow around a patient simulation session
All the observations from the previous fields studies led into the description of the various messages that would be required to be passed across the HSVO system; in order to ensure a smooth collaboration between the medical students and tutors. The various flow charts and UML communication diagrams developed in order to depict the data flow and interaction are listed bellow.
Storyboard around patient simulation and medical collaboration
We had the opportunity to put together a small cartoon scenario with our team members at NRC lead by Prof. Bruce Spencer. The cartoon describes a remote collaboration and simulation scenario between various medical practitioners. Kind thanks to my friend Adrien Sanchez Valero for creating the cartoon stills.
Dissection session: Field study
In this other "Fly in the Wall" session we now looked at a dissection session. During this session we had the opportunity to observe a foot endoscopy training session with approximately 15 medical residents. The figure on the left present the overall set up at one of the working stations. The intentions of this observations will play an important role in the future design of the camera array structure and user interface.
Interactions around a patient simulation: Workflow overview
Running a patient simulation with a mannequin requires various preparation steps. The diagram linked here present an overview of these preparation steps and interactions in terms of people and the files created before, during and after running the simulation session.
Mannequin scenario session: Field study
We started the project with a series of "Fly in the wall" sessions at the McGill Simulation Center The figure on the left shows a typical set up while running a medical patient simulation with a mannequin. The report linked bellow, describes the scenario observed, the number of players involved, their interactions and means of communication. Overall, it was rather interesting to learn the importance of having vocal and auditory communication between the instructor, mannequin operator and facilitator.
Last update: August 2015