This project involved prior contributions by M.Eng. students Daniel Sud and Nadia Hilario, and was brought to its present form by the M.Eng. research of Samuel Audet.
Figure 1(a). Before shadow removal.
Figure 1(b). After shadow removal.
Multi-projector displays are often exploited for many installations where large display surfaces are desired. A seminal example is the CAVE (Cave Automatic Virtual Environment), where projectors display images from behind the screens traditionally with the help of mirrors, a technique known as back projection. Unfortunately, it is not practical to mount and unmount a CAVE whenever needed, and consequently room space usually needs to be reserved. This drawback limits the widespread adoption of the CAVE in more fields. An alternative to back projection is front projection, such as used in the Office of the Future. However, front projection has disadvantages as well. Particularly, when a person stands between a projector and a display surface a shadow occurs. Other shadow removal methods either cannot work with dynamic video playback at 60 frames per second or require specialized hardware, but we wanted a method that could work with commodity hardware.
Our method uses a set of cameras and projectors geometrically calibrated with respect to the floor and the flat display surfaces. For calibration, we adapted existing geometric calibration methods. The cameras are used in pairs with a stereo correspondence algorithm. The recovered depth information is reprojected in the direction of the normal of the floor. Since we assume that people stand vertically on the floor, it can cope with occlusions. Instead of depth information, our system can also use tracking information of people. One way or the other, knowing the position of people on the floor, our algorithm finds in the image of a projector the region that is occluded, as depicted in Figure 2. Another projector can then automatically fill the region with equal intensity and identical color to that of the occluded projector.
We tested our method using a stereo camera from Tyzx and four projectors on one wall. We found that the recovered calibration was accurate and that the system could correctly and efficiently perform shadow removal, providing a more appealing visual experience to users of multi-projector displays. As main limitations and material for future work, the current projector arrangement does not allow for effective shadow removal of more than one user. Also, the stereo algorithm causes undesirable flickering and is not sensitive enough to work well with relatively thin objects such as arms.
Last update: 17 July 2008