Poor Man's Virtual Camera: Real-Time Simultaneous Matting and Camera Pose Estimation

augmented reality, cameras, mobile computing, camera pose estimation, real-time systems

Today's film and advertisement production heavily uses computer graphics combined with living actors by chroma keying. The matchmoving process typically takes a considerable manual effort. Semiautomatic matchmoving tools exist as well, but they still work offline and require manual check-up and correction. In this paper, we propose an instant matchmoving solution for green screen. It uses a recent technique of planar uniform marker fields. Our technique can be used in indie and professional filmmaking as a cheap and ultramobile virtual camera, and for shot prototyping and storyboard creation. The matchmoving technique based on marker fields of shades of green is very computationally efficient: we developed and present in this paper a mobile application running at 33 FPS. Our technique is thus available to anyone with a smartphone at low cost and with an easy setup, opening space for new levels of filmmakers' creative expression.