Camera Tracking (Matchmoving)

Camera Tracking (Matchmoving)

Camera tracking, also called matchmoving, is the process of reconstructing the movement of a real camera from footage. The goal is to create a virtual camera in 3D software that moves exactly like the real camera did. This virtual camera is then used to render CG elements that will match the perspective of the original footage perfectly. Without good tracking, CG elements will float, slide, or feel disconnected from the scene.

The tracking process starts by placing tracking markers in the footage. These are distinctive points that the software can follow from frame to frame. Good tracking markers are high contrast, like a dark spot on a white wall, and they should be spread across the entire frame. Modern tracking software can generate thousands of automatic tracking points using feature detection algorithms that find corners, edges, and textured areas.

Once the points are tracked, the software solves for the camera motion. It uses a mathematical process called bundle adjustment to find the camera position and rotation for every frame that best explains the movement of all the tracking points. The result is measured by the solve error, usually given in pixels. A good solve has an error of less than one pixel. Acceptable is one to two pixels. Anything above that means your CG will probably drift.

There are several reasons a solve might fail. Motion blur can blur out tracking points. Lens distortion can confuse the solver. Not enough parallax, which means the camera is not moving enough to create depth information, can cause the solver to misinterpret the scene. Sometimes you need to manually adjust tracks, remove bad ones, or add constraints like known distances to help the solver.

After solving, you get a 3D camera and a point cloud that represents the tracked scene. The point cloud is useful for approximating the geometry of the environment. You can build simple cards or geometry on top of the point cloud to act as placeholders for reflections and shadows. The final step is to check the solve by rendering a simple grid or some test objects through the solved camera and compositing them over the footage.