Object Tracking

Object Tracking

Camera tracking solves for the motion of the camera, but sometimes you need to track the motion of an object within the frame. This is called object tracking. For example, if an actor picks up a magic staff, you need to track the staff's position and rotation so you can attach a glowing CGI effect to it. Object tracking comes in several flavors, from simple planar tracking to full 3D object tracking.

Planar tracking tracks a flat surface within the footage. If an actor holds up a sign, you can track the four corners of the sign and use that to replace or augment the sign. Planar tracking works well for screens, posters, and any flat surface. The tracker follows the pattern of pixels on that surface and calculates its perspective transform over time. Tools like Mocha and the planar tracker in Nuke are built specifically for this.

Point tracking is simpler but more limited. You place a tracker on a specific point, like a dot on an actor's face, and the software follows that point from frame to frame. The result is a 2D position curve. This is useful for attaching a small element to a specific location, like a beauty mark or a button, but it does not give you rotation or scale information. You need multiple point tracks for that.

3D object tracking reconstructs the full 3D position and rotation of an object. This requires knowing the camera solve first, because the object motion is calculated relative to the camera. With a solved camera, you can place 3D tracking points on the surface of the object in 3D space and solve for its movement. This is how CG characters are attached to moving vehicles or how a CG helmet stays on an actor's head.

For complex object tracking, motion capture data is often used. Actors wear suits with reflective markers, and multiple cameras track the markers in 3D space. This gives you extremely accurate movement data that can drive CG characters. But for simpler shots without mocap, careful manual tracking with good tracking markers on the object can produce excellent results. The key is having good contrast and enough tracking points.