Video Stabilization with Matchmove

Video Stabilization with Matchmove

Video stabilization is usually thought of as a post production tool to fix shaky footage. In VFX, stabilization is often done as part of the matchmoving process. Instead of just tracking the camera motion to place CG elements, you can use the solved camera to remove unwanted motion from the footage. This gives you a stable plate that is much easier to composite on.

The process is straightforward. First, track the camera as you normally would. The solve gives you the camera position and rotation for every frame. Then you subtract the camera motion, effectively locking the camera in place. The result is footage where the background appears perfectly still, and only the moving objects in the scene show their original motion. This is called a locked off plate.

Stabilized plates are incredibly useful for compositing. If you need to add a 2D explosion or a screen replacement, doing it on a stable plate is much easier than tracking it to a moving background. You can do your compositing work on the stable plate and then add the original camera motion back at the end. This two step approach, stabilize, composite, destabilize, is a standard VFX workflow.

There is a catch. Stabilization works best when the camera motion is smooth. If the original footage is extremely shaky, stabilizing it will reveal that the edges of the frame move around a lot, and you will need to crop or scale the image to hide the missing edges. Heavy handheld shots with lots of shake can be stabilized, but you will lose some of the frame. Sometimes this is acceptable, sometimes not.

Modern tracking software like 3DEqualizer and SynthEyes have dedicated stabilization tools. You can export a stabilized image sequence directly from the tracking solve. Nuke and After Effects also have built in stabilizers that work well for simpler shots. For VFX heavy work, doing the stabilization through the matchmove solver gives you the most control and the best results because the solver understands the full 3D motion.