The director Ruben Östlund shoots wide in 4K, composing the individual shots for films like Force Majeure by, in post-production, pushing in and moving from side to side. Now, Disney Research has developed a new tool, Face Director, that offers directors even greater possibilities; it crosses the final frontier in post-production manipulation: actors’ performances. Whereas, previously, directors would shape a performance by using multiple takes or, in post, skillful editing and music, with Face Director directors can alter performance by modulating them between two extremes.

From Disney’s page:

We present a method to continuously blend between multiple facial performances of an actor, which can contain different facial expressions or emotional states. As an example, given sad and angry video takes of a scene, our method empowers a movie director to specify arbitrary weighted combinations and smooth transitions between the two takes in post-production. Our contributions include (1) a robust nonlinear audio-visual synchronization technique that exploits complementary properties of audio and visual cues to automatically determine robust, dense spatio-temporal correspondences between takes, and (2) a seamless facial blending approach that provides the director full control to interpolate timing, facial expression, and local appearance, in order to generate novel performances after filming. In contrast to most previous works, our approach operates entirely in image space, avoiding the need of 3D facial reconstruction. We demonstrate that our method can synthesize visually believable performances with applications in emotion transition, performance correction, and timing control.

Or, as Disney Research V.P. Markus Gross told Engadget:

It’s not unheard of for a director to re-shoot a crucial scene dozens of times, even 100 or more times, until satisfied,” he said. “That not only takes a lot of time — it also can be quite expensive. Now our research team has shown that a director can exert control over an actor’s performance after the shoot with just a few takes, saving both time and money.

Check out the video below for an example of software coming to an editing environment near you.