Metaphysic De-Ages Robert Zemecki’s ‘Here’ Via Generative AI

Miramax’s recently released feature, Heredirected by Robert Zemeckis and based on the graphic novel by Richard McGuire, portrays several families over time in the special place they live.

To help convey the characters at various ages throughout their lives, VFX Supervisor Kevin Baillie hired creative studio Metaphysic to digitally enhance performances via its proprietary generative AI technology. The process involved a talented team of digital characters, led by Metaphysics VFX Supervisor, Jo Plaete.

“Working with up-and-coming filmmakers like Bob Zemeckis and Kevin Baillie was a privilege,” said Plaete. “Their belief in this new technology pushed us to new heights and enabled us to deliver on their ambitious vision.”

Metaphysics’ proprietary process involves training a neural network model on a reference input with artist refinement of the results until the model is ready for production. Multiple models were trained for each actor to meet the different needs of the film; Tom Hanks is portrayed at five different ages, Robin Wright at four ages and Paul Bettany and Kelly Riley at two ages each.

Plaete and his team were able to run the workflow in real-time during filming, providing a visual reference for what the performers would look like. This allowed Zemeckis to see both the raw camera feed and the digitally enhanced feed of the actors’ younger faces while on stage, with only about a six-frame delay, and give direction accordingly.

Metaphysic also set up a camera and monitor system during filming that allowed the actors to rehearse while seeing themselves as their younger selves in real time. The “mirror of youth” system only had a two-frame delay and gave the actors feedback that helped them fine-tune their performances to better match their younger selves.

While the neural network models used for real-time output generated photorealistic results, metaphysical artists further enhanced them to hold up to cinematic 4K standards.

Source: Metaphysics

Journalist, antique shop owner, aspiring gemologist – L’Wren brings a diverse perspective to animation, with each image reflecting her diverse passions.