The film industry is undergoing a significant transformation with the integration of Artificial Intelligence (AI). From script analysis to casting decisions, visual effects creation to distribution strategies, AI is revolutionizing the film-making process and shaping the way movies are experienced. However, this is not without controversy. There is ongoing debate about how AI is affecting creativity and diversity in the industry. While some see AI as a tool to enhance and expand the creative potential of filmmakers, others worry that it could lead to homogenization and stifle individuality. As AI continues to play a larger role in the film industry, it is important to critically examine its impact and to consider how it can be harnessed to augment, rather than replace, human creativity.
The use of AI in films has revolutionised the way visual effects are created. Take for example, the critically acclaimed Avengers: Infinity War, where Marvel and Digital Domain leveraged AI to bring the supervillain Thanos to life. The AI facial capture system, Masquerade, was instrumental in capturing and replicating Josh Brolin’s performance of the character. The system utilises a head-mounted camera to translate the performance into a digital mesh, producing more natural and accurate motion capture. Previously, VFX artists had to manually tweak facial performances with animation, which is a time-consuming process.
In the upcoming movie “Here,” the AI-driven tool MetaPhysic Live will be used to de-age Tom Hanks so he can play his younger character. MetaPhysic, previously known for its deep fakes of celebrities and its appearance on America’s Got Talent (AGT), is showcasing the potential of AI-generated “Hyperreal” content. Their performance on AGT, bringing back the king, Elvis Presley, demonstrated the capability of this technology. However, this is not the first time Hollywood has been equipped with the ability to digitally recreate dead actors, as with Anthony Bourdain’s voice in the documentary “Roadrunner” or Peter Cushing and Carrie Fisher in recent “Star Wars” sequels.
AI-powered motion capture for Thanos from Avengers: Infinity War
ILM’s StageCraft is another game-changer in the film industry, offering a real-time VFX tool powered by Unreal Engine and AI to create photorealistic digital environments. Bringing the world to life in front of their eyes, makes it easier for actors to bring their performances to life. They can give more impactful and creative performances since they don’t just have a green screen to look at in the distance. This was evident in the success of The Mandalorian, which won Emmys for Outstanding Cinematography and Outstanding Special Visual Effects. Disney released a documentary on this new filmmaking technique. This tool is announced to have also been used in the 2022 Batman movie and Thor: Love and Thunder. StageCraft not only enriches the film-making experience but also streamlines production and reduces costs.
The Mandalorian crew on set using StageCraft
Let’s take a look at the synergy between human and machine. By the second season of the Mandalorian, the performance of the tool had gotten better due to insights and specific requests from team members. The improvement in the tool’s performance highlights the importance of their expertise. As these tools develop, the workflow becomes more efficient.
Some sources point to a possible decrease in jobs due to tech such as Stagecraft. However the resulting faster production times and increased content creation may also potentially result in a shift in job descriptions or the creation of new jobs rather than the loss of existing ones.
AIs in the industry are being designed to collaborate and not to replace.
Disney’s AI can create a rough storyboard and video depicting text from movie screenplays. According to researchers the AI is “valuable for screenwriting by enabling faster iteration, prototyping and proof of concept for content creators”.
“Will AI destroy creativity in the Film Industry?”
The top FAQ on Cinelytic AI’s website was “ Will Cinelytic destroy creativity in the Film Industry?”, showing a common concern among the public. Cinelytic AI was introduced in 2019 and signed a deal in 2020 with the major Hollywood studio Warner Bros to support repetitive tasks and uncertain decision-making with large datasets.
20th Century Fox’s AI Merlin, matches movies with the right genres and audiences, and ScriptBook analyses which movies will fail based on their scripts.
A major concern about using AI in this way is that people will become reliant on its accuracy and be afraid to deviate from its predictions. “Won’t it dictate all future creative endeavours?” This fear stems from the misconception that AI will only predict content as being successful if it matches a historical pattern of success. Naturally it would prioritise ‘blockbusters’ over novel ideas or ‘indies’, right?
Yes, these AIs are tools that can be used to decide which content to produce or reject. But instead of simple reject-or-not decisions, they can also be used to decide the demographics, time of year, region, and platform where audiences will be most receptive to content. This is how studios such as Warner Bros are using AI now. The impact of AI’s answers depends on the way we frame our questions.
The data supports this: AIs have not had a negative impact on creativity in the industry.
In the US for example, the number of films released by major distributors has decreased from 86% in 1999 to 66% in 2019. At the same time the number of Independent “smaller-budget” studios has grown, and the quantity of original content they produce has been increasing from 563 independent films in 2015 to 863 in 2019.
Indeed the number of sequels, prequels and content based on existing IP (Intellectual Property), has increased in the last few decades in all markets. However this increase began before the introduction of AI into the industry (most notably seen with the Avengers and the Marvel Cinematic Universe franchise). Current reliance on IP may be influenced by the emergence of streaming platforms that forced theatrical releases to turn movies into large blockbuster events to attract audiences.
Entertainment that reaches more people
In 2015, after months of production, Director Scott Mann and actor Robert de Niro perfected the dialogue and acting for the movie “Heist”. But this couldn’t translate into the dubbed release of the movie for international markets, with its jarring mismatch between speech and mouth movements. Mann was “devastated”.
Mann’s need to fix this problem led to his collaboration with researchers to create Flawless in 2018. Flawless AI created mouth movements to match dialogue flawlessly, retaining the nuance in performances while increasing the diversity of languages for content.
Flawless AI, image from a Dailymail report
In 2021 Flawless re-synced lip movements to remove swear words and turn Mann’s R-rated movie into a PG-13 rating, once again enabling it to reach a wider audience.
In the future, AI’s automatic subtitling will enable content to be shared in multiple languages, and create automatic specialised captions for hard-of-hearing people. Having content reach a wider range of audiences can bring more diversity in perspectives to the film and TV industries.
By automating repetitive tasks and reducing the time needed to create content, AI makes it easier for independent creators and filmmakers to get their projects off the ground and bring their visions to life. This opens the door for a wider variety of stories and perspectives to be brought to the screen, promoting diversity and representation in the film industry. For instance, Zone Out, a film created in just 48 hours as part of London’s Sci-Fi Challenge, was made possible through the collaboration of Sharp and Goodwin with Benjamin AI. While the film may not have won awards, it demonstrates how AI can lower the barriers to entry in the film industry, paving the way for new and exciting content.
“In a properly automated and educated world, then, machines may prove to be the true humanizing influence. It may be that machines will do the work that makes life possible and that human beings will do all the other things that make life pleasant and worthwhile ”Isaac Asimov, Robot Visions
The most controversial questions surrounding the application of AI in this field concern its effect on creativity, diversity and replacing humans at their jobs.
Yet all current evidence of the applications of AI in film and television show that it is automating the dreary parts, and augmenting the creative process. Instead of spending weeks making pixel-level corrections and acting with green-screens, humans are able to spend time on what they find pleasant and worthwhile in filmmaking.
When calculators were introduced into society at large in 1975, many of the same questions currently discussed about AI plagued educators, parents and the public alike.
“Will using it decrease inherent abilities?”, ”Will it teach people the wrong thing?”, “Will this create an unhealthy dependency?” (Pendelton in 1975 and Peterson in 1982). Yet decades later the calculator has empowered humanity without causing detrimental effects.
All technologies are tools that can be used for positive or negative outcomes.
Will AI become the terminator of boredom, or the cause of monotony?
The answer is up to us.