> Deepfakes might actually be useful in making international TV and film more accessible. Maybe. Hopefully.

What exactly is lost in translation when TV shows and films are subbed or dubbed into a new language? It’s a hard question to answer, but for the team at AI startup Flawless, it may be one we don’t have to think about in the future. The company claims it has the solution to this particular language barrier; a technical innovation that could help TV shows and films effortlessly reach new markets around the world: deepfake dubs. We often think of deepfakes as manipulating the entire image of a person or scene, but Flawless’ technology focuses on just a single element: the mouth. Customers feed the company’s software with video from a film or TV show along with dubbed dialogue recorded by humans. Flawless’ machine learning models then create new lip movements that match the translated speech and paste them automatically onto the actor’s head. “When someone’s watching this dubbed footage, they’re not jolted out of the performance by a jarring word or a mistimed mouth movement,” Flawless’ co-founder Nick Lynes tells The Verge. “It’s all about retaining the performance and retaining the original style.”

As somebody who watched a ton of anime back in the day, I got very used to subtitles. In animation I never really cared about mouth movements and things. Its drawings so it’s almost never right anyway. Most of the time though things would be changed from the original Japanese version and that was always the issue. The animated Street Fighter movie from 1994 is probably the most famous one from my friend group. Different music, lines were changed, and scenes were cut. And who can forget the old 70’s Kung Fu movies from back in the day that were so famously badly dubbed they became a whole genre of their own.

Today, I still watch a ton of international TV and film thanks to Netflix, I still prefer subtitles. Often the dubbing is so bad in terms of acting that it’s laughable. That’s what’s distracting about dubs. The person reading the english either isn’t acting or is acting badly.

The company hopes that preserving the original performance will be appealing to filmmakers who want to retain the magic of their original casting. Lynes gives the example of the 2020 Oscar award-wining Danish film Another Round, which stars Mads Mikkelsen as one of a group of teachers who experiment with low-level alcoholism to see if it improves their lives. After its success at home and on the international award circuit, the film is set to be remade for English-language audiences with Leonardo DiCaprio in the main role. The news sparked discussion about the value of such remakes. Is the Danish drinking culture that forms the film’s backbone really so alien to American audiences that a remake is required? Is Mikkelsen, an actor who’s appeared in such mainstream fare as Hannibal, Doctor Strange, and Rogue One, such an unknown that he can’t attract viewers in the US? And is the “one-inch barrier” of subtitles (to quote Parasite director Bong Joon Ho) simply too much for audiences to overcome?

I’ve never understood this. I get that subtitles can be a pain, I really do. But to completely recast and reshoot a film for English audiences… I get the feeling this is a distinctly American thing because Americans are lazy and don’t like to read. They also need complex plots simplified for them. Case in point: the Swedish film Let the Right In was fantastic in its original Swedish form. The plot? Not obvious. The simplified American version? Terrible. It lost the complexity and mystery that made the Swedish version great.

Maybe there are just certain things that aren’t for Americans. After all, Breaking Bad, considered to be one of the best modern American shows, doesn’t quite work in other developed countries and thus is distinctly American.

> ▋