Deepfakes: the next gen­er­a­tion of coun­ter­feits

Making up facts, bending the truth – this has been happening since the internet began. Whether it’s ma­nip­u­lated images, alleged news, or made up in­form­a­tion, the world wide web is full of it, making it in­creas­ingly difficult to dis­tin­guish between reality and fiction. Now coun­ter­feits have reached a new level: deepfakes.

Deepfakes first appeared in December 2017 on Reddit. One user had managed to put the faces of several celebrit­ies into porn movies in such a way that made it look shock­ingly real. For a short while, creating celebrity porn became the thing to do, but soon fake videos like these were banned on Reddit as well as other platforms such as Twitter and Discord. It didn’t stop them from spreading elsewhere though. But the question is: What makes deepfakes so special?

Usually these fakes require a lot of work and expertise. They aren’t ne­ces­sar­ily always created for shady reasons – it’s not uncommon for Hollywood movies to exchange faces, for example. Experts in the areas of cutting tech­no­logy and CGI are normally hired for these tasks. Deepfakes, however, are created by the computer itself without any ad­just­ments being made by hand.

The word “deepfakes” is a port­manteau of “deep learning” and “fake.” Deep learning is one form of machine learning. Al­gorithms are required for ex­chan­ging faces or objects with deepfakes. For deep learning to work, the al­gorithms are fed with a very, very large amount of image or video data. The more material you have of a person, the better the result should be.

Around 300 images with the chosen person’s face (at best from all possible per­spect­ives) should be enough to get a decent result. The deepfakes code contains a neural network, a so-called au­toen­coder: the network is trained to compress data in order to de­com­press it again. During de­com­pres­sion, the au­toen­coder tries to achieve a result that is as close as possible to the original. To achieve this, the networks learns to dis­tin­guish between important and un­im­port­ant data during the com­pres­sion process.

By feeding the algorithm with images of dogs, the ar­ti­fi­cial neural network learns to focus only on the dog and ignore anything in the back­ground (noise). The au­toen­coder can then create its own dog from the data. This is also how face swaps work with deepfakes: the neural network learns what the person’s face looks like and can then create it in­de­pend­ently – even if the face and mouth are moving at the same time, for example.

To ef­fect­ively swap faces, two heads need to be re­cog­niszed: the face that appears in the original material and the one that you want to exchange it with. So, one input (the encoder) and two outputs (the decoders) are used. The encoder analyszes all the material while the two decoders each generate a different output: face A or face B.

In the end, it works in such a way that the algorithm doesn’t insert face A into the video, but rather face B instead, even though it doesn’t belong there at all. This is the dif­fer­ence between other fakes where the face is cut out of an image, retouched, or adjusted, and inserted into another image. Regarding deepfakes, however, the image material isn’t copied into another image: totally new material is created. This is the only way to match the facial ex­pres­sions of the original face.

This explains why some errors occur with deepfakes: the neural networks reach their limit when it comes to unusual facial movements. If there isn’t enough material from the relevant per­spect­ive, the frame will appear blurry. The algorithm tries to generate an image from the little source material it has, but will un­for­tu­nately leave it lacking in detail.

Deepfakes first ori­gin­ated on Reddit. The website is known for also housing curious topics in its sub forums, which are known as subred­dits. A Redditor with the name “deepfakes” created a subreddit in December 2017 and published por­no­graph­ic videos featuring celebrit­ies. To do this, the anonymous user had built the afore­men­tioned algorithm, which is based on other tech­no­lo­gies such as the open source libraries, Keras and Google’s Tensor­Flow.

Within a very short time, the subreddit had over 15,000 followers. Reddit quickly put a stop to the forum and, like other companies (including the por­no­graph­ic video platform, Pornhub), banned the dis­tri­bu­tion of fake porn. But that wasn’t enough to stop deepfakes, since the code used is open source and is available to everyone. On GitHub, you can find several re­pos­it­or­ies where de­velopers work on the al­gorithms. This is how a deepfakes app, called FakeApp, was created.

The program enables even those with little computer knowledge to perform face swaps. To create deepfakes via app, you only need a powerful graphic card from Nvidia. The program uses the graphics processor (GPU) for the cal­cu­la­tions. Apart from FakeApp, deepfakes can also be created with a computer’s CPU, but this usually takes much longer.

In the meantime, the network community has found further uses (other than por­no­graphy) for face swaps based on machine learning. As you know from the internet, this tech­no­logy is used to a very large extent to create funny nonsense. It is often used to put actors in films in which they have never appeared. In a short clip from the film ad­apt­a­tion “Lord of the Rings,” for example, users replaced every actor with Nicholas Cage, and Sharon Stone was replaced by Steve Buscemi in her notorious scene from “Basic Instinct.”

Jokes like those mentioned above are harmless, but the new pos­sib­il­it­ies of video ma­nip­u­la­tion pose several chal­lenges to society. Firstly, there is the question of legality. The women appearing in these porn videos have never given their consent. Apart from the fact that this is more than ques­tion­able from a moral point of view, these videos could also wreck the person’s repu­ta­tion.

Apart from causing distress to in­di­vidu­als, deepfakes can also cause social changes. In recent years, the problems caused by so-called fake news have already started to become apparent. It is becoming in­creas­ingly difficult to dis­tin­guish between real facts and false claims. Up to now, video evidence was con­sidered a reliable in­dic­a­tion of whether a statement was correct or not, but since deepfakes came on the scene, this is no longer the case. De­cept­ively real ma­nip­u­la­tions can now be created with re­l­at­ively little effort – and not only for the purpose of en­ter­tain­ment.

Coun­ter­feit­ing is and always has been an important pro­pa­ganda tool. Deepfakes can be used to influence politics slightly. While a video of Trump’s face covering German chan­cel­lor Angela Merkel’s face is obviously fake, other politi­cians could be brought into situ­ations in which they have never been. Since machine learning can even recreate a person’s voice in a re­l­at­ively credible way, it’s scary to think what deepfakes will be able to do in the future. Fakes like these could in­ev­it­ably influence election campaigns and in­ter­na­tion­al relations.

For our society, this means that we can’t really trust the media, es­pe­cially the internet media. People are already skeptical when it comes to news, although there are others that believe every social media post they see even if it’s lacking any factual basis. In the future, we will no longer be able to believe what we’ve even seen with our own eyes!

But not all deepfakes de­vel­op­ments are de­struct­ive and foolish: deep learning can re­volu­tion­ise the creation of visual effects. At the moment, it requires a lot of work to stick the faces of actors onto the bodies of other people. For the Star Wars film “Rogue One,” the young princess Leia was created with visual effects since the actress, Carrie Fisher, was already 60 years old when the movie was released. Saying this, an internet user achieved a similar result with the help of deepfakes – according to him, it took half an hour on a normal PC. Deepfakes have the power to make visual effects in en­ter­tain­ment media faster and cheaper.

There has been spec­u­la­tion that deepfakes and the sim­pli­city as­so­ci­ated with the new types of fakes could give viewers more choice. For example, if you watch a movie in the future, it would be in­ter­est­ing to be able to select which star should play the main character. All that’s needed is a quick click before the movie begins. Something similar could also happen in the ad­vert­ising industry. Soon, celebrit­ies won’t need to fly to pho­toshoots to wear the latest designer clothes or pose for the newest perfume, they will just sell a license to allow others to use their face.