#CREATE DEEP FAKE SKIN#
Keep in mind that certain implementations of face-swap adjust the colours of the reconstructed face to match the original skin tone. If person A and B have very different skin tones, it will be difficult to merge them seamlessly. The way it works simply smooths the edges of the image in a rather primitive way. One of the biggest downsides of face-swap is how faces, once converted by the neural network, are blended back into the video. Choose people with similar skin tones.It is particularly difficult to use faces with features that go outside the face itself (such as long beards or occluding hair), as they will be cut out. Makeup, glasses and beards make the problem much more challenging. This goes beyond the simple idea concept of “face”. This is because reproducing realistic shadows requires a three-dimensional understanding of the face geometry, which needs to be learned from two-dimensional images. It also casts shadows on faces which are hard to match. Light does not only affect the colour of a scene. Use videos shot in similar lighting conditions.You can start by selecting the right videos: The easiest way to achieve that, is by reducing the complexity of the problem. If your aim is to perform face-swap on a single clip, you do not need a neural network that is able to perform face-swap in any condition. Unless you are an expert, you should strive to help the neural network as much as possible. Generally speaking, however, the more complex a problem is, the harder it is to solve it. It is impossible to know how the neural network will respond, just by saying that the pictures are taken under the same light. So what is better? Including images that are very similar to simplify the problem, or adding shots taken in different lighting conditions to better generalise how the face should look like? The answer is that it depends. The result is that the neural network will learn to reproduce the same image, rather than the same face. If all the images provided look the same, the training process might fail to capture their true variability. While that is certainly true, we also have to battle the opposite problem. This is called underfitting, and it indicates that the problem is too complex to be fully captured with the model chosen.
This takes up space that could be used to better represent facial expressions, hence reducing the overall quality. If the frames used for the training process exhibit high variability in the lighting conditions, then the encoder will be forced to include information about them in the latent representation of the face. Different lighting conditions and angles raise the complexity of the problem, as more steps are required not just to adjust the faces, but also to identify the relevant features. The neural network is not specifically designed to recognise them. The problem, however, goes beyond just faces. This explains why some of the most well-crafted deepfakes feature people with similar faces. What makes the task easier, however, is that the autoencoders can focus on the facial expressions, rather than having to re-adapt to an entirely different bone structure, skin colour, and so on. This might seem trivial at first, but keep in mind that even morphing between images from the same person is far from being trivial.
What is the simplest task that a face-swapping neural network can be asked to perform? Because it is built on autoencoders, is to reconstruct face A from face A. The neural network works in a similar fashion, as a more complex mapping has to be learned. This is easy to understand, as more “changes” are necessary to convert A into B. The complexity of such a task largely depends on how different the two faces are. Imagine you had a picture of a person A, and had to manually edit it to make it look like person B. The task of the neural network is to encode an image from person A, and reconstruct one with similar features to resemble another person B.
Let’s start by analysing the problem from a Machine Learning point of view.
#CREATE DEEP FAKE HOW TO#
Since the release of face-swapping technology, there have been countless discussions online on how to train a network to achieve photorealistic results. Training is the process which (literally!) trains a neural network to deconstruct and reconstruct faces. There are two critical processes involved in the making of any face-swap video: the training and the creation. Likewise, creating realistic face-swapped videos is hard. Like any artistic endeavour, the final result is a mixture of talent, commitment and right tools. Photoshop and After Effects are used every day by professionals, but that does not mean that just installing either of them is all it takes to create photorealistic images and videos.
0 kommentar(er)
