[NSFW] [GUIDE] VR Deepfakes

[Grrkin]

Why not correct the distortion before you start the project?Premiere and AE have a function for that:

Unless the face is on the edge of the screen I don't see a major issue.

I just feel that taking the faces that are stretched out on the top of the frame (like when the girl's on top and her face is just a stretched out stripe across the top of the frame), then smooshing that maybe 800 pixel wide face down to effectively 350 pixels to deepfake, then stretching the 350 pixel face back out to 800 pixels. I have never actually tested that to be fair though lol

I actually don't see a need unless the face is on the very very edge, like in more VR porn in certain scenes. I've also found that DFL actually does fine with straight VR footage if the face is near the center of the frame (such as a BJ scene), so if you cut around the difficult scenes and keep easy scenes, and maybe a doggy scene where the face isn't near the center... And maybe you want to try for the face at the edge of the frame shot... I think you could get away with just correcting for the extreme faces.

 

[DeepTakes1]

Thank you for creating this guide!

 

[Vurlix]

So I feel like I made a decent VR Deepfake using Miranda Cosgrove as the data_src, which You can find here..
My question is, if I'm using colab, how big should the clips be if I want to make a full video? 

For example, this one was only a 5 min clip, so I chopped it up in like 1.5min/ 2.5min intervals. But how big do you think I could use each clip to mess around with in Colab? 

I understand it's gonna take the ame amount of time in the long run, but using Google Colab Pro, I think it gives you 150gb pf HDD per session. Considering I'm using SBS making it basically two faces per frame, I know it's gonna take up more space. In your opinion, what size clips should be min/max to make a full 17min-30 min video?

Thanks in advance.

 

[rbanfield82]

Has anyone worked on or uploaded a 4-6k VR video yet? If so, how did it look?

 

[becks999]

I've tried to do a VR deepfake and now understand the difficulty with there being two faces per frame.  When I do the MANUAL RE-EXTRACT DELETED ALIGNED_DEBUG I fix all the land marks on the left side, then when I go back to view aligned_debug results there are no landmarks on the right side... So I fix those and the landmarks on the left are gone.  lol  I thought that the landmarks get saved to the extracted images in the data_dst directory but I guess not?

I'm guessing that's why suggests to split the frames into two sides, handle up to the completion of training with left side then right side... but that's where I don't understand how to get back to the two sided original.  How do the images in the data_dst and its aligned folder have to be renamed/organized and associated with the two sided original video for the merging process to make sense of whats occured?...  or does the model just know how to handle each reextracted video regardless of how they're named?  Thanks for any advice with this!

 

[Xantrax]

Hi "becks999", 

I'm not entirely sure what you mean. Could you please rephrase your question?

As I mentioned on page 2; 

[font='Open Sans', sans-serif]A typical VR scene comes in a format of for example 4096 x 2048:[/font]

1. [font='Open Sans', sans-serif]In your preferred video editing tool, literally split the scene in half and separate the left and right angle from each other into two scenes of 2048 x 2048.  [/font]
2. [font='Open Sans', sans-serif]Put the separated scenes back into the project so the left scene plays first, then the right scene plays directly after (like it was one, continuous scene). [/font]
3. [font='Open Sans', sans-serif]Render it and make sure you render your project in a format of around 2048 x 2048. [/font]

[font='Open Sans', sans-serif]Now go on and deepfake as you would normally do with any scene [/font]

[font='Open Sans', sans-serif]After you're done, reverse the separation: [/font]

1. [font='Open Sans', sans-serif]Start a new project in a format of 4096 x 2048 (or like the original file)[/font]
2. [font='Open Sans', sans-serif]Add the scene[/font]
3. [font='Open Sans', sans-serif]Find the exact frame where it's merged together in the first phase and make sure you split between the last left-angled frame and first right-angled frame.[/font]
4. [font='Open Sans', sans-serif]Move the scenes so they fill up the project completely (2048 on the left + 2048 on the right), and make sure you keep track of the left- and right angled scenes (duh..) [/font]
5. [font='Open Sans', sans-serif]Render it. [/font]

In your preferred video editing too, split the scene, then put left scene first followed by the right scene - and render them like it was one, continuous scene. 

Now you have a 'data_dst.mp4' with both the left and right scene. Extract images from data dst, extract faces, clean up data_dst/aligned, train, merge and then render again. 

When you have done this, open your video editing tool again, split the video again and put the left and right scene next to each other and render again. Make sure the left and right side is exactly aligned. 

One tip: make sure left and right scene has audio. When you play both sides at the same time you can use the audio to more easily synchronize the clips. If one side is off it will sound like a robot. 

Feel free to ask if this didn't answer your question

 

[becks999]

Thanks for the explanation. When you create the data_dst.mp4 so the left scene plays first then the right, do you mean that the frames should go L-R-L-R-L-R... or can it be all the left frames in the first half of the video then all the right side frames? For the training, I wasn't sure if the learning had to be in the order that the face would eventually be merged in.

Then when you're recreating the file in the original format how do you spice the left and right back together in a batch sort of way where it takes the left and right frames in order without manually pasting each frame together. I've done some very basic editing to cut down a video size but I didn't know you could put two frames beside each other so two videos sets of frames come together. Does that process have a certain name in video editing software? I've started using Davinci Resolve but I'm pretty clueless.

 

[armanovich]

I juste ignored the warning of multiple faces in the same Frame while merging, and it worked very well....why are you all cutting your videos in 2 parts L and R?

 

[Xantrax]

I juste ignored the warning of multiple faces in the same Frame while merging, and it worked very well....why are you all cutting your videos in 2 parts L and R?

I guess you don't need to split the scene. However when cleaning your data_dst/aligned (and in general) I personally find it easier to manage if the scene was split in two. But - personal preference. Do whatever you feel is the best. There is no definite way to do this.