[GUIDE] MVE Machine Video Editor

[INeedMoney]

If you pass a sprite at the max values then it doesn't enlarge the faces to 1024. You only get enhanced 512 images. One technique is to extract faces at 1024 and use Remini but in scenarios where the content you’re extracting faces from is not 1080p or 4k then extracting at 1024 can be detrimental to the quality. A technique I discovered is to half the dimensions when packing the sprite sheet.


When you pass this through remini it returns the maxed out resolution of 3840, 3584.


The only annoying part is to then re-modify the sprite-info.info file to extract the faces at the enlarged 1024 resolution instead of 512.


I’ve created a super simple java jar that takes care of this for you.
To run it in the command line open CMD and type

java -jar EditReminiSpriteInfo.jar

make sure you are in the same directory as the jar, otherwise list the absolute path.

It works similar to the bat files it asks you to enter the filepath of your sprite file. You can supply the absolute path this way or type 'n' and do it locally with the sprite file next to the jar file.

I had to use your half-res sprite-pack trick for 256 res, because Remini couldn't process sprites w/ 196 faces each. Your program only works on 512 res, though. What's the quickest way I can fully edit my sprite file w/o using the program?

Edit: So It looks like I need to multiply x2 every number in the sprite file except the numbers in the image filenames? Looks like the relevant ones are all preceded by a ':', and are only intervals of the original image res (256 here) starting at 0 and not exceeding (the smaller sprite dimension-image res)(0,256,512,768,1024,1280,1536) (also original sprite dimensions: 1920,1792), so total of 8 replace all commands: ":256"->":512";":512"->":1024"... but I'll have to do them in descending order to avoid repeat-doubling.

2 problems: A) I double :1280 to :2560, then when I replace all :256 to :512, :2560 becomes :5120. So you need one more "replace all" 5120->:2560) or just replace :1280 at the end after you do :256.
B) The copied face landmarks from the originals do not scale up. And when I unpack without copying face data, and then try to import it from the parent frames afterward, it doesn't import to all of them and the ones they do import to are bigger than the face. I don't know what to do about this.

 

[edelwondo]

Hi guys,

First of all, I think this is my first post in this forum, so hi everybody and thank you very much for your great work!

I'm facing an issue when trying to copy the embedded data from the original dataset to the remini magnified one. In a fresh installation of MVE 0.7.2 with only the dependencies installed (no plugins), when I push the "Copy embedded data" button, it ends unexpectedly and below error is displayed in the dev console:

Stating process D:\DFL\MachineVideoEditor\resources\external\conda\win64\condabin\conda.bat activate mve-core && python -u 
"D:\DFL\MachineVideoEditor\resources\external\python\scripts\alignment-embeding\dfl-read.py" "D:\DFL\test_remini\aligned" --copy "D:\DFL\test_remini\remini_aligned"
main.25dd6b375c5462cb40a5.js:1 
Process kill error: Error: Command failed: taskkill /pid 9132 /T /F

I tried to execute the script manually and the error is the following:

(mve-core) D:\DFL\MachineVideoEditor\resources\external\python\scripts\alignment-embeding>dfl-read.py "D:\DFL\test_remini\aligned" --copy "D:\DFL\test_remini\remini_aligned
Traceback (most recent call last):
 File "D:\DFL\MachineVideoEditor\resources\external\python\scripts\alignment-embeding\dfl-read.py", line 3, in <module>
   from PIL import Image
ModuleNotFoundError: No module named 'PIL'

My knowledge of python is very limited, but I also tried to install the Image and Pillow packages with pip (installs but no difference) and conda (conda returns an InvalidVersionSpec error) without success. Do you have any idea of what could happen?

I finally managed to fix it by uninstalling python from my machine and cleaning up manually the installation directories. Not sure why, but it was interfering with the conda enviroment. Sorry for not provide more details, again I'm not very familiar with python at this point.

After fix that I found another issue when the script tries to copy the xseg data, but I managed to fix it by commenting the code that was failing and as I'm not using masks at this point it's ok for me now.

Anyway I guess all my issues are related with my OS installation somehow, I just wanted to put it in writing here in case it happens to anyone else.

 

[vovin]

Help me understand landmarks and MVE. Unless you just have a small handful of bad alignments, is there any point in manually moving landmarks in MVE?

Xseg masking is important because you can draw 100 masks and then the xseg trainer will use those masks as reference for all the remaining frames and learn to draw in the rest... but this doesn't work for landmarks, right? You can't adjust the landmarks of a few frames and then it will learn to adjust all the others. Unless i'm wrong?

 

[angeloshredder]

Help me understand landmarks and MVE. Unless you just have a small handful of bad alignments, is there any point in manually moving landmarks in MVE?

Xseg masking is important because you can draw 100 masks and then the xseg trainer will use those masks as reference for all the remaining frames and learn to draw in the rest... but this doesn't work for landmarks, right? You can't adjust the landmarks of a few frames and then it will learn to adjust all the others. Unless i'm wrong?

you are right .. 
for landmarks it doesn't learn from your corrections .. MVE uses just a pre-trained models .. as i know , no one had made a new updated pretrained model since years

 

[xomnow]

Is there a Moron's Guide to MVE?  I'd like one.  Thanks!

I unpacked the files in E:\MachineVideoEditor and all that does is give me the following error when I click on Dependency->Install 

Attempting to download Getty Images just fails silently.  No feedback given.  I don't get it.

I get this exact same error.  I generally think I know how to install stuff, but, I too would like a moron's guide...

 

[tania01]

Is there a Moron's Guide to MVE?  I'd like one.  Thanks!

I unpacked the files in E:\MachineVideoEditor and all that does is give me the following error when I click on Dependency->Install 

Attempting to download Getty Images just fails silently.  No feedback given.  I don't get it.

getting the same thing when installing dependencies but for getty, i get the error: "project no set"

 

[light1up]

If you pass a sprite at the max values then it doesn't enlarge the faces to 1024. You only get enhanced 512 images. One technique is to extract faces at 1024 and use Remini but in scenarios where the content you’re extracting faces from is not 1080p or 4k then extracting at 1024 can be detrimental to the quality. A technique I discovered is to half the dimensions when packing the sprite sheet.


When you pass this through remini it returns the maxed out resolution of 3840, 3584.


The only annoying part is to then re-modify the sprite-info.info file to extract the faces at the enlarged 1024 resolution instead of 512.


I’ve created a super simple java jar that takes care of this for you.
To run it in the command line open CMD and type

java -jar EditReminiSpriteInfo.jar

make sure you are in the same directory as the jar, otherwise list the absolute path.

It works similar to the bat files it asks you to enter the filepath of your sprite file. You can supply the absolute path this way or type 'n' and do it locally with the sprite file next to the jar file.

I had to use your half-res sprite-pack trick for 256 res, because Remini couldn't process sprites w/ 196 faces each. Your program only works on 512 res, though. What's the quickest way I can fully edit my sprite file w/o using the program?

Edit: So It looks like I need to multiply x2 every number in the sprite file except the numbers in the image filenames? Looks like the relevant ones are all preceded by a ':', and are only intervals of the original image res (256 here) starting at 0 and not exceeding (the smaller sprite dimension-image res)(0,256,512,768,1024,1280,1536) (also original sprite dimensions: 1920,1792), so total of 8 replace all commands: ":256"->":512";":512"->":1024"... but I'll have to do them in descending order to avoid repeat-doubling.

2 problems: A) I double :1280 to :2560, then when I replace all :256 to :512, :2560 becomes :5120. So you need one more "replace all" 5120->:2560) or just replace :1280 at the end after you do :256.
B) The copied face landmarks from the originals do not scale up. And when I unpack without copying face data, and then try to import it from the parent frames afterward, it doesn't import to all of them and the ones they do import to are bigger than the face. I don't know what to do about this.

Apologies I've been a little busy.

I may have to look at this again soon because I also got a message from someone else who said


'I saw your post of using Remini to upscale 512 to 1024

The problem is that if I restore meta data then the size reverts back to it's original size.'


Perhaps this approach doesn't work or the meta data needs to be modified when this approach is done.

---

Is there a Moron's Guide to MVE?  I'd like one.  Thanks!

I unpacked the files in E:\MachineVideoEditor and all that does is give me the following error when I click on Dependency->Install 



Attempting to download Getty Images just fails silently.  No feedback given.  I don't get it.

I get this exact same error.  I generally think I know how to install stuff, but, I too would like a moron's guide...

Did you solve this? Make sure you download the correct version from the discord channel. Mine was windows 'MachineVideoEditor-0.7.2-win'.

---

Is there a Moron's Guide to MVE?  I'd like one.  Thanks!

I unpacked the files in E:\MachineVideoEditor and all that does is give me the following error when I click on Dependency->Install 



Attempting to download Getty Images just fails silently.  No feedback given.  I don't get it.

getting the same thing when installing dependencies but for getty, i get the error: "project no set"

Their discord channel is useful for help and go to Help > Dev Console to find out more about your issue.

 

[deepfakeid]

Hi there,
I'm new to this MVE, short question tho... can it produce aligned debug images just like if we extract the sequence from DFL?
Thanks

 

[light1up]

Hi there,
I'm new to this MVE, short question tho... can it produce aligned debug images just like if we extract the sequence from DFL?
Thanks

data_src/dst 'util add landmarks debug images' .bat file will cover this for you.

 

[deepfakeid]

Hi there,
I'm new to this MVE, short question tho... can it produce aligned debug images just like if we extract the sequence from DFL?
Thanks

data_src/dst 'util add landmarks debug images' .bat file will cover this for you.

Why I never thought about this... thank you three thousand!

 

[SPT]

Hi, currently using MVE for a project and there's one step I have a problem with :

1/Using MVE to generate pics for Remini, everything is fine.

2\Corrected landmarks on 50 of the resulting Remini faces for now, but in the end I will have to do it for ~750 pics.

So I wanted to check with the 50 images if they worked with DFL, just copying those in the MVE folder to data_dst/aligned
Of course it didn't work.
I'm not sure why.

To be able to edit alignments, I had to change "face" to "frames" on the right menu.

So do I have to finish editing landmarks on the 750 pics first then simply put them back to "face" on the right menu and "save alignment data" on the current tab, and then just copy pics from this folder to data_dst\aligned ?
Or is there some exporting to do first ?

for info I modified the landmarks on the first 50 pictures just using "apply & next" each time.
For landmark model, "dlib" is selected. Is that the right one or should I have selected "fan" ?

MVE seems to remember every edits I did with no issue, no problem on this front. Just don't know how I should proceed so that DFL accept the faces as DFL files.

Thanks to whoever can explain this to me.

 

[shaniafox]

hey guys! Thanks for the tips. Really worked for me. But I have one issue, the whole MVE is set on some fixed display resolution. How do i change it?

Right now, my screen is 1366x768 but the MVE application is on some other resolution

 

[pronto]

So do I have to finish editing landmarks on the 750 pics first then simply put them back to "face" on the right menu and "save alignment data" on the current tab, and then just copy pics from this folder to data_dst\aligned ?
Or is there some exporting to do first ?

I did the same mistake a couple of times, I think you need to select/highlight the modified frames (not faces tab) and then use the rightside 'detection management' menu to export/extract the changes. You get face extract options when you're on the frames tab.

 

[SPT]

So do I have to finish editing landmarks on the 750 pics first then simply put them back to "face" on the right menu and "save alignment data" on the current tab, and then just copy pics from this folder to data_dst\aligned ?
Or is there some exporting to do first ?

I did the same mistake a couple of times, I think you need to select/highlight the modified frames (not faces tab) and then use the rightside 'detection management' menu to export/extract the changes.  You get face extract options when you're on the frames tab.

Thank you. I went to MVE's discord and found a tutorial explaining this and now it works.

 

[shaniafox]

hey guys! Thanks for the tips. Really worked for me. But I have one issue, the whole MVE is set on some fixed display resolution. How do i change it?

Right now, my screen is 1366x768 but the MVE application is on some other resolution

Just an update since there was no answer. 

 

[INeedMoney]

Video tutorial showing how to automatically, variably extract all faces at each face's native size, instead of extracting them all at one resolution (i.e. 512, 1024).

(Lower total file size than extracting all at the largest face's resolution, and/or better quality of larger faces that are extracted at lower resolutions.)

 

[INeedMoney]

How to edit XSeg in MVE:

• Apply xseg mask to faceset.
• In MVE, open project, open images (aligned), open top tab on the right, set to face, click import face data. (wait)
• Click button on top that looks like a face, "Open XSeg masks". (wait)
• Your faces should all show their yellow xseg masks now. Scroll through to check them. (Click the button left of the "Open XSeg masks" button, "Show segmentation polygons" to see your drawn xseg labels if you made any yet.)
• To edit/create a label, right click a face and click "Edit Alignments".
• Above the face, switch from "Landmarks" to "Segmentation".
• On the right, click "Load generated mask". This will create a new label based on the applied mask.
• Move the yellow slider under "Point approximation" to the right. This will reduce the amount of points in the label to make it more manageable to edit in xseg editor.
• Reduce opacity to see the face edges better.
• You can use the two brushes to add/erase the mask. Or with the "draw polygon" tools, you can draw a more precise line, and then right click to add or subtract (based on which one you're using) the mask inside the shape that you draw.
• Click "Save" under the face when you're done.
• When you're done making labels, Save your aligned tab, then in the top right tab again, click "Embed Mask Polygons". (wait) Sometimes MVE is buggy and it won't embed the data and all your xseg work will be wasted. So to avoid this, select the face and click "Embed Mask Polygons" every time you create a new label. Make sure you see the progress go from 0 to 1 (or however many you selected) before it disappears. You can even double check by running xseg edit to make sure the labels are embedded.
• In DFL run the xseg edit bat, to more precisely check and fine-tune the labels. (ctrl+a or d to see only your labels. ctrl+click to add/subtract points.)
• Train xseg for a few minutes or longer if needed.
• Restart from the top of this list until your satisfied. (Don't forget to apply xseg after training!)

 

[tonobu]

Hello.

Is there a way to see DFL original alignments and masks in MVE? I want to use the 'face graph' tool to optimize my src by deleting redundant faces and complete missing faces.

Thanks.

 

[SPT]

I've made a mini tutorial to fix landmarks with MVE. Nothing new but said differently so it may help those who didn't get the previous tutorials.

//deep.whitecatchel.ru/literotica/forums/thread-how-to-fix-face-landmarks-with-mve

 

[SPT]

For info there's currently a very annoying bug with MVE's latest version. Basically when you save face landmarks in a tab/folder, it doesn't save everything, and if you want to save everything, you have to load face data, save, close tab, re open re load face data again and again until everything is saved. Most of the time it saves ~100 images more each time, but sometimes it's a random number.


So keep previous version for now if it works, until this one is fixed.