Ideas for DFL

[SPT]

Hello, , would it be possible to have filenames like this in the preview ? So that we instantly know the name of the file when we see a wrong image we forgot to delete. Instead of looking inside the whole folder.


Another idea, maybe more complicated to do :




A way to view the images causing these peaks in the graph, so that we can correct or delete them.


What do you think ?

 

[Grrkin]

I just want the ability to extract faces at higher resolutions. Would chip in to pay for that if enough other ppl wanted to try to bribe Iperov. Lol

Future proofing reasons, really bummed me out so many face sets at 256, and 512 will be like that very quickly with the speed of things.

---

Ps. Also moved the thread to the discussion forum, it fits better.

 

[sharital]

I just want the ability to extract faces at higher resolutions. Would chip in to pay for that if enough other ppl wanted to try to bribe Iperov. Lol

Future proofing reasons, really bummed me out so many face sets at 256, and 512 will be like that very quickly with the speed of things.

---

Ps. Also moved the thread to the discussion forum, it fits better.

---

The majority of video SRC out there to get faces from is 1080p.  4K just isn't available enough in reasonable quantities.  So with getting most videos at 1080p there isn't really any need for anything above 512 for the foreseeable future.

Working in 4K would be fun occasionally, but for folks creating a lot of videos that is a ton of downloading/uploading 2GB+ files over and over and over.  I don't think we'll need more than 512 res for a couple years.

---

Hello, , would it be possible to have filenames like this in the preview ? So that we instantly know the name of the file when we see a wrong image we forgot to delete. Instead of looking inside the whole folder.


Another idea, maybe more complicated to do :




A way to view the images causing these peaks in the graph, so that we can correct or delete them.


What do you think ?

I love idea #1.   Same thing (file name) needs to be in the XSEG training window as well, so when you see those wonky masks you know where to go to fix it.

 

[Grrkin]

The majority of video SRC out there to get faces from is 1080p.  4K just isn't available enough in reasonable quantities.  So with getting most videos at 1080p there isn't really any need for anything above 512 for the foreseeable future.

Working in 4K would be fun occasionally, but for folks creating a lot of videos that is a ton of downloading/uploading 2GB+ files over and over and over.  I don't think we'll need more than 512 res for a couple years.

The majority of video SRC's that I choose to get faces from are 1080p. 4k just isn't something I can find or want to use, and I do not care about having this face set for the future. I think 512 is good enough for me.
Working in 4k is fun and a lot of work, but for folks like me, it's too much. I don't think I will need more than 512 for many years.
Fixed.

 

[morphmorpheus]

"SPT"  Upvote

"Grrkin" Please fix mine 

 

[BeakFakes]

Hi. I would wish that the improt of Faceset.pak in google colab for pretraining would be easier. At the moment I put the Faceset.pak in the model folder and upload it. Then I delete /pretrain/Faceset.pak and move the Faceset.pak from /workspace/model there.
A direct import would be a great relief!

 

[DarkAlchy]

Yes, I was saying the same thing about idea #1 just a few days ago myself. I see a really weird photo but out of 25k photos there is no way I will ever be able to find the bad ones

 

[SPT]

Another idea : Can we have "5) data_dst faceset MANUAL RE-EXTRACT DELETED ALIGNED_DEBUG" but for pictures deleted from data_dst\aligned ? I know you can do that via Powershell, but I have yet to look into this. Can't someone just make a .bat for it ? It would probably need to have pictures inside data_dst\aligned have their original filename, but besides this, with my very limited coding knowledge, I don't see why it wouldn't be do-able with a .bat file. I'm also incapable of doing it myself so I might be wrong

 

[angeloshredder]

Hello, , would it be possible to have filenames like this in the preview ? So that we instantly know the name of the file when we see a wrong image we forgot to delete. Instead of looking inside the whole face

strange !!!  , but i had this same idea today after looking at the preview screen for more than 3 hours watching faces and pressing "space" and "P"   
Also the woman at left is so beautiful , the kind of beauty that i like

 

[SPT]

Hello, , would it be possible to have filenames like this in the preview ? So that we instantly know the name of the file when we see a wrong image we forgot to delete. Instead of looking inside the whole face

strange !!!  , but i had this same idea today after looking at the preview screen for more than 3 hours watching faces and pressing "space" and "P"   
Also the woman at left is so beautiful , the kind of beauty that i like

Thank you. Google Hailee Steinfeld.

 

[SPT]

New idea : Intelligent interpolating.

There's always some undetected or unproperly detected faces. Some are fixable with manual extract, some aren't.

So in my idea we should have a way to mark the unfixable faces, for example in manual re-extract GUI. Let's say DEL or any other shortcut marks these pics.

Then, in merging, there's a new option to either basically cut these frames from the merging process (and cutting the audio track in the same place too) or, merger will replace these unfixable frames with a smooth morph/transition/interpolation. I guess it would use a system similar to Hybrid or Twixtor, but just working for intervals of a few frames of unfixable faces. If there's 6 unfixable faces, this option would cut these and replace the 6 frames with 1-3 frames of transition/morph between first ok face before unfixable and next ok frame after.

For example :
ok 01 - unfix01 - unfix02 - unfix03 - unfix04 - unfix05 - unfix06 - ok 02

is replaced by
ok 01 - transi01 - transi02 - ok 02

Also while cutting the audio track approriately.


This is all stuff you can do in external apps, but having a way to merge without these unfixable frames would accelerate the workflow. No need to view the whole vid and search for those frame to cut.

Anyways, probably a lot of complex coding required for this, but if is out of ideas one day...

 

[Groggy4]

"iperov" Could it be possible in XSeg editor to make "shift" toggle through different viewing speeds? When watching for holes in learned mask, default is fine. But with overlay it would help slowing it down to maybe 1/3 because of all the extra details to consider. Or just a new keystroke for that purpose.

 

[papolk]

Merger:
Shift face vertically
Sometimes you just need a little more chin/forehead (or less)

Resize face y/x axis separetely, sometimes you need to stretch it.

 

[TheLukaDragar]

Using colab to process merged frames (remote gpu) like avatarfy via ngrok example here: 
github:  
colab: 


Thanks!

[img=400x189]

 

[iperov]

Merger:
Shift face vertically
Sometimes you just need a little more chin/forehead (or less)

impossible, because local coordinate system (red debug frame) is not precise across timeline, thus shifted face will shaking

 

[sharital]

Merger:
Shift face vertically
Sometimes you just need a little more chin/forehead (or less)

Resize face y/x axis separetely, sometimes you need to stretch it.

iperov addressed your first thing, but for the second thing as long as you have a video where the person's head angle doesn't change much, you can just use video editing before you extract to change the width or height to stretch the face.  I've done this before.

 

[SPT]

Another idea :

Sometimes when we merge, whatever color transfer method we use,there's always frames here and there not looking great. For example I'm using idt-m, which is perfect 95% of the time, but produces white blurbs on some frames.

So I would like to have a tool to select these frames, and re-do the merging of only these ones with RCT or any other color transfer method able correct this problem.

Maybe knows a trick to do it, but if not it could be a nice improvement. Even for things other than color transfer btw, maybe some frames needs way more super res, or more motion blur... whatever.

 

[TMBDF]

Another idea :

Sometimes when we merge, whatever color transfer method we use,there's always frames here and there not looking great. For example I'm using idt-m, which is perfect 95% of the time, but produces white blurbs on some frames.

So I would like to have a tool to select these frames, and re-do the merging of only these ones with RCT or any other color transfer method able correct this problem.

Maybe knows a trick to do it, but if not it could be a nice improvement. Even for things other than color transfer btw, maybe some frames needs way more super res, or more motion blur... whatever.

I think the best way would be to add an option similar to keyframing to the merger where we can set from which frame to which one use certain settings, be able to set merging setting for few of them and then either have them change right away as soon as we get to that frame/keyframe or have few options to interpolate the setting values (similar to those you typically have in video editing software).

I still stand for my ideas on improving the face type coverage, if not new face types (which according to my test while can be improved to waste less space it's not much) could be improved by adding new landmarks for hairline (or it's approximation if hair is obstructing it). This would solve a lot of issues where faces sometimes are too small or big on the image and maximize effectiveness of the model resolution by always having face fill the entire space (from hairline to chin for whole face, from half of the chin to half of the forehead for full face and obviously head would remain as it is so it covers hair and all). I'm just not sure what adding new landmarks would take (incompatibility with previous datasets? rewrite of more code? incompatibility of previous models?)

 

[SPT]

Another one :

Fast preview of merging.

It would work like a preview screenshot for video in MPC or any video player able to make previews screenshots : you choose the number of previews you want, and the program divides the whole video by this number so you get screenshots at different times in the vid.

Example :

You have a 30000 frames vid, you choose 30, the program renders 1 every 1000 frames with your merging settings and you have a good overview of what it looks like at different points in the video.

Because sometimes it looks great in the first 300 frames and then it's shit after that and you have no way other than re-render the whole thing with different settings.


It might even be a simple enough thing that you could just create a .bat file for it (since it would be the same thing as "merge SAEHD" but limited to whatever number of previews you want). But I'm so illiterate in coding that I can't do it myself.

 

[Groggy4]

"SPT" You could always split the dst up in section based on scenes, and merge them as such. I usually do that when having a blowjob scene at the end where you wanna check most frames. But an automatic system sounds easier.