Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hi Pedro, On Tue, Jul 31, 2018 at 1:22 PM, Pedro Arthur wrote: > 2018-07-28 14:31 GMT-03:00 Pedro Arthur : > > 2018-07-27 14:24 GMT-03:00 Rostislav Pehlivanov : > >> On 27 July 2018 at 18:12, Rostislav Pehlivanov > wrote: > >> > >>> > >>> And the coding style is just the tip, there are dozens of things I > >>> disagree with, such as the custom float to 8 bit conversions, when > filters > >>> already exist which take in floats (zscale). > >>> > >> > >> I should probably expand on what I mean by the custom float conversion: > >> make the filter accept a float pixel format, such as > AV_PIX_FMT_GBRPF32. We > >> have one, we use it in vf_tonemap and vf_zscale. Users will need to > specify > >> a filter to convert to float first, which will match what they have to > do > >> for vf_zscale and vf_tonemap. If you need to process YUV then add a > float > >> YUV format, but please, don't do custom conversions in filters. > > It seems there is a patch adding YUV float formats in the ml, I asked > > the student to make a patch removing the conversion and supporting > > only float formats when the yuv float format get in the tree. > Just to update on the matter, there are currently two patches on ml, > one which fixes the code style and one which removes a huge amount of > stored data from the sr filter. > We are working on removing the format conversion from sr filter, > adding the needed floating formats to swscale. Thanks for taking the time/effort to address some of our concerns. Ronald ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
2018-07-28 14:31 GMT-03:00 Pedro Arthur : > 2018-07-27 14:24 GMT-03:00 Rostislav Pehlivanov : >> On 27 July 2018 at 18:12, Rostislav Pehlivanov wrote: >> >>> >>> And the coding style is just the tip, there are dozens of things I >>> disagree with, such as the custom float to 8 bit conversions, when filters >>> already exist which take in floats (zscale). >>> >> >> I should probably expand on what I mean by the custom float conversion: >> make the filter accept a float pixel format, such as AV_PIX_FMT_GBRPF32. We >> have one, we use it in vf_tonemap and vf_zscale. Users will need to specify >> a filter to convert to float first, which will match what they have to do >> for vf_zscale and vf_tonemap. If you need to process YUV then add a float >> YUV format, but please, don't do custom conversions in filters. > It seems there is a patch adding YUV float formats in the ml, I asked > the student to make a patch removing the conversion and supporting > only float formats when the yuv float format get in the tree. Just to update on the matter, there are currently two patches on ml, one which fixes the code style and one which removes a huge amount of stored data from the sr filter. We are working on removing the format conversion from sr filter, adding the needed floating formats to swscale. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
2018-07-27 14:24 GMT-03:00 Rostislav Pehlivanov : > On 27 July 2018 at 18:12, Rostislav Pehlivanov wrote: > >> >> And the coding style is just the tip, there are dozens of things I >> disagree with, such as the custom float to 8 bit conversions, when filters >> already exist which take in floats (zscale). >> > > I should probably expand on what I mean by the custom float conversion: > make the filter accept a float pixel format, such as AV_PIX_FMT_GBRPF32. We > have one, we use it in vf_tonemap and vf_zscale. Users will need to specify > a filter to convert to float first, which will match what they have to do > for vf_zscale and vf_tonemap. If you need to process YUV then add a float > YUV format, but please, don't do custom conversions in filters. It seems there is a patch adding YUV float formats in the ml, I asked the student to make a patch removing the conversion and supporting only float formats when the yuv float format get in the tree. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hi, On Fri, Jul 27, 2018 at 10:59 AM, Pedro Arthur wrote: > 2018-07-27 11:31 GMT-03:00 Ronald S. Bultje : > > Hi, > > > > On Fri, Jul 27, 2018 at 10:16 AM, Pedro Arthur > wrote: > > > >> Hi, > >> > >> 2018-07-27 8:12 GMT-03:00 Ronald S. Bultje : > >> > > >> > NN is different. The exposed software *uses* but does not *generate* > the > >> > coefficients. In fact, the meaning of most coefficients is completely > >> > opaque. They are just multiplier and addition constants with no > obvious > >> > relationship to reality, other than that they were generated by some > >> > magical process that tells us that statistically (??), this bears > >> > resemblence to some trained form of reality. You want to adjust > behaviour > >> > for a defined different outcome? You'll have to retrain. That's all > fine, > >> > don't get me wrong, but we need to know how to retrain since it's not > >> > obvious to us. Just tell us what process and what data were used to > get > >> the > >> > outcome so that we can get a different outcome if we so choose (that > *is* > >> > freedom to modify)? > >> > > >> > Please document (and make available) all such information for the > >> > re-training. > >> We already did it [1] the repo provides all scripts to download the > >> data and run the training and also generates the header with weights > >> readily to use in ffmpeg. > >> When Jean-Baptist raised this issue we worked to provide all the > >> necessary scripts to perform training, reading the replies above, it > >> seems no one is aware of it (not even J-B). > >> Isn't [1] enough? whats is missing then? why no one asked for whatever > >> may be missing in the original email when we provided the link to the > >> repo? > >> > >> > >> [1] - https://github.com/HighVoltageRocknRoll/sr > > > > > > It's probably because the documentation is not in FFmpeg, so how could we > > be aware of it? > > > > $ git grep -i rocknroll . > > $ pwd > > /Users/ronaldbultje/Projects/ffmpeg > > $ git log --oneline -1 > > 536bcc3 avcodec: add missing files missed in previous commits > > > > It sounds to me like we agree that documentation for re-training should > be > > provided, and the miscommunication was more about where this should be > > and/or how to make it discoverable. I would encourage you to make it as > > discoverable as possible by including the documentation inside the FFmpeg > > source tree, possibly inside the filter's source code file. > > > > (I understand that it may not be practical for the actual retraining data > > and scripts to be hosted on videolan/FFmpeg infrastructure.) > A small description with a link for the repo in the header with the > weights is enough? If it answers the questions (after the obvious "hey, that looks cool, let me dig deeper") "what on earth is this" and/or "how can I do that myself?", then maybe, yes? It might also be useful to briefly refer to (or repeat) the same in public documentation, e.g. in filters.texi, so that users who start reading about nn filters and want to pass their own coefficients can read easily how to generate them. Freedoms aren't just intended for developers, they're also meant for users. Ronald ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
On 27 July 2018 at 18:12, Rostislav Pehlivanov wrote: > > And the coding style is just the tip, there are dozens of things I > disagree with, such as the custom float to 8 bit conversions, when filters > already exist which take in floats (zscale). > I should probably expand on what I mean by the custom float conversion: make the filter accept a float pixel format, such as AV_PIX_FMT_GBRPF32. We have one, we use it in vf_tonemap and vf_zscale. Users will need to specify a filter to convert to float first, which will match what they have to do for vf_zscale and vf_tonemap. If you need to process YUV then add a float YUV format, but please, don't do custom conversions in filters. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
On 27 July 2018 at 17:48, Pedro Arthur wrote: > 2018-07-27 12:07 GMT-03:00 Rostislav Pehlivanov : > > On 27 July 2018 at 15:11, James Almer wrote: > > > >> On 7/27/2018 8:04 AM, Rostislav Pehlivanov wrote: > >> > Reference - IRC and j-b's earlier email. > >> > Coding style issues: > >> > DNNModel* ff_dnn_load_model_native(const char* model_filename) > >> > > >> > We never ever do stupid things like put the asterix first. The author > of > >> > the patch should have known better and the patch should have been > >> checked. > >> > Even a glance could have told you its wrong. > >> > >> Tone it down. It's a style issue. New contributors don't always know > >> things like that and we always tell them to fix it. It's the reviewer > >> who should have pointed it out, and if they missed it then it's > >> harmless. It's not like he used a public suffix like av_, where it can > >> be a problem. > >> > > > > It shows the code wasn't reviewed properly. These style issues are > > propagated throughout all the DNN framework, which was many thousands of > > lines of code and shows that not even a glimpse was spent on the actual > > code. We don't commit such huge patches without at least some form of > > review, even if months have passed and no one has bothered to yet. > The style issue is my fault, I did not ensure the student were aware > of the coding style. > That does not mean that other aspects of the code were not properly > reviewed, I spent a good amount of time reviewing and testing the > code. > The code style issue is easily fixable, the student will already > provide a fix, but if it is too much urgent I could fix it myself in a > few minutes. > Doesn't matter how easy it is to fix, its too late, the code landed, you pushed it and you didn't even bother to look at it. This is unacceptable, especially for such a big patchset, and unfair to the rest of developers. Ping the patchset, and keep pinging it until someone reviews it. Don't make it your authority to do so especially as its new code for which no maintainers exist. And the coding style is just the tip, there are dozens of things I disagree with, such as the custom float to 8 bit conversions, when filters already exist which take in floats (zscale). I don't see how reverting the code and asking for "proper" review, > which no one (in almost 3 months) besides me did, will do any good. > After reverting will you be compromised (or anyone else) to review it? > or the code will get forgotten waiting for review until no one cares > about it? > Yes, I'll review it myself. It is not fair to propose a patch, and push it within 24h, reverting > the student work of almost 3 months where no one has complained before > of issues to be fixed. > What isn't fair is that this literal unreviewed code dump is sitting in our repository at all, without review from any developer, but only with the reassurance that it compiles and that it mostly does the things it should. I don't want to keep pointing out issues upon issues with code in our repository, a nice clean start is what's needed, which means like I said a temporary revert. I'd rather have that than a tens of patches needed to actually fix the whole thing, when a few well formatted emails would suffice. The code style issue will be addressed. > We already provide a repo with training scripts, and it will be > referenced in the code. > > Objectively, what else needs to be fixed? let me know and give the > student at least a few days so he can provide a fix. Remove the internal tables. All of them. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
On 7/27/2018 12:07 PM, Rostislav Pehlivanov wrote: > On 27 July 2018 at 15:11, James Almer wrote: > >> On 7/27/2018 8:04 AM, Rostislav Pehlivanov wrote: >>> Reference - IRC and j-b's earlier email. >>> Coding style issues: >>> DNNModel* ff_dnn_load_model_native(const char* model_filename) >>> >>> We never ever do stupid things like put the asterix first. The author of >>> the patch should have known better and the patch should have been >> checked. >>> Even a glance could have told you its wrong. >> >> Tone it down. It's a style issue. New contributors don't always know >> things like that and we always tell them to fix it. It's the reviewer >> who should have pointed it out, and if they missed it then it's >> harmless. It's not like he used a public suffix like av_, where it can >> be a problem. >> > > It shows the code wasn't reviewed properly. These style issues are > propagated throughout all the DNN framework, which was many thousands of > lines of code and shows that not even a glimpse was spent on the actual > code. We don't commit such huge patches without at least some form of > review, even if months have passed and no one has bothered to yet. > > > >> >>> I described them in the sentence above. >>> But I'm not willing to wait for a potential fix, and especially not for a >>> whole bunch of them rewriting everything. The whole code needs to be >> thrown >>> out and thoroughly reviewed properly, by at least yourself and one other >>> person, preferably before gsoc ends. >>> You should start coordinating with your student on how to fix everything >>> mentioned and then resend the patchsets once fixed. I'll apply the revert >>> patch tomorrow. >> >> No, you wont. Not until this has been discussed. > > > Yes I will, unless someone objects I still intend to. No one has yet. I just objected, and Ronald is trying to find a way to solve this without getting to the point of reverting. You'll not apply this patch until there's a consensus, and much less after a day just because you don't like it. This is a collaborative project, and revert wars are not welcome. > > > I don't know what got >> to you but you're acting like someone pushed code that would get a cop >> on your doorstep. Calm down for once in your life, you're seemingly >> angry in every other email you write, and you're not helping making this >> project a friendly place at all for new and old contributors alike. >> > > I didn't intend to sound such but nevertheless the facts I mentioned > remain, and I can't really think of a way to present them better. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
2018-07-27 12:07 GMT-03:00 Rostislav Pehlivanov : > On 27 July 2018 at 15:11, James Almer wrote: > >> On 7/27/2018 8:04 AM, Rostislav Pehlivanov wrote: >> > Reference - IRC and j-b's earlier email. >> > Coding style issues: >> > DNNModel* ff_dnn_load_model_native(const char* model_filename) >> > >> > We never ever do stupid things like put the asterix first. The author of >> > the patch should have known better and the patch should have been >> checked. >> > Even a glance could have told you its wrong. >> >> Tone it down. It's a style issue. New contributors don't always know >> things like that and we always tell them to fix it. It's the reviewer >> who should have pointed it out, and if they missed it then it's >> harmless. It's not like he used a public suffix like av_, where it can >> be a problem. >> > > It shows the code wasn't reviewed properly. These style issues are > propagated throughout all the DNN framework, which was many thousands of > lines of code and shows that not even a glimpse was spent on the actual > code. We don't commit such huge patches without at least some form of > review, even if months have passed and no one has bothered to yet. The style issue is my fault, I did not ensure the student were aware of the coding style. That does not mean that other aspects of the code were not properly reviewed, I spent a good amount of time reviewing and testing the code. The code style issue is easily fixable, the student will already provide a fix, but if it is too much urgent I could fix it myself in a few minutes. I don't see how reverting the code and asking for "proper" review, which no one (in almost 3 months) besides me did, will do any good. After reverting will you be compromised (or anyone else) to review it? or the code will get forgotten waiting for review until no one cares about it? It is not fair to propose a patch, and push it within 24h, reverting the student work of almost 3 months where no one has complained before of issues to be fixed. The code style issue will be addressed. We already provide a repo with training scripts, and it will be referenced in the code. Objectively, what else needs to be fixed? let me know and give the student at least a few days so he can provide a fix. > > > >> >> > I described them in the sentence above. >> > But I'm not willing to wait for a potential fix, and especially not for a >> > whole bunch of them rewriting everything. The whole code needs to be >> thrown >> > out and thoroughly reviewed properly, by at least yourself and one other >> > person, preferably before gsoc ends. >> > You should start coordinating with your student on how to fix everything >> > mentioned and then resend the patchsets once fixed. I'll apply the revert >> > patch tomorrow. >> >> No, you wont. Not until this has been discussed. > > > Yes I will, unless someone objects I still intend to. No one has yet. > > > I don't know what got >> to you but you're acting like someone pushed code that would get a cop >> on your doorstep. Calm down for once in your life, you're seemingly >> angry in every other email you write, and you're not helping making this >> project a friendly place at all for new and old contributors alike. >> > > I didn't intend to sound such but nevertheless the facts I mentioned > remain, and I can't really think of a way to present them better. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
On 27 July 2018 at 15:11, James Almer wrote: > On 7/27/2018 8:04 AM, Rostislav Pehlivanov wrote: > > Reference - IRC and j-b's earlier email. > > Coding style issues: > > DNNModel* ff_dnn_load_model_native(const char* model_filename) > > > > We never ever do stupid things like put the asterix first. The author of > > the patch should have known better and the patch should have been > checked. > > Even a glance could have told you its wrong. > > Tone it down. It's a style issue. New contributors don't always know > things like that and we always tell them to fix it. It's the reviewer > who should have pointed it out, and if they missed it then it's > harmless. It's not like he used a public suffix like av_, where it can > be a problem. > It shows the code wasn't reviewed properly. These style issues are propagated throughout all the DNN framework, which was many thousands of lines of code and shows that not even a glimpse was spent on the actual code. We don't commit such huge patches without at least some form of review, even if months have passed and no one has bothered to yet. > > > I described them in the sentence above. > > But I'm not willing to wait for a potential fix, and especially not for a > > whole bunch of them rewriting everything. The whole code needs to be > thrown > > out and thoroughly reviewed properly, by at least yourself and one other > > person, preferably before gsoc ends. > > You should start coordinating with your student on how to fix everything > > mentioned and then resend the patchsets once fixed. I'll apply the revert > > patch tomorrow. > > No, you wont. Not until this has been discussed. Yes I will, unless someone objects I still intend to. No one has yet. I don't know what got > to you but you're acting like someone pushed code that would get a cop > on your doorstep. Calm down for once in your life, you're seemingly > angry in every other email you write, and you're not helping making this > project a friendly place at all for new and old contributors alike. > I didn't intend to sound such but nevertheless the facts I mentioned remain, and I can't really think of a way to present them better. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
On Fri, 27 Jul 2018, at 16:59, Pedro Arthur wrote: > 2018-07-27 11:31 GMT-03:00 Ronald S. Bultje : > > Hi, > > > > On Fri, Jul 27, 2018 at 10:16 AM, Pedro Arthur wrote: > > > >> Hi, > >> > >> 2018-07-27 8:12 GMT-03:00 Ronald S. Bultje : > >> > > >> > NN is different. The exposed software *uses* but does not *generate* the > >> > coefficients. In fact, the meaning of most coefficients is completely > >> > opaque. They are just multiplier and addition constants with no obvious > >> > relationship to reality, other than that they were generated by some > >> > magical process that tells us that statistically (??), this bears > >> > resemblence to some trained form of reality. You want to adjust behaviour > >> > for a defined different outcome? You'll have to retrain. That's all fine, > >> > don't get me wrong, but we need to know how to retrain since it's not > >> > obvious to us. Just tell us what process and what data were used to get > >> the > >> > outcome so that we can get a different outcome if we so choose (that *is* > >> > freedom to modify)? > >> > > >> > Please document (and make available) all such information for the > >> > re-training. > >> We already did it [1] the repo provides all scripts to download the > >> data and run the training and also generates the header with weights > >> readily to use in ffmpeg. > >> When Jean-Baptist raised this issue we worked to provide all the > >> necessary scripts to perform training, reading the replies above, it > >> seems no one is aware of it (not even J-B). > >> Isn't [1] enough? whats is missing then? why no one asked for whatever > >> may be missing in the original email when we provided the link to the > >> repo? > >> > >> > >> [1] - https://github.com/HighVoltageRocknRoll/sr > > > > > > It's probably because the documentation is not in FFmpeg, so how could we > > be aware of it? > > > > $ git grep -i rocknroll . > > $ pwd > > /Users/ronaldbultje/Projects/ffmpeg > > $ git log --oneline -1 > > 536bcc3 avcodec: add missing files missed in previous commits > > > > It sounds to me like we agree that documentation for re-training should be > > provided, and the miscommunication was more about where this should be > > and/or how to make it discoverable. I would encourage you to make it as > > discoverable as possible by including the documentation inside the FFmpeg > > source tree, possibly inside the filter's source code file. > > > > (I understand that it may not be practical for the actual retraining data > > and scripts to be hosted on videolan/FFmpeg infrastructure.) > A small description with a link for the repo in the header with the > weights is enough? And the README is empty. -- Jean-Baptiste Kempf - President +33 672 704 734 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
2018-07-27 11:31 GMT-03:00 Ronald S. Bultje : > Hi, > > On Fri, Jul 27, 2018 at 10:16 AM, Pedro Arthur wrote: > >> Hi, >> >> 2018-07-27 8:12 GMT-03:00 Ronald S. Bultje : >> > >> > NN is different. The exposed software *uses* but does not *generate* the >> > coefficients. In fact, the meaning of most coefficients is completely >> > opaque. They are just multiplier and addition constants with no obvious >> > relationship to reality, other than that they were generated by some >> > magical process that tells us that statistically (??), this bears >> > resemblence to some trained form of reality. You want to adjust behaviour >> > for a defined different outcome? You'll have to retrain. That's all fine, >> > don't get me wrong, but we need to know how to retrain since it's not >> > obvious to us. Just tell us what process and what data were used to get >> the >> > outcome so that we can get a different outcome if we so choose (that *is* >> > freedom to modify)? >> > >> > Please document (and make available) all such information for the >> > re-training. >> We already did it [1] the repo provides all scripts to download the >> data and run the training and also generates the header with weights >> readily to use in ffmpeg. >> When Jean-Baptist raised this issue we worked to provide all the >> necessary scripts to perform training, reading the replies above, it >> seems no one is aware of it (not even J-B). >> Isn't [1] enough? whats is missing then? why no one asked for whatever >> may be missing in the original email when we provided the link to the >> repo? >> >> >> [1] - https://github.com/HighVoltageRocknRoll/sr > > > It's probably because the documentation is not in FFmpeg, so how could we > be aware of it? > > $ git grep -i rocknroll . > $ pwd > /Users/ronaldbultje/Projects/ffmpeg > $ git log --oneline -1 > 536bcc3 avcodec: add missing files missed in previous commits > > It sounds to me like we agree that documentation for re-training should be > provided, and the miscommunication was more about where this should be > and/or how to make it discoverable. I would encourage you to make it as > discoverable as possible by including the documentation inside the FFmpeg > source tree, possibly inside the filter's source code file. > > (I understand that it may not be practical for the actual retraining data > and scripts to be hosted on videolan/FFmpeg infrastructure.) A small description with a link for the repo in the header with the weights is enough? > > Ronald > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hi, On Fri, Jul 27, 2018 at 10:16 AM, Pedro Arthur wrote: > Hi, > > 2018-07-27 8:12 GMT-03:00 Ronald S. Bultje : > > > > NN is different. The exposed software *uses* but does not *generate* the > > coefficients. In fact, the meaning of most coefficients is completely > > opaque. They are just multiplier and addition constants with no obvious > > relationship to reality, other than that they were generated by some > > magical process that tells us that statistically (??), this bears > > resemblence to some trained form of reality. You want to adjust behaviour > > for a defined different outcome? You'll have to retrain. That's all fine, > > don't get me wrong, but we need to know how to retrain since it's not > > obvious to us. Just tell us what process and what data were used to get > the > > outcome so that we can get a different outcome if we so choose (that *is* > > freedom to modify)? > > > > Please document (and make available) all such information for the > > re-training. > We already did it [1] the repo provides all scripts to download the > data and run the training and also generates the header with weights > readily to use in ffmpeg. > When Jean-Baptist raised this issue we worked to provide all the > necessary scripts to perform training, reading the replies above, it > seems no one is aware of it (not even J-B). > Isn't [1] enough? whats is missing then? why no one asked for whatever > may be missing in the original email when we provided the link to the > repo? > > > [1] - https://github.com/HighVoltageRocknRoll/sr It's probably because the documentation is not in FFmpeg, so how could we be aware of it? $ git grep -i rocknroll . $ pwd /Users/ronaldbultje/Projects/ffmpeg $ git log --oneline -1 536bcc3 avcodec: add missing files missed in previous commits It sounds to me like we agree that documentation for re-training should be provided, and the miscommunication was more about where this should be and/or how to make it discoverable. I would encourage you to make it as discoverable as possible by including the documentation inside the FFmpeg source tree, possibly inside the filter's source code file. (I understand that it may not be practical for the actual retraining data and scripts to be hosted on videolan/FFmpeg infrastructure.) Ronald ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hi, 2018-07-27 8:12 GMT-03:00 Ronald S. Bultje : > > NN is different. The exposed software *uses* but does not *generate* the > coefficients. In fact, the meaning of most coefficients is completely > opaque. They are just multiplier and addition constants with no obvious > relationship to reality, other than that they were generated by some > magical process that tells us that statistically (??), this bears > resemblence to some trained form of reality. You want to adjust behaviour > for a defined different outcome? You'll have to retrain. That's all fine, > don't get me wrong, but we need to know how to retrain since it's not > obvious to us. Just tell us what process and what data were used to get the > outcome so that we can get a different outcome if we so choose (that *is* > freedom to modify)? > > Please document (and make available) all such information for the > re-training. We already did it [1] the repo provides all scripts to download the data and run the training and also generates the header with weights readily to use in ffmpeg. When Jean-Baptist raised this issue we worked to provide all the necessary scripts to perform training, reading the replies above, it seems no one is aware of it (not even J-B). Isn't [1] enough? whats is missing then? why no one asked for whatever may be missing in the original email when we provided the link to the repo? [1] - https://github.com/HighVoltageRocknRoll/sr ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
On 7/27/2018 8:04 AM, Rostislav Pehlivanov wrote: > Reference - IRC and j-b's earlier email. > Coding style issues: > DNNModel* ff_dnn_load_model_native(const char* model_filename) > > We never ever do stupid things like put the asterix first. The author of > the patch should have known better and the patch should have been checked. > Even a glance could have told you its wrong. Tone it down. It's a style issue. New contributors don't always know things like that and we always tell them to fix it. It's the reviewer who should have pointed it out, and if they missed it then it's harmless. It's not like he used a public suffix like av_, where it can be a problem. > I described them in the sentence above. > But I'm not willing to wait for a potential fix, and especially not for a > whole bunch of them rewriting everything. The whole code needs to be thrown > out and thoroughly reviewed properly, by at least yourself and one other > person, preferably before gsoc ends. > You should start coordinating with your student on how to fix everything > mentioned and then resend the patchsets once fixed. I'll apply the revert > patch tomorrow. No, you wont. Not until this has been discussed. I don't know what got to you but you're acting like someone pushed code that would get a cop on your doorstep. Calm down for once in your life, you're seemingly angry in every other email you write, and you're not helping making this project a friendly place at all for new and old contributors alike. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Rostislav Pehlivanov (2018-07-27): > Reference - IRC I think I agree with you on the issue at hand, but this point raises again an issue I would like to emphasize: Not everybody in the project does, or even can, participate in IRC discussions. And while these discussions are archived, they are not adapted for reading afterward, they are not curated by their authors like mails. Therefore, please remember that IRC discussions cannot be considered consensus. You can of course discuss things on IRC, but before calling their result consensus, somebody needs to summarize the arguments to bring them on the mailing-list and let them be discussed here. Regards, -- Nicolas George signature.asc Description: Digital signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hi, On Fri, Jul 27, 2018 at 7:02 AM, Ronald S. Bultje wrote: > Hi, > > On Thu, Jul 26, 2018 at 11:49 PM, Pedro Arthur > wrote: > >> Taking the vp9 as example, sure the coeficients are obtained by the >> 'poly3' but the real data are the polynomial coeficients, does any one >> asks where these polynomial coeficients came from, reproducibility, >> etc? Your comparison does not seems fair to me. > > > The ~1.02 in VP9/AV1 and 6 in HEVC? They are stepsizes (in different > units). It's exactly clear where they come from and it's obvious how to > adjust them. You want more than 52 qps at the same range in HEVC? Increase > the 6. > > How do I adjust the NN coefficients to get a defined adjustment in > behaviour? > Actually, since I know you'll answer like you don't get it, let me explain it in more detail to you: the adjustment in quantizer, derivation of the coefficients if you will, is all defined by the process itself. The external tools used to validate are the actual encoding software itself (the model). You call this training? Sure, but then the software created by the training *is* the training software in itself. But in all reasonableness, it's clear what each value means and how its adjustments affects the process - that is, if you are willing to understand. NN is different. The exposed software *uses* but does not *generate* the coefficients. In fact, the meaning of most coefficients is completely opaque. They are just multiplier and addition constants with no obvious relationship to reality, other than that they were generated by some magical process that tells us that statistically (??), this bears resemblence to some trained form of reality. You want to adjust behaviour for a defined different outcome? You'll have to retrain. That's all fine, don't get me wrong, but we need to know how to retrain since it's not obvious to us. Just tell us what process and what data were used to get the outcome so that we can get a different outcome if we so choose (that *is* freedom to modify)? Please document (and make available) all such information for the re-training. There's a second, orthogonal, concern: many of these filters that we're seeing now are filters to convert X to Y are do process Z on data A. That's great. I love it. I have no problem with it. I see a filter that converts 8bit RGB to 10bit HDR RGB and it's fantastic, it's a little bit like colorspace (which I wrote) but maybe it can actually get closer to intended (rather than artifacted) source, which would be fantastic. I hope it works. But: how do we know that it does? For example, for the colorspace filter, it's clear, because all coefficients come from the reference documentation. For the NN filter version, how do I know it works? Seriously. If I were an anonymous Russian submitting such a filter to the American elections, how can I guarantee that it does what I say it does rather than just randomly garbling the lower bits? How would I know the difference? Again, I need to know how it was trained. Lastly, if all filters do the same but with a different set of coefs, then why not simply have one filter and loadable coefs? Maybe subclassing etc. for better documentation - it would make SIMD a lot easier since we only have to write it once. Ronald ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
On 27 July 2018 at 03:04, Pedro Arthur wrote: > Hi, > > I'm surprised with this patch, there wasn't any concern raised in the > patch review process. > It slipped through, and we (and you) should have known better. > 2018-07-26 16:26 GMT-03:00 Rostislav Pehlivanov : > > As discussed recently, the vf_sr filter and the DNN framework have an > > issue: unreproducable weights and questionable license, as well as > > overall unfitting coding style to the rest of the project. > I think I'm not aware of these discussions could you provide a > reference? I also don't understand why the coding style is unfitting > (again no concern was raised). > Reference - IRC and j-b's earlier email. Coding style issues: DNNModel* ff_dnn_load_model_native(const char* model_filename) We never ever do stupid things like put the asterix first. The author of the patch should have known better and the patch should have been checked. Even a glance could have told you its wrong. > > > The vf_sr filter in particular has weights embedded which weight the > > libavfilter binary by a bit and cannot currently be reproduced. > > There's an overall consensus that NN filters should accept external > > weights only, as the nnedi filter currently does. > > > > So, temporarily remove both until the coding style issues have been > > fixed with the framework and the filter has been modified to accept > > external weights. > What are these issues so we can fix them? > I described them in the sentence above. But I'm not willing to wait for a potential fix, and especially not for a whole bunch of them rewriting everything. The whole code needs to be thrown out and thoroughly reviewed properly, by at least yourself and one other person, preferably before gsoc ends. You should start coordinating with your student on how to fix everything mentioned and then resend the patchsets once fixed. I'll apply the revert patch tomorrow. > > > Also, there's a discussion by the Debian folks as to whether to treat > > pretrained NNs as non-free[0], hence its not just our project that's > > affected by the questionable license of distributing pretrained NN > > weights. > > > > Due to the weight of the patch (more than 1mb!) I've uploaded it to > > https://0x0.st/sVEH.patch if anyone wants to test it. The change stat > > is printed below. > > > > [0]: https://lwn.net/Articles/760142/ > I took the time to read the whole discussion and in my opinion it is > flawed, except for the storage requirement, which Sergey already > worked on a patch to reduce the stored data. > > > I think before any discussion, it should be clear what is the ffmpeg > policy on adding data, and what is considered data, and it should be > consistent. > > I'll try to address the topics in the above discussion. > > First what is data? is it expected that all data stored should be > easily reproducible? > Easily? No. Always? Absolutely. I guess not, what is the point in storing data that is easily reproducible? > We do it for speed reasons. Large tables take time to generate on startup. The entire humanity is built on previous stored knowledge, namely > data, do we require each time one is going to use some form of > knowledge to reproduce it? that is, proof everything one is using? > The answer is no, the whole point in storing data is that you had once > worked hard to proof it works and onwards just use it and believe it > works. That does not mean it is imposible to proof everything. > > I think the above fits perfectly with NN weights as data. > > The next point is the reproducibility, one should be reasonable, it is > hard to reproduce bit by bit of NN stored data but is totally doable > to achieve similar results following the same training metodology > used. > Very well, provide code and references to sources to reproduce please. > Then there is the question what is open-source, once again one should > be reasonable. > The NN model is public available, everything is documented, the math > machinery is also widely available and documented. > There is also a repository containing everything one needs to train > the NN and achieve similar results, the trainig data is public also. > The training software is open-source, the user could also use any > machine learning framework of their choice to perform the training > since the model is documented, there is nothing locking one to a > specific software or hardware. > I can't see what is not open. > The process to replicate the weights. Provide script(s) and references to sources. Does we impose all requiriments imposed for NN weights on all other > data stored in ffmpeg? > I guess not, once more one should be consistent. > > > If you compare NN weights with quantization tables they are pretty > similar, both can be obtained from a training process over a dataset > so it achieves better results (quality/compression). Are quantization > tables evil? I don't think so. > It seems people is afraid of NN just because we give it
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hi, On Thu, Jul 26, 2018 at 11:49 PM, Pedro Arthur wrote: > Taking the vp9 as example, sure the coeficients are obtained by the > 'poly3' but the real data are the polynomial coeficients, does any one > asks where these polynomial coeficients came from, reproducibility, > etc? Your comparison does not seems fair to me. The ~1.02 in VP9/AV1 and 6 in HEVC? They are stepsizes (in different units). It's exactly clear where they come from and it's obvious how to adjust them. You want more than 52 qps at the same range in HEVC? Increase the 6. How do I adjust the NN coefficients to get a defined adjustment in behaviour? Ronald ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
2018-07-27 4:04 GMT+02:00, Pedro Arthur : > Does we impose all requiriments imposed for NN weights > on all other data stored in ffmpeg? (As explained before:) No, we clearly do not. Carl Eugen ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hello, On Fri, 27 Jul 2018, at 05:49, Pedro Arthur wrote: > > Are NN weights a single, one-line (10-character) expression? Please > > elaborate. Why isn't that 10-character function documented anywhere? > I think you missed the point, I wrote "can be obtained from a training > process over a dataset so it achieves better results > (quality/compression)". Yes, but that is the point I made days ago already, and you did not address. You MUST provide a process to produce data that is similar. Else, it is not free software. It might be cool and fun; but that is not free software. -- Jean-Baptiste Kempf - President +33 672 704 734 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
2018-07-26 23:19 GMT-03:00 Ronald S. Bultje : > Hi, > > On Thu, Jul 26, 2018 at 10:04 PM, Pedro Arthur wrote: > >> If you compare NN weights with quantization tables they are pretty >> similar > > > https://chromium.googlesource.com/webm/libvpx/+/3b9c19aaa7b8830a896c5f578a3ce6c6a7953947%5E%21/#F0 > > So, that one tiny single function is how VP9/AV1 quant tables are generated. > > Or, the HEVC/H264 ones, they are even simpler: exp2(qp/6). > > Are NN weights a single, one-line (10-character) expression? Please > elaborate. Why isn't that 10-character function documented anywhere? I think you missed the point, I wrote "can be obtained from a training process over a dataset so it achieves better results (quality/compression)". Taking the vp9 as example, sure the coeficients are obtained by the 'poly3' but the real data are the polynomial coeficients, does any one asks where these polynomial coeficients came from, reproducibility, etc? Your comparison does not seems fair to me. > > Ronald > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hi, On Thu, Jul 26, 2018 at 10:04 PM, Pedro Arthur wrote: > If you compare NN weights with quantization tables they are pretty > similar https://chromium.googlesource.com/webm/libvpx/+/3b9c19aaa7b8830a896c5f578a3ce6c6a7953947%5E%21/#F0 So, that one tiny single function is how VP9/AV1 quant tables are generated. Or, the HEVC/H264 ones, they are even simpler: exp2(qp/6). Are NN weights a single, one-line (10-character) expression? Please elaborate. Why isn't that 10-character function documented anywhere? Ronald ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hi, I'm surprised with this patch, there wasn't any concern raised in the patch review process. 2018-07-26 16:26 GMT-03:00 Rostislav Pehlivanov : > As discussed recently, the vf_sr filter and the DNN framework have an > issue: unreproducable weights and questionable license, as well as > overall unfitting coding style to the rest of the project. I think I'm not aware of these discussions could you provide a reference? I also don't understand why the coding style is unfitting (again no concern was raised). > > The vf_sr filter in particular has weights embedded which weight the > libavfilter binary by a bit and cannot currently be reproduced. > There's an overall consensus that NN filters should accept external > weights only, as the nnedi filter currently does. > > So, temporarily remove both until the coding style issues have been > fixed with the framework and the filter has been modified to accept > external weights. What are these issues so we can fix them? > > Also, there's a discussion by the Debian folks as to whether to treat > pretrained NNs as non-free[0], hence its not just our project that's > affected by the questionable license of distributing pretrained NN > weights. > > Due to the weight of the patch (more than 1mb!) I've uploaded it to > https://0x0.st/sVEH.patch if anyone wants to test it. The change stat > is printed below. > > [0]: https://lwn.net/Articles/760142/ I took the time to read the whole discussion and in my opinion it is flawed, except for the storage requirement, which Sergey already worked on a patch to reduce the stored data. I think before any discussion, it should be clear what is the ffmpeg policy on adding data, and what is considered data, and it should be consistent. I'll try to address the topics in the above discussion. First what is data? is it expected that all data stored should be easily reproducible? I guess not, what is the point in storing data that is easily reproducible? The entire humanity is built on previous stored knowledge, namely data, do we require each time one is going to use some form of knowledge to reproduce it? that is, proof everything one is using? The answer is no, the whole point in storing data is that you had once worked hard to proof it works and onwards just use it and believe it works. That does not mean it is imposible to proof everything. I think the above fits perfectly with NN weights as data. The next point is the reproducibility, one should be reasonable, it is hard to reproduce bit by bit of NN stored data but is totally doable to achieve similar results following the same training metodology used. Then there is the question what is open-source, once again one should be reasonable. The NN model is public available, everything is documented, the math machinery is also widely available and documented. There is also a repository containing everything one needs to train the NN and achieve similar results, the trainig data is public also. The training software is open-source, the user could also use any machine learning framework of their choice to perform the training since the model is documented, there is nothing locking one to a specific software or hardware. I can't see what is not open. Does we impose all requiriments imposed for NN weights on all other data stored in ffmpeg? I guess not, once more one should be consistent. If you compare NN weights with quantization tables they are pretty similar, both can be obtained from a training process over a dataset so it achieves better results (quality/compression). Are quantization tables evil? I don't think so. It seems people is afraid of NN just because we give it a fancy name, while it is just tables of data as we always used in our code. > > Signed-off-by: Rostislav Pehlivanov > > Rostislav Pehlivanov (1): > libavfilter: temporarily remove DNN framework and vf_sr filter > > Changelog| 1 - > configure| 8 - > libavfilter/Makefile | 3 - > libavfilter/allfilters.c | 1 - > libavfilter/dnn_backend_native.c | 495 -- > libavfilter/dnn_backend_native.h |40 - > libavfilter/dnn_backend_tf.c | 325 - > libavfilter/dnn_backend_tf.h |40 - > libavfilter/dnn_espcn.h | 12637 - > libavfilter/dnn_interface.c |60 - > libavfilter/dnn_interface.h |63 - > libavfilter/dnn_srcnn.h | 4957 --- > libavfilter/vf_sr.c | 354 - > 13 files changed, 18984 deletions(-) > delete mode 100644 libavfilter/dnn_backend_native.c > delete mode 100644 libavfilter/dnn_backend_native.h > delete mode 100644 libavfilter/dnn_backend_tf.c > delete mode 100644 libavfilter/dnn_backend_tf.h > delete mode 100644 libavfilter/dnn_espcn.h > delete mode 100644 libavfilter/dnn_interface.c > delete mode 100644 libavfilter/dnn_interface.h > delete mode 100644 libavfilter/dnn_srcnn.h > delete mode 100644 lib
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
2018-07-26 21:26 GMT+02:00, Rostislav Pehlivanov : > There's an overall consensus that NN filters should accept > external weights only Do you mean an overall consensus on irc? I ask because the patch in question was sent several times for review, and I don't remember a comment concerning internal weights. When the issue was brought up on the mailing list, at least one developer defended the internal weights iirc. (No opinion here.) Carl Eugen ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
On 7/26/18, Thilo Borgmann wrote: > Hi, > > Am 26.07.18 um 21:26 schrieb Rostislav Pehlivanov: >> As discussed recently, the vf_sr filter and the DNN framework have an >> issue: unreproducable weights and questionable license, as well as >> overall unfitting coding style to the rest of the project. >> >> The vf_sr filter in particular has weights embedded which weight the >> libavfilter binary by a bit and cannot currently be reproduced. >> There's an overall consensus that NN filters should accept external >> weights only, as the nnedi filter currently does. >> >> So, temporarily remove both until the coding style issues have been >> fixed with the framework and the filter has been modified to accept >> external weights. >> >> Also, there's a discussion by the Debian folks as to whether to treat >> pretrained NNs as non-free[0], hence its not just our project that's >> affected by the questionable license of distributing pretrained NN >> weights. > > I personally don't have a good feeling with pre-trained NNs in the codebase, > too. However, I do not care much about what solution we take, but Mina's > GSoC project also depends on the NN code that comes with this and therefore > I'd encourage everyone to make up their mind to find a suitable solution > sometime soonish. > > Maybe for the time-being, we might only accept such code reading in > externally provided NNs and/or the ability to train using FFmpeg itself. (Or > ask one of our kind users with compute power to generate some for us) IIRC mentioned filter already supports external files. It just have internal one too. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
Hi, Am 26.07.18 um 21:26 schrieb Rostislav Pehlivanov: > As discussed recently, the vf_sr filter and the DNN framework have an > issue: unreproducable weights and questionable license, as well as > overall unfitting coding style to the rest of the project. > > The vf_sr filter in particular has weights embedded which weight the > libavfilter binary by a bit and cannot currently be reproduced. > There's an overall consensus that NN filters should accept external > weights only, as the nnedi filter currently does. > > So, temporarily remove both until the coding style issues have been > fixed with the framework and the filter has been modified to accept > external weights. > > Also, there's a discussion by the Debian folks as to whether to treat > pretrained NNs as non-free[0], hence its not just our project that's > affected by the questionable license of distributing pretrained NN > weights. I personally don't have a good feeling with pre-trained NNs in the codebase, too. However, I do not care much about what solution we take, but Mina's GSoC project also depends on the NN code that comes with this and therefore I'd encourage everyone to make up their mind to find a suitable solution sometime soonish. Maybe for the time-being, we might only accept such code reading in externally provided NNs and/or the ability to train using FFmpeg itself. (Or ask one of our kind users with compute power to generate some for us) > Due to the weight of the patch (more than 1mb!) I've uploaded it to > https://0x0.st/sVEH.patch if anyone wants to test it. The change stat > is printed below. > > [0]: https://lwn.net/Articles/760142/ > > Signed-off-by: Rostislav Pehlivanov > > Rostislav Pehlivanov (1): > libavfilter: temporarily remove DNN framework and vf_sr filter > > Changelog| 1 - > configure| 8 - > libavfilter/Makefile | 3 - > libavfilter/allfilters.c | 1 - > libavfilter/dnn_backend_native.c | 495 -- > libavfilter/dnn_backend_native.h |40 - > libavfilter/dnn_backend_tf.c | 325 - > libavfilter/dnn_backend_tf.h |40 - > libavfilter/dnn_espcn.h | 12637 - > libavfilter/dnn_interface.c |60 - > libavfilter/dnn_interface.h |63 - > libavfilter/dnn_srcnn.h | 4957 --- > libavfilter/vf_sr.c | 354 - > 13 files changed, 18984 deletions(-) > delete mode 100644 libavfilter/dnn_backend_native.c > delete mode 100644 libavfilter/dnn_backend_native.h > delete mode 100644 libavfilter/dnn_backend_tf.c > delete mode 100644 libavfilter/dnn_backend_tf.h > delete mode 100644 libavfilter/dnn_espcn.h > delete mode 100644 libavfilter/dnn_interface.c > delete mode 100644 libavfilter/dnn_interface.h > delete mode 100644 libavfilter/dnn_srcnn.h > delete mode 100644 libavfilter/vf_sr.c > -Thilo ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter
As discussed recently, the vf_sr filter and the DNN framework have an issue: unreproducable weights and questionable license, as well as overall unfitting coding style to the rest of the project. The vf_sr filter in particular has weights embedded which weight the libavfilter binary by a bit and cannot currently be reproduced. There's an overall consensus that NN filters should accept external weights only, as the nnedi filter currently does. So, temporarily remove both until the coding style issues have been fixed with the framework and the filter has been modified to accept external weights. Also, there's a discussion by the Debian folks as to whether to treat pretrained NNs as non-free[0], hence its not just our project that's affected by the questionable license of distributing pretrained NN weights. Due to the weight of the patch (more than 1mb!) I've uploaded it to https://0x0.st/sVEH.patch if anyone wants to test it. The change stat is printed below. [0]: https://lwn.net/Articles/760142/ Signed-off-by: Rostislav Pehlivanov Rostislav Pehlivanov (1): libavfilter: temporarily remove DNN framework and vf_sr filter Changelog| 1 - configure| 8 - libavfilter/Makefile | 3 - libavfilter/allfilters.c | 1 - libavfilter/dnn_backend_native.c | 495 -- libavfilter/dnn_backend_native.h |40 - libavfilter/dnn_backend_tf.c | 325 - libavfilter/dnn_backend_tf.h |40 - libavfilter/dnn_espcn.h | 12637 - libavfilter/dnn_interface.c |60 - libavfilter/dnn_interface.h |63 - libavfilter/dnn_srcnn.h | 4957 --- libavfilter/vf_sr.c | 354 - 13 files changed, 18984 deletions(-) delete mode 100644 libavfilter/dnn_backend_native.c delete mode 100644 libavfilter/dnn_backend_native.h delete mode 100644 libavfilter/dnn_backend_tf.c delete mode 100644 libavfilter/dnn_backend_tf.h delete mode 100644 libavfilter/dnn_espcn.h delete mode 100644 libavfilter/dnn_interface.c delete mode 100644 libavfilter/dnn_interface.h delete mode 100644 libavfilter/dnn_srcnn.h delete mode 100644 libavfilter/vf_sr.c -- 2.18.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
