Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On 2015-10-21 12:18, wm4 wrote: > with size_t/ptrdiff_t > being 128 bit, and a new "long long long int" type (I swear, they will > do it, even if that type name looks horrible). Please no! Just require a C99 style uint128_t/int128_t type. signature.asc Description: OpenPGP digital signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On 2015-10-21 14:44, Clément Bœsch wrote: > On Wed, Oct 21, 2015 at 06:00:21AM -0400, Ganesh Ajjanagadde wrote: > [...] >> why don't you spend 5 minutes trying to outline to beginners like me >> what is "actually important" in your view? >> > > According to the first 100 answers of the survey, the majority of the > users want... speed optimisation above anything. I can't give more info > for now, I'll need to pay to get the stats taking into account all votes. I know you just said you can't give more detail but can I ask this: did these people asking for more speed say where they want it? signature.asc Description: OpenPGP digital signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Wed, Oct 21, 2015 at 1:24 AM, Timothy Guwrote: > On Tue, Oct 20, 2015 at 7:09 PM Ganesh Ajjanagadde wrote: > >> Hi all, >> >> It is known that there exist at least certain parts of the codebase >> that do not work correctly if ints are 64 bits. One of them I noticed >> was in avutil/intmath.h: ff_ctz_c does not compute the right thing if >> int is 64 bits. This is true both before and after the De-Bruijn >> optimization. >> >> A more interesting (and serious) question is whether FATE is broken if >> int's are 64 bits. I did some digging, and found the following from >> "The Definitive Guide to GCC" - On GCC prior to 4.x, there was a flag >> -mint64, documented as follows (see e.g >> https://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc.pdf): >> "Force int and long types to be 64 bits wide. See -mlong32 for an >> explanation of the default and the way that the pointer size is >> determined". >> >> This should be helpful in setting up a FATE client to test (and >> possibly fix) bad code that assumed int = 32 bits. I myself can't >> easily run such an outdated GCC, but I noticed a bunch of clients >> running GCC < 4.0 where this may be set up. >> > > For lack of milder wording, this is a terrible idea. > > FATE is set up, not to test every single possible combination of compiler, > compiler flags, and operating system, but to make sure FFmpeg works on the > ones people actually use. There are already not many people using GCC 3, > and the addition of a flag that completely changes the API is nowhere near > realistic. Int being 64 bits is not as unrealistic as one can expect. Sure, right now we have no such platform. However, computer architecture people definitely do talk seriously about moving to 128 bit (or wider) architectures. On the GCC/Glibc mailing lists, people do discuss these things every now and then. Furthermore, how does this change the API? On an int = 64 bits platform (with all libraries built accordingly), people would still call/pass in int's when they are needed, right? > > Sometimes it's necessary to draw a line between "what's written in ISO/IEC > 9899" and "real world." I admit it is "theoretical" at the moment, but Michael did express some interest in testing this idea, suggesting some level of consideration in FFmpeg. In the "real world", a lot of "undefined behavior" has well defined, consistent semantics on all platforms we care about. For instance, negative left shifts are handled by both GCC and clang in the same way. Does that mean we should just ignore it? Not by any means. FFmpeg cares about portability and robustness, with good reason. In other words: I recall Nicolas mentioning that FFmpeg could do with using width specific types more liberally. If I see a patch which causes additional potential trouble if/when int's become 64 bits, I will raise the point. > > Plus Mark's point on technical possibility of it (thank GOD GCC didn't do > something as crazy as this on other platforms). Indeed, it is very fortunate that GCC does not do this on other platforms. That does not change whether we should test it or not. > > Timothy > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
Ganesh Ajjanagadde mit.edu> writes: > Since you seem to be an "expert" on what things > affect this decade, why don't you spend 5 minutes > trying to outline to beginners like me what is > "actually important" in your view? Important is whatever you want to work on. That can be DTS-XLL or any random ticket or a new format or a new decoder or a new filter. But if the new format or the new decoder is an invention of you it is less likely to find fans (but see a64). (And consider to remember that if people do not comment it does not always mean they agree, so don't shoot those who do comment even if you don't like it.) Carl Eugen ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Wed, Oct 21, 2015 at 6:18 AM, wm4wrote: > On Wed, 21 Oct 2015 06:00:21 -0400 > Ganesh Ajjanagadde wrote: > >> I don't "expect support" from anyone. Frankly, I have dealt with >> enough mud from you, who seems to take great pleasure in criticizing >> or otherwise derailing every patch/discussion of mine to not care >> about things either way - if you don't want me, just say the word, and >> I will be gone. And your own stance on this point is not even clear - >> GCC 20 is not going to hit the shelves this decade. > > Nobody is throwing mud at you. But your patches are mostly > "theoretical" changes, often without much proof that they are really > necessary, or cosmetic changes. While many of these changes are quite > ok and actually improve the code base a bit, they also generate a LOT > of traffic and discussion. This can be exhausting, especially if you > have to care about other things. So yes, it's all kind of noisy and > annoying. Sorry. No problem - thanks for speaking up. > > This doesn't mean you should go away, or that we don't want you, or > that you should stop doing what you do. On the opposite, I think you're > a promising new developer, and you should definitely stay. > > Maybe you should feel encouraged to get on the next level. For example, > you could enter the fuzzing-security-bugs business, or the > rewrite-inline-asm-as-yasm business. Or pick anything that looks > interesting to you; you already have enough experience with this project > that you will find your way. You will learn new useful things, and the > result will be useful for everyone. It should also be more fun than > fixing theoretical standards issues. Got it. > >> Since you seem to be an "expert" on what things affect this decade, >> why don't you spend 5 minutes trying to outline to beginners like me >> what is "actually important" in your view? > > Important is fixing bugs that actually could affect people right now. > I do agree that ffmpeg should follow standards, and that code assuming > sizeof(int)==4 is buggy and should be fixed. But is it important enough > that we should go and turn around every single line of code? Or, > alternatively, depend on an ancient compiler, and work through > potentially hundreds of FATE failures? And only because someone _could_ > decide that new architectures will use 64 bit ints? I don't think so. Ok, I will send patches if I see issues while working on some other thing, but not do a mass cleanup. Thanks. > > The likeliness of this happening is very low anyway. Just look at > Windows: they decided that even the "long" type should remain 32 bit, > because making it 64 bit (like on Unix) would break way too much code. > The most likely scenario is that a 128 bit architecture will have the > same types and sizes as 64 bit systems, just with size_t/ptrdiff_t > being 128 bit, and a new "long long long int" type (I swear, they will > do it, even if that type name looks horrible). > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Wed, Oct 21, 2015 at 5:31 AM, Hendrik Leppkeswrote: > On Wed, Oct 21, 2015 at 11:10 AM, Ganesh Ajjanagadde wrote: >> On Wed, Oct 21, 2015 at 1:24 AM, Timothy Gu wrote: >>> On Tue, Oct 20, 2015 at 7:09 PM Ganesh Ajjanagadde wrote: >>> Hi all, It is known that there exist at least certain parts of the codebase that do not work correctly if ints are 64 bits. One of them I noticed was in avutil/intmath.h: ff_ctz_c does not compute the right thing if int is 64 bits. This is true both before and after the De-Bruijn optimization. A more interesting (and serious) question is whether FATE is broken if int's are 64 bits. I did some digging, and found the following from "The Definitive Guide to GCC" - On GCC prior to 4.x, there was a flag -mint64, documented as follows (see e.g https://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc.pdf): "Force int and long types to be 64 bits wide. See -mlong32 for an explanation of the default and the way that the pointer size is determined". This should be helpful in setting up a FATE client to test (and possibly fix) bad code that assumed int = 32 bits. I myself can't easily run such an outdated GCC, but I noticed a bunch of clients running GCC < 4.0 where this may be set up. >>> >>> For lack of milder wording, this is a terrible idea. >>> >>> FATE is set up, not to test every single possible combination of compiler, >>> compiler flags, and operating system, but to make sure FFmpeg works on the >>> ones people actually use. There are already not many people using GCC 3, >>> and the addition of a flag that completely changes the API is nowhere near >>> realistic. >> >> Int being 64 bits is not as unrealistic as one can expect. Sure, right >> now we have no such platform. However, computer architecture people >> definitely do talk seriously about moving to 128 bit (or wider) >> architectures. On the GCC/Glibc mailing lists, people do discuss these >> things every now and then. Furthermore, how does this change the API? >> On an int = 64 bits platform (with all libraries built accordingly), >> people would still call/pass in int's when they are needed, right? >> >>> >>> Sometimes it's necessary to draw a line between "what's written in ISO/IEC >>> 9899" and "real world." >> >> I admit it is "theoretical" at the moment, but Michael did express >> some interest in testing this idea, suggesting some level of >> consideration in FFmpeg. In the "real world", a lot of "undefined >> behavior" has well defined, consistent semantics on all platforms we >> care about. For instance, negative left shifts are handled by both GCC >> and clang in the same way. Does that mean we should just ignore it? >> Not by any means. FFmpeg cares about portability and robustness, with >> good reason. >> > > I think an important point to remember is that we have plenty of areas > that could use improvement right now, on platforms relevant today. > We don't need to keep ourselves busy worrying about some theoretical > future system that may or may not exist in a decade or so. > > If you want to waste your time on that, noone is going to stop you, > just don't expect much support from other developers, whose time may > be much better spent on things affecting this decade. I don't "expect support" from anyone. Frankly, I have dealt with enough mud from you, who seems to take great pleasure in criticizing or otherwise derailing every patch/discussion of mine to not care about things either way - if you don't want me, just say the word, and I will be gone. And your own stance on this point is not even clear - GCC 20 is not going to hit the shelves this decade. Since you seem to be an "expert" on what things affect this decade, why don't you spend 5 minutes trying to outline to beginners like me what is "actually important" in your view? > > - Hendrik > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Wed, 21 Oct 2015 06:00:21 -0400 Ganesh Ajjanagaddewrote: > I don't "expect support" from anyone. Frankly, I have dealt with > enough mud from you, who seems to take great pleasure in criticizing > or otherwise derailing every patch/discussion of mine to not care > about things either way - if you don't want me, just say the word, and > I will be gone. And your own stance on this point is not even clear - > GCC 20 is not going to hit the shelves this decade. Nobody is throwing mud at you. But your patches are mostly "theoretical" changes, often without much proof that they are really necessary, or cosmetic changes. While many of these changes are quite ok and actually improve the code base a bit, they also generate a LOT of traffic and discussion. This can be exhausting, especially if you have to care about other things. So yes, it's all kind of noisy and annoying. Sorry. This doesn't mean you should go away, or that we don't want you, or that you should stop doing what you do. On the opposite, I think you're a promising new developer, and you should definitely stay. Maybe you should feel encouraged to get on the next level. For example, you could enter the fuzzing-security-bugs business, or the rewrite-inline-asm-as-yasm business. Or pick anything that looks interesting to you; you already have enough experience with this project that you will find your way. You will learn new useful things, and the result will be useful for everyone. It should also be more fun than fixing theoretical standards issues. > Since you seem to be an "expert" on what things affect this decade, > why don't you spend 5 minutes trying to outline to beginners like me > what is "actually important" in your view? Important is fixing bugs that actually could affect people right now. I do agree that ffmpeg should follow standards, and that code assuming sizeof(int)==4 is buggy and should be fixed. But is it important enough that we should go and turn around every single line of code? Or, alternatively, depend on an ancient compiler, and work through potentially hundreds of FATE failures? And only because someone _could_ decide that new architectures will use 64 bit ints? I don't think so. The likeliness of this happening is very low anyway. Just look at Windows: they decided that even the "long" type should remain 32 bit, because making it 64 bit (like on Unix) would break way too much code. The most likely scenario is that a 128 bit architecture will have the same types and sizes as 64 bit systems, just with size_t/ptrdiff_t being 128 bit, and a new "long long long int" type (I swear, they will do it, even if that type name looks horrible). ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Wed, Oct 21, 2015 at 11:10 AM, Ganesh Ajjanagaddewrote: > On Wed, Oct 21, 2015 at 1:24 AM, Timothy Gu wrote: >> On Tue, Oct 20, 2015 at 7:09 PM Ganesh Ajjanagadde wrote: >> >>> Hi all, >>> >>> It is known that there exist at least certain parts of the codebase >>> that do not work correctly if ints are 64 bits. One of them I noticed >>> was in avutil/intmath.h: ff_ctz_c does not compute the right thing if >>> int is 64 bits. This is true both before and after the De-Bruijn >>> optimization. >>> >>> A more interesting (and serious) question is whether FATE is broken if >>> int's are 64 bits. I did some digging, and found the following from >>> "The Definitive Guide to GCC" - On GCC prior to 4.x, there was a flag >>> -mint64, documented as follows (see e.g >>> https://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc.pdf): >>> "Force int and long types to be 64 bits wide. See -mlong32 for an >>> explanation of the default and the way that the pointer size is >>> determined". >>> >>> This should be helpful in setting up a FATE client to test (and >>> possibly fix) bad code that assumed int = 32 bits. I myself can't >>> easily run such an outdated GCC, but I noticed a bunch of clients >>> running GCC < 4.0 where this may be set up. >>> >> >> For lack of milder wording, this is a terrible idea. >> >> FATE is set up, not to test every single possible combination of compiler, >> compiler flags, and operating system, but to make sure FFmpeg works on the >> ones people actually use. There are already not many people using GCC 3, >> and the addition of a flag that completely changes the API is nowhere near >> realistic. > > Int being 64 bits is not as unrealistic as one can expect. Sure, right > now we have no such platform. However, computer architecture people > definitely do talk seriously about moving to 128 bit (or wider) > architectures. On the GCC/Glibc mailing lists, people do discuss these > things every now and then. Furthermore, how does this change the API? > On an int = 64 bits platform (with all libraries built accordingly), > people would still call/pass in int's when they are needed, right? > >> >> Sometimes it's necessary to draw a line between "what's written in ISO/IEC >> 9899" and "real world." > > I admit it is "theoretical" at the moment, but Michael did express > some interest in testing this idea, suggesting some level of > consideration in FFmpeg. In the "real world", a lot of "undefined > behavior" has well defined, consistent semantics on all platforms we > care about. For instance, negative left shifts are handled by both GCC > and clang in the same way. Does that mean we should just ignore it? > Not by any means. FFmpeg cares about portability and robustness, with > good reason. > I think an important point to remember is that we have plenty of areas that could use improvement right now, on platforms relevant today. We don't need to keep ourselves busy worrying about some theoretical future system that may or may not exist in a decade or so. If you want to waste your time on that, noone is going to stop you, just don't expect much support from other developers, whose time may be much better spent on things affecting this decade. - Hendrik ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Wed, Oct 21, 2015 at 6:10 AM, Carl Eugen Hoyoswrote: > Ganesh Ajjanagadde mit.edu> writes: > >> Since you seem to be an "expert" on what things >> affect this decade, why don't you spend 5 minutes >> trying to outline to beginners like me what is >> "actually important" in your view? > > Important is whatever you want to work on. > That can be DTS-XLL or any random ticket or a > new format or a new decoder or a new filter. Then by definition the int = 64 bits is something that has some importance. The fact that Michael asked about it, and I am interested to some (minor) degree confirms this according to your definition. > > But if the new format or the new decoder is an > invention of you it is less likely to find fans > (but see a64). > > (And consider to remember that if people do not > comment it does not always mean they agree, so > don't shoot those who do comment even if you > don't like it.) I have not "shot" anybody, just paid back in the same manner of tone. I will refrain from it in future, thanks. > > Carl Eugen > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Wed, Oct 21, 2015 at 06:00:21AM -0400, Ganesh Ajjanagadde wrote: [...] > why don't you spend 5 minutes trying to outline to beginners like me > what is "actually important" in your view? > According to the first 100 answers of the survey, the majority of the users want... speed optimisation above anything. I can't give more info for now, I'll need to pay to get the stats taking into account all votes. -- Clément B. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Wed, Oct 21, 2015 at 1:14 AM, Mark Harris wrote: > On Tue, Oct 20, 2015 at 7:08 PM, Ganesh Ajjanagaddewrote: >> Hi all, >> >> It is known that there exist at least certain parts of the codebase >> that do not work correctly if ints are 64 bits. One of them I noticed >> was in avutil/intmath.h: ff_ctz_c does not compute the right thing if >> int is 64 bits. This is true both before and after the De-Bruijn >> optimization. >> >> A more interesting (and serious) question is whether FATE is broken if >> int's are 64 bits. I did some digging, and found the following from >> "The Definitive Guide to GCC" - On GCC prior to 4.x, there was a flag >> -mint64, documented as follows (see e.g >> https://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc.pdf): >> "Force int and long types to be 64 bits wide. See -mlong32 for an >> explanation of the default and the way that the pointer size is >> determined". >> >> This should be helpful in setting up a FATE client to test (and >> possibly fix) bad code that assumed int = 32 bits. I myself can't >> easily run such an outdated GCC, but I noticed a bunch of clients >> running GCC < 4.0 where this may be set up. > > This option was only valid on MIPS targets, not on x86_64 or other > common platforms. It produced code that was incompatible with the > ABI, which made it not very useful if you want to link with system > libraries. It was therefore removed. > > Although it would be possible to modify gcc to add a new x86_64 target > where int is 64 bits, you would probably have the same ABI issues > because libc and other system libraries would be assuming a 32-bit > int. Well, can't you make the whole platform use 64 bit ints, e.g building libc and other system libraries from source to use 64 bit ints? Please note that this is just to get a more complete understanding, not necessarily that this should be done. > >> >> Regards, >> Ganesh > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Tue, Oct 20, 2015 at 7:08 PM, Ganesh Ajjanagaddewrote: > Hi all, > > It is known that there exist at least certain parts of the codebase > that do not work correctly if ints are 64 bits. One of them I noticed > was in avutil/intmath.h: ff_ctz_c does not compute the right thing if > int is 64 bits. This is true both before and after the De-Bruijn > optimization. > > A more interesting (and serious) question is whether FATE is broken if > int's are 64 bits. I did some digging, and found the following from > "The Definitive Guide to GCC" - On GCC prior to 4.x, there was a flag > -mint64, documented as follows (see e.g > https://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc.pdf): > "Force int and long types to be 64 bits wide. See -mlong32 for an > explanation of the default and the way that the pointer size is > determined". > > This should be helpful in setting up a FATE client to test (and > possibly fix) bad code that assumed int = 32 bits. I myself can't > easily run such an outdated GCC, but I noticed a bunch of clients > running GCC < 4.0 where this may be set up. This option was only valid on MIPS targets, not on x86_64 or other common platforms. It produced code that was incompatible with the ABI, which made it not very useful if you want to link with system libraries. It was therefore removed. Although it would be possible to modify gcc to add a new x86_64 target where int is 64 bits, you would probably have the same ABI issues because libc and other system libraries would be assuming a 32-bit int. > > Regards, > Ganesh ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] forcing ints to be 64 bits, possible new FATE client idea
On Tue, Oct 20, 2015 at 7:09 PM Ganesh Ajjanagaddewrote: > Hi all, > > It is known that there exist at least certain parts of the codebase > that do not work correctly if ints are 64 bits. One of them I noticed > was in avutil/intmath.h: ff_ctz_c does not compute the right thing if > int is 64 bits. This is true both before and after the De-Bruijn > optimization. > > A more interesting (and serious) question is whether FATE is broken if > int's are 64 bits. I did some digging, and found the following from > "The Definitive Guide to GCC" - On GCC prior to 4.x, there was a flag > -mint64, documented as follows (see e.g > https://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc.pdf): > "Force int and long types to be 64 bits wide. See -mlong32 for an > explanation of the default and the way that the pointer size is > determined". > > This should be helpful in setting up a FATE client to test (and > possibly fix) bad code that assumed int = 32 bits. I myself can't > easily run such an outdated GCC, but I noticed a bunch of clients > running GCC < 4.0 where this may be set up. > For lack of milder wording, this is a terrible idea. FATE is set up, not to test every single possible combination of compiler, compiler flags, and operating system, but to make sure FFmpeg works on the ones people actually use. There are already not many people using GCC 3, and the addition of a flag that completely changes the API is nowhere near realistic. Sometimes it's necessary to draw a line between "what's written in ISO/IEC 9899" and "real world." Plus Mark's point on technical possibility of it (thank GOD GCC didn't do something as crazy as this on other platforms). Timothy ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel