Re: Arithmetic coding in Fedora libjpeg (bug #639531)
On Sun, Oct 03, 2010 at 01:52:28PM +0530, Mukund Sivaraman wrote: > Hi all > > In reply to Gregory Maxwell: > >> It's well known around the Internet that to achieve compatibility you > >> should be conservative in what you send and liberal in what you > >> accept. Applied to JPEG: Use only Huffman coding when encoding ? > >> except maybe if you know that all recipients can handle arithmetic > >> coding ? but support both encodings when decoding. > > >Absolutely. This is an excellent argument and I think is sufficient to > >justify the inclusion alone. > > Thank you everyone for the replies. I did not know earlier that Fedora > was switching to libjpeg-turbo. I have created a new bug now: > > https://bugzilla.redhat.com/show_bug.cgi?id=639672 > Add support for decoding arithmetic coded files in libjpeg-turbo > > It contains a patch against upstream libjpeg-turbo HEAD, and the patch > has been submitted upstream too: > > https://sourceforge.net/tracker/?func=detail&aid=3080268&group_id=303195&atid=1278160 > > This patch has been tested against some arithmetic coded images. > > I wish libjpeg-turbo also accepts to include creating arithmetic coded > images too, as the project is not really going to affect creation of > such images if an application wants to do so. But this can be saved > for another argument, and doesn't stand in the way of decode support. We decided to accept arithmetic method into libjpeg-turbo upstream as soon as Fedora legal says it's fine. Regards, Adam -- Adam Tkac, Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
Björn Persson wrote: > Tom Lane wrote: >> I believe that they'd be best advised to say "no", because at this point >> one of JPEG's principal attractions is near-universal compatibility. >> Throwing A/C into the mix will throw that away, for what really is a >> very marginal gain in compression efficiency. > > It's well known around the Internet that to achieve compatibility you should > be conservative in what you send and liberal in what you accept. Applied to > JPEG: Use only Huffman coding when encoding – except maybe if you know that > all > recipients can handle arithmetic coding – but support both encodings when > decoding. > Well said. Adding decoding support can do no harm. -- Roberto Ragusamail at robertoragusa.it -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
Hi all In reply to Gregory Maxwell: >> It's well known around the Internet that to achieve compatibility you >> should be conservative in what you send and liberal in what you >> accept. Applied to JPEG: Use only Huffman coding when encoding ? >> except maybe if you know that all recipients can handle arithmetic >> coding ? but support both encodings when decoding. >Absolutely. This is an excellent argument and I think is sufficient to >justify the inclusion alone. Thank you everyone for the replies. I did not know earlier that Fedora was switching to libjpeg-turbo. I have created a new bug now: https://bugzilla.redhat.com/show_bug.cgi?id=639672 Add support for decoding arithmetic coded files in libjpeg-turbo It contains a patch against upstream libjpeg-turbo HEAD, and the patch has been submitted upstream too: https://sourceforge.net/tracker/?func=detail&aid=3080268&group_id=303195&atid=1278160 This patch has been tested against some arithmetic coded images. I wish libjpeg-turbo also accepts to include creating arithmetic coded images too, as the project is not really going to affect creation of such images if an application wants to do so. But this can be saved for another argument, and doesn't stand in the way of decode support. Mukund -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
Björn Persson wrote: > It's well known around the Internet that to achieve compatibility you > should be conservative in what you send and liberal in what you accept. > Applied to JPEG: Use only Huffman coding when encoding – except maybe if > you know that all recipients can handle arithmetic coding – but support > both encodings when decoding. +1 I can see not adding the support to the ENCODING portion, also because that appears to imply an ABI change (extra flag), but the DECODING portion of the patch should be merged. The patch against the original libjpeg libjpeg-turbo is derived from should probably work, it might not be "turbo"-optimized, but given how rare those files are, that shouldn't be a problem. It's still better to be able to decode the files slowly than not at all. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
On Sat, Oct 2, 2010 at 1:34 PM, Paul F. Johnson wrote: > Hi, > >> "You shall not create images with arithmetic coding" is like saying "You >> shall not create images of the flying sphagetti monster." It's not up to >> Fedora to make this choice for me. > > It is though - you have chosen to use Fedora therefore have to live with > the decisions the Fedora legal people (I'm assuming they said no to > arithmetic coding) have said goes. The relevant patent expired last year. I believe the SUN OMS team had researched this extensively as they were using the JPEG arithmetic coder in their aggressively researched royalty free video codec design. If someone doing legal research on this needs more information, you can contact me offlist and I can connect you with people who have researched this topic and may be willing to provide some useful pointers. 2010/10/2 Björn Persson : > It's well known around the Internet that to achieve compatibility you should > be conservative in what you send and liberal in what you accept. Applied to > JPEG: Use only Huffman coding when encoding – except maybe if you know that > all > recipients can handle arithmetic coding – but support both encodings when > decoding. Absolutely. This is an excellent argument and I think is sufficient to justify the inclusion alone. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
Tom Lane wrote: > I believe that they'd be best advised to say "no", because at this point > one of JPEG's principal attractions is near-universal compatibility. > Throwing A/C into the mix will throw that away, for what really is a > very marginal gain in compression efficiency. It's well known around the Internet that to achieve compatibility you should be conservative in what you send and liberal in what you accept. Applied to JPEG: Use only Huffman coding when encoding – except maybe if you know that all recipients can handle arithmetic coding – but support both encodings when decoding. Björn Persson signature.asc Description: This is a digitally signed message part. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
Kevin Fenzi writes: > On Sat, 2 Oct 2010 22:53:53 +0530 > Mukund Sivaraman wrote: >> Arithmetic coding in Fedora libjpeg (bug #639531) > My thoughts: > - We should not be making this change in stable releases even if > otherwise like the idea. It could well cause issues and problems with > a fragile library used by a lot of things. Don't worry, there's exactly 0 chance of that. > - In f14+ we are not using libjpeg. We are using libjpeg-turbo. See: > http://fedoraproject.org/wiki/Features/libjpeg-turbo Yes. So actually the OP is pestering the wrong person; he should be trying to convice the libjpeg-turbo maintainers to expend work on this. I believe that they'd be best advised to say "no", because at this point one of JPEG's principal attractions is near-universal compatibility. Throwing A/C into the mix will throw that away, for what really is a very marginal gain in compression efficiency. If you want a new not-compatible-with-anything image format, you may as well adopt JPEG 2000, or some other format that's less than 20 years old. The A/C option to original JPEG is something that missed its chance because of patents. Resurrecting it from the dead now is just an exercise in misplaced priorities. But having said that, it's not my decision to make; it's the libjpeg-turbo authors' decision whether to expend effort in that direction. regards, tom lane -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
Peter Lemenkov wrote: > Just for the record, AFAIK there are only two countries which allows > software patents - USA and South Korea. That's not the whole story. The situation is quite weird in countries that have signed the European Patent Convention. The convention and the national laws all state clearly that computer programs are not patentable, and yet software patents are granted on a daily basis, justified with some irrational reasoning that makes sense only to mentally deranged patent lawyers and some gullible politicians. Tens of thousands of these illegal patents have been granted. Several years ago the patent lobby attempted to push through an EU directive to legalize software patents. The directive was eventally dropped after years of massive campaigning for and against, but that only means that the granting of illegal patents continues as before. Such a patent might not hold up in court if it were contested by a good lawyer and the judge had some basic understanding of how computers work, but that doesn't make the patents harmless. Due to the high cost and the uncertain outcome of a patent lawsuit, small companies often have no choice but to pay the license fees. Björn Persson signature.asc Description: This is a digitally signed message part. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
On Sat, 2 Oct 2010 22:53:53 +0530 Mukund Sivaraman wrote: > Arithmetic coding in Fedora libjpeg (bug #639531) ...snip long thing... My thoughts: - We should not be making this change in stable releases even if otherwise like the idea. It could well cause issues and problems with a fragile library used by a lot of things. - In f14+ we are not using libjpeg. We are using libjpeg-turbo. See: http://fedoraproject.org/wiki/Features/libjpeg-turbo So, any request to enable this support should be filed there and looked at from it's perspective. Do they have support? Does it cause any issues? Does upstream enable it? - For the legal side, make your (new libjpeg-turbo) bug block FE_LEGAL so it can be looked at. kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
Hi, > > Remember, Fedora is not just for use in your country - there are piles > > of countries it's used in, each with their own insane patent > > regulations. The fedora stance is that if it offends, it's out. > > Just for the record, AFAIK there are only two countries which allows > software patents - USA and South Korea. Yep - and it's the US which has some of the more insane ones (and therefore has to be the most careful over). I wonder how long it will be before Apple's current insane one over backing up to a local repository will be allowed to stand - effectively it means you can't burn to a CD! Paul -- Vertraue mir, ich weiss, was ich mache... -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
2010/10/2 Paul F. Johnson : > Remember, Fedora is not just for use in your country - there are piles > of countries it's used in, each with their own insane patent > regulations. The fedora stance is that if it offends, it's out. Just for the record, AFAIK there are only two countries which allows software patents - USA and South Korea. -- With best regards, Peter Lemenkov. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Arithmetic coding in Fedora libjpeg (bug #639531)
Hi, > "You shall not create images with arithmetic coding" is like saying "You > shall not create images of the flying sphagetti monster." It's not up to > Fedora to make this choice for me. It is though - you have chosen to use Fedora therefore have to live with the decisions the Fedora legal people (I'm assuming they said no to arithmetic coding) have said goes. Remember, Fedora is not just for use in your country - there are piles of countries it's used in, each with their own insane patent regulations. The fedora stance is that if it offends, it's out. If you want a prime example, look at either DeCSS or mp3 support. Plenty of distros have it, but they're both out of Fedora. Doesn't matter that in the authenticity of the mp3 patent is still in doubt, Fedora plays it safe. rpmfusion may supply DeCSS and mp3 support, but they're not part of Fedora. This may be one avenue to look at - see if rpmfusion can supply an add-on package for this. Paul -- Vertraue mir, ich weiss, was ich mache... -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Arithmetic coding in Fedora libjpeg (bug #639531)
Arithmetic coding in Fedora libjpeg (bug #639531) I want libjpeg packages in future Fedora distributions to support reading arithmetic coded JPEG files. Huffman coding and arithmetic are two different bit coding methods that are available in the JPEG standard. Huffman is simpler to implement than arithmetic coding and historically hasn't suffered from software patents. Arithmetic coding (as used in JPEG) had been covered by software patents, but these patents have now apparently expired. [Fedora legal could check this.] The vast majority of JPEG files so far used Huffman coding because of the software patents issue. libjpeg version 7 (one of the forks of the old IJG software) and newer enable support for arithmetic coding by default. With older versions of libjpeg (such as the ones bundled by Fedora up to release 13), support can be added in the form of a patch. No application code modifications are required. (I'm not sure if it's ABI compatible though, but on a re-reading of the libjpeg headers, it might well be.) Arithmetic coding is in every case a superior bit coding method than Huffman, because arithmetic coding doesn't use an integral number of bits to represent codes like Huffman does. So JPEG files created with arithmetic coding are always smaller than those created with Huffman coding. Software such as jpegtran (as distributed with libjpeg 7 and above) can losslessly convert Huffman coded images to arithmetic coded and vice versa. The lossy behavior of JPEG does not happen at this (bit coding) layer of the format. The libjpeg package maintainer seems to be satisfied in closing the bug rather than providing a good rationale for doing so. But I will be the devil's advocate myself. You see, .jpg files are assumed to be an ubiquitous file format. They _just work_ everywhere. If you have a .jpg file, it is understood you aren't going to have problems when using it. But arithmetic coded files do not open everywhere. In fact, they rarely work in most applications, devices, etc. today, because pretty much everyone is using an old libjpeg with no support for it due to the historical software patents issue. As an example, in the context of GIMP creating such files, there were some comments that people would start losing faith in JPEG if they see incompatibilites, that such files are crippled JPEGs [sic], that people who use devices such as digital photo frames will suffer, that I should instead blog about free software and the evils of software patents, that this option should never be available in a dialog to the end-user, etc. * Fedora is not the only software that creates JPEG files. The nearest first implementation of a .jpg file creator program is cjpeg which is a part of libjpeg (the fork at ijg.org), and even that allows creating arithmetic coded files now. It would be arrogance to think that just because Fedora creates these files, the world of JPEG is going to suffer. Or that if Fedora doesn't do it, nobody else would. * The situation with arithmetic coding in the context of incompatibility with existing embedded decoders is just like H.264 and profiles. The iPhone for example does not play all kinds of H.264 files (it supports only the baseline profile). But MPEG content creation software does allow you to create files that use baseline as well as other profiles. It doesn't say "Hey, you the user are pretty stupid, so I'm not just turning this option off, but I'm not making it available to you." It would be arrogance to not provide an option to the end-user, just because I think the user may not know what he/she is doing. * Due to the patents issue, we had to assume that Huffman was the only bit coding method of this standard, whereas it isn't so. Arithmetic coding has been in the standard all these umpteen years, and some people used it. There was always the odd JPEG content that used arithmetic coding and which still complied with the standards. These are not broken JPEGs. Also quoting from the bug: > In any case, there are practical and political reasons not to encourage the > spread of the arithmetic-coding variant of JPEG. It's not compatible with > most > implementations and it doesn't offer enough benefit to be worth creating an > enormous compatibility problem. So even if the code existed I would resist > turning it on. I think the author of libjpeg 8 has made a very serious error > by adding support for it. I have replied on the bug about it. Some of the text in my initial bug description is incorrect, and I have corrected it in this email. It is not upto Fedora to make political decisions when there is no good reason for it. This goes against the values of free software. "You shall not create images with arithmetic coding" is like saying "You shall not create images of the flying sphagetti monster." It's not up