Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-04 Thread Adam Tkac
On Sun, Oct 03, 2010 at 01:52:28PM +0530, Mukund Sivaraman wrote:
 Hi all
 
 In reply to Gregory Maxwell:
  It's well known around the Internet that to achieve compatibility you
  should be conservative in what you send and liberal in what you
  accept. Applied to JPEG: Use only Huffman coding when encoding ?
  except maybe if you know that all recipients can handle arithmetic
  coding ? but support both encodings when decoding.
 
 Absolutely. This is an excellent argument and I think is sufficient to
 justify the inclusion alone.
 
 Thank you everyone for the replies. I did not know earlier that Fedora
 was switching to libjpeg-turbo.  I have created a new bug now:
 
 https://bugzilla.redhat.com/show_bug.cgi?id=639672
 Add support for decoding arithmetic coded files in libjpeg-turbo
 
 It contains a patch against upstream libjpeg-turbo HEAD, and the patch
 has been submitted upstream too:
 
 https://sourceforge.net/tracker/?func=detailaid=3080268group_id=303195atid=1278160
 
 This patch has been tested against some arithmetic coded images.
 
 I wish libjpeg-turbo also accepts to include creating arithmetic coded
 images too, as the project is not really going to affect creation of
 such images if an application wants to do so.  But this can be saved
 for another argument, and doesn't stand in the way of decode support.

We decided to accept arithmetic method into libjpeg-turbo upstream as
soon as Fedora legal says it's fine.

Regards, Adam

-- 
Adam Tkac, Red Hat, Inc.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-03 Thread Mukund Sivaraman
Hi all

In reply to Gregory Maxwell:
 It's well known around the Internet that to achieve compatibility you
 should be conservative in what you send and liberal in what you
 accept. Applied to JPEG: Use only Huffman coding when encoding ?
 except maybe if you know that all recipients can handle arithmetic
 coding ? but support both encodings when decoding.

Absolutely. This is an excellent argument and I think is sufficient to
justify the inclusion alone.

Thank you everyone for the replies. I did not know earlier that Fedora
was switching to libjpeg-turbo.  I have created a new bug now:

https://bugzilla.redhat.com/show_bug.cgi?id=639672
Add support for decoding arithmetic coded files in libjpeg-turbo

It contains a patch against upstream libjpeg-turbo HEAD, and the patch
has been submitted upstream too:

https://sourceforge.net/tracker/?func=detailaid=3080268group_id=303195atid=1278160

This patch has been tested against some arithmetic coded images.

I wish libjpeg-turbo also accepts to include creating arithmetic coded
images too, as the project is not really going to affect creation of
such images if an application wants to do so.  But this can be saved
for another argument, and doesn't stand in the way of decode support.

Mukund
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-03 Thread Roberto Ragusa
Björn Persson wrote:
 Tom Lane wrote:
 I believe that they'd be best advised to say no, because at this point
 one of JPEG's principal attractions is near-universal compatibility.
 Throwing A/C into the mix will throw that away, for what really is a
 very marginal gain in compression efficiency.
 
 It's well known around the Internet that to achieve compatibility you should 
 be conservative in what you send and liberal in what you accept. Applied to 
 JPEG: Use only Huffman coding when encoding – except maybe if you know that 
 all 
 recipients can handle arithmetic coding – but support both encodings when 
 decoding.
 

Well said.
Adding decoding support can do no harm.

-- 
   Roberto Ragusamail at robertoragusa.it
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Mukund Sivaraman
Arithmetic coding in Fedora libjpeg (bug #639531)

I want libjpeg packages in future Fedora distributions to support
reading arithmetic coded JPEG files.

Huffman coding and arithmetic are two different bit coding methods that
are available in the JPEG standard. Huffman is simpler to implement than
arithmetic coding and historically hasn't suffered from software
patents. Arithmetic coding (as used in JPEG) had been covered by
software patents, but these patents have now apparently expired. [Fedora
legal could check this.]

The vast majority of JPEG files so far used Huffman coding because of
the software patents issue. libjpeg version 7 (one of the forks of the
old IJG software) and newer enable support for arithmetic coding by
default. With older versions of libjpeg (such as the ones bundled by
Fedora up to release 13), support can be added in the form of a
patch. No application code modifications are required.  (I'm not sure if
it's ABI compatible though, but on a re-reading of the libjpeg headers,
it might well be.)

Arithmetic coding is in every case a superior bit coding method than
Huffman, because arithmetic coding doesn't use an integral number of
bits to represent codes like Huffman does. So JPEG files created with
arithmetic coding are always smaller than those created with Huffman
coding.

Software such as jpegtran (as distributed with libjpeg 7 and above) can
losslessly convert Huffman coded images to arithmetic coded and vice
versa. The lossy behavior of JPEG does not happen at this (bit coding)
layer of the format.

The libjpeg package maintainer seems to be satisfied in closing the bug
rather than providing a good rationale for doing so. But I will be the
devil's advocate myself.

You see, .jpg files are assumed to be an ubiquitous file format. They
_just work_ everywhere. If you have a .jpg file, it is understood you
aren't going to have problems when using it. But arithmetic coded files
do not open everywhere. In fact, they rarely work in most applications,
devices, etc. today, because pretty much everyone is using an old
libjpeg with no support for it due to the historical software patents
issue.

As an example, in the context of GIMP creating such files, there were
some comments that people would start losing faith in JPEG if they see
incompatibilites, that such files are crippled JPEGs [sic], that people
who use devices such as digital photo frames will suffer, that I should
instead blog about free software and the evils of software patents, that
this option should never be available in a dialog to the end-user, etc.

* Fedora is not the only software that creates JPEG files. The nearest
  first implementation of a .jpg file creator program is cjpeg which is
  a part of libjpeg (the fork at ijg.org), and even that allows creating
  arithmetic coded files now. It would be arrogance to think that just
  because Fedora creates these files, the world of JPEG is going to
  suffer. Or that if Fedora doesn't do it, nobody else would.

* The situation with arithmetic coding in the context of incompatibility
  with existing embedded decoders is just like H.264 and profiles. The
  iPhone for example does not play all kinds of H.264 files (it supports
  only the baseline profile). But MPEG content creation software does
  allow you to create files that use baseline as well as other
  profiles. It doesn't say Hey, you the user are pretty stupid, so I'm
  not just turning this option off, but I'm not making it available to
  you. It would be arrogance to not provide an option to the end-user,
  just because I think the user may not know what he/she is doing.

* Due to the patents issue, we had to assume that Huffman was the only
  bit coding method of this standard, whereas it isn't so. Arithmetic
  coding has been in the standard all these umpteen years, and some
  people used it. There was always the odd JPEG content that used
  arithmetic coding and which still complied with the standards. These
  are not broken JPEGs.

Also quoting from the bug:
 In any case, there are practical and political reasons not to encourage the
 spread of the arithmetic-coding variant of JPEG.  It's not compatible with 
 most
 implementations and it doesn't offer enough benefit to be worth creating an
 enormous compatibility problem.  So even if the code existed I would resist
 turning it on.  I think the author of libjpeg 8 has made a very serious error
 by adding support for it.

I have replied on the bug about it. Some of the text in my initial bug
description is incorrect, and I have corrected it in this email.

It is not upto Fedora to make political decisions when there is no good
reason for it. This goes against the values of free software.

You shall not create images with arithmetic coding is like saying You
shall not create images of the flying sphagetti monster. It's not up to
Fedora to make this choice for me.

As libjpeg is a shared library on the system, if it can support
arithmetic coding, it should

Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Paul F. Johnson
Hi,

 You shall not create images with arithmetic coding is like saying You
 shall not create images of the flying sphagetti monster. It's not up to
 Fedora to make this choice for me.

It is though - you have chosen to use Fedora therefore have to live with
the decisions the Fedora legal people (I'm assuming they said no to
arithmetic coding) have said goes.

Remember, Fedora is not just for use in your country - there are piles
of countries it's used in, each with their own insane patent
regulations. The fedora stance is that if it offends, it's out.

If you want a prime example, look at either DeCSS or mp3 support. Plenty
of distros have it, but they're both out of Fedora. Doesn't matter that
in the authenticity of the mp3 patent is still in doubt, Fedora plays it
safe. rpmfusion may supply DeCSS and mp3 support, but they're not part
of Fedora. This may be one avenue to look at - see if rpmfusion can
supply an add-on package for this.

Paul

-- 
Vertraue mir, ich weiss, was ich mache...

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Peter Lemenkov
2010/10/2 Paul F. Johnson p...@all-the-johnsons.co.uk:

 Remember, Fedora is not just for use in your country - there are piles
 of countries it's used in, each with their own insane patent
 regulations. The fedora stance is that if it offends, it's out.

Just for the record, AFAIK there are only two countries which allows
software patents - USA and South Korea.
-- 
With best regards, Peter Lemenkov.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Paul F. Johnson
Hi,

  Remember, Fedora is not just for use in your country - there are piles
  of countries it's used in, each with their own insane patent
  regulations. The fedora stance is that if it offends, it's out.
 
 Just for the record, AFAIK there are only two countries which allows
 software patents - USA and South Korea.

Yep - and it's the US which has some of the more insane ones (and
therefore has to be the most careful over). 

I wonder how long it will be before Apple's current insane one over
backing up to a local repository will be allowed to stand - effectively
it means you can't burn to a CD!

Paul

-- 
Vertraue mir, ich weiss, was ich mache...

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Kevin Fenzi
On Sat, 2 Oct 2010 22:53:53 +0530
Mukund Sivaraman m...@banu.com wrote:

 Arithmetic coding in Fedora libjpeg (bug #639531)

...snip long thing...

My thoughts: 

- We should not be making this change in stable releases even if
  otherwise like the idea. It could well cause issues and problems with
  a fragile library used by a lot of things. 

- In f14+ we are not using libjpeg. We are using libjpeg-turbo. See: 
http://fedoraproject.org/wiki/Features/libjpeg-turbo
So, any request to enable this support should be filed there and looked
at from it's perspective. Do they have support? Does it cause any
issues? Does upstream enable it? 

- For the legal side, make your (new libjpeg-turbo) bug block FE_LEGAL
  so it can be looked at. 

kevin


signature.asc
Description: PGP signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Björn Persson
Peter Lemenkov wrote:
 Just for the record, AFAIK there are only two countries which allows
 software patents - USA and South Korea.

That's not the whole story. The situation is quite weird in countries that 
have signed the European Patent Convention. The convention and the national 
laws all state clearly that computer programs are not patentable, and yet 
software patents are granted on a daily basis, justified with some irrational 
reasoning that makes sense only to mentally deranged patent lawyers and some 
gullible politicians. Tens of thousands of these illegal patents have been 
granted.

Several years ago the patent lobby attempted to push through an EU directive 
to legalize software patents. The directive was eventally dropped after years 
of massive campaigning for and against, but that only means that the granting 
of illegal patents continues as before.

Such a patent might not hold up in court if it were contested by a good lawyer 
and the judge had some basic understanding of how computers work, but that 
doesn't make the patents harmless. Due to the high cost and the uncertain 
outcome of a patent lawsuit, small companies often have no choice but to pay 
the license fees.

Björn Persson


signature.asc
Description: This is a digitally signed message part.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Tom Lane
Kevin Fenzi ke...@scrye.com writes:
 On Sat, 2 Oct 2010 22:53:53 +0530
 Mukund Sivaraman m...@banu.com wrote:
 Arithmetic coding in Fedora libjpeg (bug #639531)

 My thoughts: 

 - We should not be making this change in stable releases even if
   otherwise like the idea. It could well cause issues and problems with
   a fragile library used by a lot of things. 

Don't worry, there's exactly 0 chance of that.

 - In f14+ we are not using libjpeg. We are using libjpeg-turbo. See: 
 http://fedoraproject.org/wiki/Features/libjpeg-turbo

Yes.  So actually the OP is pestering the wrong person; he should be
trying to convice the libjpeg-turbo maintainers to expend work on this.

I believe that they'd be best advised to say no, because at this point
one of JPEG's principal attractions is near-universal compatibility.
Throwing A/C into the mix will throw that away, for what really is a
very marginal gain in compression efficiency.  If you want a new
not-compatible-with-anything image format, you may as well adopt JPEG
2000, or some other format that's less than 20 years old.  The A/C
option to original JPEG is something that missed its chance because of
patents.  Resurrecting it from the dead now is just an exercise in
misplaced priorities.

But having said that, it's not my decision to make; it's the
libjpeg-turbo authors' decision whether to expend effort in that
direction.

regards, tom lane
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Björn Persson
Tom Lane wrote:
 I believe that they'd be best advised to say no, because at this point
 one of JPEG's principal attractions is near-universal compatibility.
 Throwing A/C into the mix will throw that away, for what really is a
 very marginal gain in compression efficiency.

It's well known around the Internet that to achieve compatibility you should 
be conservative in what you send and liberal in what you accept. Applied to 
JPEG: Use only Huffman coding when encoding – except maybe if you know that all 
recipients can handle arithmetic coding – but support both encodings when 
decoding.

Björn Persson


signature.asc
Description: This is a digitally signed message part.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Gregory Maxwell
On Sat, Oct 2, 2010 at 1:34 PM, Paul F. Johnson
p...@all-the-johnsons.co.uk wrote:
 Hi,

 You shall not create images with arithmetic coding is like saying You
 shall not create images of the flying sphagetti monster. It's not up to
 Fedora to make this choice for me.

 It is though - you have chosen to use Fedora therefore have to live with
 the decisions the Fedora legal people (I'm assuming they said no to
 arithmetic coding) have said goes.

The relevant patent expired last year. I believe the SUN OMS team had
researched this extensively as they were using the JPEG arithmetic
coder in their aggressively researched royalty free video codec
design.

If someone doing legal research on this needs more information, you
can contact me offlist and I can connect you with people who have
researched this topic and may be willing to provide some useful
pointers.

2010/10/2 Björn Persson bj...@xn--rombobjrn-67a.se:
 It's well known around the Internet that to achieve compatibility you should
 be conservative in what you send and liberal in what you accept. Applied to
 JPEG: Use only Huffman coding when encoding – except maybe if you know that 
 all
 recipients can handle arithmetic coding – but support both encodings when
 decoding.

Absolutely. This is an excellent argument and I think is sufficient to
justify the inclusion alone.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Arithmetic coding in Fedora libjpeg (bug #639531)

2010-10-02 Thread Kevin Kofler
Björn Persson wrote:
 It's well known around the Internet that to achieve compatibility you
 should be conservative in what you send and liberal in what you accept.
 Applied to JPEG: Use only Huffman coding when encoding – except maybe if
 you know that all recipients can handle arithmetic coding – but support
 both encodings when decoding.

+1

I can see not adding the support to the ENCODING portion, also because that 
appears to imply an ABI change (extra flag), but the DECODING portion of the 
patch should be merged.

The patch against the original libjpeg libjpeg-turbo is derived from should 
probably work, it might not be turbo-optimized, but given how rare those 
files are, that shouldn't be a problem. It's still better to be able to 
decode the files slowly than not at all.

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel