If people are thinking of the problem as inherently having 16 bit unsigned
integer values, then converting to half float for output to exr, and finally
converting back to canonical uint16, perhaps they are barking up the wrong tree
by considering OpenEXR at all.
The whole idea behind OpenEXR is for extended range -- values that may be
negative, or very small, or greater than 1. The kinds of things you would
usually use a 'float' for if they were in memory. The use of half precision is
itself a form of lossy compression, halving the size by accepting a loss of
precision (but an acceptable and roughly perceptually uniform loss, since
vision sensitivity is essentially logarithmic).
OpenEXR is meant to store float data, not integer data. You wouldn't expect a
uint16 image to have very good accuracy or compression by going through a
'half' conversion. It's a mismatch of method and task.
-- lg
> On Dec 5, 2016, at 5:41 PM, Conrad Gaunt <helloconradga...@googlemail.com>
> wrote:
>
> hi.
>
> ive been testing image formats for a few months, and in depth. new to openexr.
>
> i'm currently starting the optimizing of my lossless 16bit+ compression
> system
>
> some observations.
>
> remember, if you're source material is 16bit integer, the half float
> conversion is very lossy, although it has more conservative properties when
> manipulated down the post pipe in some regards,
>
> one thing all formats suffer from .. they are always going out of date . most
> were invented in the 90s
>
> my classical Huffman coder beats j2k and png on 48bit photographic material,
> and PIZ
>
> PNG uses Huffman coding .. and can compress 16bit values .. but the
> compression system in png is only 8bit. same with TIF.
>
> all these are by products of formats beings developed back when computers
> had 640kb of ram..
>
> when you convert 16bit integer to 16bit half float, and normalise between
> 0..1 , there is half as much data recorded as when you normalise between -1
> ..1 .. and you're not wasting the sign bit !
>
> I don't know what openexr convention says. anyway, normalising between 0..1
> makes my compressor produce files that are significantly smaller than the
> compressed integer source.
>
> I'm not sure what the correct normalization procedure for openexr is to be
> honest
>
> try normalising all possible 65536 integer values into half floats. then
> convert them back to integer, and print a list of the original inputs and
> outputs (into a text file) . quite interesting
>
> k.r
>
>
>
>
>
>
>
>
> On Sun, Dec 4, 2016 at 5:43 AM, <openexr-devel-requ...@nongnu.org
> <mailto:openexr-devel-requ...@nongnu.org>> wrote:
> Send Openexr-devel mailing list submissions to
> openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org>
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.nongnu.org/mailman/listinfo/openexr-devel
> <https://lists.nongnu.org/mailman/listinfo/openexr-devel>
> or, via email, send a message with subject or body 'help' to
> openexr-devel-requ...@nongnu.org
> <mailto:openexr-devel-requ...@nongnu.org>
>
> You can reach the person managing the list at
> openexr-devel-ow...@nongnu.org <mailto:openexr-devel-ow...@nongnu.org>
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Openexr-devel digest..."
>
> Today's Topics:
>
> 1. OpenEXR - lossless compression problems (David Ku??k)
> 2. Re: OpenEXR - lossless compression problems (Peter Hillman)
>
>
> ---------- Forwarded message ----------
> From: "David Kuťák" <davedes...@email.cz <mailto:davedes...@email.cz>>
> To: <openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org>>
> Cc:
> Date: Sat, 3 Dec 2016 20:40:10 +0100
> Subject: [Openexr-devel] OpenEXR - lossless compression problems
> Hello,
>
>
>
> I am currently working on my bachelor thesis in which I am comparing image
> compression algorithms (available in different image file formats).
>
> For my work, only file formats supporting lossless compression and 16 bit
> depth are appropriate.
>
> During my exploration of possible formats, I found an OpenEXR and decided to
> include it in my tests.
>
> However, I am not able to make it work properly.
>
>
>
> The input image and output image (created during compression) are not same –
> the colors seem to be different (shifted).
>
> I thought it might be gamma problem but even after gamma correction, I am not
> able to retrieve correct output image.
>
> Although visually the image seems to be very close to the input one (after
> gamma correction), there is still a minor difference which is unacceptable.
>
>
>
> So I’d like to ask you for a help – what I am doing is written below (some
> undefined wars like imageWidth, etc. are correct for sure).
>
>
>
> 1) I create my output file
>
> RgbaOutputFile outputFile(argv[3], imageWidth, imageHeight, WRITE_RGB,
> 1.0f,
>
> V2f(0, 0), 1.0f, INCREASING_Y,
> compressionType, globalThreadCount());
>
>
>
> 2) Then I create a pixelData vector
>
> std::vector<Rgba> pixelData(imageWidth * imageHeight);
>
>
>
> 3) I fill this vector with data retrieved from my test images (these data are
> pixel values (i.e. numbers from interval <0,255/65535> – depending on bit
> depth – which I normalize to inverval <0,1.0> and pass to half ctor and then
> as an Rgba struct to appropriate pos in pixelData vector).
>
>
>
> 4) Then I set framebuffer and write pixels to file (shown in code below). I
> think that my problem occurs somewhere in those two methods (EXR must be
> doing some operations with pixel data).
>
> outputFile.setFrameBuffer(pixelData.data(), 1, imageWidth);
>
> outputFile.writePixels(imageHeight);
>
>
>
> To sum up my problem – is this the correct way to pass image data to OpenEXR
> and retrieve the same image (but in EXR format)? I really don’t know why the
> color problem appears.
>
>
>
> Another question I’d like to ask is that I got the idea that the compression
> is done in writePixels method. Is that’s right? If it’s true, is there any
> possibility to compress image in memory without writing it to file?
>
>
>
> Thank you for your help!
>
>
>
> David
>
>
>
> ---------- Forwarded message ----------
> From: Peter Hillman <pet...@wetafx.co.nz <mailto:pet...@wetafx.co.nz>>
> To: "David Kuťák" <davedes...@email.cz <mailto:davedes...@email.cz>>,
> "openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org>"
> <openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org>>
> Cc:
> Date: Sun, 4 Dec 2016 00:33:39 +0000
> Subject: Re: [Openexr-devel] OpenEXR - lossless compression problems
> Hi David,
>
>
> Your code looks valid, and as the image looks quite close to correct it's
> unlikely there's a bug in your code.
>
> Perhaps the difference you are seeing is due to the fact that OpenEXR's 16
> bit half is a floating point number. Converting a 16 bit unsigned integer n
> to a half float using n/65535 will cause rounding error.
>
>
> From a quick test it seems you get 7169 unique bit patterns in a half float
> with that approach, which means you are losing information. The other
> patterns used by 16 bit half mainly store negative numbers and numbers above
> 1, which you aren't using here. These come about because OpenEXR is intended
> for storing HDR images. With a 16 bit unsigned int sRGB PNG, the darkest grey
> is 20 stops darker than the maximum white; with a 16 bit half linear OpenEXR
> that's closer to 40 stops.
>
>
> The 8 bit version of n/255 will also cause rounding error, but in that case
> each input value is represented uniquely and you can fix the error by
> rounding to the nearest integer when you denormalize.
>
> The openexr.com <http://openexr.com/> website has some sample HDR images you
> could use to experiment with OpenEXR compression schemes, but it is hard to
> make a rigorous comparison of PNG (for example) to OpenEXR because they set
> out to achieve different things working with different kinds of image data.
>
>
> Your second question:
>
>
> > I got the idea that the compression is done in writePixels method. Is
> > that’s right? If it’s true, is there any possibility to compress image in
> > memory without writing it to file?
>
> Yes, this is correct-writePixels is the only access to the compression
> schemes from the API. It is possible to generate an OpenEXR file entirely in
> memory, but is non-trivial: you have to subclass OStream and override the
> methods to write data into your memory array rather than to file. If you need
> to read it back, you need to make an IStream too.
>
> You then pass an instance of your new OStream to the OutputFile constructor
> when you write it. The OpenEXR library will call your methods when it wants
> to write data. You might start here:
>
> <https://github.com/openexr/openexr/blob/develop/OpenEXR/IlmImfExamples/lowLevelIoExamples.cpp>
> https://github.com/openexr/openexr/blob/develop/OpenEXR/IlmImfExamples/lowLevelIoExamples.cpp
>
> <https://github.com/openexr/openexr/blob/develop/OpenEXR/IlmImfExamples/lowLevelIoExamples.cpp>
>
> Actually, if you just want to know how big the file would be with a given
> scheme, you could be sneaky and not store the image at all: just subclass
> OStream with a class that has its own "file pointer" which is read by
> readp(), set by seekp() and incremented by write(). You'll also need to track
> the maximum value the "file pointer" ever has, which would be the file size
> you need (not the final value of "file pointer") Doing this won't let you
> analyse errors in lossy schemes, but would let you measure the lossless
> schemes' efficiency.
>
> <https://github.com/openexr/openexr/blob/develop/OpenEXR/IlmImfExamples/lowLevelIoExamples.cpp>
>
>
> From: Openexr-devel <openexr-devel-bounces+peterh=wetafx.co...@nongnu.org
> <mailto:wetafx.co...@nongnu.org>> on behalf of David Kuťák
> <davedes...@email.cz <mailto:davedes...@email.cz>>
> Sent: Sunday, 4 December 2016 8:40 a.m.
> To: openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org>
> Subject: [Openexr-devel] OpenEXR - lossless compression problems
>
> Hello,
>
> I am currently working on my bachelor thesis in which I am comparing image
> compression algorithms (available in different image file formats).
> For my work, only file formats supporting lossless compression and 16 bit
> depth are appropriate.
> During my exploration of possible formats, I found an OpenEXR and decided to
> include it in my tests.
> However, I am not able to make it work properly.
>
> The input image and output image (created during compression) are not same –
> the colors seem to be different (shifted).
> I thought it might be gamma problem but even after gamma correction, I am not
> able to retrieve correct output image.
> Although visually the image seems to be very close to the input one (after
> gamma correction), there is still a minor difference which is unacceptable.
>
> So I’d like to ask you for a help – what I am doing is written below (some
> undefined wars like imageWidth, etc. are correct for sure).
>
> 1) I create my output file
> RgbaOutputFile outputFile(argv[3], imageWidth, imageHeight, WRITE_RGB,
> 1.0f,
> V2f(0, 0), 1.0f, INCREASING_Y,
> compressionType, globalThreadCount());
>
> 2) Then I create a pixelData vector
> std::vector<Rgba> pixelData(imageWidth * imageHeight);
>
> 3) I fill this vector with data retrieved from my test images (these data are
> pixel values (i.e. numbers from interval <0,255/65535> – depending on bit
> depth – which I normalize to inverval <0,1.0> and pass to half ctor and then
> as an Rgba struct to appropriate pos in pixelData vector).
>
> 4) Then I set framebuffer and write pixels to file (shown in code below). I
> think that my problem occurs somewhere in those two methods (EXR must be
> doing some operations with pixel data).
> outputFile.setFrameBuffer(pixelData.data(), 1, imageWidth);
> outputFile.writePixels(imageHeight);
>
> To sum up my problem – is this the correct way to pass image data to OpenEXR
> and retrieve the same image (but in EXR format)? I really don’t know why the
> color problem appears.
>
> Another question I’d like to ask is that I got the idea that the compression
> is done in writePixels method. Is that’s right? If it’s true, is there any
> possibility to compress image in memory without writing it to file?
>
> Thank you for your help!
>
> David
>
> _______________________________________________
> Openexr-devel mailing list
> Openexr-devel@nongnu.org <mailto:Openexr-devel@nongnu.org>
> https://lists.nongnu.org/mailman/listinfo/openexr-devel
> <https://lists.nongnu.org/mailman/listinfo/openexr-devel>
>
>
>
>
> --
> conrad gaunt
>
> www.cheapredhire.com <http://www.cheapredhire.com/>
> tel.07835915253
> _______________________________________________
> Openexr-devel mailing list
> Openexr-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/openexr-devel
--
Larry Gritz
l...@larrygritz.com
_______________________________________________
Openexr-devel mailing list
Openexr-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/openexr-devel
