Hello,
I think I got the point. However, one more question just come to my mind – OpenEXR should support not just 16bit half float, but also 32 bit float and 32bit unsigned int (referred to as FLOAT / UINT in documentation). These other data types are not widely mentioned in documentation and Rgba interface seems to support only half float (as it’s probably most logical choice). But is it possible to save pixel data directly in UINT / FLOAT (and thus not to use half float) using OpenEXR? This way there should be no problem with losing information when reading 16 bit unsigned int data and converting them to UINT / FLOAT (although the compression ratio might be unsatisfying). Or are those additional formats like UINT / FLOAT used just for „metadata“ (like depth, etc.)? Thank you. David From: Openexr-devel [mailto:openexr-devel-bounces+davedesign=email...@nongnu.org] On Behalf Of Larry Gritz Sent: Tuesday, December 6, 2016 6:44 AM To: openexr-devel@nongnu.org Subject: Re: [Openexr-devel] OpenEXR - lossless compression problems (Peter Hillman) If people are thinking of the problem as inherently having 16 bit unsigned integer values, then converting to half float for output to exr, and finally converting back to canonical uint16, perhaps they are barking up the wrong tree by considering OpenEXR at all. The whole idea behind OpenEXR is for extended range -- values that may be negative, or very small, or greater than 1. The kinds of things you would usually use a 'float' for if they were in memory. The use of half precision is itself a form of lossy compression, halving the size by accepting a loss of precision (but an acceptable and roughly perceptually uniform loss, since vision sensitivity is essentially logarithmic). OpenEXR is meant to store float data, not integer data. You wouldn't expect a uint16 image to have very good accuracy or compression by going through a 'half' conversion. It's a mismatch of method and task. -- lg On Dec 5, 2016, at 5:41 PM, Conrad Gaunt <helloconradga...@googlemail.com <mailto:helloconradga...@googlemail.com> > wrote: hi. ive been testing image formats for a few months, and in depth. new to openexr. i'm currently starting the optimizing of my lossless 16bit+ compression system some observations. remember, if you're source material is 16bit integer, the half float conversion is very lossy, although it has more conservative properties when manipulated down the post pipe in some regards, one thing all formats suffer from .. they are always going out of date . most were invented in the 90s my classical Huffman coder beats j2k and png on 48bit photographic material, and PIZ PNG uses Huffman coding .. and can compress 16bit values .. but the compression system in png is only 8bit. same with TIF. all these are by products of formats beings developed back when computers had 640kb of ram.. when you convert 16bit integer to 16bit half float, and normalise between 0..1 , there is half as much data recorded as when you normalise between -1 ..1 .. and you're not wasting the sign bit ! I don't know what openexr convention says. anyway, normalising between 0..1 makes my compressor produce files that are significantly smaller than the compressed integer source. I'm not sure what the correct normalization procedure for openexr is to be honest try normalising all possible 65536 integer values into half floats. then convert them back to integer, and print a list of the original inputs and outputs (into a text file) . quite interesting k.r On Sun, Dec 4, 2016 at 5:43 AM, <openexr-devel-requ...@nongnu.org> wrote: Send Openexr-devel mailing list submissions to openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org> To subscribe or unsubscribe via the World Wide Web, visit https://lists.nongnu.org/mailman/listinfo/openexr-devel or, via email, send a message with subject or body 'help' to openexr-devel-requ...@nongnu.org <mailto:openexr-devel-requ...@nongnu.org> You can reach the person managing the list at openexr-devel-ow...@nongnu.org <mailto:openexr-devel-ow...@nongnu.org> When replying, please edit your Subject line so it is more specific than "Re: Contents of Openexr-devel digest..." Today's Topics: 1. OpenEXR - lossless compression problems (David Ku??k) 2. Re: OpenEXR - lossless compression problems (Peter Hillman) ---------- Forwarded message ---------- From: "David Kuťák" <davedes...@email.cz <mailto:davedes...@email.cz> > To: <openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org> > Cc: Date: Sat, 3 Dec 2016 20:40:10 +0100 Subject: [Openexr-devel] OpenEXR - lossless compression problems Hello, I am currently working on my bachelor thesis in which I am comparing image compression algorithms (available in different image file formats). For my work, only file formats supporting lossless compression and 16 bit depth are appropriate. During my exploration of possible formats, I found an OpenEXR and decided to include it in my tests. However, I am not able to make it work properly. The input image and output image (created during compression) are not same – the colors seem to be different (shifted). I thought it might be gamma problem but even after gamma correction, I am not able to retrieve correct output image. Although visually the image seems to be very close to the input one (after gamma correction), there is still a minor difference which is unacceptable. So I’d like to ask you for a help – what I am doing is written below (some undefined wars like imageWidth, etc. are correct for sure). 1) I create my output file RgbaOutputFile outputFile(argv[3], imageWidth, imageHeight, WRITE_RGB, 1.0f, V2f(0, 0), 1.0f, INCREASING_Y, compressionType, globalThreadCount()); 2) Then I create a pixelData vector std::vector<Rgba> pixelData(imageWidth * imageHeight); 3) I fill this vector with data retrieved from my test images (these data are pixel values (i.e. numbers from interval <0,255/65535> – depending on bit depth – which I normalize to inverval <0,1.0> and pass to half ctor and then as an Rgba struct to appropriate pos in pixelData vector). 4) Then I set framebuffer and write pixels to file (shown in code below). I think that my problem occurs somewhere in those two methods (EXR must be doing some operations with pixel data). outputFile.setFrameBuffer(pixelData.data(), 1, imageWidth); outputFile.writePixels(imageHeight); To sum up my problem – is this the correct way to pass image data to OpenEXR and retrieve the same image (but in EXR format)? I really don’t know why the color problem appears. Another question I’d like to ask is that I got the idea that the compression is done in writePixels method. Is that’s right? If it’s true, is there any possibility to compress image in memory without writing it to file? Thank you for your help! David ---------- Forwarded message ---------- From: Peter Hillman <pet...@wetafx.co.nz <mailto:pet...@wetafx.co.nz> > To: "David Kuťák" <davedes...@email.cz <mailto:davedes...@email.cz> >, "openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org> " <openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org> > Cc: Date: Sun, 4 Dec 2016 00:33:39 +0000 Subject: Re: [Openexr-devel] OpenEXR - lossless compression problems Hi David, Your code looks valid, and as the image looks quite close to correct it's unlikely there's a bug in your code. Perhaps the difference you are seeing is due to the fact that OpenEXR's 16 bit half is a floating point number. Converting a 16 bit unsigned integer n to a half float using n/65535 will cause rounding error. >From a quick test it seems you get 7169 unique bit patterns in a half float >with that approach, which means you are losing information. The other patterns >used by 16 bit half mainly store negative numbers and numbers above 1, which >you aren't using here. These come about because OpenEXR is intended for >storing HDR images. With a 16 bit unsigned int sRGB PNG, the darkest grey is >20 stops darker than the maximum white; with a 16 bit half linear OpenEXR >that's closer to 40 stops. The 8 bit version of n/255 will also cause rounding error, but in that case each input value is represented uniquely and you can fix the error by rounding to the nearest integer when you denormalize. The openexr.com <http://openexr.com/> website has some sample HDR images you could use to experiment with OpenEXR compression schemes, but it is hard to make a rigorous comparison of PNG (for example) to OpenEXR because they set out to achieve different things working with different kinds of image data. Your second question: > I got the idea that the compression is done in writePixels method. Is that’s > right? If it’s true, is there any possibility to compress image in memory > without writing it to file? Yes, this is correct-writePixels is the only access to the compression schemes from the API. It is possible to generate an OpenEXR file entirely in memory, but is non-trivial: you have to subclass OStream and override the methods to write data into your memory array rather than to file. If you need to read it back, you need to make an IStream too. You then pass an instance of your new OStream to the OutputFile constructor when you write it. The OpenEXR library will call your methods when it wants to write data. You might start here: <https://github.com/openexr/openexr/blob/develop/OpenEXR/IlmImfExamples/lowLevelIoExamples.cpp> https://github.com/openexr/openexr/blob/develop/OpenEXR/IlmImfExamples/lowLevelIoExamples.cpp Actually, if you just want to know how big the file would be with a given scheme, you could be sneaky and not store the image at all: just subclass OStream with a class that has its own "file pointer" which is read by readp(), set by seekp() and incremented by write(). You'll also need to track the maximum value the "file pointer" ever has, which would be the file size you need (not the final value of "file pointer") Doing this won't let you analyse errors in lossy schemes, but would let you measure the lossless schemes' efficiency. _____ From: Openexr-devel <openexr-devel-bounces+peterh=wetafx.co...@nongnu.org <mailto:wetafx.co...@nongnu.org> > on behalf of David Kuťák <davedes...@email.cz <mailto:davedes...@email.cz> > Sent: Sunday, 4 December 2016 8:40 a.m. To: openexr-devel@nongnu.org <mailto:openexr-devel@nongnu.org> Subject: [Openexr-devel] OpenEXR - lossless compression problems Hello, I am currently working on my bachelor thesis in which I am comparing image compression algorithms (available in different image file formats). For my work, only file formats supporting lossless compression and 16 bit depth are appropriate. During my exploration of possible formats, I found an OpenEXR and decided to include it in my tests. However, I am not able to make it work properly. The input image and output image (created during compression) are not same – the colors seem to be different (shifted). I thought it might be gamma problem but even after gamma correction, I am not able to retrieve correct output image. Although visually the image seems to be very close to the input one (after gamma correction), there is still a minor difference which is unacceptable. So I’d like to ask you for a help – what I am doing is written below (some undefined wars like imageWidth, etc. are correct for sure). 1) I create my output file RgbaOutputFile outputFile(argv[3], imageWidth, imageHeight, WRITE_RGB, 1.0f, V2f(0, 0), 1.0f, INCREASING_Y, compressionType, globalThreadCount()); 2) Then I create a pixelData vector std::vector<Rgba> pixelData(imageWidth * imageHeight); 3) I fill this vector with data retrieved from my test images (these data are pixel values (i.e. numbers from interval <0,255/65535> – depending on bit depth – which I normalize to inverval <0,1.0> and pass to half ctor and then as an Rgba struct to appropriate pos in pixelData vector). 4) Then I set framebuffer and write pixels to file (shown in code below). I think that my problem occurs somewhere in those two methods (EXR must be doing some operations with pixel data). outputFile.setFrameBuffer(pixelData.data(), 1, imageWidth); outputFile.writePixels(imageHeight); To sum up my problem – is this the correct way to pass image data to OpenEXR and retrieve the same image (but in EXR format)? I really don’t know why the color problem appears. Another question I’d like to ask is that I got the idea that the compression is done in writePixels method. Is that’s right? If it’s true, is there any possibility to compress image in memory without writing it to file? Thank you for your help! David _______________________________________________ Openexr-devel mailing list Openexr-devel@nongnu.org <mailto:Openexr-devel@nongnu.org> https://lists.nongnu.org/mailman/listinfo/openexr-devel -- conrad gaunt www.cheapredhire.com <http://www.cheapredhire.com/> tel.07835915253 _______________________________________________ Openexr-devel mailing list Openexr-devel@nongnu.org <mailto:Openexr-devel@nongnu.org> https://lists.nongnu.org/mailman/listinfo/openexr-devel -- Larry Gritz l...@larrygritz.com <mailto:l...@larrygritz.com>
_______________________________________________ Openexr-devel mailing list Openexr-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/openexr-devel
