Was it always like this, in 1.x as well? Some of my code seems to bend over
backwards to not allow random order writes unless a specific request was made
for random Y order. I don't remember now why I wrote it that way, but the code
is structured as if at some time in the past, the library would not have done
the buffering without RANDOM_Y lineOrder. Was I just always wrong about this?
-- lg
On Apr 11, 2014, at 10:13 AM, Larry Gritz <l...@larrygritz.com> wrote:
> OK, so let me restate it to make sure I have it right:
>
> For scanline files, storage in the file is either increasing or decreasing
> (determined by lineOrder); libIlmImf will accept scanlines in any order, but
> if they are sent other than with the line order you specified, the library
> will buffer internally.
>
> For tiled files, libIlmImf will accept tiles in any order, buffering if they
> are not in canonical order, UNLESS you set lineOrder to RANDOM_Y, in which
> case it really will store it in random order rather than buffering (but at
> the cost of reading that file in canonical order having more seeks than if it
> had been written in canonical tile order).
>
> Do I have that all correct?
>
>
>
> On Apr 10, 2014, at 5:10 PM, Peter Hillman <pet...@wetafx.co.nz> wrote:
>
>> RANDOM_Y is not supported for scanline images - you either start at
>> datawindow.min.y and work to datawindow.max.y or vice versa. You can deliver
>> scanlines out of order: they'll be buffered before writing.
>>
>> With tiled images, you can write tiles in any order, but you must set
>> RANDOM_Y to allow it write without buffering.
>>
>>
>>
>> On 11/04/14 12:02, Larry Gritz wrote:
>>> Uh oh.
>>>
>>> I knew about RANDOM_Y for scanline images. Are you saying that for tiled
>>> images, if we don't set lineorder to RANDOM_Y, then there's internal
>>> buffering if we send the tiles in anything but canonical order?
>>>
>>>
>>>
>>> On Apr 10, 2014, at 4:54 PM, Peter Hillman <pet...@wetafx.co.nz> wrote:
>>>
>>>> In general with OpenEXR, you cannot write to a file from different threads
>>>> unless they have a shared framebuffer. Calling "setFrameBuffer" from one
>>>> thread while another is writing may cause a crash.
>>>>
>>>> Even if you do have a shared framebuffer, the threads lock to serialise
>>>> the writes: only one "writePixels" or "writeTile" request goes through at
>>>> once.
>>>> If you don't use the IlmThread library to run your threads it may be wise
>>>> to implement your own lock mechanism around writePixel and writeTile calls.
>>>>
>>>>
>>>> There are two ways of writing tiled data: if you use
>>>> xTileCoords/yTileCoords, then pixel(0,0) is the bottom left corner of each
>>>> tile, implying the image buffer pointed to by the slice is the same size
>>>> as the tile.
>>>> if you don't do that, then pixel(0,0) is the bottom left corner of the
>>>> displayWindow, implying the image buffer pointed to be the slice is the
>>>> size of the dataWindow, accounting for offset data windows appropriately.
>>>>
>>>> In your case, it would seem like you want the second option: preallocate
>>>> your dataWindow sized array of pointers-to-samples, and call
>>>> setFrameBuffer before any rendering begins.
>>>> Then each thread sets the pointers to the samples for the tile it is going
>>>> to write and fills in the corresponding part of the samplecount array
>>>> before calling writeTile.
>>>>
>>>> The pointers need not be valid when setFrameBuffer is called. When you
>>>> call writeTile, only the pointers and sampleCount data for the pixels
>>>> being written need to be valid, and need not remain valid afterwards. So
>>>> yes, you can delete data for the actual sample data of a tile after
>>>> calling writeTile, though you'll need to keep the sampleCount array and
>>>> the pointers-to-samples array for each channel.
>>>>
>>>> Note that if you are writing tiles in anything other than RANDOM_Y order,
>>>> then whenever your threads deliver tiles out of order OpenEXR will have to
>>>> buffer data so it can write the tiles sequentially.
>>>> I suspect this may well be the case from multithreaded write applications.
>>>> With deep data this could take a significant amount of memory.
>>>> You may wish to enforce RANDOM_Y order to allow the library to write tiles
>>>> to disk as soon as you call writeTile. If your read pattern is generally
>>>> also random, rather than in 'scanline order', there'll be little
>>>> performance hit.
>>>>
>>>> You might also consider having one worker thread managing the writing of
>>>> tile data and marshalling into the format required for the DeepSlices, and
>>>> have other threads compute data and signal the worker thread when they
>>>> have tiles ready for processing.
>>>>
>>>> Yes, you can use an Array2D<float*> for your data points (dataZ and dataA
>>>> in the example). Each value is a pointer to the first sample of the
>>>> channel data for each pixel.
>>>> Subsequent samples are found using the offset provided. That means you can
>>>> store pixel data in separate or interleaved channels.
>>>>
>>>>
>>>>
>>>> On 11/04/14 02:16, Michel Lerenard wrote:
>>>>> Hi,
>>>>>
>>>>> I'm trying to write deep data into an EXR file and am having trouble
>>>>> understanding how the API works.
>>>>> I'd like to be able to write tiles from different threads. The
>>>>> application I'm working on (Isotropix Clarisse) runs several threads,
>>>>> each one render a part of an image.
>>>>> I'd like to write deep data when the current tile is completely computed.
>>>>>
>>>>> I'm using the "Reading And Writing Image Files" documentation, and the
>>>>> section about Deep Tile File writing on page 26 is confusing to me.
>>>>>
>>>>> From what I understand, I need a DeepFrameBuffer to write an image. It
>>>>> seems it should be the same for all threads, as a DeepTiledOutputFile can
>>>>> only use one. Is it thread safe ? Can I use the same FrameBuffer from all
>>>>> threads ?
>>>>> I was puzzled by the type of dataZ and data0, reading the next example
>>>>> (reading a deep tiled file) I understand that the arrays are only storing
>>>>> pointers to deep data for a particular pixel, and that they could as well
>>>>> be Array2D<float*>. Is that correct ?
>>>>>
>>>>> If it is, this means that each one of my threads should set pointers in
>>>>> the framebuffer DeepSlice for each pixel of the tile it's computing, then
>>>>> call writeTile. After that can I delete the data and reset the pointers
>>>>> in the DeepSlice ?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Michel
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Openexr-devel mailing list
>>>>> Openexr-devel@nongnu.org
>>>>> https://lists.nongnu.org/mailman/listinfo/openexr-devel
>>>>
>>>> _______________________________________________
>>>> Openexr-devel mailing list
>>>> Openexr-devel@nongnu.org
>>>> https://lists.nongnu.org/mailman/listinfo/openexr-devel
>>> --
>>> Larry Gritz
>>> l...@larrygritz.com
>>>
>>>
>>>
>>
>
> --
> Larry Gritz
> l...@larrygritz.com
>
>
>
>
> _______________________________________________
> Openexr-devel mailing list
> Openexr-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/openexr-devel
--
Larry Gritz
l...@larrygritz.com
_______________________________________________
Openexr-devel mailing list
Openexr-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/openexr-devel
