I'd be interested in any performance bottlenecks that people run into with the multi-part path. One of the bigger wins we saw was with stereo image pipelines where only one view was being asked for from the file, a simple seek to the requested part vs a not an insignificant amount of IO.
-Piotr On 7 March 2016 at 12:25, Larry Gritz <[email protected]> wrote: > I want to add that while I'm happy to add this flexibility to the OIIO > API, I make NO GUARANTEES about libIlmImf (or any other format libraries) > performance with random part access. I don't know how it buffers internally > or whether keeping 200 parts alive in an open file will result in massive > memory allocation internal to the library that I can't control. > > -- lg > > > > On Mar 7, 2016, at 12:18 PM, Larry Gritz <[email protected]> wrote: > > > > Thanks for the additional input, Pete. > > > > So is the current proposal is: > > > > * ImageOutput::supports("random_subimage_access") will return true for > formats that support writing to subimages other than the currently > designated one (set by the last open() call). > > > > * Add a variant of ImageOutput::write_tiles that accepts an "int > subimage" parameter to specify which subimage the tile belongs to. > > > > Is that sufficient to meet your needs? > > > > > > > >> On Mar 7, 2016, at 12:10 PM, Pete Black <[email protected]> wrote: > >> > >> Hi Larry, > >> > >> I personally can see a need to write entire image parts in a more > flexible fashion, rather than tiles - I cant talk about the details > on-list, but recently I had to deal with stereo pairs of EXRS each with > 400+ channels, and selectively output multipart stereo EXRs with 200+ parts > per frame - I could do the job with OIIO, but having more flexible APIs for > multipart would be nice. > >> > >> Thanks > >> > >> -Pete > >> > >> > >>> On 8/03/2016, at 8:48 AM, Larry Gritz <[email protected]> wrote: > >>> > >>> Wow, I guess I never fully realized that OpenEXR's API allowed for > this! OpenEXR's addition of "multi-part" was a latecomer, compared to TIFF > which has allowed multiple images in a file, but not random read or write > access to them, and I guess I never reexamined those assumptions when > OpenEXR added the ability. > >>> > >>> Yes, in light of that, I think it does make sense to extend the API to > have a variant of write_tiles that lets you specify the subimage. Of > course, since most image formats (such as TIFF) do not allow this > flexibility, apps will be cautioned to only use it for output formats for > which supports("random_subimage_access") is true, otherwise it would be an > error to write tiles to anything but the currently-opened subimage. > >>> > >>> Do you think we need write_scanlines similarly modified? write_image? > Or is the practical use case for this always going to be tiles? > >>> > >>> What about input? Do you think it's important to allow reading of a > tile from one part in a random access way, but without needing a > seek_subimage to set the current subimage? > >>> > >>> Let me think a bit about how to structure this. It seems to me that > this is going to have to be a master-only feature, since adding it is going > to break link compatibility with the existing ImageOutput class API and > vtable. > >>> > >>> -- lg > >>> > >>> > >>>> On Mar 7, 2016, at 3:38 AM, Ramon Montoya Vozmediano < > [email protected]> wrote: > >>>> > >>>> Hi Larry, > >>>> > >>>> OIIO supports output to multipart EXR images, and they have to be > written out like this: > >>>> > >>>> open(multipart file with n parts) > >>>> write_tiles(...) > >>>> > >>>> open(...AppendSubimage) > >>>> write_tiles(...) > >>>> > >>>> open(...AppendSubimage) > >>>> write_tiles(...) > >>>> > >>>> close() > >>>> > >>>> So you write out the first subimage, then the next, etc... > >>>> > >>>> The motivation for this email is that with this setup, when you are > rendering, you need to keep all the tiles in memory before they can be > written out. > >>>> > >>>> OpenEXR itself lets you write out tiles for any part in any order. > >>>> > >>>> So, when using the EXR API directly, a renderer could write out tiles > as soon as they are done, and the physical layout of the multipart image > would end up looking like this: > >>>> > >>>> [chunk offset index] > >>>> [part 0, data for tile (0,0)] > >>>> [part 1, data for tile (0,0)] > >>>> [part 2, data for tile (0,0)] > >>>> ... > >>>> [part 0, data for tile (0,1)] > >>>> [part 1, data for tile (0,1)] > >>>> [part 2, data for tile (0,1)] > >>>> ... > >>>> > >>>> It looks like supporting this means extending OIIO's API, so that you > can set the part you want to write to, without having to close/open a > "subimage". > >>>> > >>>> Another way to look at this conceptually, is to consider an EXR part > as a channel grouping rather than a tiff style subimage. > >>>> > >>>> Have you considered extending OIIO's API in this direction? > >>>> > >>>> Best, > >>>> > >>>> r > >>>> > >>> > >>> -- > >>> Larry Gritz > >>> [email protected] > >>> > >>> > >>> _______________________________________________ > >>> Oiio-dev mailing list > >>> [email protected] > >>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > >> > >> _______________________________________________ > >> Oiio-dev mailing list > >> [email protected] > >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > > > > -- > > Larry Gritz > > [email protected] > > > > > > _______________________________________________ > > Oiio-dev mailing list > > [email protected] > > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > > -- > Larry Gritz > [email protected] > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
