> Do you think we need write_scanlines similarly modified? write_image? Or is the practical use case for this always going to be tiles?
The format does support those cases, I think at least write_scanlines should also be modified. The only reason to modify write_image would be for consistency. For our needs adding the new write_tiles / write_scanlines would be sufficient in the way you propose. > What about input? Do you think it's important to allow reading of a tile from one part in a random access way, but without needing a seek_subimage to set the current subimage? Sounds like with seek_subimage you can already read in a random way. The IO actions from the OpenEXR library would be the same in both cases (find the chunk in the table of offsets and get it). So I think it's really a case of evaluating if the new interface would save OIIO some overhead or if it would simplify subimage access for OIIO users. > I want to add that while I'm happy to add this flexibility to the OIIO API, I make NO GUARANTEES about libIlmImf (or any other format libraries) performance with random part access. I don't know how it buffers internally or whether keeping 200 parts alive in an open file will result in massive memory allocation internal to the library that I can't control. True. The physical layout of the multipart EXR seems to be specifically designed to allow that kind of setup, so I imagine it's not bad. r On Mon, Mar 7, 2016 at 9:45 PM, Piotr Stanczyk <[email protected]> wrote: > I'd be interested in any performance bottlenecks that people run into with > the multi-part path. One of the bigger wins we saw was with stereo image > pipelines where only one view was being asked for from the file, a simple > seek to the requested part vs a not an insignificant amount of IO. > > -Piotr > > On 7 March 2016 at 12:25, Larry Gritz <[email protected]> wrote: > >> I want to add that while I'm happy to add this flexibility to the OIIO >> API, I make NO GUARANTEES about libIlmImf (or any other format libraries) >> performance with random part access. I don't know how it buffers internally >> or whether keeping 200 parts alive in an open file will result in massive >> memory allocation internal to the library that I can't control. >> >> -- lg >> >> >> > On Mar 7, 2016, at 12:18 PM, Larry Gritz <[email protected]> wrote: >> > >> > Thanks for the additional input, Pete. >> > >> > So is the current proposal is: >> > >> > * ImageOutput::supports("random_subimage_access") will return true for >> formats that support writing to subimages other than the currently >> designated one (set by the last open() call). >> > >> > * Add a variant of ImageOutput::write_tiles that accepts an "int >> subimage" parameter to specify which subimage the tile belongs to. >> > >> > Is that sufficient to meet your needs? >> > >> > >> > >> >> On Mar 7, 2016, at 12:10 PM, Pete Black <[email protected]> wrote: >> >> >> >> Hi Larry, >> >> >> >> I personally can see a need to write entire image parts in a more >> flexible fashion, rather than tiles - I cant talk about the details >> on-list, but recently I had to deal with stereo pairs of EXRS each with >> 400+ channels, and selectively output multipart stereo EXRs with 200+ parts >> per frame - I could do the job with OIIO, but having more flexible APIs for >> multipart would be nice. >> >> >> >> Thanks >> >> >> >> -Pete >> >> >> >> >> >>> On 8/03/2016, at 8:48 AM, Larry Gritz <[email protected]> wrote: >> >>> >> >>> Wow, I guess I never fully realized that OpenEXR's API allowed for >> this! OpenEXR's addition of "multi-part" was a latecomer, compared to TIFF >> which has allowed multiple images in a file, but not random read or write >> access to them, and I guess I never reexamined those assumptions when >> OpenEXR added the ability. >> >>> >> >>> Yes, in light of that, I think it does make sense to extend the API >> to have a variant of write_tiles that lets you specify the subimage. Of >> course, since most image formats (such as TIFF) do not allow this >> flexibility, apps will be cautioned to only use it for output formats for >> which supports("random_subimage_access") is true, otherwise it would be an >> error to write tiles to anything but the currently-opened subimage. >> >>> >> >>> Do you think we need write_scanlines similarly modified? write_image? >> Or is the practical use case for this always going to be tiles? >> >>> >> >>> What about input? Do you think it's important to allow reading of a >> tile from one part in a random access way, but without needing a >> seek_subimage to set the current subimage? >> >>> >> >>> Let me think a bit about how to structure this. It seems to me that >> this is going to have to be a master-only feature, since adding it is going >> to break link compatibility with the existing ImageOutput class API and >> vtable. >> >>> >> >>> -- lg >> >>> >> >>> >> >>>> On Mar 7, 2016, at 3:38 AM, Ramon Montoya Vozmediano < >> [email protected]> wrote: >> >>>> >> >>>> Hi Larry, >> >>>> >> >>>> OIIO supports output to multipart EXR images, and they have to be >> written out like this: >> >>>> >> >>>> open(multipart file with n parts) >> >>>> write_tiles(...) >> >>>> >> >>>> open(...AppendSubimage) >> >>>> write_tiles(...) >> >>>> >> >>>> open(...AppendSubimage) >> >>>> write_tiles(...) >> >>>> >> >>>> close() >> >>>> >> >>>> So you write out the first subimage, then the next, etc... >> >>>> >> >>>> The motivation for this email is that with this setup, when you are >> rendering, you need to keep all the tiles in memory before they can be >> written out. >> >>>> >> >>>> OpenEXR itself lets you write out tiles for any part in any order. >> >>>> >> >>>> So, when using the EXR API directly, a renderer could write out >> tiles as soon as they are done, and the physical layout of the multipart >> image would end up looking like this: >> >>>> >> >>>> [chunk offset index] >> >>>> [part 0, data for tile (0,0)] >> >>>> [part 1, data for tile (0,0)] >> >>>> [part 2, data for tile (0,0)] >> >>>> ... >> >>>> [part 0, data for tile (0,1)] >> >>>> [part 1, data for tile (0,1)] >> >>>> [part 2, data for tile (0,1)] >> >>>> ... >> >>>> >> >>>> It looks like supporting this means extending OIIO's API, so that >> you can set the part you want to write to, without having to close/open a >> "subimage". >> >>>> >> >>>> Another way to look at this conceptually, is to consider an EXR part >> as a channel grouping rather than a tiff style subimage. >> >>>> >> >>>> Have you considered extending OIIO's API in this direction? >> >>>> >> >>>> Best, >> >>>> >> >>>> r >> >>>> >> >>> >> >>> -- >> >>> Larry Gritz >> >>> [email protected] >> >>> >> >>> >> >>> _______________________________________________ >> >>> Oiio-dev mailing list >> >>> [email protected] >> >>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> >> >> >> _______________________________________________ >> >> Oiio-dev mailing list >> >> [email protected] >> >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> > >> > -- >> > Larry Gritz >> > [email protected] >> > >> > >> > _______________________________________________ >> > Oiio-dev mailing list >> > [email protected] >> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> >> -- >> Larry Gritz >> [email protected] >> >> >> _______________________________________________ >> Oiio-dev mailing list >> [email protected] >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > >
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
