By the way, for those following along, if you're wondering how to make oiiotool
take these very same shortcuts:
oiiotool -i:ch=R,G,B:type=uint8:now=1 foo.exr ...
> On May 14, 2022, at 10:59 AM, Larry Gritz <l...@larrygritz.com> wrote:
>
> I think that the problem here is that the ImageBuf will read in all the
> channels, even though the get_pixels only copies out the 3 channels you need.
> The I/O itself happens "lazily", triggered by the get_pixels call itself
> needing pixel values, but at that point, it still doesn't know that you won't
> subsequently try to access other channels in the ImageBuf, so the channel
> restriction of get_pixels doesn't turn into a channel restriction for how the
> file is read and stored.
>
> Additionally, the ImageBuf will by default store the pixels in whatever the
> "widest" data type of the file itself has (in the case of an OpenEXR, that
> would mean half at best, and maybe float depending on what's in your file),
> even though ultimately you only want UINT8 data in your program. (And if the
> file is large, there's also probably yet another copy, or partial copy, in
> the underlying ImageCache.)
>
> But I think you can fix all of these issues by doing an explicit
> ImageBuf::read() before the first time you access the pixels, using the
> version of read() that lets you select the channel range and data type:
>
> buf.read(/*subimage*/ 0, /*miplevel*/ 0,
> /*chbegin*/ 15, /*chend*/ 18,
> /*forceread*/ true, /*convert*/ TypeDesc::UINT8);
> // ... then ...
>
> // BEWARE! now channels 15... of the file are stored in 0... of the buf!
> ROI roi(0, xres, 0, yres, 0, 1, /*chans:*/ 0, 4);
> buf.get_pixels(roi, TypeDesc::UINT8, &pixels[0]);
>
> Now the buf will internally store only 3 channels of UINT8.
>
> See where I passed true for the "forceread" parameter? That forces it to read
> the image right then, skipping use of the ImageCache and storing the entire
> image in memory (well, the channels you asked for, I mean). So the ImageBuf
> itself holds the channels you asked for, without an extra copy in the
> ImageCache. The ImageCache is usually very helpful, especially if you only
> need a small portion of a large image at any one time by one thread. The one
> case where it's wasteful is if you need an entire image all at once, and it
> can fit into memory so there is no benefit to reading it piece by piece only
> as needed.
>
> And given that we have the whole image loaded into the ImageBuf without any
> ImageCache backing, it's possible that you can do one more better thing,
> depending on how you use the data. With the forced read, the ImageBuf itself
> holds channels 15,16,17 as UINT8, contiguously in memory starting at address
> localpixels(). So do you really need the pixels to be copied (and stored
> again) in your std::vector? Or can you directly use the ImageBuf's internal
> storage of the pixels?
>
> You can test if an ImageBuf's pixels are all in memory:
>
> if (buf.storage() == ImageBuf::LOCALBUFFER)
> address = (const uint8*) buf.localpixels();
>
> Actually, I think localpixels() will return nullptr if it doesn't use local
> storage, so you don't need the separate check of storage(), but I still
> wanted to point that out.
>
> (I think everything I have written is true. Please double check it.)
>
>
>> On May 14, 2022, at 8:28 AM, ████刮 <15952002...@qq.com
>> <mailto:15952002...@qq.com>> wrote:
>>
>> Hello.
>> I'm trying to load OpenEXR sequence to memory then draw them with Opengl.
>> but these EXR files are so big with many sub-channels in(one file with 300M
>> and more than 30 channels).
>> i just want to load the "Beauty" pass.
>>
>> here is what i was tried:
>>
>> ImageBuf buf = ImageBuf::ImageBuf(exr_filepath);
>> const ImageSpec& spec = buf.spec();
>> int xres = spec.width;
>> int yres = spec.height;
>>
>> // i just want to load the RGB data from the "Beauty" pass
>> std::vector<UINT8> pixels(xres * yres * 3);
>>
>> // for example the Beauty pass start with index of 15
>> ROI roi(0, xres, 0, yres, 0, 1, /*chans:*/ 15, 18);
>> buf.get_pixels(roi, TypeDesc::UINT8, &pixels[0]);
>>
>> it works. but cost a huge amount of memory use with
>> multi-threading(sometimes more than 80G) during loading all files.
>> here is my quesion:
>> like what i did. is this way to load all data together then save the "Beauty
>> data" from them to the std::vector?
>> or just load the "Beauty" pass without cache all exr data first?
>> is this memory usage normal?
>>
>> and i have checked the document. it looks like the ImageCache class is what
>> i want. can i use this class in my situation?
>> is there anywhere i can find more example about this class?
>>
>> thanks!
>> kong
>>
>> _______________________________________________
>> Oiio-dev mailing list
>> Oiio-dev@lists.openimageio.org <mailto:Oiio-dev@lists.openimageio.org>
>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
>
> --
> Larry Gritz
> l...@larrygritz.com <mailto:l...@larrygritz.com>
>
>
>
>
>
> _______________________________________________
> Oiio-dev mailing list
> Oiio-dev@lists.openimageio.org
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
--
Larry Gritz
l...@larrygritz.com
_______________________________________________
Oiio-dev mailing list
Oiio-dev@lists.openimageio.org
http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
