I think that the problem here is that the ImageBuf will read in all the
channels, even though the get_pixels only copies out the 3 channels you need.
The I/O itself happens "lazily", triggered by the get_pixels call itself
needing pixel values, but at that point, it still doesn't know that you won't
subsequently try to access other channels in the ImageBuf, so the channel
restriction of get_pixels doesn't turn into a channel restriction for how the
file is read and stored.
Additionally, the ImageBuf will by default store the pixels in whatever the
"widest" data type of the file itself has (in the case of an OpenEXR, that
would mean half at best, and maybe float depending on what's in your file),
even though ultimately you only want UINT8 data in your program. (And if the
file is large, there's also probably yet another copy, or partial copy, in the
underlying ImageCache.)
But I think you can fix all of these issues by doing an explicit
ImageBuf::read() before the first time you access the pixels, using the version
of read() that lets you select the channel range and data type:
buf.read(/*subimage*/ 0, /*miplevel*/ 0,
/*chbegin*/ 15, /*chend*/ 18,
/*forceread*/ true, /*convert*/ TypeDesc::UINT8);
// ... then ...
// BEWARE! now channels 15... of the file are stored in 0... of the buf!
ROI roi(0, xres, 0, yres, 0, 1, /*chans:*/ 0, 4);
buf.get_pixels(roi, TypeDesc::UINT8, &pixels[0]);
Now the buf will internally store only 3 channels of UINT8.
See where I passed true for the "forceread" parameter? That forces it to read
the image right then, skipping use of the ImageCache and storing the entire
image in memory (well, the channels you asked for, I mean). So the ImageBuf
itself holds the channels you asked for, without an extra copy in the
ImageCache. The ImageCache is usually very helpful, especially if you only need
a small portion of a large image at any one time by one thread. The one case
where it's wasteful is if you need an entire image all at once, and it can fit
into memory so there is no benefit to reading it piece by piece only as needed.
And given that we have the whole image loaded into the ImageBuf without any
ImageCache backing, it's possible that you can do one more better thing,
depending on how you use the data. With the forced read, the ImageBuf itself
holds channels 15,16,17 as UINT8, contiguously in memory starting at address
localpixels(). So do you really need the pixels to be copied (and stored again)
in your std::vector? Or can you directly use the ImageBuf's internal storage of
the pixels?
You can test if an ImageBuf's pixels are all in memory:
if (buf.storage() == ImageBuf::LOCALBUFFER)
address = (const uint8*) buf.localpixels();
Actually, I think localpixels() will return nullptr if it doesn't use local
storage, so you don't need the separate check of storage(), but I still wanted
to point that out.
(I think everything I have written is true. Please double check it.)
> On May 14, 2022, at 8:28 AM, ████刮 <15952002...@qq.com> wrote:
>
> Hello.
> I'm trying to load OpenEXR sequence to memory then draw them with Opengl.
> but these EXR files are so big with many sub-channels in(one file with 300M
> and more than 30 channels).
> i just want to load the "Beauty" pass.
>
> here is what i was tried:
>
> ImageBuf buf = ImageBuf::ImageBuf(exr_filepath);
> const ImageSpec& spec = buf.spec();
> int xres = spec.width;
> int yres = spec.height;
>
> // i just want to load the RGB data from the "Beauty" pass
> std::vector<UINT8> pixels(xres * yres * 3);
>
> // for example the Beauty pass start with index of 15
> ROI roi(0, xres, 0, yres, 0, 1, /*chans:*/ 15, 18);
> buf.get_pixels(roi, TypeDesc::UINT8, &pixels[0]);
>
> it works. but cost a huge amount of memory use with multi-threading(sometimes
> more than 80G) during loading all files.
> here is my quesion:
> like what i did. is this way to load all data together then save the "Beauty
> data" from them to the std::vector?
> or just load the "Beauty" pass without cache all exr data first?
> is this memory usage normal?
>
> and i have checked the document. it looks like the ImageCache class is what i
> want. can i use this class in my situation?
> is there anywhere i can find more example about this class?
>
> thanks!
> kong
>
> _______________________________________________
> Oiio-dev mailing list
> Oiio-dev@lists.openimageio.org
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
--
Larry Gritz
l...@larrygritz.com
_______________________________________________
Oiio-dev mailing list
Oiio-dev@lists.openimageio.org
http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
