Thanks, Larry! My EXR textures are already mipmapped and tiled so I don't think we're using any extra memory on auto-tile, but it certainly was true that we were using OIIO with the default 100 file handle cache size. We'll give it a go with a much higher value.
I'm still surprised that libIlmImf requires that much more memory. If our handle cache size was limited to 100 files at a time, that's an extra 38.75 MB per texture. Even if it's caching all 1957 texture handles at once that's still an extra 1.98 MB per texture. On Fri, Sep 16, 2016 at 5:56 PM, Larry Gritz <l...@larrygritz.com> wrote: > See how there were 1957 unique textures, but you created 144168 > ImageInputs? That means your file (handle) cache was too small, you were > accessing the files in an incoherent order and the "working set" was much > larger than the file cache size (100), so you ended up closing and > reopening files a massive number of times (73 times each, on average). > > The solution to this is to raise the TextureSystem's (that is, the > underlying ImageCache's) "max_open_files" significantly. The default of 100 > is extremely conservative -- trying to keep you out of trouble on operating > systems with a low handle count per process, or with ImageCache's file > operations being potentially only a small portion of your process's file > needs. > > On a modern Linux system, you should be able to have several thousand > files open simultaneously. I would do that and see what it does to all your > figures. At the very least, it should substantially lower the "file open > time" and maybe the file I/O time overall. > > But, it may make the memory issues worse -- if libIlmImf is internally > holding more memory per open file than libtiff does (my tests indicate this > as well), then having more files open at a time may exacerbate the problem. > I haven't tested this for a while, though, but I think that Karl is correct > that part of this is that each of the threads in OpenEXR's pool is holding > a bit of working memory, and that adds up, so the higher your thread count, > the more memory use overhead there will be for libIlmImf, and it's very > hard for the app to account for that or control it. > > It is simply a fact that on a per-thread, per-file basis, libIlmImf uses > more overhead memory per open file than libtiff does. I have not looked > into it deeply enough to know if that's a good design tradeoff or not (for > example, maybe that memory is put to good use, and helps speed up some > operations). > > This is the kind of thing you would never notice reading OpenEXR files > sequentially, but in something like a texture system where you may have > thousands of files open simultaneously, an extra 256KB of overhead per file > adds up fast. > > > > On Sep 16, 2016, at 3:34 AM, Søren Ragsdale <sor...@gmail.com> wrote: > > > > Hello, OpenEXR devs. I've been doing some comparative rendering tests > I've found something a bit surprising. > > > > TIFF and EXR texture access *times* seems more or less the same, which > is fine because the underlying data is equivalent. (Same data type, > compression, tile size, etc.) But the RAM overhead seems much higher for > EXRs. We've got a 9GB render using TIFFs and a 13GB render using EXRs. > > > > Does anyone have some theories why EXR texture access is requiring 4GB > more memory? > > > > > > Prman-20.11, OSL shaders, OIIO/TIFF textures: > > real 00:21:46 > > VmRSS 9,063.45 MB > > OpenImageIO ImageCache statistics (shared) ver 1.7.3dev > > Options: max_memory_MB=4000.0 max_open_files=100 autotile=64 > > autoscanline=0 automip=1 forcefloat=0 accept_untiled=1 > > accept_unmipped=1 read_before_insert=0 deduplicate=1 > > unassociatedalpha=0 failure_retries=0 > > Images : 1957 unique > > ImageInputs : 136432 created, 100 current, 796 peak > > Total size of all images referenced : 166.0 GB > > Read from disk : 55.5 GB > > File I/O time : 7h 2m 33.9s (16m 54.2s average per thread) > > File open time only : 27m 44.0s > > > > > > Prman-20.11, OSL shaders, OIIO/EXR textures: > > real 00:21:14 > > VmRSS 12,938.83 MB > > OpenImageIO ImageCache statistics (shared) ver 1.7.3dev > > Options: max_memory_MB=4000.0 max_open_files=100 autotile=64 > > autoscanline=0 automip=1 forcefloat=0 accept_untiled=1 > > accept_unmipped=1 read_before_insert=0 deduplicate=1 > > unassociatedalpha=0 failure_retries=0 > > Images : 1957 unique > > ImageInputs : 133168 created, 100 current, 771 peak > > Total size of all images referenced : 166.0 GB > > Read from disk : 55.5 GB > > File I/O time : 6h 15m 42.1s (15m 1.7s average per thread) > > File open time only : 1m 22.5s > > > > _______________________________________________ > > Openexr-devel mailing list > > Openexr-devel@nongnu.org > > https://lists.nongnu.org/mailman/listinfo/openexr-devel > > -- > Larry Gritz > l...@larrygritz.com > > >
_______________________________________________ Openexr-devel mailing list Openexr-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/openexr-devel
