I was using 32x32 tile size (not that I'd expect it to make *that* much difference), and my tests between EXR and TIFF were at full float only (couldn't be bothered to get half working for TIFF).
But I noticed the discrepancy between OpenEXR 1.7 and 2.0 in terms of speed with both half and full float EXRs (zip, 3-channel, 32x32 tile size). I'm assuming you reverted IlmBase back to 1.x as well (not sure if this matters, but...)? Peter On 5 January 2016 at 08:44, Karl Rasche <[email protected]> wrote: > > To throw in another data point; I tried to repro the tiff/exr discrepancy > using the recipe that Larry posted earlier. > > Using local files, 1 thread, gcc 4.8.1, oiio master, and the > system-provided libtiff (3.9.4), I'm *not* able to trip the problem > (provided I'm looking at the right numbers in the testtex output -- I'm > using the first time that is listed next to the thread efficiency). I also > don't see any pref regression (at least with the 1 thread case, which may > be expected) between 1.7.1 and 2.2. > > > * Type (# thr) OpenEXR 2.2 OpenEXR 1.7.1* > exr-half-zip (1) 77.1 s 77.7 s > exr-half-piz (1) 79.6 s 110.1 s > tif-half-zip (1) 75.3 s 75.8 s > > > The PIZ difference between 1.7.1 and 2.2 is expected, since part of the > 2.2 changes included an optimization of the static Huffman decoder used by > PIZ. For tiled reads in 2.2, PIZ @ 64x64 tiles is still a bit slower than > ZIP in other tests I've done, mostly due to the overhead of decoder > initialization (which is negligible for scanline reads, but totally > dominates for tiled reads). Since we're talking performance, I suspect > tiled PIZ could go a fair bit faster with some TLC. But that's another > thread. > > Karl > > > On Mon, Jan 4, 2016 at 9:20 AM, Piotr Stanczyk <[email protected]> > wrote: > >> Thanks for the suggestion. I'll take some time to look at what could >> have caused a regression / change >> >> On 2 January 2016 at 22:28, Larry Gritz <[email protected]> wrote: >> >>> That's a fascinating clue! >>> >>> I'm not using "the latest", but I'm certainly using 2.x (2.1 I think?) >>> because of our need for deep files. It never occurred me to test with 1.x, >>> but it will be easy to go back to up the OIIO-based synthetic benchmark I >>> used for my original findings and rerun it after building against OpenEXR >>> 1.7. >>> >>> This is very hopeful! If it used to be fast to read OpenEXR and now is >>> slow, that means that the speed problem may not be inherent in the file >>> layout itself or the basic architecture of the library, but could just be a >>> performance regression in the library that went undetected and could be >>> fixed. That would really be great for everybody. >>> >>> I'll look into this as soon as I'm back at work on Monday and will >>> report back. >>> >>> Thanks, Peter. >>> >>> -- lg >>> >>> >>> On Jan 2, 2016, at 10:06 PM, Peter Pearson <[email protected]> >>> wrote: >>> >>> Hi, >>> >>> This is really a continuation of Larry's deviation (but a very useful >>> and timely one!) in the '.exr vs .tx' thread - I've only just subscribed to >>> this list, so can't respond directly to that. >>> >>> I've seen pretty much the same issue regarding read speeds, with TIFF >>> being significantly faster (around 1.5-1.7x) in my tests within a renderer >>> - incoherent access for varying mipmap levels (or subimages in TIFF's >>> case), on a local disk on Linux. >>> >>> There's also this bug: >>> https://github.com/openexr/openexr/pull/170 >>> >>> that in my case severely affects concurrent access when using worker >>> threads to do the reading work - without Larry's patch, 3/4 threads is the >>> best I can hope for, after that there's so much contention that CPU usage >>> just dwindles over time - with the patch, I can push 24 threads easily, and >>> wall-clock time goes down by orders of magnitude. >>> >>> Ignoring the above patch for a moment though, I've previously (2/3 years >>> ago) found that at least within PRMan, OpenEXR reading was noticeably >>> faster than TIFF reading, even when reading half for EXR and 8-bit for >>> TIFF, so TIFF technically has an unfair advantage. This was over NFS on a >>> saturated network - on a local disk, TIFF could sometimes win. At the time, >>> profiling (at the NFS level) seemed to indicate that the stat() call done >>> within TIFFSetDirectory() to change mipmap levels caused >>> disproportionate slowdowns for TIFF, as the raw reading and decompression >>> was generally faster than EXR (although EXR compresses more in general, and >>> things like tile size and planar config will obviously affect the >>> comparison one way or another). However, as the studio I was with at the >>> time used PRMan, we were using Pixar's format which was faster and more >>> compact than EXR, so just used Pixar's format instead of investigating >>> further. >>> >>> I've always since assumed based on this that OpenEXR was the better >>> format than TIFF for rendering, but I've recently been doing some work on >>> texture reading for a renderer, and as well as noticing the scalability >>> issue Larry found, have also found that TIFF is faster than OpenEXR is >>> using zip compression - which as far as I'm aware (ignoring DWA type for >>> the moment) is the fastest for tiled? >>> >>> I've done a quick test today with IlmBase 1.0.2 and OpenEXR 1.7.0 just >>> as a "I'm sure it didn't used to be *this* bad" sanity-check, and indeed - >>> as well as the IlmBase ThreadPool bug above not seeming to exist - reading >>> of the same EXRs with 1.7.0 vs 2.0.1 is almost twice as fast, for the >>> wall-clock time of reading them in a renderer. >>> >>> Given that I've tested this in a renderer, it's a bit difficult to give >>> exact timings, but based on wall-clock texture read times, I'm roughly >>> seeing OpenEXR 1.7 read times be 50-70% of the OpenEXR 2.0.1 ones. >>> >>> So it seems (possibly pending more investigation and timings) that there >>> has been a regression in read speeds since OpenEXR 2.0 (and currently, as >>> I'm assuming Larry was testing with the latest OpenEXR version?) >>> >>> Cheers, >>> Peter >>> >>> >>> >>> _______________________________________________ >>> Oiio-dev mailing list >>> [email protected] >>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>> >>> >>> -- >>> Larry Gritz >>> [email protected] >>> >>> >>> >>> _______________________________________________ >>> Oiio-dev mailing list >>> [email protected] >>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>> >>> >> >> _______________________________________________ >> Oiio-dev mailing list >> [email protected] >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> >> > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > >
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
