was Just saying if someone would like I think there has been in depth discussion about it I believe. Wasn't meaning to contradict you if it came off like that. 3 weeks no days off shortest day 13 hours brain left last week.
Randy S. Little http://www.rslittle.com/ http://www.imdb.com/name/nm2325729/ On Fri, Sep 11, 2015 at 4:37 PM, Nathan Rusch <[email protected]> wrote: > Yup, and that's part of what I said. I just wanted to point out that > multiple file sequences *does* introduce its own form of overhead, but it's > still almost always the better option. > > > *From:* Randy Little <[email protected]> > *Sent:* Friday, September 11, 2015 4:29 PM > *To:* Nuke user discussion <[email protected]> > *Subject:* Re: [Nuke-users] Multipart multichannel exrs vs. separate exrs > > Can't speak to multipart at all but there has always been discussion on > the list about monolithic multichannel files not being faster in > traditional networked workflows. I"m sure I can find the archives and > repost those results. (pretty sure actually data) > > Randy S. Little > http://www.rslittle.com/ > http://www.imdb.com/name/nm2325729/ > > > > On Fri, Sep 11, 2015 at 3:38 PM, Nathan Rusch <[email protected]> > wrote: > >> It has to *open* the file, yes, but opening does not imply reading. >> >> One of the main points of a multi-part file is to allow discrete access >> to each part. As far as I know, hardly any renderers currently support >> writing multi-part files though, and your 3D guys may not know how to set >> them up even if they could, so chances are you're looking at 1.x files. >> >> To partially echo Thorsten, I think in theory multi-part files should be >> faster than separate file sequences (I say "in theory" because I haven't >> done any benchmarking of my own). >> >> Multiple file sequences generally beat monolithic single-part (i.e. >> multi-layer EXR 1.x) files because of Nuke's selective channel access >> behavior. In other words, you are rarely reading all of your AOVs at the >> same time and same place in the node tree, but you are still paying the >> decompression/read cost for all of them whenever one is read. Incremental >> improvements to Nuke's caching system may have improved the situation with >> mutli-layer 1.x files somewhat though. >> >> On the other side, using multiple sequences trades the >> "read-and-decompress-everything" requirement for the overhead of requiring >> Nuke to at least open and read each file's header in order to determine >> what channels are available (regardless of what channels are actually being >> read). If you use network storage like most people, this overhead can >> increase quite a bit due to latency, cluster cache eviction, etc. However, >> it's still generally better than the alternative. >> >> On paper, multi-part files should ideally give you the best of both >> worlds: encapsulation alongside piecemeal data access. To inspect the >> available channels, Nuke only needs to open a single file, though it will >> still need to read the header for each part (I'm assuming there's a jump >> table at the top of the file, but I'm not sure). You can then also seek to >> and read the data for any one layer without touching data for any others >> (assuming each is stored in its own part). This relies on smart reader >> plugin design on the Nuke side, as well as support for parallel access to >> different parts in the same file. >> >> -Nathan >> >> >> *From:* Randy Little <[email protected]> >> *Sent:* Friday, September 11, 2015 2:57 PM >> *To:* Nuke user discussion <[email protected]> >> *Subject:* Re: [Nuke-users] Multipart multichannel exrs vs. separate exrs >> >> Has to load the file to read it always. Can't just load part of a >> little endian file pretty sure. I know here I can load a beauty pass and >> scrub it and play around with it and do all kinds of fun things. When some >> 3d guy decides to put 50 passes (10-15) it take longer to load and they are >> don't scrub. >> >> Randy S. Little >> http://www.rslittle.com/ >> http://www.imdb.com/name/nm2325729/ >> >> >> >> On Fri, Sep 11, 2015 at 1:53 PM, Thorsten Kaufmann < >> [email protected]> wrote: >> >>> This is not really true with multipart EXR files anymore. If files are >>> interleaved by layer rathern than by scanline you loose the benefits but >>> gain performance compared to multiple sequences. I have yet to do a full >>> performance check but the boost seems to be rather big if you do not use >>> all channels (if you use all channels it should not matter anyways, should >>> it). >>> >>> >>> >>> I expect multichannel to still have some overhead maybe, but given the >>> downsides of splitting exrs (too many open file handles anyone?) This is a >>> use case decicsion i would say. >>> >>> >>> >>> Cheers, >>> >>> Thorsten >>> >>> >>> >>> --- >>> Thorsten Kaufmann >>> Production Pipeline Architect >>> >>> Mackevision Medien Design GmbH >>> Forststraße 7 >>> 70174 Stuttgart >>> >>> T +49 711 93 30 48 606 >>> F +49 711 93 30 48 90 >>> M +49 151 19 55 55 02 >>> >>> [email protected] >>> www.mackevision.com >>> >>> Geschäftsführer: Armin Pohl, Joachim Lincke >>> HRB 243735 Amtsgericht Stuttgart >>> >>> --- >>> *VFX:* Game of Thrones, Season 5 – VFX making of reel >>> <https://vimeo.com/133433110>. >>> *TWITTER | ADOBE BEHANCE:* Follow us on Twitter >>> <https://twitter.com/Mackevision> and Adobe Behance >>> <https://www.behance.net/mackevision>. >>> ------------------------------ >>> *Von:* [email protected] < >>> [email protected]> im Auftrag von Randy >>> Little <[email protected]> >>> *Gesendet:* Freitag, 11. September 2015 22:06 >>> *An:* Nuke user discussion >>> *Betreff:* Re: [Nuke-users] Multipart multichannel exrs vs. separate >>> exrs >>> >>> big huge all passes in a exr is slow. Has to read entire file (75MB+) >>> to find the channel you want that might be 200KB every single time you want >>> to deal with that little mask channel. They also seem to take longer to >>> render out of 3d. We usually break them up by math type and try not to >>> stuff 20 passes into a file. >>> >>> Randy S. Little >>> http://www.rslittle.com/ >>> http://www.imdb.com/name/nm2325729/ >>> >>> >>> >>> On Fri, Sep 11, 2015 at 2:31 AM, Johannes Hezer <[email protected] >>> > wrote: >>> >>>> Hey everyone, >>>> >>>> A bit of a survey question... Is anyone using multipart exrs ? >>>> We have been using them for 2 projects and we have not done any >>>> performance profiling, but I am not 100% sure if it is a speed bost or not, >>>> compared to multichannel 1.xxx exrs. >>>> I was hoping might come close to splitted exrs (each layer in a single >>>> file) ... >>>> >>>> Looking forward to some input >>>> >>>> Cheers >>>> Johannes >>>> >>>> Von meinem iPhone gesendet >>>> >>>> ____ ESET 12236 (20150911) ____ >>>> The message was checked by ESET Mail Security. >>>> >>>> _______________________________________________ >>>> Nuke-users mailing list >>>> [email protected], http://forums.thefoundry.co.uk/ >>>> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >>>> >>> >>> >>> _______________________________________________ >>> Nuke-users mailing list >>> [email protected], http://forums.thefoundry.co.uk/ >>> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >>> >> >> ------------------------------ >> _______________________________________________ >> Nuke-users mailing list >> [email protected], http://forums.thefoundry.co.uk/ >> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >> >> _______________________________________________ >> Nuke-users mailing list >> [email protected], http://forums.thefoundry.co.uk/ >> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >> > > > ------------------------------ > _______________________________________________ > Nuke-users mailing list > [email protected], http://forums.thefoundry.co.uk/ > http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users > > > _______________________________________________ > Nuke-users mailing list > [email protected], http://forums.thefoundry.co.uk/ > http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >
_______________________________________________ Nuke-users mailing list [email protected], http://forums.thefoundry.co.uk/ http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users
