As a 3D lighter I prefer having it split into multiple files. Sure it's nice to deliver one big exr to comp because then you know all the available channels will show up in nuke. Easy and quick for compositor to bring into the app. Neat.
But I see a few drawbacks, at least with the way the compositors used the resulting passes. Some would just use the beauty pass and then maybe add/subtract some lighting passes or use a mask or two. Making the majority of the passes redundant But since they were still included, because they might be nice to have, they mostly just caused a strain on the network. Albeit this was a few years ago, and exr might have improved since. But even so, when the compositor requested a new special pass. Like a second more broad ambient occlusion pass. It became a bit of a hassle to copy that one pass into the previously delivered main file in nuke and then write out a new main file. I know this I deviating from the original topic, which is faster, multipart or monolithic? And the answer is of course "it depends". Does the compositor use all the passes and does the monolith only include the very essentials then that may be the faster route. Do you include a lot of masks and special utility passes, then size of the file grows and the load on the network unnecessary. Excuse the long post, and for possibly just stating the obvious. Den 12 sep 2015 2:06 fm skrev "Randy Little" <[email protected]>: > was Just saying if someone would like I think there has been in depth > discussion about it I believe. Wasn't meaning to contradict you if it came > off like that. 3 weeks no days off shortest day 13 hours brain left last > week. > > Randy S. Little > http://www.rslittle.com/ > http://www.imdb.com/name/nm2325729/ > > > > On Fri, Sep 11, 2015 at 4:37 PM, Nathan Rusch <[email protected]> > wrote: > >> Yup, and that's part of what I said. I just wanted to point out that >> multiple file sequences *does* introduce its own form of overhead, but it's >> still almost always the better option. >> >> >> *From:* Randy Little <[email protected]> >> *Sent:* Friday, September 11, 2015 4:29 PM >> *To:* Nuke user discussion <[email protected]> >> *Subject:* Re: [Nuke-users] Multipart multichannel exrs vs. separate exrs >> >> Can't speak to multipart at all but there has always been discussion on >> the list about monolithic multichannel files not being faster in >> traditional networked workflows. I"m sure I can find the archives and >> repost those results. (pretty sure actually data) >> >> Randy S. Little >> http://www.rslittle.com/ >> http://www.imdb.com/name/nm2325729/ >> >> >> >> On Fri, Sep 11, 2015 at 3:38 PM, Nathan Rusch <[email protected]> >> wrote: >> >>> It has to *open* the file, yes, but opening does not imply reading. >>> >>> One of the main points of a multi-part file is to allow discrete access >>> to each part. As far as I know, hardly any renderers currently support >>> writing multi-part files though, and your 3D guys may not know how to set >>> them up even if they could, so chances are you're looking at 1.x files. >>> >>> To partially echo Thorsten, I think in theory multi-part files should be >>> faster than separate file sequences (I say "in theory" because I haven't >>> done any benchmarking of my own). >>> >>> Multiple file sequences generally beat monolithic single-part (i.e. >>> multi-layer EXR 1.x) files because of Nuke's selective channel access >>> behavior. In other words, you are rarely reading all of your AOVs at the >>> same time and same place in the node tree, but you are still paying the >>> decompression/read cost for all of them whenever one is read. Incremental >>> improvements to Nuke's caching system may have improved the situation with >>> mutli-layer 1.x files somewhat though. >>> >>> On the other side, using multiple sequences trades the >>> "read-and-decompress-everything" requirement for the overhead of requiring >>> Nuke to at least open and read each file's header in order to determine >>> what channels are available (regardless of what channels are actually being >>> read). If you use network storage like most people, this overhead can >>> increase quite a bit due to latency, cluster cache eviction, etc. However, >>> it's still generally better than the alternative. >>> >>> On paper, multi-part files should ideally give you the best of both >>> worlds: encapsulation alongside piecemeal data access. To inspect the >>> available channels, Nuke only needs to open a single file, though it will >>> still need to read the header for each part (I'm assuming there's a jump >>> table at the top of the file, but I'm not sure). You can then also seek to >>> and read the data for any one layer without touching data for any others >>> (assuming each is stored in its own part). This relies on smart reader >>> plugin design on the Nuke side, as well as support for parallel access to >>> different parts in the same file. >>> >>> -Nathan >>> >>> >>> *From:* Randy Little <[email protected]> >>> *Sent:* Friday, September 11, 2015 2:57 PM >>> *To:* Nuke user discussion <[email protected]> >>> *Subject:* Re: [Nuke-users] Multipart multichannel exrs vs. separate >>> exrs >>> >>> Has to load the file to read it always. Can't just load part of a >>> little endian file pretty sure. I know here I can load a beauty pass and >>> scrub it and play around with it and do all kinds of fun things. When some >>> 3d guy decides to put 50 passes (10-15) it take longer to load and they are >>> don't scrub. >>> >>> Randy S. Little >>> http://www.rslittle.com/ >>> http://www.imdb.com/name/nm2325729/ >>> >>> >>> >>> On Fri, Sep 11, 2015 at 1:53 PM, Thorsten Kaufmann < >>> [email protected]> wrote: >>> >>>> This is not really true with multipart EXR files anymore. If files are >>>> interleaved by layer rathern than by scanline you loose the benefits but >>>> gain performance compared to multiple sequences. I have yet to do a full >>>> performance check but the boost seems to be rather big if you do not use >>>> all channels (if you use all channels it should not matter anyways, should >>>> it). >>>> >>>> >>>> >>>> I expect multichannel to still have some overhead maybe, but given the >>>> downsides of splitting exrs (too many open file handles anyone?) This is a >>>> use case decicsion i would say. >>>> >>>> >>>> >>>> Cheers, >>>> >>>> Thorsten >>>> >>>> >>>> >>>> --- >>>> Thorsten Kaufmann >>>> Production Pipeline Architect >>>> >>>> Mackevision Medien Design GmbH >>>> Forststraße 7 >>>> 70174 Stuttgart >>>> >>>> T +49 711 93 30 48 606 >>>> F +49 711 93 30 48 90 >>>> M +49 151 19 55 55 02 >>>> >>>> [email protected] >>>> www.mackevision.com >>>> >>>> Geschäftsführer: Armin Pohl, Joachim Lincke >>>> HRB 243735 Amtsgericht Stuttgart >>>> >>>> --- >>>> *VFX:* Game of Thrones, Season 5 – VFX making of reel >>>> <https://vimeo.com/133433110>. >>>> *TWITTER | ADOBE BEHANCE:* Follow us on Twitter >>>> <https://twitter.com/Mackevision> and Adobe Behance >>>> <https://www.behance.net/mackevision>. >>>> ------------------------------ >>>> *Von:* [email protected] < >>>> [email protected]> im Auftrag von Randy >>>> Little <[email protected]> >>>> *Gesendet:* Freitag, 11. September 2015 22:06 >>>> *An:* Nuke user discussion >>>> *Betreff:* Re: [Nuke-users] Multipart multichannel exrs vs. separate >>>> exrs >>>> >>>> big huge all passes in a exr is slow. Has to read entire file (75MB+) >>>> to find the channel you want that might be 200KB every single time you want >>>> to deal with that little mask channel. They also seem to take longer to >>>> render out of 3d. We usually break them up by math type and try not to >>>> stuff 20 passes into a file. >>>> >>>> Randy S. Little >>>> http://www.rslittle.com/ >>>> http://www.imdb.com/name/nm2325729/ >>>> >>>> >>>> >>>> On Fri, Sep 11, 2015 at 2:31 AM, Johannes Hezer < >>>> [email protected]> wrote: >>>> >>>>> Hey everyone, >>>>> >>>>> A bit of a survey question... Is anyone using multipart exrs ? >>>>> We have been using them for 2 projects and we have not done any >>>>> performance profiling, but I am not 100% sure if it is a speed bost or >>>>> not, >>>>> compared to multichannel 1.xxx exrs. >>>>> I was hoping might come close to splitted exrs (each layer in a single >>>>> file) ... >>>>> >>>>> Looking forward to some input >>>>> >>>>> Cheers >>>>> Johannes >>>>> >>>>> Von meinem iPhone gesendet >>>>> >>>>> ____ ESET 12236 (20150911) ____ >>>>> The message was checked by ESET Mail Security. >>>>> >>>>> _______________________________________________ >>>>> Nuke-users mailing list >>>>> [email protected], http://forums.thefoundry.co.uk/ >>>>> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Nuke-users mailing list >>>> [email protected], http://forums.thefoundry.co.uk/ >>>> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >>>> >>> >>> ------------------------------ >>> _______________________________________________ >>> Nuke-users mailing list >>> [email protected], http://forums.thefoundry.co.uk/ >>> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >>> >>> _______________________________________________ >>> Nuke-users mailing list >>> [email protected], http://forums.thefoundry.co.uk/ >>> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >>> >> >> >> ------------------------------ >> _______________________________________________ >> Nuke-users mailing list >> [email protected], http://forums.thefoundry.co.uk/ >> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >> >> >> _______________________________________________ >> Nuke-users mailing list >> [email protected], http://forums.thefoundry.co.uk/ >> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >> > > > _______________________________________________ > Nuke-users mailing list > [email protected], http://forums.thefoundry.co.uk/ > http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >
_______________________________________________ Nuke-users mailing list [email protected], http://forums.thefoundry.co.uk/ http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users
