Can't speak to multipart at all but there has always been discussion on the list about monolithic multichannel files not being faster in traditional networked workflows. I"m sure I can find the archives and repost those results. (pretty sure actually data)
Randy S. Little http://www.rslittle.com/ http://www.imdb.com/name/nm2325729/ On Fri, Sep 11, 2015 at 3:38 PM, Nathan Rusch <[email protected]> wrote: > It has to *open* the file, yes, but opening does not imply reading. > > One of the main points of a multi-part file is to allow discrete access to > each part. As far as I know, hardly any renderers currently support writing > multi-part files though, and your 3D guys may not know how to set them up > even if they could, so chances are you're looking at 1.x files. > > To partially echo Thorsten, I think in theory multi-part files should be > faster than separate file sequences (I say "in theory" because I haven't > done any benchmarking of my own). > > Multiple file sequences generally beat monolithic single-part (i.e. > multi-layer EXR 1.x) files because of Nuke's selective channel access > behavior. In other words, you are rarely reading all of your AOVs at the > same time and same place in the node tree, but you are still paying the > decompression/read cost for all of them whenever one is read. Incremental > improvements to Nuke's caching system may have improved the situation with > mutli-layer 1.x files somewhat though. > > On the other side, using multiple sequences trades the > "read-and-decompress-everything" requirement for the overhead of requiring > Nuke to at least open and read each file's header in order to determine > what channels are available (regardless of what channels are actually being > read). If you use network storage like most people, this overhead can > increase quite a bit due to latency, cluster cache eviction, etc. However, > it's still generally better than the alternative. > > On paper, multi-part files should ideally give you the best of both > worlds: encapsulation alongside piecemeal data access. To inspect the > available channels, Nuke only needs to open a single file, though it will > still need to read the header for each part (I'm assuming there's a jump > table at the top of the file, but I'm not sure). You can then also seek to > and read the data for any one layer without touching data for any others > (assuming each is stored in its own part). This relies on smart reader > plugin design on the Nuke side, as well as support for parallel access to > different parts in the same file. > > -Nathan > > > *From:* Randy Little <[email protected]> > *Sent:* Friday, September 11, 2015 2:57 PM > *To:* Nuke user discussion <[email protected]> > *Subject:* Re: [Nuke-users] Multipart multichannel exrs vs. separate exrs > > Has to load the file to read it always. Can't just load part of a little > endian file pretty sure. I know here I can load a beauty pass and scrub > it and play around with it and do all kinds of fun things. When some 3d guy > decides to put 50 passes (10-15) it take longer to load and they are don't > scrub. > > Randy S. Little > http://www.rslittle.com/ > http://www.imdb.com/name/nm2325729/ > > > > On Fri, Sep 11, 2015 at 1:53 PM, Thorsten Kaufmann < > [email protected]> wrote: > >> This is not really true with multipart EXR files anymore. If files are >> interleaved by layer rathern than by scanline you loose the benefits but >> gain performance compared to multiple sequences. I have yet to do a full >> performance check but the boost seems to be rather big if you do not use >> all channels (if you use all channels it should not matter anyways, should >> it). >> >> >> >> I expect multichannel to still have some overhead maybe, but given the >> downsides of splitting exrs (too many open file handles anyone?) This is a >> use case decicsion i would say. >> >> >> >> Cheers, >> >> Thorsten >> >> >> >> --- >> Thorsten Kaufmann >> Production Pipeline Architect >> >> Mackevision Medien Design GmbH >> Forststraße 7 >> 70174 Stuttgart >> >> T +49 711 93 30 48 606 >> F +49 711 93 30 48 90 >> M +49 151 19 55 55 02 >> >> [email protected] >> www.mackevision.com >> >> Geschäftsführer: Armin Pohl, Joachim Lincke >> HRB 243735 Amtsgericht Stuttgart >> >> --- >> *VFX:* Game of Thrones, Season 5 – VFX making of reel >> <https://vimeo.com/133433110>. >> *TWITTER | ADOBE BEHANCE:* Follow us on Twitter >> <https://twitter.com/Mackevision> and Adobe Behance >> <https://www.behance.net/mackevision>. >> ------------------------------ >> *Von:* [email protected] < >> [email protected]> im Auftrag von Randy Little >> <[email protected]> >> *Gesendet:* Freitag, 11. September 2015 22:06 >> *An:* Nuke user discussion >> *Betreff:* Re: [Nuke-users] Multipart multichannel exrs vs. separate exrs >> >> big huge all passes in a exr is slow. Has to read entire file (75MB+) >> to find the channel you want that might be 200KB every single time you want >> to deal with that little mask channel. They also seem to take longer to >> render out of 3d. We usually break them up by math type and try not to >> stuff 20 passes into a file. >> >> Randy S. Little >> http://www.rslittle.com/ >> http://www.imdb.com/name/nm2325729/ >> >> >> >> On Fri, Sep 11, 2015 at 2:31 AM, Johannes Hezer <[email protected]> >> wrote: >> >>> Hey everyone, >>> >>> A bit of a survey question... Is anyone using multipart exrs ? >>> We have been using them for 2 projects and we have not done any >>> performance profiling, but I am not 100% sure if it is a speed bost or not, >>> compared to multichannel 1.xxx exrs. >>> I was hoping might come close to splitted exrs (each layer in a single >>> file) ... >>> >>> Looking forward to some input >>> >>> Cheers >>> Johannes >>> >>> Von meinem iPhone gesendet >>> >>> ____ ESET 12236 (20150911) ____ >>> The message was checked by ESET Mail Security. >>> >>> _______________________________________________ >>> Nuke-users mailing list >>> [email protected], http://forums.thefoundry.co.uk/ >>> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >>> >> >> >> _______________________________________________ >> Nuke-users mailing list >> [email protected], http://forums.thefoundry.co.uk/ >> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >> > > > ------------------------------ > _______________________________________________ > Nuke-users mailing list > [email protected], http://forums.thefoundry.co.uk/ > http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users > > > _______________________________________________ > Nuke-users mailing list > [email protected], http://forums.thefoundry.co.uk/ > http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users >
_______________________________________________ Nuke-users mailing list [email protected], http://forums.thefoundry.co.uk/ http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-users
