On Tue, 16 Oct 2007 16:37:41 BST, Robert Jessop wrote: >Hi there. I did a search of the mailing list archives looking for >something similar to this, but I didn't find anything so apologies if >this has been discussed before. > >I'm investigating using Hadoop for distributed rendering. The Mapper >would define the tiles to be rendered and the nodes would render them >using the scene data (which is for the sake of argument, all wrapped >up in one big binary file on the HDFS). The reducer would take the >output tiles and stitch them together to form the final image. I have >a system that does this already but it doesn't have any of the >advantages of a distributed file system, there are lots of IO >bottlenecks and communication overheads. All the code is currently in >C++. > >Does this sound like a good use for Hadoop? >-Rob >
We did almost exactly this with our whole Internet map (http://www.isi.edu/ant/address/whole_internet/). By "almost": we rendered the tiles in the reduce phase, but wrote them out-of-band (direct to the file system), rather than as direct reduce output. Since we were writing pngs, I didn't want to have to decapsulate them from the reducer output stream. We were using hadoop streaming; perhaps this goes away if we had directly written a Java reducer with custom output, but that seems unnecessary. We then did the stitching together on a single workstation after the fact. About 2.8 billion records in through a custom map IO function, with ~43k tiles output over ~19 hours on 52 cores. Then maybe 4 hours stiching things together on a single box, and ~36 hours printing on a single printer. (By the way, if anyone knows how to stream huge images at an HP Designjet 800PS printer without blowing up memory, please let me know. The postscript redbook commands to control paper advance and non-cutting seem to be ignored.) You don't say what kind of rendering you're doing. If it's movie-style 3-D rendering, I expect you'd need some work to get benefit from HDFS or other file systems---the input needed for rendering (compositing, textures, models, etc.) is not an obvious fit for map/reduce, at least to me. -John Heidemann
