Sorry. I meant to answer that. The short answer is that hadoop is often reasonable for this sort of problem, BUT
* if you have lots of little files, you may do better otherwise * if you can't handle batch-oriented, merge-based designs, then map-reduce itself isn't going to help you much * if you need some kind of work-flow, hadoop won't help (but it won't hurt either) As hbase comes of age, then the first two problems will moderate. Adding a simple queuing system like SQS on the side is an easy way to deal with the third issue. In much of my work, I have lots of big files composed of concatenations of little files, I can work in hourly batches and my workflow requirements are minimal so I am very happy with Hadoop. For video file serving, we opted not to go with Hadoop because we have too many files, didn't need map-reduce and our systems guys liked perl better than java (go figure). On 12/21/07 12:24 PM, "John Heidemann" <[EMAIL PROTECTED]> wrote: > > He's not saying "is Hadoop optimal" for things that aren't really > map/reduce, but "is it reasonable" for those things? > (Kirk, is that right?)
