On Mon, Sep 5, 2011 at 11:47 AM, Lance Norskog <[email protected]> wrote:
> Haha. Part of the abuse is "Google envy". (Google "Sigmund Freud" to fully > understand this.) > > I'm finding inherent difficulty in documenting map/reduce code, and > assimilating an existing job. Haven't seen a "UML for Map/Reduce" yet; > Hamake is the cleanest "everything in one file" description, and it only > stores half of what's going on. > I think Pig deserves a lot of credit for being the closest to a real high-level language for M/R. It's still funky to write these things in M/R, even with that level of help. > > Mahout's "in-memory" code is all single-threaded, and is bifurcated from > the > map/reduce versions. A few places have custom multi-threading shoehorned > in. > You can't buy a stationary single-processor computer. We bought an 8-core > server 1.5 years ago for under 5 grand. We can't easily write > multi-processor java for it. If Mahout wants to stay M/R focused it could > Well, the idea is that the parallelism comes one level above. You can handle N simultaneous requests from callers at once on N cores. I think this is a pretty good theory.
