On Friday, January 30, 2015 at 5:00:31 PM UTC-5, Jason Wolfe wrote: Thanks for the recommendation. For now we're looking for a simple > low-level interface to MR, but we're also keeping an eye on parkour and > pigpen for more complex tasks down the road. Can you explain why I might > prefer parkour to pigpen or vice-versa? >
Parkour actually *is* a low-level interface to MR. It just exposes that interface though relatively Clojure-idiomatic and composable abstractions, which can make it look higher-level than it really is. At Parkour's core is the support required for MR tasks to invoke a regular Clojure var-bound function in place of the `.run` method of a `Mapper` or `Reducer` class. Everything else in Parkour is built to make using that primitive, low-level interface more composable, convenient, and pleasant; but ultimately nothing *replaces* that interface -- your Parkour MR Clojure task code runs in exactly the way equivalent raw Hadoop MR Java task code would. Parkour's documentation includes a "motivation" document describing the project motivation in the face of the Clojure-Hadoop integration projects which existed when I started Parkour (including clojure-hadoop): https://github.com/damballa/parkour/blob/master/doc/motivation.md . It doesn't yet cover PigPen, although I certainly should add a section. I honestly haven't evaluated PigPen in detail, but the approach of compiling Clojure code to Pig seems excessively complex to me, to the point of only be worth it for organizations which have already made a significant investment in Pig. -Marshall -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
