Thanks for releasing this, I personally had to re-invent such functionality over clojure-hadoop
Did you happen to test this over AWS EMR? On Monday, November 4, 2013 3:55:23 PM UTC+2, Marshall Bockrath-Vandegrift wrote: > > I’m pleased to announce the first public release of Parkour, a library > for writing Hadoop MapReduce applications in idiomatic Clojure. Parkour > takes your Clojure code’s functional gymnastics and sends it > free-running across the urban environment of your Hadoop cluster. > > https://github.com/damballa/parkour/ > > Parkour aims to provide deep Clojure integration for Hadoop. Programs > using Parkour are normal Clojure programs, using standard Clojure > functions instead of new framework abstractions. Programs using Parkour > are also full Hadoop programs, with complete access to absolutely > everything possible in raw Java Hadoop MapReduce. If you know Clojure, > and you know Hadoop, then you’re most of the way to knowing Parkour. > > Here is the core of the obligatory “word count” MapReduce program, > written using Parkour: > > (defn mapper > [conf] > (fn [context input] > (->> (mr/vals input) > (r/mapcat #(str/split % #"\s+")) > (r/map #(-> [% 1]))))) > > (defn reducer > [conf] > (fn [context input] > (->> (mr/keyvalgroups input) > (r/map (fn [[word counts]] > [word (r/reduce + 0 counts)]))))) > > (defn word-count > [dseq dsink] > (-> (pg/input dseq) > (pg/map #'mapper) > (pg/partition [Text LongWritable]) > (pg/combine #'reducer) > (pg/reduce #'reducer) > (pg/output dsink))) > > Parkour includes detailed documentation, ranging from a quickstart > introduction through detailed discussions of several specific aspects: > > https://github.com/damballa/parkour/#documentation > > Although this is the first public release of Parkour, the Damballa R&D > team has been using it extensively since beginning serious development > earlier this year. We do also use and will continue to use Cascalog, > but we’ve found that Parkour’s simpler model and more direct Hadoop > integration is a better fit for many problems. > > I am personally incredibly excited about this release. I will be at > this year’s Clojure/conj, and will be more than happy to discuss Parkour > in detail with those interested. > > Questions and pull requests welcome! > > -- > Marshall Bockrath-Vandegrift <lla...@damballa.com <javascript:>> > Principal Software Engineer, Damballa R&D > > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
