Hey Victor-- Silly question: is there a specific version of gradle I should use? I downloaded 1.3 and ran gradle build inside of crackle-core, and I got an exception on the compileClojure step:
Caused by: java.lang.NoSuchMethodError: org.gradle.util.GUtil.join(Ljava/util/Collection;Ljava/lang/String;)Ljava/lang/String; Sorry for the sporadic feedback; I'm trying to play with this whenever I can find a free moment. Thanks! Josh On Sun, Dec 2, 2012 at 5:33 AM, Victor Iacoban <[email protected]>wrote: > Hi, > > I've change the way crackle users will be defining remote functions, this > allowed me to for nicer and more fluent dsl > I still did not get rid of crunch types influence in crackle dsl but at > least now it's harder to get it wrong and easier to debug > > I think this is pretty close to what I'm trying to achieve. Next steps are > going to be in the direction of documentation, more validation, error > condition checks, tests and crackle hbase. > > here it is, any comments are welcome: > > (ns crackle.example > (:require [crackle.from :as from]) > (:require [crackle.to :as to]) > (:use crackle.core)) > ;====== word count example ===============(fn-mapcat split-words [line] > :strings > (clojure.string/split line #"\s+")) > (defn count-words [input-path output-path] > (pipeline (from/text-file input-path) > (split-words) > (count-values) > (to/text-file output-path))) > ;====== average bytes by ip example ======(fn-map parse-line [line] > [:strings :clojure] > (let [parts (clojure.string/split line #"\s+")] > (pair-of (first parts) [(read-string (second parts)) 1]))) > (fn-combine sum-bytes-and-counts [value1 value2] > [(+ (first value1) (first value2)) (+ (second value1) (second value2))]) > (fn-mapv compute-average [value] :ints > (int (apply / value))) > (defn count-bytes-by-ip [input-path output-path] > (pipeline (from/text-file input-path) > (parse-line) > (group-by-key) > (sum-bytes-and-counts) > (compute-average) > (to/text-file output-path))) > > -- victor > -- Director of Data Science Cloudera <http://www.cloudera.com> Twitter: @josh_wills <http://twitter.com/josh_wills>
