Re: Accumulate results without state?
I rewrote it using reduce as Jarrod suggested. It looks good to my newbie eyes now, but I'm interested in any other feedback. (defn failures [file] (with-open [rdr (clojure.java.io/reader file)] (reduce (fn [accu input] (if (re-matches #^[^,]+,[^,]+,FAIL,.*$ input) (conj accu (second (string/split input #,))) accu)) [] (line-seq rdr Dean On Tuesday, March 4, 2014 4:21:47 PM UTC-5, Dean Laskin wrote: I'm comparing two large files based on specific fields in the files. Is there a functional way of accumulating the results without atoms? (And since I'm a newbie, any other advice is appreciated!) (let [sun (atom []) fri (atom [])] (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! sun conj (nth (string/split line #,) 1) (with-open [rdr (clojure.java.io/reader /dev/errors-friday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! fri conj (nth (string/split line #,) 1) (println (nth (data/diff (set @fri) (set @sun)) 1)) ) Thanks, Dean -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Accumulate results without state?
I'm comparing two large files based on specific fields in the files. Is there a functional way of accumulating the results without atoms? (And since I'm a newbie, any other advice is appreciated!) (let [sun (atom []) fri (atom [])] (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! sun conj (nth (string/split line #,) 1) (with-open [rdr (clojure.java.io/reader /dev/errors-friday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! fri conj (nth (string/split line #,) 1) (println (nth (data/diff (set @fri) (set @sun)) 1)) ) Thanks, Dean -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Accumulate results without state?
Something like (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (- (line-seq rdr) (filter #(re-matches #...)) (map #(nth (string/split % #,) 1 Will do what you want. It's laziness to the rescue. On Tue, Mar 4, 2014 at 3:21 PM, Dean Laskin wdlas...@gmail.com wrote: I'm comparing two large files based on specific fields in the files. Is there a functional way of accumulating the results without atoms? (And since I'm a newbie, any other advice is appreciated!) (let [sun (atom []) fri (atom [])] (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! sun conj (nth (string/split line #,) 1) (with-open [rdr (clojure.java.io/reader /dev/errors-friday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! fri conj (nth (string/split line #,) 1) (println (nth (data/diff (set @fri) (set @sun)) 1)) ) Thanks, Dean -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Accumulate results without state?
Awesome, thanks Jason! It looks much cleaner. Dean On Tuesday, March 4, 2014 4:47:12 PM UTC-5, Jason Felice wrote: Something like (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (- (line-seq rdr) (filter #(re-matches #...)) (map #(nth (string/split % #,) 1 Will do what you want. It's laziness to the rescue. On Tue, Mar 4, 2014 at 3:21 PM, Dean Laskin wdla...@gmail.comjavascript: wrote: I'm comparing two large files based on specific fields in the files. Is there a functional way of accumulating the results without atoms? (And since I'm a newbie, any other advice is appreciated!) (let [sun (atom []) fri (atom [])] (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! sun conj (nth (string/split line #,) 1) (with-open [rdr (clojure.java.io/reader /dev/errors-friday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! fri conj (nth (string/split line #,) 1) (println (nth (data/diff (set @fri) (set @sun)) 1)) ) Thanks, Dean -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.comjavascript: Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com javascript: For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Accumulate results without state?
Your welcome! I just realized that, in this case, filter and map are good enough, but the general answer for threading state through the processing of a sequence is to use `reduce`. Coming from another language, it takes a while to realize how often the answer is to use reduce. -Jason On Tue, Mar 4, 2014 at 4:07 PM, Dean Laskin wdlas...@gmail.com wrote: Awesome, thanks Jason! It looks much cleaner. Dean On Tuesday, March 4, 2014 4:47:12 PM UTC-5, Jason Felice wrote: Something like (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (- (line-seq rdr) (filter #(re-matches #...)) (map #(nth (string/split % #,) 1 Will do what you want. It's laziness to the rescue. On Tue, Mar 4, 2014 at 3:21 PM, Dean Laskin wdla...@gmail.com wrote: I'm comparing two large files based on specific fields in the files. Is there a functional way of accumulating the results without atoms? (And since I'm a newbie, any other advice is appreciated!) (let [sun (atom []) fri (atom [])] (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! sun conj (nth (string/split line #,) 1) (with-open [rdr (clojure.java.io/reader /dev/errors-friday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! fri conj (nth (string/split line #,) 1) (println (nth (data/diff (set @fri) (set @sun)) 1)) ) Thanks, Dean -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Accumulate results without state?
I do something very similar to this where I parse large CSVs and keep/discard rows of the CSV based on the values of columns. Like you I had a hard time getting how to use reduce beyond trivial implementations. I spent some time messing with reduce and learning how to construct more advanced reductions. The result of that learning became this blog post: http://jarrodswart.com/clojure-like-im-five-reduce-functions/, I hope it helps you too. On Tuesday, March 4, 2014 5:14:49 PM UTC-5, Jason Felice wrote: Your welcome! I just realized that, in this case, filter and map are good enough, but the general answer for threading state through the processing of a sequence is to use `reduce`. Coming from another language, it takes a while to realize how often the answer is to use reduce. -Jason On Tue, Mar 4, 2014 at 4:07 PM, Dean Laskin wdla...@gmail.comjavascript: wrote: Awesome, thanks Jason! It looks much cleaner. Dean On Tuesday, March 4, 2014 4:47:12 PM UTC-5, Jason Felice wrote: Something like (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (- (line-seq rdr) (filter #(re-matches #...)) (map #(nth (string/split % #,) 1 Will do what you want. It's laziness to the rescue. On Tue, Mar 4, 2014 at 3:21 PM, Dean Laskin wdla...@gmail.com wrote: I'm comparing two large files based on specific fields in the files. Is there a functional way of accumulating the results without atoms? (And since I'm a newbie, any other advice is appreciated!) (let [sun (atom []) fri (atom [])] (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! sun conj (nth (string/split line #,) 1) (with-open [rdr (clojure.java.io/reader /dev/errors-friday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! fri conj (nth (string/split line #,) 1) (println (nth (data/diff (set @fri) (set @sun)) 1)) ) Thanks, Dean -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.comjavascript: Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com javascript: For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Accumulate results without state?
Hello, On Tuesday, March 4, 2014 10:47:12 PM UTC+1, Jason Felice wrote: Something like (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (- (line-seq rdr) (filter #(re-matches #...)) (map #(nth (string/split % #,) 1 Will do what you want. It's laziness to the rescue. Just a heads up: laziness is dangerous when using I/O. In this case, with-open will close the reader before all lines have been read, and throw an IOException for trying to read from a closed reader. Add a `doall` to the end of the threading, and it should be safe. Or use `slurp` to read the whole file, avoiding the problem to begin with. As another suggestion: You can use a for comprehension, if you consider it more readable. It's relatively similar to your imperative version. (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (doall (for [line (line-seq rdr) :when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line)] (second (string/split line #,) -- Jean Niklas -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Accumulate results without state?
Hi! An other possible solution is using state monad from https://github.com/clojure/algo.monads Greetings. Andrey 2014-03-05 0:48 GMT+01:00 Jean Niklas L'orange jeann...@hypirion.com: Hello, On Tuesday, March 4, 2014 10:47:12 PM UTC+1, Jason Felice wrote: Something like (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (- (line-seq rdr) (filter #(re-matches #...)) (map #(nth (string/split % #,) 1 Will do what you want. It's laziness to the rescue. Just a heads up: laziness is dangerous when using I/O. In this case, with-open will close the reader before all lines have been read, and throw an IOException for trying to read from a closed reader. Add a `doall` to the end of the threading, and it should be safe. Or use `slurp` to read the whole file, avoiding the problem to begin with. As another suggestion: You can use a for comprehension, if you consider it more readable. It's relatively similar to your imperative version. (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (doall (for [line (line-seq rdr) :when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line)] (second (string/split line #,) -- Jean Niklas -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Andrey Antukh - Андрей Антух - andrei.anto...@kaleidos.net / n...@niwi.be http://www.niwi.be http://www.niwi.be/page/about/ https://github.com/niwibe -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Accumulate results without state?
It depends on what you want to do. If you really want to reduce the collection to an error count, then this will do it. (let [count-if (comp count filter)] (count-if #(re-matches pattern %) (line-seq rdr))) On Tuesday, March 4, 2014 4:21:47 PM UTC-5, Dean Laskin wrote: I'm comparing two large files based on specific fields in the files. Is there a functional way of accumulating the results without atoms? (And since I'm a newbie, any other advice is appreciated!) (let [sun (atom []) fri (atom [])] (with-open [rdr (clojure.java.io/reader /dev/errors-sunday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! sun conj (nth (string/split line #,) 1) (with-open [rdr (clojure.java.io/reader /dev/errors-friday.csv)] (doseq [line (line-seq rdr)] (when (re-matches #^[^,]+,[^,]+,FAIL,.*$ line) (swap! fri conj (nth (string/split line #,) 1) (println (nth (data/diff (set @fri) (set @sun)) 1)) ) Thanks, Dean -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
