Re: Reducers exception when calling into

2017-05-12 Thread Matt Grimm
Thanks very much for the quick response. From your hints, I tried switching uses of into to r/foldcat and it worked. On Thursday, May 11, 2017 at 4:44:37 PM UTC-6, Alex Miller wrote: > > The symptoms sound like you are running into a case where there are > multiple protocol branches that apply

Reducers exception when calling into

2017-05-11 Thread Alex Miller
The symptoms sound like you are running into a case where there are multiple protocol branches that apply - this is a case where the behavior is undefined and can vary between JVM executions. Looks like it's going down the seq reduce path. I'm not at a computer to check more precisely what the

Reducers exception when calling into

2017-05-11 Thread Matt Grimm
Hi, I've run into a bizarre issue with reducers that is consistently reproducible, but only on one set of systems. On a separate system with the same OS and java, the error is not reproducible. The exception messages seem to indicate that I've tried to access a reducible like a seq before

Re: reducers and sets/PersistentSet

2015-11-10 Thread Alex Miller
; On Tuesday, 10 November 2015 16:26:58 UTC, Alex Miller wrote: >> >> Persistent sets are really persistent maps under the hood, so in >> principle doesn't seem like there's any reason that shouldn't work. >> >> On Tuesday, November 10, 2015 at 10:23:19 AM UTC-6, retnu

reducers and sets/PersistentSet

2015-11-10 Thread retnuH
Hi there, I was playing with reducers recently, and was wondering why PersistentSets aren't foldable (i.e. don't take advantage of the Fork/Join stuff? Is there some fundamental reason why it wouldn't work (or is especially difficult), or just a case that no one has gotten around

Re: reducers and sets/PersistentSet

2015-11-10 Thread Alex Miller
Persistent sets are really persistent maps under the hood, so in principle doesn't seem like there's any reason that shouldn't work. On Tuesday, November 10, 2015 at 10:23:19 AM UTC-6, retnuH wrote: > > Hi there, I was playing with reducers recently, and was wondering why > Persi

Re: reducers and sets/PersistentSet

2015-11-10 Thread retnuH
od, so in principle > doesn't seem like there's any reason that shouldn't work. > > On Tuesday, November 10, 2015 at 10:23:19 AM UTC-6, retnuH wrote: >> >> Hi there, I was playing with reducers recently, and was wondering why >> PersistentSets aren't foldable (i.e. don'

Re: reducers question

2015-07-15 Thread Erik Assum
ways to increase the performance of Clojure programs and I’ve run into an interesting issue with using the reducers library. I’d love any help! The code I’m working on is at https://github.com/flyingmachine/quil-pix/tree/4cce95390f5ac7a206cc14a8ec5a4a2492c813fc The issue I’m having

Re: reducers question

2015-07-15 Thread Daniel Higginbotham
Sweet, thanks everyone for your responses! This was very helpful! r/foldcat did the trick. It looks like it's faster than pmap by about 10%. Also, I'm happy to have my sanity back. Thank you! Daniel On Wednesday, July 15, 2015 at 12:38:39 AM UTC-4, James Reeves wrote: Reducers and pmap have

Re: reducers question

2015-07-15 Thread Alan Busby
On Wed, Jul 15, 2015 at 12:02 PM, Daniel Higginbotham nonrecurs...@gmail.com wrote: Surely reducers/map should be faster, but it doesn’t seem like I’ve done something wrong? This is driving me crazy :) It is much faster, and runs in parallel, if you use fold. Try this; (defn fold-into-vec

reducers question

2015-07-14 Thread Daniel Higginbotham
I’ve been trying to better understand ways to increase the performance of Clojure programs and I’ve run into an interesting issue with using the reducers library. I’d love any help! The code I’m working on is at https://github.com/flyingmachine/quil-pix/tree

reducers question

2015-07-14 Thread Francis Avila
Reducers can perform fewer allocations than lazy seqs, but that does not automatically translate into a big speed difference, especially with your trivial example. Try comparing chained lazy seq ops (map, mapcat, filter, take, drop, etc) to an equivalent chain of reducer calls: you may see

Re: reducers question

2015-07-14 Thread James Reeves
Reducers and pmap have different approaches to concurrency. pmap works a lot like map, except that instead of evaluating the seq one item at a time, it spins up N threads which work on the next N items in parallel. N is 2 + the number of CPU cores you have. The reducers library works in a rather

Re: are there any real examples on Github of how to use reducers?

2015-05-22 Thread Marshall Bockrath-Vandegrift
piastkra...@gmail.com writes: Can anyone point me to a project on Github that is using Clojure 1.5 reducerers? The Parkour project provides an interface to Hadoop MapReduce primarily in terms of tasks as functions over task input chunks as clojure.core.reducers reducible collections:

are there any real examples on Github of how to use reducers?

2015-05-21 Thread piastkrakow
Can anyone point me to a project on Github that is using Clojure 1.5 reducerers? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please

Re: are there any real examples on Github of how to use reducers?

2015-05-21 Thread Max Countryman
Kyle Kingsbury’s linearizability checker “Knossos” uses reducers. (See the core, history, and util namespaces.) https://github.com/aphyr/knossos On May 21, 2015, at 19:55, piastkra...@gmail.com wrote: Can anyone point me to a project on Github that is using Clojure 1.5 reducerers

Reducers question

2015-02-09 Thread Aaron Cohen
I'm not sure if the 4-arity fold function is working as expected. My understanding is that it should be equivalent to the 3-arity version, but with specified parallelism. However: (r/fold r/cat r/append! [1 2 3]) = [1 2 3] (r/fold 1 r/cat r/append! [1 2 3]) = #Cat

Re: Reducers question

2015-02-09 Thread Thomas Heller
of 1 (eg, [[1] [2] [3]]) and then handed off to reducers, the default of n=512 is larger than your input so you'll just use regular reduce. /thomas On Monday, February 9, 2015 at 10:38:59 AM UTC+1, Aaron Cohen wrote: I'm not sure if the 4-arity fold function is working as expected. My

Re: Very strange behaviour in reducers

2014-01-17 Thread Yves Parès
Ok, but apparently I'm not the only one to be puzzled by this behaviour, which we may regard as an inconsistency. Just to know, is it planned to be changed? (so that I can note it and remember it if I have an error that appears in this part of my code after a clojure upgrade) Le mardi 14

Re: Very strange behaviour in reducers

2014-01-15 Thread icamts
Similar threads are https://groups.google.com/forum/?hl=en#!topic/clojure/A1gW_BB_4MU https://groups.google.com/forum/?hl=en#!topic/clojure-dev/scvyi2Cwk7g Luca Il giorno martedì 14 gennaio 2014 16:43:10 UTC+1, Yves Parès ha scritto: Hello! When mapping and reducing a map, it seems

Very strange behaviour in reducers

2014-01-14 Thread Yves Parès
Hello! When mapping and reducing a map, it seems arguments are passed differently to the function that is mapped depending on whether you reduce with r/reduce or r/fold: (r/fold + (r/map (fn [k v] (inc v)) {:a 4 :b 5})) But: (r/reduce + (r/map (fn [[k v]] (inc v)) {:a 4 :b 5})) The

Re: Very strange behaviour in reducers

2014-01-14 Thread Filip Štaffa
Hi Yves, I guess it is because r/reduce calls r/reduce-kv (which is specific for map), while r/fold it does not have such special handling Filip On Tuesday, January 14, 2014 4:43:10 PM UTC+1, Yves Parès wrote: Hello! When mapping and reducing a map, it seems arguments are passed differently

Re: lazy seq from reducers with core.async

2013-09-02 Thread Jozef Wagner
On Sunday, August 25, 2013 7:47:14 PM UTC+2, Jozef Wagner wrote: Hi, A distinctive feature of reducers is that reducing is a one-shot thing. The common understanding is that reducers are fast for cases where you want to process whole collection at once, but for infinite and lazy seqs, you

Re: lazy seq from reducers with core.async

2013-08-26 Thread Timothy Baldridge
Although this also ties up a thread for every reduce. Not exactly efficient IMO. Timothy On Sun, Aug 25, 2013 at 11:50 PM, Alan Busby thebu...@gmail.com wrote: On Mon, Aug 26, 2013 at 1:37 PM, Timothy Baldridge tbaldri...@gmail.comwrote: Since reducers use fork-join pools, Reducers

lazy seq from reducers with core.async

2013-08-25 Thread Jozef Wagner
Hi, A distinctive feature of reducers is that reducing is a one-shot thing. The common understanding is that reducers are fast for cases where you want to process whole collection at once, but for infinite and lazy seqs, you have to use good old seqs. With core.async, it is now easy to create

Re: lazy seq from reducers with core.async

2013-08-25 Thread Mikera
Nice idea Jozef! Hmmm this is another example of why nil-as-end-of-channel is a slightly problematic design decision for core.async: it makes this kind of code much more fiddly. On Monday, 26 August 2013 01:47:14 UTC+8, Jozef Wagner wrote: Hi, A distinctive feature of reducers

Re: lazy seq from reducers with core.async

2013-08-25 Thread Alan Busby
wrote: Hi, A distinctive feature of reducers is that reducing is a one-shot thing. The common understanding is that reducers are fast for cases where you want to process whole collection at once, but for infinite and lazy seqs, you have to use good old seqs. With core.async, it is now easy

Re: lazy seq from reducers with core.async

2013-08-25 Thread Timothy Baldridge
Although this looks like it might work, it's not exactly a good idea. Look at what you're doing inside the reducer, a call to !! will block a OS level thread until someone takes from the channel. Since reducers use fork-join pools, I wouldn't be surprised if it caused some serious side-effects

Re: lazy seq from reducers with core.async

2013-08-25 Thread Alan Busby
On Mon, Aug 26, 2013 at 1:37 PM, Timothy Baldridge tbaldri...@gmail.comwrote: Since reducers use fork-join pools, Reducers use multiple threads and fork-join pools when called with fold on vectors (anything else?), not reduce. By making the single producer thread of the reducer block on writes

Re: butlast with reducers

2013-08-23 Thread Jozef Wagner
. . . 100) user (last x) 100 JW On Thursday, August 8, 2013 4:46:37 PM UTC+2, Jozef Wagner wrote: Wow, thank you very much! A perfect solution. At the end, wouldn't be good if the reducers would implement alongside CollReduce also a Seqable interface, so that reducers could be used

Re: butlast with reducers

2013-08-23 Thread Jozef Wagner
+ x) ;; using reducer 5050 user (clojure.core.reducers/fold + x) ;; using folder 5050 JW On Thursday, August 8, 2013 4:46:37 PM UTC+2, Jozef Wagner wrote: Wow, thank you very much! A perfect solution. At the end, wouldn't be good if the reducers would implement alongside CollReduce also

Re: butlast with reducers

2013-08-23 Thread Jozef Wagner
(clojure.core.reducers/fold + x) 5050 Best, JW On Thursday, August 8, 2013 4:46:37 PM UTC+2, Jozef Wagner wrote: Wow, thank you very much! A perfect solution. At the end, wouldn't be good if the reducers would implement alongside CollReduce also a Seqable interface, so that reducers could be used

Re: butlast with reducers

2013-08-09 Thread Christophe Grand
to implement efficient butlast (and drop-last, take-last) with reducers? The only solution I can think of needs additional reduce to compute count, which may often be undesirable. Or is it OK to say that reducers are not designed for such cases? JW -- -- You received this message because

butlast with reducers

2013-08-08 Thread Jozef Wagner
Is it possible to implement efficient butlast (and drop-last, take-last) with reducers? The only solution I can think of needs additional reduce to compute count, which may often be undesirable. Or is it OK to say that reducers are not designed for such cases? JW -- -- You received

Re: butlast with reducers

2013-08-08 Thread Christophe Grand
of a mutable array/buffer would be preferable. hth, Christophe On Thu, Aug 8, 2013 at 1:00 PM, Jozef Wagner jozef.wag...@gmail.com wrote: Is it possible to implement efficient butlast (and drop-last, take-last) with reducers? The only solution I can think of needs additional reduce to compute count

Re: butlast with reducers

2013-08-08 Thread Christophe Grand
of a mutable array/buffer would be preferable. hth, Christophe On Thu, Aug 8, 2013 at 1:00 PM, Jozef Wagner jozef.wag...@gmail.comwrote: Is it possible to implement efficient butlast (and drop-last, take-last) with reducers? The only solution I can think of needs additional reduce

Re: butlast with reducers

2013-08-08 Thread Jozef Wagner
Wow, thank you very much! A perfect solution. At the end, wouldn't be good if the reducers would implement alongside CollReduce also a Seqable interface, so that reducers could be used as a drop in replacement for today's sequence functions (map, filter, ...)? CollReduce implements 'eager

Re: cannot import namespace reducers

2013-07-24 Thread Johannes Brauer
if I remember correctly I solved the problem by reinstalling Java 7 from Oracle. Thereafter my $JAVA_HOME points to: /Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home Johannes Am 24.07.2013 um 03:02 schrieb Keith Maynard kpmayn...@gmail.commailto:kpmayn...@gmail.com : Please spam

Re: cannot import namespace reducers

2013-07-23 Thread Keith Maynard
Please spam the list!!! I am sure anyone who receives that message is probably running Mac OS X 10.7.x or later and trying to unravel the mess between Java 6 and Java 7. Please post recommendations. I finally got Eclipse to see the jdl1.7.0_25.jdk now how do I get the OS to replace the Apple

cannot import namespace reducers

2013-06-16 Thread Johannes
Hi, trying (require '[clojure.core.reducers :as r]) at the repl prompt the error message CompilerException java.lang.ClassNotFoundException: jsr166y.ForkJoinPool, compiling:(clojure/core/reducers.clj:56:21) What is going wrong? Johannes -- -- You received this message because you are

Re: cannot import namespace reducers

2013-06-16 Thread László Török
Hi, there are two ways to deal with this: a) use Java 7 2013/6/16 Johannes bra...@nordakademie.de Hi, trying (require '[clojure.core.reducers :as r]) at the repl prompt the error message CompilerException java.lang.ClassNotFoundException: jsr166y.ForkJoinPool,

Re: cannot import namespace reducers

2013-06-16 Thread Mayank Jain
Try (:require '[clojure.core.reducers :as r]) i.e. :require On Mon, Jun 17, 2013 at 1:38 AM, Johannes bra...@nordakademie.de wrote: Hi, trying (require '[clojure.core.reducers :as r]) at the repl prompt the error message CompilerException java.lang.ClassNotFoundException:

Re: cannot import namespace reducers

2013-06-16 Thread László Török
.. sorry, gmail's new annoying keyboard shortcut b) include the dependency to the forkjoin library [1] that is not included in Java6 Las [1] http://mavenhub.com/mvn/central/org.coconut.forkjoin/jsr166y/070108 2013/6/16 László Török ltoro...@gmail.com Hi, there are two ways to deal with

Re: cannot import namespace reducers

2013-06-16 Thread László Török
(require '[clojure.core.reducers :as r]) is correct. (:require '[clojure.core.reducers :as r]) only works within the ns macro: (ns 'yournamespace (:require '[clojure.core.reducers :as r])) Las 2013/6/16 Mayank Jain firesof...@gmail.com Try (:require '[clojure.core.reducers :as r])

Re: cannot import namespace reducers

2013-06-16 Thread Johannes Brauer
thank you, Las, for the quick tip. I will give Java 7 a try. I hope there are no problems on Mac OS 10.8.4 Johannes Am 16.06.2013 um 22:15 schrieb László Török ltoro...@gmail.commailto:ltoro...@gmail.com : .. sorry, gmail's new annoying keyboard shortcut b) include the dependency to the

Re: cannot import namespace reducers

2013-06-16 Thread László Török
I'm on Java7 and OS X 10.8.4, no problem over here. :) 2013/6/16 Johannes Brauer bra...@nordakademie.de thank you, Las, for the quick tip. I will give Java 7 a try. I hope there are no problems on Mac OS 10.8.4 Johannes Am 16.06.2013 um 22:15 schrieb László Török ltoro...@gmail.com :

Re: cannot import namespace reducers

2013-06-16 Thread Johannes Brauer
now, I've Java 7 installed and get another error message: Exception namespace 'clojure.core.reducers' not found clojure.core/load-lib (core.clj:5380) any further hints? Johannes Am 16.06.2013 um 22:43 schrieb Johannes Brauer bra...@nordakademie.demailto:bra...@nordakademie.de : thank you,

Re: cannot import namespace reducers

2013-06-16 Thread László Török
are you on clojure 1.5+ ? If I launch the REPL using lein repl and then (require 'clojure.core.reducers) it works ok for me. 2013/6/16 Johannes Brauer bra...@nordakademie.de now, I've Java 7 installed and get another error message: Exception namespace 'clojure.core.reducers' not found

Re: cannot import namespace reducers

2013-06-16 Thread Johannes Brauer
I am on clojure 1.5.1 and I use lein repl. But after (require '[clojure.core.reducers :as r]) I still get the same error message as with Java 6: CompilerException java.lang.ClassNotFoundException: jsr166y.ForkJoinPool, compiling:(clojure/core/reducers.clj:56:21) A second input of (require

Re: cannot import namespace reducers

2013-06-16 Thread László Török
Then you are having a path problem. Your lein is still using java6 Check your $PATH and $JAVA_HOME env. variables. As this not strictly clojure related, let's not spam the list, im happy to help off list Sent from my phone On Jun 16, 2013 11:22 PM, Johannes Brauer bra...@nordakademie.de wrote:

Re: Iota, reducers, word counting

2013-05-03 Thread Catonano
Additionally you cannot use transients at the moment with fold. That´s a pity. I suppose I could explicitly split the input vector in parts and then call pvalues with a function implementing my word counting on each subpart. And then merging the results from each subpart But the reducers idiom is more

Re: Iota, reducers, word counting

2013-05-01 Thread Catonano
pvalues with a function implementing my word counting on each subpart. And then merging the results from each subpart But the reducers idiom is more beautiful because the modifications to the traditional map reduce version would be smaller, with that you haven´t got to explicitly split the input

Iota, reducers, word counting

2013-04-30 Thread Catonano
Hello people, I would like you to take a look at the question I posted here http://meta.stackoverflow.com/questions/178326/why-is-this-question-not-good-enough I´m trying to implement a word count of a large file using the machinery by Iota ( https://github.com/thebusby/iota#readme ) just as an

Re: Iota, reducers, word counting

2013-04-30 Thread Meikel Brandmeyer (kotarak)
for the subtask. In our case an empty map. (This is were your example blows up.) To combine two maps we use merge-with. The update-in is basically the reduce function. The reducers library provides the monoid helper to ease the pain a little. (Although it doesn't buy us much in your example.) (defn my

Re: Reducers newbie question

2013-04-27 Thread Stanislav Yurin
Yep, thanks, my bad. I got the point. Actually, what I was trying to do, is to prototype multithreaded i/o operation via reducers. And then use fold to regulate number of concurrent operations. But now something tells me I am doing not very clever thing. On Friday, April 26, 2013 5:27:46 PM

Re: Reducers newbie question

2013-04-27 Thread Alan Busby
On Sat, Apr 27, 2013 at 7:35 PM, Stanislav Yurin jusk...@gmail.com wrote: Actually, what I was trying to do, is to prototype multithreaded i/o operation via reducers. And then use fold to regulate number of concurrent operations. But now something tells me I am doing not very clever thing

Re: Reducers newbie question

2013-04-27 Thread Stanislav Yurin
Yurin jus...@gmail.comjavascript: wrote: Actually, what I was trying to do, is to prototype multithreaded i/o operation via reducers. And then use fold to regulate number of concurrent operations. But now something tells me I am doing not very clever thing. I'm not entirely sure what you

Re: Reducers newbie question

2013-04-27 Thread Alan Busby
On Sat, Apr 27, 2013 at 10:01 PM, Stanislav Yurin jusk...@gmail.com wrote: By the way, fold function has [n combinef reducef coll] implementation, where n is number of elements collection is folded by. 512 is just the default. Yep I misspoke there, but it is still one of the tricks to be

Re: Reducers newbie question

2013-04-27 Thread Stanislav Yurin
Indeed, that was my really bad example with 100. On Saturday, April 27, 2013 4:19:07 PM UTC+3, Alan Busby wrote: On Sat, Apr 27, 2013 at 10:01 PM, Stanislav Yurin jus...@gmail.comjavascript: wrote: By the way, fold function has [n combinef reducef coll] implementation, where n is

Reducers newbie question

2013-04-26 Thread Stanislav Yurin
I was assuming that following code will fold in parallel, but it is reduced sequentially (require '[clojure.core.reducers :as r]) (defn test1 [x] (Thread/sleep 1000) (println (str Finished: x)) x) (def xxx (r/map test1 (range 100))) (r/fold + xxx) What am I doing wrong? Thanks. -- -- You

Re: Reducers newbie question

2013-04-26 Thread László Török
Hi, Not sure what you are trying to do, but xxx is a lazy seq, thus it can only be consumed sequentially and fold falls back to reduce You need a vector. Las Sent from my phone On Apr 26, 2013 4:46 PM, Stanislav Yurin jusk...@gmail.com wrote: I was assuming that following code will fold in

Re: Reducers newbie question

2013-04-26 Thread Alan Busby
Some additional pointers here (this is a little old though); http://www.thebusby.com/2012/07/tips-tricks-with-clojure-reducers.html On Fri, Apr 26, 2013 at 11:51 PM, László Török ltoro...@gmail.com wrote: Hi, Not sure what you are trying to do, but xxx is a lazy seq, thus it can only be

Re: Reducers newbie question

2013-04-26 Thread Jim - FooBar();
+1 ! I use 'fold-into-vec' regularly :) Jim On 26/04/13 18:07, Alan Busby wrote: Some additional pointers here (this is a little old though); http://www.thebusby.com/2012/07/tips-tricks-with-clojure-reducers.html On Fri, Apr 26, 2013 at 11:51 PM, László Török ltoro...@gmail.com

Re: would FixedThreadPool-backed reducers make sense?

2013-03-30 Thread Gary Verhaegen
on reducers (and it's intuitively true anyway) that FJ is not well suited to all workloads: uniform ones would do just fine with a fixed allocation of tasks to threads. I believe the tradeoff in that case is that one has to manage parallelism very explicitly. Now, is it feasible to provide

would FixedThreadPool-backed reducers make sense?

2013-03-28 Thread vemv
I recall from Rich's presentation on reducers (and it's intuitively true anyway) that FJ is not well suited to all workloads: uniform ones would do just fine with a fixed allocation of tasks to threads. I believe the tradeoff in that case is that one has to manage parallelism very explicitly

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-10 Thread Alan Busby
Hi Bernard, I'm going to see if I can add some performance metrics in the test code to try and determine where various bottlenecks may exist. If that goes well I'll see if I can add a branch that returns Byte[] instead of String for iota/vec. I suspect this will just require modifying

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-08 Thread Alan Busby
, Alan Busby wrote: With the release of Clojure 1.5 and it's new reducers, I figured this would be a good time to release a library to help with file IO for reducers. As reducers can only operate in parallel over specific collections, like vec, it requires some work to use them against files

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-08 Thread bernardH
Hi Alan, On Friday, March 8, 2013 4:02:18 PM UTC+1, Alan Busby wrote: Hi Bernard, I'd certainly like to add support for binary files, but as I haven't had a need for it myself I haven't had a good place to start. As Java NIO's mmap() doesn't support ranges over 2GB, I've had to paste

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-07 Thread bernardH
Hi, On Wednesday, March 6, 2013 2:53:26 PM UTC+1, Alan Busby wrote: With the release of Clojure 1.5 and it's new reducers, I figured this would be a good time to release a library to help with file IO for reducers. As reducers can only operate in parallel over specific collections, like

ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-06 Thread Alan Busby
Hi all, With the release of Clojure 1.5 and it's new reducers, I figured this would be a good time to release a library to help with file IO for reducers. As reducers can only operate in parallel over specific collections, like vec, it requires some work to use them against files. Iota wraps

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-06 Thread Michael Klishin
2013/3/6 Alan Busby thebu...@thebusby.com Code is available on Github here: https://github.com/thebusby/iota Library is available on Clojars here: https://clojars.org/iota Alan, Please add dependency information to the README. Otherwise beginners won't be able to use your project. If you

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-06 Thread kovas boguta
Very cool! Been wanting to play with something like this. On Wed, Mar 6, 2013 at 5:53 AM, Alan Busby thebu...@thebusby.com wrote: Hi all, With the release of Clojure 1.5 and it's new reducers, I figured this would be a good time to release a library to help with file IO for reducers

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-06 Thread Paul deGrandis
I agree, this is a really cool project. Nice work! -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first

Re: reduce, reduce-kv, map, mapv, reducers/map and nil

2013-01-07 Thread Wolodja Wentland
on different branches as well. This would be perfect for reducers, but from a quick look it didn't appear that any of the key internals were exposed to be taken advantage of. Unfortunately I haven't discovered a way to facilitate more efficient merging without modifying the actual Clojure core

Re: reduce, reduce-kv, map, mapv, reducers/map and nil

2013-01-07 Thread Alan Busby
node stored the number of child nodes, then you can assign different threads to work on different branches as well. This would be perfect for reducers, but from a quick look it didn't appear that any of the key internals were exposed to be taken advantage of. Unfortunately I haven't

Re: group-by vs. reducers?

2012-12-07 Thread Balint Erdi
Hey, Reducers is fascinating and quite complex at the same time. I feel like best practices around it has not quite solidified yet. Here is how I made your example work: (ns group-by-reducers.core (:require [clojure.core.reducers :as r :only [fold reduce map]]) (:require [criterium.core

Re: group-by vs. reducers?

2012-12-07 Thread Balint Erdi
BTW I understood the most about reducers (still not quite there yet, though :) ) from Rich Hickey's talk at EuroClojure: https://vimeo.com/45561411 On Friday, December 7, 2012 10:21:59 AM UTC+1, Balint Erdi wrote: Hey, Reducers is fascinating and quite complex at the same time. I feel like

Re: group-by vs. reducers?

2012-12-07 Thread Christophe Grand
Elapsed time: 725.995 msecs Elapsed time: 494.038 msecs Run 4 Elapsed time: 647.53 msecs Elapsed time: 731.085 msecs Elapsed time: 538.649 msecs hth, Christophe On Fri, Dec 7, 2012 at 10:30 AM, Balint Erdi balint.e...@gmail.com wrote: BTW I understood the most about reducers (still not quite

Re: group-by vs. reducers?

2012-12-07 Thread László Török
understood the most about reducers (still not quite there yet, though :) ) from Rich Hickey's talk at EuroClojure: https://vimeo.com/45561411 On Friday, December 7, 2012 10:21:59 AM UTC+1, Balint Erdi wrote: Hey, Reducers is fascinating and quite complex at the same time. I feel like best

Re: group-by vs. reducers?

2012-12-07 Thread Christophe Grand
, Christophe On Fri, Dec 7, 2012 at 10:30 AM, Balint Erdi balint.e...@gmail.comwrote: BTW I understood the most about reducers (still not quite there yet, though :) ) from Rich Hickey's talk at EuroClojure: https://vimeo.com/45561411 On Friday, December 7, 2012 10:21:59 AM UTC+1, Balint Erdi wrote

Re: group-by vs. reducers?

2012-12-07 Thread Balint Erdi
Thank you, Christophe, that makes total sense. Interestingly, on my machine, the reducers version is still slower: Run # 0 Elapsed time: 1105.586 msecs Elapsed time: 685.8 msecs Elapsed time: 549.969 msecs Run # 1 Elapsed time: 497.831 msecs Elapsed time: 489.932 msecs Elapsed time: 520.38 msecs

group-by vs. reducers?

2012-12-03 Thread László Török
Hi, As I was trying to wrap my head around the reducers library[1], I thought implementing group-by would be a good exercise to gain some insight. After spending a few hours with it, I'm still pretty much clueless, so hope to find someone here to help me out: So if I understood the reducer

Re: reduce, reduce-kv, map, mapv, reducers/map and nil

2012-11-01 Thread Alan Busby
On Mon, Oct 29, 2012 at 10:00 PM, Wolodja Wentland babi...@gmail.com wrote: I find this behaviour quite unfortunate because I now have to explicitly test for nil? and ensure consistent behaviour. This inconsistency violates the principle of least-surprise and I am not sure if the current

Re: reduce, reduce-kv, map, mapv, reducers/map and nil

2012-11-01 Thread Wolodja Wentland
On Thu, Nov 01, 2012 at 15:11 +0900, Alan Busby wrote: On Mon, Oct 29, 2012 at 10:00 PM, Wolodja Wentland babi...@gmail.com wrote: I find this behaviour quite unfortunate because I now have to explicitly test for nil? and ensure consistent behaviour. This inconsistency violates the

Re: reduce, reduce-kv, map, mapv, reducers/map and nil

2012-11-01 Thread Alan Busby
tries, and can pair each branch together recursively. If each branch node stored the number of child nodes, then you can assign different threads to work on different branches as well. This would be perfect for reducers, but from a quick look it didn't appear that any of the key internals were

Re: reduce, reduce-kv, map, mapv, reducers/map and nil

2012-10-31 Thread Wolodja Wentland
On Tue, Oct 30, 2012 at 15:17 +0100, Herwig Hochleitner wrote: I've also run into this. Maybe this is just an oversight, since clojure handles nils gracefully almost everywhere else. Should CollFold and IKVReduce be extended to nil, or is there some rationale against it? I would

Re: reduce, reduce-kv, map, mapv, reducers/map and nil

2012-10-31 Thread Herwig Hochleitner
Created an issue: http://dev.clojure.org/jira/browse/CLJ-1098 Don't know if patch is welcome, but fix should be trivial, so not much is lost if declined. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to

Re: reduce, reduce-kv, map, mapv, reducers/map and nil

2012-10-30 Thread abp
I find this behaviour quite unfortunate because I now have to explicitly test for nil? and ensure consistent behaviour. Yes, especially unfortunate considering that Rich said the reducers lib could be used as a drop in replacement for core to speed up programs, or something along

Re: reduce, reduce-kv, map, mapv, reducers/map and nil

2012-10-30 Thread Herwig Hochleitner
I've also run into this. Maybe this is just an oversight, since clojure handles nils gracefully almost everywhere else. Should CollFold and IKVReduce be extended to nil, or is there some rationale against it? -- You received this message because you are subscribed to the Google Groups Clojure

reduce, reduce-kv, map, mapv, reducers/map and nil

2012-10-29 Thread Wolodja Wentland
Hi all, I am currently testing performance of different reduce and map implementations in my programs and have problems because their treatment of nil is different. The normal clojure.core implementations of reduce and map work well when called on nil, but reduce-kv and functions in

trampoline not compatible with reducers?

2012-10-16 Thread Jim - FooBar();
depth Can anyone shine some light? Is it something that has to do with reducers? Jim ps: it works fine without trampoline and '#' behind 'r/reduce'. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure

Re: trampoline not compatible with reducers?

2012-10-16 Thread Kevin Downey
)) #(r/reduce my-max (r/map (fn [child] (minimize (:tree child) (dec d))) (:children tree)] (trampoline minimize tree (int depth Can anyone shine some light? Is it something that has to do with reducers? Jim ps: it works fine without

Re: trampoline not compatible with reducers?

2012-10-16 Thread Jim - FooBar();
On 16/10/12 19:15, Kevin Downey wrote: you are declaring the functions return doubles, but in fact returning functions or doubles yes you're right (my bad) but the same thing happens without the type-hinting - albeit in a different place and different originating function: ClassCastException

Re: trampoline not compatible with reducers?

2012-10-16 Thread Kevin Downey
if you look further down the stacktrace (where it refers to your code instead of clojure.lang.Numbers.lt) it will give you line numbers in your code to look at. you are calling these trampolined functions without trampoline. On Tue, Oct 16, 2012 at 11:24 AM, Jim - FooBar(); jimpil1...@gmail.com

Re: trampoline not compatible with reducers?

2012-10-16 Thread Jim - FooBar();
'my-min' and 'my-max' simply wrap core/min core/max. You mean i have to trampoline these calls as well? return something like this? : #(r/reduce (trampoline my-max) ;;was my-max (r/map (fn [child] (minimize (:tree child) (dec d))) (:children tree))) Jim

Re: trampoline not compatible with reducers?

2012-10-16 Thread Alan Malloy
Reducers don't enter into the picture at all: this code is just as broken with clojure.core/reduce and clojure.core/map. The immediate problem that Kevin is trying to draw your attention to is that you are calling (reduce max (map my-fn coll)), where my-fn sometimes returns a number, and sometimes

Reducers reduce my performance

2012-09-14 Thread Tassilo Horn
Hi all, I have some code which uses a lot of map/mapcat/filter stuff and is totally eager (result is always an ordered set). That looked to me like a good candidate for reducers. Basically, my code enables me to write something like this: --8---cut here---start

Re: Reducers reduce my performance

2012-09-14 Thread Kevin Downey
On Fri, Sep 14, 2012 at 4:22 AM, Tassilo Horn t...@gnu.org wrote: Hi all, I have some code which uses a lot of map/mapcat/filter stuff and is totally eager (result is always an ordered set). That looked to me like a good candidate for reducers. Basically, my code enables me to write

Re: Reducers reduce my performance

2012-09-14 Thread Tassilo Horn
Kevin Downey redc...@gmail.com writes: Hi Kevin, This is the new version using reducers (:as r). The problem here is that to stop the iteration, one has to reduce the intermediate result in every step and check if new reachable vertices (n) could be found. If so, we need to do another

Re: real-world usage of reducers?

2012-08-24 Thread nicolas.o...@gmail.com
-Move ~game p# %) (getMoves p#))) team#)] (into [] tmvs#)) ) You should return tmvs# and not (into [] ...). The beauty of reducers is that they are composable without putting results in intermediate sequence. So, as tmvs# is going to be passed to another reducer transformer, you don't need

  1   2   >