Re: Adding up numbers, quickly
Thanks all again for the advice. For the record, I ended up writing the loop in Java - being constrained on the type of the data (it ultimately comes from Java code), and not wanting to pay the (newly reduced!) price of converting to vectorz for just this operation, this seemed like the best way. Jony -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding up numbers, quickly
On Friday, October 17, 2014 1:21:55 PM UTC+2, Jony Hudson wrote: Thanks all again for the advice. For the record, I ended up writing the loop in Java - being constrained on the type of the data (it ultimately comes from Java code), and not wanting to pay the (newly reduced!) price of converting to vectorz for just this operation, this seemed like the best way. Jony If the data came from java (presumably as arrays), I would have considered using http://clojuredocs.org/clojure.core/areduce On a side note, not wanting to hijack your thread, I'm wondering if someone interested in summing many numbers in Clojure had considered implementing the https://en.wikipedia.org/wiki/Kahan_summation_algorithm. Best Regards Bernard -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding up numbers, quickly
This is a case where I would definitely recommend using core.matrix / vectorz-clj (let [a (array :vectorz (range 1000)) b (array :vectorz (range 1000 2000))] (criterium/quick-bench (esum (sub a b = Execution time mean : 2.266914 µs Or it's even faster if you notice that you can switch the operation order: (let [a (array :vectorz (range 1000)) b (array :vectorz (range 1000 2000))] (c/quick-bench (- (esum a) (esum b = Execution time mean : 1.798722 µs On Thursday, 16 October 2014 03:53:40 UTC+8, Jony Hudson wrote: Hi all, another performance question ... this time about arithmetic on vectors :-) Let's say I have two vectors of numbers (specifically, things with type clojure.lang.PersistentVector, containing things of type java.lang.Double). And let's say I want to sum the differences [1] between corresponding elements of the lists i.e. (a1 - b1) + (a2 - b2) + ... Any suggestions on how to do this quickly. What I find is that if I use the 'obvious' high-level construction: (reduce + (mapv #(- %1 %2) a b)) then it goes pretty slowly. On my laptop I measure about 180us for 1000 element lists. If I try using `loop`: (loop [sum 0.0 i 0] (if ( i 1000) (recur (+ sum (- (nth a i) (nth b i))) (inc i)) sum)) it does better, at about 100us. But these are still a ways off what I might think is the best that could be done. If I run a similar computation in Java, it takes about 8us, which ties up with what my gut feeling would be. So ... does anyone have any advice on closing the gap here? Thanks in advance, again, Jony [1] Actually I want to sum the absolute differences, but that brings in java interop which I want to leave out lest it confuse matters. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding up numbers, quickly
Thanks @Mikera. I had a look at using core.matrix, but what I couldn't figure out is how to quickly get the data into core.matrix form. For instance, if I take my PersistentList of Doubles from the example then getting it into a vectors array takes: (criterium/quick-bench (array :vectorz a)) Execution time mean : 70.764422 µs so for the example above I'm no better off overall. Obviously, the ideal would be to use vectorz arrays all through the application, but I'm not sure that's going to be so easy in this case. Is there a quick way to marshall data into a vectorz vector? Jony -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding up numbers, quickly
On Thursday, 16 October 2014 18:04:39 UTC+8, Jony Hudson wrote: Thanks @Mikera. I had a look at using core.matrix, but what I couldn't figure out is how to quickly get the data into core.matrix form. For instance, if I take my PersistentList of Doubles from the example then getting it into a vectors array takes: (criterium/quick-bench (array :vectorz a)) Execution time mean : 70.764422 µs so for the example above I'm no better off overall. Obviously, the ideal would be to use vectorz arrays all through the application, but I'm not sure that's going to be so easy in this case. Is there a quick way to marshall data into a vectorz vector? Hmmm I think you've actually hit a performance issue in the conversion of Clojure vectors to Vectorz format. I just patched this in the latest core.matrix snapshot, the conversion is now around 20µs -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Adding up numbers, quickly
Hi all, another performance question ... this time about arithmetic on vectors :-) Let's say I have two vectors of numbers (specifically, things with type clojure.lang.PersistentVector, containing things of type java.lang.Double). And let's say I want to sum the differences [1] between corresponding elements of the lists i.e. (a1 - b1) + (a2 - b2) + ... Any suggestions on how to do this quickly. What I find is that if I use the 'obvious' high-level construction: (reduce + (mapv #(- %1 %2) a b)) then it goes pretty slowly. On my laptop I measure about 180us for 1000 element lists. If I try using `loop`: (loop [sum 0.0 i 0] (if ( i 1000) (recur (+ sum (- (nth a i) (nth b i))) (inc i)) sum)) it does better, at about 100us. But these are still a ways off what I might think is the best that could be done. If I run a similar computation in Java, it takes about 8us, which ties up with what my gut feeling would be. So ... does anyone have any advice on closing the gap here? Thanks in advance, again, Jony [1] Actually I want to sum the absolute differences, but that brings in java interop which I want to leave out lest it confuse matters. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding up numbers, quickly
Slowdowns wrt math are caused mainly by boxing and range check, see this thread [1] [1] https://groups.google.com/d/msg/clojure/kcx5eKdMxcs/Wy4_IHrSEaMJ Jozef On Wednesday, October 15, 2014 9:53:40 PM UTC+2, Jony Hudson wrote: Hi all, another performance question ... this time about arithmetic on vectors :-) Let's say I have two vectors of numbers (specifically, things with type clojure.lang.PersistentVector, containing things of type java.lang.Double). And let's say I want to sum the differences [1] between corresponding elements of the lists i.e. (a1 - b1) + (a2 - b2) + ... Any suggestions on how to do this quickly. What I find is that if I use the 'obvious' high-level construction: (reduce + (mapv #(- %1 %2) a b)) then it goes pretty slowly. On my laptop I measure about 180us for 1000 element lists. If I try using `loop`: (loop [sum 0.0 i 0] (if ( i 1000) (recur (+ sum (- (nth a i) (nth b i))) (inc i)) sum)) it does better, at about 100us. But these are still a ways off what I might think is the best that could be done. If I run a similar computation in Java, it takes about 8us, which ties up with what my gut feeling would be. So ... does anyone have any advice on closing the gap here? Thanks in advance, again, Jony [1] Actually I want to sum the absolute differences, but that brings in java interop which I want to leave out lest it confuse matters. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding up numbers, quickly
Thanks, that's really useful! Jony On Wednesday, 15 October 2014 20:53:40 UTC+1, Jony Hudson wrote: Hi all, another performance question ... this time about arithmetic on vectors :-) Let's say I have two vectors of numbers (specifically, things with type clojure.lang.PersistentVector, containing things of type java.lang.Double). And let's say I want to sum the differences [1] between corresponding elements of the lists i.e. (a1 - b1) + (a2 - b2) + ... Any suggestions on how to do this quickly. What I find is that if I use the 'obvious' high-level construction: (reduce + (mapv #(- %1 %2) a b)) then it goes pretty slowly. On my laptop I measure about 180us for 1000 element lists. If I try using `loop`: (loop [sum 0.0 i 0] (if ( i 1000) (recur (+ sum (- (nth a i) (nth b i))) (inc i)) sum)) it does better, at about 100us. But these are still a ways off what I might think is the best that could be done. If I run a similar computation in Java, it takes about 8us, which ties up with what my gut feeling would be. So ... does anyone have any advice on closing the gap here? Thanks in advance, again, Jony [1] Actually I want to sum the absolute differences, but that brings in java interop which I want to leave out lest it confuse matters. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding up numbers, quickly
There are primitive vectors. Extraordinary clever. http://clojuredocs.org/clojure.core/vector-of /Linus Den 16 okt 2014 00:02 skrev Jony Hudson jonyepsi...@gmail.com: Thanks, that's really useful! Jony On Wednesday, 15 October 2014 20:53:40 UTC+1, Jony Hudson wrote: Hi all, another performance question ... this time about arithmetic on vectors :-) Let's say I have two vectors of numbers (specifically, things with type clojure.lang.PersistentVector, containing things of type java.lang.Double). And let's say I want to sum the differences [1] between corresponding elements of the lists i.e. (a1 - b1) + (a2 - b2) + ... Any suggestions on how to do this quickly. What I find is that if I use the 'obvious' high-level construction: (reduce + (mapv #(- %1 %2) a b)) then it goes pretty slowly. On my laptop I measure about 180us for 1000 element lists. If I try using `loop`: (loop [sum 0.0 i 0] (if ( i 1000) (recur (+ sum (- (nth a i) (nth b i))) (inc i)) sum)) it does better, at about 100us. But these are still a ways off what I might think is the best that could be done. If I run a similar computation in Java, it takes about 8us, which ties up with what my gut feeling would be. So ... does anyone have any advice on closing the gap here? Thanks in advance, again, Jony [1] Actually I want to sum the absolute differences, but that brings in java interop which I want to leave out lest it confuse matters. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
