Re: Clojure generates unnecessary and slow type-checks

2013-07-03 Thread Jim
After following Jason's suggestion to use BigInteger and .multiply instead of BigInt and * I too am getting speed on par with Java (108-109 miroseconds on my RPI). Therefore, I consider this issue closed :) @Stuart thanks for running the benchmark yourself...maybe I could have save you

Re: Clojure generates unnecessary and slow type-checks

2013-07-02 Thread Dmitry Groshev
Jason, can you please help me with this: I'm not sure if the protocol-based implementation can give users any help writing new core operations efficiently (say, making a new array with c[i] = a[i] + b[i]^2 / 2) -- unless there's some clever way of combining protocols with macros (hmmm).

Re: Clojure generates unnecessary and slow type-checks

2013-07-02 Thread Jim - FooBar();
I'm really sorry for coming back to this but even after everything we learned I'm still not able to get performance equal to java in a simple factorial benchmark. I'd like to think that I'm doing all the correct things to keep the comparison fair...observe this: benchmarks.core= (crit/bench (jf!

Re: Clojure generates unnecessary and slow type-checks

2013-07-02 Thread Leon Barrett
Try longs instead of ints? Clojure doesn't support local ints, so you may be casting longs to ints a lot. On Tue, Jul 2, 2013 at 11:05 AM, Jim - FooBar(); jimpil1...@gmail.comwrote: I'm really sorry for coming back to this but even after everything we learned I'm still not able to get

Re: Clojure generates unnecessary and slow type-checks

2013-07-02 Thread Jim - FooBar();
with a long counter it needs slightly longer (216 microseconds in average instead of 196)! any other ideas? On 02/07/13 19:11, Leon Barrett wrote: Try longs instead of ints? Clojure doesn't support local ints, so you may be casting longs to ints a lot. On Tue, Jul 2, 2013 at 11:05 AM, Jim -

Re: Clojure generates unnecessary and slow type-checks

2013-07-02 Thread Leon Barrett
I got nothin'. Stare at the bytecode? On Tue, Jul 2, 2013 at 11:21 AM, Jim - FooBar(); jimpil1...@gmail.comwrote: with a long counter it needs slightly longer (216 microseconds in average instead of 196)! any other ideas? On 02/07/13 19:11, Leon Barrett wrote: Try longs instead of

Re: Clojure generates unnecessary and slow type-checks

2013-07-02 Thread Jason Wolfe
I guess this is not connected to our issue. You're using Clojure's BigInt, which is probably a bit slower than BigInteger if you know you want it to be big: user (class 1N) clojure.lang.BigInt Have you tried translating your Java code directly to see if that helps? On Tue, Jul 2, 2013 at

Re: Clojure generates unnecessary and slow type-checks

2013-07-02 Thread Stuart Sierra
Hi Jim, I cannot reproduce your results. I see Clojure and Java with similar performance when they are both using java.lang.BigInteger. Clojure's arbitrary-precision integer defaults to clojure.lang.BigInt, which in my test is about 12% slower than java.lang.BigInteger. See

Re: Clojure generates unnecessary and slow type-checks

2013-06-29 Thread Jason Wolfe
This looks very interesting, thanks for sharing! I'll think about it a bit while working on our codebase and see if I can contribute any good examples. On Fri, Jun 28, 2013 at 7:17 AM, Mikera mike.r.anderson...@gmail.comwrote: On Thursday, 20 June 2013 08:45:47 UTC+1, Jason Wolfe wrote: On

Re: Clojure generates unnecessary and slow type-checks

2013-06-28 Thread Mikera
On Thursday, 20 June 2013 08:45:47 UTC+1, Jason Wolfe wrote: On Saturday, June 15, 2013 4:37:06 AM UTC-7, Mikera wrote: On Friday, 14 June 2013 18:15:34 UTC+1, Jason Wolfe wrote: Hey Mikera, I did look at core.matrix awhile ago, but I'll take another look. Right now, flop is just

Re: Clojure generates unnecessary and slow type-checks

2013-06-20 Thread Jason Wolfe
On Saturday, June 15, 2013 4:37:06 AM UTC-7, Mikera wrote: On Friday, 14 June 2013 18:15:34 UTC+1, Jason Wolfe wrote: Hey Mikera, I did look at core.matrix awhile ago, but I'll take another look. Right now, flop is just trying to make it easy to write *arbitrary* array operations

Re: Clojure generates unnecessary and slow type-checks

2013-06-19 Thread Stuart Sierra
Jason Wolfe wrote: We thought we were being very careful Sorry, didn't mean to imply that you weren't. ;) It was me who wasn't careful: when I started investigating this, I used a dead-code loop similar to the Gist I posted, which made it look like Clojure 1.2 was much faster than 1.5. I

Re: Clojure generates unnecessary and slow type-checks

2013-06-18 Thread Rich Hickey
Could you please try your tests against master? Here, I went from 145ms to 85ms going from 1.5.1 to 1.6.0 master. If it's the same for you, someone can git bisect and figure out what's up. Thanks, Rich On Thursday, June 13, 2013 3:02:56 PM UTC-4, Leon Barrett wrote: Hi. I've been working

Re: Clojure generates unnecessary and slow type-checks

2013-06-18 Thread Stuart Sierra
One has to be very careful with this kind of micro-benchmarking on the JVM. Dead-code elimination can easily make something seem fast simply because it's not doing anything. For example, in Java: https://gist.github.com/stuartsierra/5807356 Being careful not to have dead code, I get about the

Re: Clojure generates unnecessary and slow type-checks

2013-06-18 Thread Jason Wolfe
Hi Rich, I don't see any difference between 1.5.1 and 1.6.0 master. The good news is that Stuart Sierra nailed the problem above -- I had no idea that leiningen messed with jvm-opts in the host process, and that seems to have been solely responsible for the performance issues (so we're at

Re: Clojure generates unnecessary and slow type-checks

2013-06-18 Thread Jason Wolfe
Hi Stuart, On Tuesday, June 18, 2013 11:12:28 AM UTC-7, Stuart Sierra wrote: One has to be very careful with this kind of micro-benchmarking on the JVM. Dead-code elimination can easily make something seem fast simply because it's not doing anything. For example, in Java:

Re: Clojure generates unnecessary and slow type-checks

2013-06-18 Thread Stuart Sierra
For the record, it was Rich who discovered, and told me, that Leiningen was adding extra JVM args. :) -S On Tuesday, June 18, 2013 2:29:00 PM UTC-4, Jason Wolfe wrote: The good news is that Stuart Sierra nailed the problem above -- -- You received this message because you are subscribed to

Re: Clojure generates unnecessary and slow type-checks

2013-06-18 Thread Leon Barrett
Holy crap, the ^:replace seems to improve my example to the performance of Java! I'm looking further, but that may be the key. On Tue, Jun 18, 2013 at 11:12 AM, Stuart Sierra the.stuart.sie...@gmail.com wrote: One has to be very careful with this kind of micro-benchmarking on the JVM.

Re: Clojure generates unnecessary and slow type-checks

2013-06-18 Thread Stuart Sierra
Great. Glad we found it! Since this was so confusing, I've filed an issue with Leiningen to make :jvm-opts in a project.clj override any Leiningen defaults: https://github.com/technomancy/leiningen/pull/1230 -S -- -- You received this message because you are subscribed to the Google Groups

Re: Clojure generates unnecessary and slow type-checks

2013-06-18 Thread Jason Wolfe
Yes, I agree that this is very confusing behavior -- I just chimed in on the pull request. Thanks again for your help and this follow-up. On Tue, Jun 18, 2013 at 12:03 PM, Stuart Sierra the.stuart.sie...@gmail.com wrote: Great. Glad we found it! Since this was so confusing, I've filed an

Re: Clojure generates unnecessary and slow type-checks

2013-06-16 Thread Jason Wolfe
On Fri, Jun 14, 2013 at 5:46 PM, Mikhail Kryshen mikh...@kryshen.net wrote: JIT will probably remove unnecessary checkcast instructions. What looks suspicious to me is that i and asize are converted to longs (notice i2l opcodes). I noticed earlier that loops with long counters are measurably

Re: Clojure generates unnecessary and slow type-checks

2013-06-16 Thread Jason Wolfe
On Sat, Jun 15, 2013 at 4:37 AM, Mikera mike.r.anderson...@gmail.com wrote: On Friday, 14 June 2013 18:15:34 UTC+1, Jason Wolfe wrote: Hey Mikera, I did look at core.matrix awhile ago, but I'll take another look. Right now, flop is just trying to make it easy to write *arbitrary* array

Re: Clojure generates unnecessary and slow type-checks

2013-06-15 Thread Mikera
On Friday, 14 June 2013 18:15:34 UTC+1, Jason Wolfe wrote: Hey Mikera, I did look at core.matrix awhile ago, but I'll take another look. Right now, flop is just trying to make it easy to write *arbitrary* array operations compactly, while minimizing the chance of getting

Re: Clojure generates unnecessary and slow type-checks

2013-06-14 Thread Glen Mailer
This doesn't really answer your question directly, but is there a reason you need to keep this in clojure, or are you just aiming to establish why this is happening? My understanding was that for performance critical code the general advice is to drop down to raw java? Glen -- -- You

Re: Clojure generates unnecessary and slow type-checks

2013-06-14 Thread Jason Wolfe
Thanks for your response. I attempted to answer this in my clarification, but our goal is to attack this 'general advice' and make it possible to get the same speed for array handling in natural-seeming Clojure without writing Java. In particular, we want to create macros that make it easy to

Re: Clojure generates unnecessary and slow type-checks

2013-06-14 Thread David Nolen
Maybe this is an unintended side effect of the changes around unboxed support in loop/recur? On Thu, Jun 13, 2013 at 10:50 PM, Jason Wolfe ja...@w01fe.com wrote: Taking a step back, the core problem we're trying to solve is just to sum an array's values as quickly as in Java. (We really want

Re: Clojure generates unnecessary and slow type-checks

2013-06-14 Thread Mikera
Hi Jason, Have you guys taken a look at core.matrix for any of this stuff? We're also shooting for near-Java-parity for all of the core operations on large double arrays. (use 'clojure.core.matrix) (require '[criterium.core :as c]) (let [a (double-array (range 1))] (c/quick-bench

Re: Clojure generates unnecessary and slow type-checks

2013-06-14 Thread Jason Wolfe
Hey Mikera, I did look at core.matrix awhile ago, but I'll take another look. Right now, flop is just trying to make it easy to write *arbitrary* array operations compactly, while minimizing the chance of getting worse-than-Java performance. This used to be very tricky to get right when flop

Re: Clojure generates unnecessary and slow type-checks

2013-06-14 Thread Mikhail Kryshen
JIT will probably remove unnecessary checkcast instructions. What looks suspicious to me is that i and asize are converted to longs (notice i2l opcodes). I noticed earlier that loops with long counters are measurably slower for the same number of iterations (probably, HotSpot does not apply some

Clojure generates unnecessary and slow type-checks

2013-06-13 Thread Leon Barrett
Hi. I've been working with people at Prismatic to optimize some simple math code in Clojure. However, it seems that Clojure generates an unnecessary type check that slows our (otherwise-optimized) code by 50%. Is there a good way to avoid this, is it a bug in Clojure 1.5.1, or something else?

Re: Clojure generates unnecessary and slow type-checks

2013-06-13 Thread Jason Wolfe
Taking a step back, the core problem we're trying to solve is just to sum an array's values as quickly as in Java. (We really want to write a fancier macro that allows arbitrary computations beyond summing that can't be achieved by just calling into Java, but this simpler task gets at the crux