Re: Clojure performance tests and clojure a little slower than Java

[Andy Fingerhut]

I have added a script that uses the Java version of the benchmark
programs to generate the large files that were in the distribution
file I gave a link to earlier, so it is much smaller.  I've also
published it on github and added a COPYING file that makes the
licenses more explicit (revised BSD for everything, some copyright by
me, the rest by the owner of the benchmark web site).  You can get it
here:

git://github.com/jafingerhut/clojure-benchmarks.git

Again, submissions for new benchmark programs, or improvements to
existing ones, are welcome.

Thanks,
Andy


On Jul 28, 8:52 am, AndyF  wrote:
> Anybody is welcome to take the Clojure sources and do what they will
> with them -- publish the original or modified code under their own
> license, sell it, etc.
>
> For the Perl, Common Lisp, Haskell, etc. sources, those are published
> under this license.  It isn't my code.
>
> http://shootout.alioth.debian.org/u32q/miscfile.php?file=license&titl...
>
> Andy
>
> On Jul 28, 5:09 am, Berlin Brown  wrote:
>
> > Thanks AndyF, do you mind if I use some of your examples and put my
> > own spin on them.  I was curious and want to run my own tests and see
> > if I get similar output?
>
> > On Jul 28, 12:03 am, ataggart  wrote:
>
> > > On Jul 27, 10:06 am, BerlinBrown  wrote:
>
> > > > So far I have found that clojure is about
> > > > 5-10 times as slow as comparable code in Clojure.
>
> > > I infer you mean clojure execution times are about 5-10 times longer
> > > than in java.  It depends on what you're doing, but that sounds
> > > plausible.
>
> > > Here's are some slides of a comparison between a number of JVM
> > > languages:
>
> > >http://www.slideshare.net/michael.galpin/performance-comparisons-of-d...
>
> > > Note that in the "reversible" section, he has a loop in the clojure
> > > code with unnecessary Integer object creation which ends up doubling
> > > the execution time.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Good memory profilers (program or human)?

[Andy Fingerhut]

I'm gradually adding a few more Clojure benchmark programs to my
repository here:

git://github.com/jafingerhut/clojure-benchmarks.git

The one I wrote for the "reverse-complement" benchmark is here:

http://github.com/jafingerhut/clojure-benchmarks/blob/4ab4f41c6f963445e1e0973a668ac64939be1c7f/rcomp/revcomp.clj-4.clj

revcomp.clj-4.clj is the best I've got so far, but it runs out of
memory on the full size benchmark.

If you clone the repository, and successfully run the init.sh script
to generate the big input and expected output files, the file rcomp/
long-input.txt contains 3 DNA sequences in FASTA format. The first is
50,000,000 characters long, the second is 75,000,000 characters long,
and the third is 125,000,000 characters long. Each needs to be
reversed, have each character replaced with a different one, and
printed out, so we need to store each of the strings one at a time,
but it is acceptable to deallocate/garbage-collect the previous one
when starting on the next. I think my code should be doing that, but I
don't know how to verify that.

I've read that a Java String takes 2 bytes per character, plus about
38 bytes of overhead per string. That is about 250 Mbytes for the
longest one. I also read in a seq of lines, and these long strings are
split into lines with 60 characters (plus a newline) each. Thus the
string's data needs to be stored at least twice temporarily -- once
for the many 60-character strings, plus the final long one.  Also, the
Java StringBuilder that Clojure's (str ...) function uses probably
needs to be copied and reallocated periodically as it outgrows its
current allocation. So I could imagine needing about 3 * 250 Mbytes
temporarily, but that doesn't explain why my 1536 Mbytes of JVM memory
are being exhausted.

It would be possible to improve things by not creating all of the
separate strings, one for each line, and then concatenating them
together. But first I'd like to explain why it is using so much,
because I must be missing something.

Thank,
Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Good memory profilers (program or human)?

[Andy Fingerhut]

I thought I'd follow up my own question with some programs that I
should have already known about for memory profiling, which were
already installed on my Mac as part of the standard Java installation
from Apple (who are just passing them on from Sun, I'm sure), but I
didn't know about them:

jconsole -- Good for seeing how fast a java process is allocating
memory, and garbage collecting.

jmap -- Good for a quick summary of the above, or with the "-histo"
option, a much more detailed list of what kinds of objects are taking
up the most memory.

I also learned that Clojure's cons and lazy cons structures take up 48
bytes per element (at least on a Mac with 'java -client' and java
1.6.0_), which gets significant when your program has sequences
of several millions of elements about.

I've updated my github repo with a pretty decent version of the
reverse-complement benchmark in Clojure.  It isn't as sequence-y as it
could be, but the more sequence-y version generates and collects
garbage so fast that it really slows things down significantly.  Same
lesson from other flavors of Lisp, I guess -- you can write the
straightforward easy-to-write-and-test-and-understand code that conses
a lot (i.e. allocates memory quickly that typically becomes garbage
quite soon), or you can write the more loopy code that doesn't, but
typically starts to merge many things that you'd otherwise prefer to
separate into different functions.  Just compare revcomp.clj-5.clj and
revcomp.clj-6.clj in my git repo for an example.

The nice thing is that when you don't need the "uglier" code, Clojure
and other Lisps usually let you write code much more concisely than
lower level languages.  Get it working first, then optimize it.  Since
I'm comparing run times of the Clojure programs versus those submitted
to the language shootout benchmark web site, some of which appear
quite contorted in order to gain performance, I wanted to do some
optimizations that you wouldn't necessarily want to do otherwise.

git://github.com/jafingerhut/clojure-benchmarks.git

You can see my latest run time results here.  I've got 4 benchmarks
written in Clojure so far, with my current versions being 6x, 8x, 12x,
and 15x more CPU time than the Java programs submitted to the language
shootout benchmark web site.

http://github.com/jafingerhut/clojure-benchmarks/blob/20d21bc169d52ca52d6a8281536838662c54e854/RESULTS

I could make some of these significantly closer in speed to the Java
versions, but I suspect that they will start looking more and more
like the Java versions if I do, except with Clojure syntax for Java
calls.  I'm happy to be proved wrong on that, if someone finds better
Clojure versions than I've got.

Thanks,
Andy


On Jul 30, 11:00 am, Andy Fingerhut 
wrote:
> I'm gradually adding a few more Clojure benchmark programs to my
> repository here:
>
> git://github.com/jafingerhut/clojure-benchmarks.git
>
> The one I wrote for the "reverse-complement" benchmark is here:
>
> http://github.com/jafingerhut/clojure-benchmarks/blob/4ab4f41c6f96344...
>
> revcomp.clj-4.clj is the best I've got so far, but it runs out of
> memory on the full size benchmark.
>
> If you clone the repository, and successfully run the init.sh script
> to generate the big input and expected output files, the file rcomp/
> long-input.txt contains 3 DNA sequences in FASTA format. The first is
> 50,000,000 characters long, the second is 75,000,000 characters long,
> and the third is 125,000,000 characters long. Each needs to be
> reversed, have each character replaced with a different one, and
> printed out, so we need to store each of the strings one at a time,
> but it is acceptable to deallocate/garbage-collect the previous one
> when starting on the next. I think my code should be doing that, but I
> don't know how to verify that.
>
> I've read that a Java String takes 2 bytes per character, plus about
> 38 bytes of overhead per string. That is about 250 Mbytes for the
> longest one. I also read in a seq of lines, and these long strings are
> split into lines with 60 characters (plus a newline) each. Thus the
> string's data needs to be stored at least twice temporarily -- once
> for the many 60-character strings, plus the final long one.  Also, the
> Java StringBuilder that Clojure's (str ...) function uses probably
> needs to be copied and reallocated periodically as it outgrows its
> current allocation. So I could imagine needing about 3 * 250 Mbytes
> temporarily, but that doesn't explain why my 1536 Mbytes of JVM memory
> are being exhausted.
>
> It would be possible to improve things by not creating all of the
> separate strings, one for each line, and then concatenating them
> together. But first I'd like to expla

Re: Transient Data Structures

[Andy Fingerhut]

I like it.  I can see significant use of these to help speed up some
of the benchmark programs I've been hacking on:

git://github.com/jafingerhut/clojure-benchmarks.git

and more importantly, that means they can be good in optimizing useful
code, too :-)

I was pondering this question "If a pure function mutates some local
data in order to produce an immutable return value, is that ok?" that
Rich posed just a day or three ago, when thinking about whether there
was a way to automatically determine whether a function is pure /
"referentially transparent" or not (at least for a useful subset of
actual Clojure code).  In particular, Clojure's str function does this
already with StringBuilder.append:

(defn str
;; doc string deleted for brevity
  ([] "")
  ([#^Object x]
   (if (nil? x) "" (. x (toString
  ([x & ys]
 ((fn [#^StringBuilder sb more]
  (if more
(recur (. sb  (append (str (first more (next more))
(str sb)))
  (new StringBuilder #^String (str x)) ys)))

Very nice to see the idea generalized to Clojure data structures, too,
and the safety nets built into it in case someone forgets to call
"persistent!" on a data structure before using it in another thread.

Thanks,
Andy


On Aug 3, 2:25 pm, Rich Hickey  wrote:
> I've been doing some work on Transient Data Structures. You can read
> about them here:
>
> http://clojure.org/transients
>
> Feedback welcome,
>
> Rich

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

pmap uses more parallelism than intended due to use of "eager" map?

[Andy Fingerhut]

I was looking into the question raised in the "Question about pmap"
thread, and noticed that on my Mac and on a Linux virtual machine, a
recent git version of clojure (about 1 week old) seems to use more
parallelism in 'pmap' than its source code in core.clj would imply is
the intent.  The code there seems to imply that the desired number of
threads to run at once is 2 more than the number of available
processors.  However, when I run it, it always fires off one thread
for every element of the collection, right at the beginning,
regardless of the number of available processors.

I created my own slightly modified version of pmap, where the only
difference is that the version of 'map' used has been changed to 'my-
lazy-map', which is explicitly lazy.  See this link for the source
code of the test I was using.  Ignore modified-pmap1.  Look at
modified-pmap2.

http://github.com/jafingerhut/clojure-benchmarks/blob/0227501c6c53736db05facd647c09e1d4e9e77b3/misc/pmap-testing.clj

You can try this out on your system with a command line like this:

clj pmap-test.clj 10 1

You might need to adjust the second number to be more or less,
depending upon the speed of your machine.  It would be good to make it
take at least a couple of seconds.  The first number is the length of
the collection over which map/pmap is run, and for most obvious
results should be about 3 to 4 times the number of processors on your
machine.

I suspect that the issue is that the version of map used by pmap in
core.clj is eager, i.e. not lazy, and thus all of the called to future
are performed as soon as the last of these lines in the pmap function
is executed:

(defn pmap
  ;; doc string deleted for brevity
  ([f coll]
   (let [n (+ 2 (.. Runtime getRuntime availableProcessors))
 rets (map #(future (f %)) coll)

but I don't know yet where the implementation of map that is "active"
at that point in the source code is.  Perhaps it is implemented in
Java?

Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Question about pmap

[Andy Fingerhut]

Johann:

Could it be that your CPU has a single floating-point unit shared by 4
cores on a single die, and thus only 2 floating-point units total for
all 8 of your cores?  If so, then that fact, plus the fact that each
core has its own separate ALU for integer operations, would seem to
explain the results you are seeing.  I see similar results on my 2-
core Intel Core 2 Duo when running your test.

I tried to check out that possibility by doing some searches on the
Internet about the Xeon, but couldn't quickly find a definitive
answer.  I'm hoping someone else on the group might know the answer
off the top of their head.

Andy


On Aug 4, 1:59 am, Johann Kraus  wrote:
> > My guess would be you're seeing the overhead for pmap since the
> > (inc 0.1) computation is so cheap.  From the docs for pmap:
> >   "Only useful for computationally intensive functions where the time of
> >   f dominates the coordination overhead."
>
> I don't think so, as the cheap computation (inc 0.1) is packed into
> the long running (dotimes). And it does work for using (inc 0), where
> I get a perfect linear speedup. Just by using (inc 0.1) instead of
> (inc 0) the speedup is rapidly getting down while the CPU load is next
> to the limit.
>
> > Maybe try this with something more expensive to compute?  Also, making
> > the result a function of the parameter can't hurt since there's a chance
> > the JVM may optimize away the inc of a constant as it stands.
>
> I also would expect the JVM to optimize away something, but for me it
> seems it does not, because again in case of (inc 0) everything works
> perfect.
>
> The point is:
> Why do I get a linear speedup with (inc 0) -> 8 times and not with
> (inc 0.1) -> 2 times, while the CPU load is 800% and 700%,
> respectively?
> __
> Johann

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: pmap uses more parallelism than intended due to use of "eager" map?

[Andy Fingerhut]

Ugh.  A couple minutes of more careful searching and I had the
answer.  It isn't because the map used by pmap is eager, it is because
it optimizes for input collections that are chunked.

It seems worth considering modifying pmap to either:

(1) use a lazy version of map, with no optimization for chunked
collections.  That's pretty quick and easy.

or

(2) modify the pmap implementation so that even if map optimized for
chunked collections, pmap doesn't use more parallelism than intended.
That would require a more significant code change in pmap.

Thanks,
Andy


On Aug 4, 1:31 pm, Andy Fingerhut 
wrote:
> I was looking into the question raised in the "Question about pmap"
> thread, and noticed that on my Mac and on a Linux virtual machine, a
> recent git version of clojure (about 1 week old) seems to use more
> parallelism in 'pmap' than its source code in core.clj would imply is
> the intent.  The code there seems to imply that the desired number of
> threads to run at once is 2 more than the number of available
> processors.  However, when I run it, it always fires off one thread
> for every element of the collection, right at the beginning,
> regardless of the number of available processors.
>
> I created my own slightly modified version of pmap, where the only
> difference is that the version of 'map' used has been changed to 'my-
> lazy-map', which is explicitly lazy.  See this link for the source
> code of the test I was using.  Ignore modified-pmap1.  Look at
> modified-pmap2.
>
> http://github.com/jafingerhut/clojure-benchmarks/blob/0227501c6c53736...
>
> You can try this out on your system with a command line like this:
>
> clj pmap-test.clj 10 1
>
> You might need to adjust the second number to be more or less,
> depending upon the speed of your machine.  It would be good to make it
> take at least a couple of seconds.  The first number is the length of
> the collection over which map/pmap is run, and for most obvious
> results should be about 3 to 4 times the number of processors on your
> machine.
>
> I suspect that the issue is that the version of map used by pmap in
> core.clj is eager, i.e. not lazy, and thus all of the called to future
> are performed as soon as the last of these lines in the pmap function
> is executed:
>
> (defn pmap
>   ;; doc string deleted for brevity
>   ([f coll]
>    (let [n (+ 2 (.. Runtime getRuntime availableProcessors))
>          rets (map #(future (f %)) coll)
>
> but I don't know yet where the implementation of map that is "active"
> at that point in the source code is.  Perhaps it is implemented in
> Java?
>
> Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Question about pmap

[Andy Fingerhut]

On Aug 5, 6:09 am, Rich Hickey  wrote:
> On Wed, Aug 5, 2009 at 8:29 AM, Johann Kraus wrote:
>
> >> Could it be that your CPU has a single floating-point unit shared by 4
> >> cores on a single die, and thus only 2 floating-point units total for
> >> all 8 of your cores?  If so, then that fact, plus the fact that each
> >> core has its own separate ALU for integer operations, would seem to
> >> explain the results you are seeing.
>
> > Exactly, this would explain the behaviour. But unfortunately it is not
> > the case. I implemented a small example using Java (Java Threads) and
> > C (PThreads) and both times I get a linear speedup. See the attached
> > code below. The cores only share 12 MB cache, but this should be
> > enough memory for my micro-benchmark. Seeing the linear speedup in
> > Java and C, I would negate a hardware limitation.
>
> > _
> > Johann
>
>
> I looked briefly at your problem and don't see anything right off the
> bat. Do you have a profiler and could you try that out? I'm
> interested.
> Rich

I ran these tests on my iMac with 2.16 GHz Intel Core 2 Duo (2 cores)
using latest Clojure and clojure-contrib from git as of some time on
Aug 4, 2009.  The Java implementation is from Apple, version 1.6.0_13.

--
For int, there are 64 "jobs" run, each of which consists of doing
(inc 0) 1,000,000,000 times.  See pmap-batch.sh and pmap-testing.clj
for details.

http://github.com/jafingerhut/clojure-benchmarks/blob/398688c71525964ba4c2d55d0e487b7618efdc8b/misc/pmap-batch.sh

http://github.com/jafingerhut/clojure-benchmarks/blob/398688c71525964ba4c2d55d0e487b7618efdc8b/misc/pmap-testing.clj

Yes, yes, I know.  I should really use a library for command line
argument parsing to avoid so much repetitive code.  I may do that some
day.


Results for int 1 thread - jobs run sequentially

"Elapsed time: 267547.789 msecs"
real   269.22
user   268.61
sys  1.79

int 2 threads - jobs run in 2 threads using modified-pmap, which
limits the number of futures causing threads to run jobs to be at most
2 at a time.

"Elapsed time: 177428.626 msecs"
real   179.14
user   330.30
sys 15.46

Comment: Elapsed time with 2 threads is about 2/3 of elapsed time with
1 thread.  Not as good as the 1/2 as we'd like with a 2 core machine,
but better than not being faster at all.

--
For double, there are 16 "jobs" run, each of which consists of doing
(inc 0.1) 1,000,000,000 times.

double 1 thread

"Elapsed time: 258659.424 msecs"
real   263.28
user   247.29
sys 12.17

double 2 threads

"Elapsed time: 229382.68 msecs"
Dumping CPU usage by sampling running threads ... done.
real   231.05
user   380.79
sys 11.49

Comment: Elapsed time with 2 threads is about 7/8 of elapsed time with
1 thread.  Hardly any improvement at all for something that should be
"embarrassingly parallel", and the user time reported by Mac OS X's
/usr/bin/time increased by a factor of about 1.5.  That seems like way
too much overhead for thread coordination.


Here are hprof output files for the "double 1 thread" and "double 2
threads" tests:

http://github.com/jafingerhut/clojure-benchmarks/blob/51d499c2679c2d5ed42a65adcef6d9e8bc3e1aad/misc/pmap-batch-output/pmap-double-1-hprof.txt

http://github.com/jafingerhut/clojure-benchmarks/blob/51d499c2679c2d5ed42a65adcef6d9e8bc3e1aad/misc/pmap-batch-output/pmap-double-2-hprof.txt

In both cases, over 98% of the time is spent in
java.lang.Double.valueOf(double d).  See the files for the full stack
backtraces if you are curious.

I don't see any reason why that method should have any kind of
contention or worse performance when running on 2 cores vs. 1 core,
but I don't know the guts of how it is implemented.  At least in
OpenJDK all it does is "return new Double(d)", where d is the double
arg to valueOf().  Is there any reason why "new" might exhibit
contention between parallel threads?

Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Question about pmap

[Andy Fingerhut]

On Aug 6, 10:00 am, Bradbev  wrote:
> On Aug 6, 3:07 am, Andy Fingerhut 
> wrote:
>
>
>
> > On Aug 5, 6:09 am, Rich Hickey  wrote:
>
> > > On Wed, Aug 5, 2009 at 8:29 AM, Johann Kraus 
> > > wrote:
>
> > > >> Could it be that your CPU has a single floating-point unit shared by 4
> > > >> cores on a single die, and thus only 2 floating-point units total for
> > > >> all 8 of your cores?  If so, then that fact, plus the fact that each
> > > >> core has its own separate ALU for integer operations, would seem to
> > > >> explain the results you are seeing.
>
> > > > Exactly, this would explain the behaviour. But unfortunately it is not
> > > > the case. I implemented a small example using Java (Java Threads) and
> > > > C (PThreads) and both times I get a linear speedup. See the attached
> > > > code below. The cores only share 12 MB cache, but this should be
> > > > enough memory for my micro-benchmark. Seeing the linear speedup in
> > > > Java and C, I would negate a hardware limitation.
>
> > > > _
> > > > Johann
>
> > > I looked briefly at your problem and don't see anything right off the
> > > bat. Do you have a profiler and could you try that out? I'm
> > > interested.
> > > Rich
>
> > I ran these tests on my iMac with 2.16 GHz Intel Core 2 Duo (2 cores)
> > using latest Clojure and clojure-contrib from git as of some time on
> > Aug 4, 2009.  The Java implementation is from Apple, version 1.6.0_13.
>
> > --
> > For int, there are 64 "jobs" run, each of which consists of doing
> > (inc 0) 1,000,000,000 times.  See pmap-batch.sh and pmap-testing.clj
> > for details.
>
> >http://github.com/jafingerhut/clojure-benchmarks/blob/398688c71525964...
>
> >http://github.com/jafingerhut/clojure-benchmarks/blob/398688c71525964...
>
> > Yes, yes, I know.  I should really use a library for command line
> > argument parsing to avoid so much repetitive code.  I may do that some
> > day.
>
> > Results for int 1 thread - jobs run sequentially
>
> > "Elapsed time: 267547.789 msecs"
> > real       269.22
> > user       268.61
> > sys          1.79
>
> > int 2 threads - jobs run in 2 threads using modified-pmap, which
> > limits the number of futures causing threads to run jobs to be at most
> > 2 at a time.
>
> > "Elapsed time: 177428.626 msecs"
> > real       179.14
> > user       330.30
> > sys         15.46
>
> > Comment: Elapsed time with 2 threads is about 2/3 of elapsed time with
> > 1 thread.  Not as good as the 1/2 as we'd like with a 2 core machine,
> > but better than not being faster at all.
>
> > --
> > For double, there are 16 "jobs" run, each of which consists of doing
> > (inc 0.1) 1,000,000,000 times.
>
> > double 1 thread
>
> > "Elapsed time: 258659.424 msecs"
> > real       263.28
> > user       247.29
> > sys         12.17
>
> > double 2 threads
>
> > "Elapsed time: 229382.68 msecs"
> > Dumping CPU usage by sampling running threads ... done.
> > real       231.05
> > user       380.79
> > sys         11.49
>
> > Comment: Elapsed time with 2 threads is about 7/8 of elapsed time with
> > 1 thread.  Hardly any improvement at all for something that should be
> > "embarrassingly parallel", and the user time reported by Mac OS X's
> > /usr/bin/time increased by a factor of about 1.5.  That seems like way
> > too much overhead for thread coordination.
>
> > Here are hprof output files for the "double 1 thread" and "double 2
> > threads" tests:
>
> >http://github.com/jafingerhut/clojure-benchmarks/blob/51d499c2679c2d5...
>
> >http://github.com/jafingerhut/clojure-benchmarks/blob/51d499c2679c2d5...
>
> > In both cases, over 98% of the time is spent in
> > java.lang.Double.valueOf(double d).  See the files for the full stack
> > backtraces if you are curious.
>
> > I don't see any reason why that method should have any kind of
> > contention or worse performance when running on 2 cores vs. 1 core,
> > but I don't know the guts of how it is implemented.  At least in
> > OpenJDK all it does is "return new Double(d)", where d is the double
> > arg to valueOf().  Is there any reason why "

Re: Transient Data Structures

[Andy Fingerhut]

On Aug 6, 4:53 am, Rich Hickey  wrote:
> On Aug 5, 10:10 pm, Luc Prefontaine 
> wrote:
>
> > I like this very much... that's the kind of clever optimizations that
> > preserves Clojure principles and
> > can yield significant performance increases. This could also help
> > dealing with performance critics
> > in these small mutable languages "benchmarks" that newbies attempt to
> > clone in Clojure.
>
> > Thank's Rich !
>
> You're welcome!
>
> And special thanks to Christophe Grand, who (quickly!) applied the
> same technique to the hash maps and contributed that yesterday. So
> now, in the master branch, vectors and hash maps support transients.
> Everyone please try them out (where appropriate :).
>
> Rich

Thank you, Christophe!  I've been wanting to try those out.

I made changes to 3 lines of my Clojure program for the k-nucleotide
benchmark, which spends most of its time in a function tally-dna-subs-
with-len that creates a hash map counting the number of times that
each of a bunch of length k strings occurs in a long string.  Source
here, if you're curious:

http://github.com/jafingerhut/clojure-benchmarks/blob/38e1f592ca3befe94a0674a5f5a43d952cd369b3/knuc/knucleotide.clj-7.clj

It went from about 19 minutes down to about 12 minutes.  Excellent
improvement for a small change to the code.  That brings it down to
about 7.7 times the run time of the Java version from the language
shootout web site.

Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Clojure performance tests and clojure a little slower than Java

[Andy Fingerhut]

On Aug 6, 3:28 pm, André Thieme  wrote:
> On 27 Jul., 23:26, AndyF  wrote:
>
> Hello Andy, could you please update the following table?
>
>
>
> >         |  sbcl  |  perl  |   ghc  |  java |   clj
> > -
> > mand-   | wrong  | out of |  32.7  |  28.6  | 340.4
> > elbrot  | output | mem    |  59.3  |  54.4  | 350.5
> >         |        | (?)    |   0.8  |   0.4  |   4.7
>
> > k-nuc-  | 190.9  | 306.0  |  90.5  |  52.4  | 1677.6 (27m 57.6s)
> > leotide | 187.9  | 302.7  | 130.8  |  89.6  | 2245.1 (37m 25.1s)
> >         |   2.4  |   1.9  |   4.6  |   1.8  |   24.2 (    24.2s)
>
> When I understand you correctly, you now use Transients
> in those benchmarks, which improved their runtime behaviour.

You are correct.  I've updated that file:

http://github.com/jafingerhut/clojure-benchmarks/blob/bb9755bdeeccae84a9b09fbf34e45f6d45d4b627/RESULTS

I've attempted to always put the best Clojure program results I've
found so far on the same line with the run times for the programs in
other languages, and then sometimes I will also include the second
best Clojure results on a separate group of lines after that.

Thanks,
Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Question about pmap

[Andy Fingerhut]

On Aug 6, 11:51 am, John Harrop  wrote:
> Cache misses are a possibility; try the integer version with long, so the
> size of the data is the same as with double.
> The other possibility I'd consider likely is that the JDK you were using
> implements caching in Double.valueOf(double). This could be dealt with if
> Clojure boxing directly called new Double(double). Caching in valueOf
> methods has been known to cause trouble in other situations, involving Java,
> primitive boxing, and performance. Besides the caching wrecking memory
> access-pattern locality, causing CPU cache misses, it likely involves
> acquiring and releasing a global lock on the Double cache, which will kill
> multithreaded performance particularly.

Interesting how the caching can mess performance up, especially in
multi-threaded situations.  I don't know yet how to confirm whether my
implementation uses that caching, but I did see a mention of it in
some docs for Double.valueOf(double), and didn't think about the
possibility of that introducing locking.  In this particular test, I
doubt it would increase non-locality much if at all, since we are
always providing the same value (0.1) to the method, so the cache
should have exactly 1 element in it the whole time.  But if it were
synchronized, that would definitely bring down the performance of this
particular test, or in general any test with a significant number of
float or double operations, down closer to that of a single core.

If that is the performance bottleneck in this case, is there any way
around it?  Perhaps I could try locally changing my Clojure Java
source to use new Double instead of Double.valueOf(double) and it
might actually be faster.

I'll try out some variations with different types like long and float,
and report back when I've got them.

Thanks,
Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Clojure performance tests and clojure a little slower than Java

[Andy Fingerhut]

On Aug 6, 6:49 pm, John Harrop  wrote:
> On Thu, Aug 6, 2009 at 6:57 PM, Andy Fingerhut <
>
> andy_finger...@alum.wustl.edu> wrote:
> > You are correct.  I've updated that file:
>
> >http://github.com/jafingerhut/clojure-benchmarks/blob/bb9755bdeeccae8...
>
> Could you post the Mandelbrot code you use? Because I know for a fact that
> Clojure can do FP calcs as fast as native C code, given the -server vm and a
> loop/recur with unboxed doubles, and my understanding of Mandelbrot is that
> it's just FP calcs.

Here is the root of the clojure-benchmarks github directory tree.
They are all in there:

http://github.com/jafingerhut/clojure-benchmarks/tree/master

You can get specific files by browsing the directory tree on the web
site, or if you use git, you can git clone the whole thing and have a
local copy on your computer:

git clone git://github.com/jafingerhut/clojure-benchmarks.git

It is mandelbrot/mandelbrot.clj-1.clj.  There is a
mandelbrot.clj-2.clj that is the same, except it uses my modified-pmap
to compute separate rows in parallel.  I'm still investigating reasons
why parallelism isn't getting me much bang for the buck in the
"questions about pmap" thread before worrying performance turning more
complex programs using floats/doubles in parallel.

I did go back to the archives for a discussion in early April, 2009
when others on this list were discussing how to tweak the core of the
mandelbrot code (the function "dot" in my version), and I used the
fastest of those versions I could find, but I may have missed some
opportunities for speeding it up, and I definitely welcome any and all
improved versions.  I'd recommend comparing the speed on your machine
vs. the Java version that is in the same directory as the Clojure
source.

Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Test a random function

[Andy Fingerhut]

On Aug 7, 6:17 pm, Sean Devlin  wrote:
> Ok, I need some help.  I'm writing some tests for c.c.seq-utils, and I
> ran into a problem defining a test for both shuffle and rand-elt.
>
> Does anyone here have any experience writing tests for random
> functions?  Am I going to need to use serious statistics the answer
> this?
>
> Ideas?

It depends upon what you want to test.  If you want to test that rand-
elt selects an item from the seq s independently on each run, and with
equal probability for all elements, then that is effectively like
testing rand-int for the same properties, and yes, you will have to
get into some statistics to do so.

However, if you back off from testing the random part of the function,
there are some "invariants" that are always true that you could test,
such as:

(1) the value x returned by rand-elt should always be a member of the
seq s

(2) the value returned by shuffle should always have the same number
of each item as the input seq s.  That is, if you evaluate the
function 'tally' below on the input seq s and on the returned seq t,
you should always find (= (tally s) (tally t)) is true.

(defn tally [s]
  (loop [s s
 t (transient {})]
(if-let [s (seq s)]
  (let [item (first s)]
(recur (next s) (assoc! t item (inc (get t item 0)
  (persistent! t

Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Clojure performance tests and clojure a little slower than Java

[Andy Fingerhut]

On Aug 7, 5:14 pm, John Harrop  wrote:
> Your core loop seems to be:
> (loop [zr (double 0.0)
>          zi (double 0.0)
>          zr2 (double 0.0)
>          zi2 (double 0.0)
>          iterations-remaining iterations-remaining]
>     (if (and (not (neg? iterations-remaining))
>              (< (+ zr2 zi2) limit-square))
>       (let [new-zi (double (+ (* (* f2 zr) zi) pi))
>             new-zr (double (+ (- zr2 zi2) pr))
>             new-zr2 (double (* new-zr new-zr))
>             new-zi2 (double (* new-zi new-zi))]
>         (recur new-zr new-zi new-zr2 new-zi2 (dec iterations-remaining)))
>
> What I suggest is
>
> (loop [zr (double 0.0)
>        zi (double 0.0)
>        i (int (inc iterations-remaining))]
>   (let [zr2 (* zr zr)
>         zi2 (* zi zi)]
>     (if (and (not (= 0 i)) (< (+ zr2 zi2 limit-square)))
>        (recur (+ (- zr2 zi2) pr) (+ (* (* f2 zr) zi) pi) (unchecked-inc i))
>        (whatever...
>
> * Same calculations
> * Less items in recur rebind
> * Counter is also primitive
>
> (untested, may need slight tweakage)

Needed unchecked-dec in place of unchecked-inc, and I used (zero? i)
instead of (= 0 i) (not sure if that makes much difference), and the
results are much better -- the time went down to a little less than
half of what it was before.

http://github.com/jafingerhut/clojure-benchmarks/blob/fe10ef25ec17b77dd03f6d1ccff4f35273764f3b/RESULTS

I also tried it with 4 parallel threads on my 2-core machine, and this
problem parallelizes significantly better than the simpler problem in
the "questions about pmap" discussion thread -- nearly half the
elapsed time, and only 6% more total CPU time for the 4-thread version
vs. the sequential version.

Thanks!
Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Question about pmap

[Andy Fingerhut]

Johann, if you are still following this thread, could you try running
this Clojure program on your 8 core machine?

http://github.com/jafingerhut/clojure-benchmarks/blob/3e45bd8f6c3eba47f982a0f6083493a9f076d0e9/misc/pmap-testing.clj

These first set of parameters below will do 8 jobs sequentially, each
doing 10^10 (inc c)'s, where c is a double primitive.  The second will
do the 8 jobs in parallel, hopefully finishing about 8 times faster on
your machine:

java -cp  clojure.main pmap-testing.clj double2 8
100 1
java -cp  clojure.main pmap-testing.clj double2 8
100 8

If you replace double2 with double1, it should reproduce your initial
test case with (inc 0.1) in the inner loop -- the one that started
this thread about why there wasn't much speedup.


I created some Clojure and Java functions that are as similar as I
know how to make them, but frankly I don't really know whether my JVM
implementation (Apple's java 1.6.0_13) is using 'new Double', or a
cache as mentioned by John Harrop earlier in this discussion, in its
implementation of Double.valueOf(double).  I've found that the
performance is very similar to a Java program that uses 'new Double'
explicitly in its inner loop.

In the results linked below, the 'sequential' runs are for calling the
function named twice in a row, whereas the 'parallel' runs are for
having two parallel threads, on my 2 core machine, each of which calls
the named function once.  The total amount of computation in each case
should be the same, so you'd hope that the parallel case would finish
in about half the time, with about the same amount of total CPU time
being utilized.

That isn't what happens for the Java code with 'new Double' (called
NewDoubleTest), or for the Clojure code that has (inc 0.1) and calls
Double.valueOf(double) down in its implementation (called spin-
double1).  The parallel case only saves about 14% of the elapsed time
with 2 cores, and takes about 50% more total CPU time.

The more typical code that takes a single double value, initializes
it, and then adds 1 to it each time through the inner loop (Java
DoubleTest, Clojure spin-double2), is significantly faster than the
versions mentioned above.  Plus they exhibit the expected speedup from
using 2 cores instead of 1.

http://github.com/jafingerhut/clojure-benchmarks/blob/3e45bd8f6c3eba47f982a0f6083493a9f076d0e9/misc/RESULTS

I'm not sure how to determine why calling 'new Double' each time
through NewDoubleTest's inner loop causes 2 threads to perform not
much better than 1.  The best possible explanation I've heard is from
Nicolas Oury -- perhaps we are measuring the bandwidth from cache to
main memory, not raw computational ability of the processor cores.

Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Clojure performance tests and clojure a little slower than Java

[Andy Fingerhut]

On Aug 8, 2:16 pm, John Harrop  wrote:
> On Sat, Aug 8, 2009 at 5:23 AM, Mark Engelberg 
> wrote:
>
>
>
> > On Fri, Aug 7, 2009 at 5:14 PM, John Harrop wrote:
> > >     (if (and (not (= 0 i)) (< (+ zr2 zi2 limit-square)))
>
> > I believe that (zero? i) is faster than (= 0 i).
>
> On primitive ints? Have you tested it?

I have, now, at your suggestion.  Here are the results on my iMac,
2.16 GHz Intel Core 2 Duo, Mac OS X 10.5.8, Java version:

% java -version
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03-211)
Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02-83, mixed mode)

All are for the mandelbrot.clj-3.clj program here:

http://github.com/jafingerhut/clojure-benchmarks/blob/3e45bd8f6c3eba47f982a0f6083493a9f076d0e9/mandelbrot/mandelbrot.clj-3.clj

(By the way, these links should not be to blank pages, as you
experienced earlier.  I can follow them when reading in the Clojure
group web page, or in my email client.  Perhaps the software you are
using to read these messages is truncating the URLs, and thus they are
not going to the intended place?  Sorry they are so long, which tends
to cause these kinds of issues.)

I did two runs for each version, with the only difference between them
being replacing the (zero? i) expression in function 'dot' with a
different expression, as indicated below.  (zero? i) is a clear winner
in reducing run time.

If it makes a difference, I'm also using latest github clojure and
clojure-contrib as of about August 6, 2009.  Here is what the REPL
says about the definition of zero?

% clj
Clojure 1.1.0-alpha-SNAPSHOT
user=> (use 'clojure.contrib.repl-utils)
nil
user=> (source zero?)
(defn zero?
  "Returns true if num is zero, else false"
  {:tag Boolean
   :inline (fn [x] `(. clojure.lang.Numbers (isZero ~x)))}
  [x] (. clojure.lang.Numbers (isZero x)))



using (zero? i) in dot:

real2m38.378s
user2m40.601s
sys 0m1.875s

real2m30.614s
user2m33.081s
sys 0m1.853s

using (= 0 i) in dot:

real3m18.556s
user3m20.334s
sys 0m2.060s

real3m20.425s
user3m23.392s
sys 0m1.734s

using (== 0 i) in dot:

real3m21.701s
user3m24.043s
sys 0m1.982s

real3m22.441s
user3m24.420s
sys 0m2.013s

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Pure-functional N-body benchmark implementation

[Andy Fingerhut]

On Aug 10, 11:35 am, fft1976  wrote:
> On Aug 10, 4:46 am, Jarkko Oranen  wrote:
>
> > I'm not going to start optimising,
>
> Somebody'd better!
>
> You always hear this dogma that one should write "elegant" code first
> and optimize later, and when you do that, a few little changes can
> make Clojure as fast as Java.
>
> Here's your chance to show it :-)
>
> > but somehow I doubt that is
> > the bottleneck
>
> The locality of changes is what I doubt.

I've got a functional version in my clojure-benchmarks git repository,
too, and it is currently 138 times slower on my machine than the Java
version.  I haven't looked in detail at your code, but at a high level
it appears to be similar to yours in structure -- i.e., it doesn't try
to go overboard with float/double declarations, and uses only Clojure
data structures.  I haven't tried optimizing it yet, just because I've
done more optimizing with some of the other benchmarks first.

It would be nice to have a Clojure version under 10 times Java's run
time here, without resorting to Java data structures and methods.
There may be better motivational techniques than using the word
"dogma" and daring people to do better :-)

Then again, perhaps this may be an example where the best thing for
someone working on a real project is to take the inner loop code and
write it in Java, the way some people take inner loops in C progams
and hand-code them in assembly -- because they are in a situation
where it is worth the time to optimize.  I would point out that some
of these benchmark programs in Haskell, for example, are not pure
functional implementations, and people have gone to (what seems to me)
great effort to optimize the heck out of their programs in ways that
most Haskell programmers would probably find excessive.  Even some of
the C and C++ implementations seem to pull out __builtin_memmove calls
and explicitly using SSE macros in order to tweak out a few more
milliseconds of lower run time.

Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Pure-functional N-body benchmark implementation

[Andy Fingerhut]

On Aug 10, 2:19 pm, Jonathan Smith  wrote:
> 1.) use something mutable
> 2.) unroll all the loops (mapping is a loop)
> 3.) try not to coerce between seq/vec/hash-map too much.
>
> in real world, stuff like the shootout is pretty useless, as generally
> you'd reach for a better algorithm rather than implementing the
> shackled, crippled, naive algorithms that the benchmark forces you to
> implement.
>
> (Not that they aren't useful to some extent, just that language
> productivity and how fast you can iterate your software design in the
> language is, IMO, a much better indicator of a good language than
> micro benchmarking).

I agree that they are useful to some extent.  In the real world, you
would use a better algorithm, but those better algorithms can be
implemented in any language, too.  As you say, a higher level language
that lets you iterate more quickly on multiple implementations of
different algorithms has advantages.  At the end of the day, there is
something in the language implementations that can enable faster
execution times than others.  A micro-benchmark can be useful to tell
you that if your application needs to do lots of double number
crunching, you definitely shouldn't be implementing it in Perl, for
example -- at least not that part of your application.

By the way, fft1976, this Clojure version of the n-body benchmark
takes about half the time of your code on my machine (for a shorter
test than the one for the benchmark web site -- I haven't compared yet
on the full length test since it takes about an hour for my version).

http://github.com/jafingerhut/clojure-benchmarks/blob/3e45bd8f6c3eba47f982a0f6083493a9f076d0e9/n-body/nbody.clj-5.clj

It doesn't use any mutable data structures, but it does use vectors,
has some double declarations, and avoids using map for operations on
pairs of 3d vectors, so it has some optimizations.

Using the new transient vectors recently added by Rich, and also
either using transient maps, or changing the implementation of planet
data to use vectors instead of the maps that I use, would probably
speed things up some more.  I also still have a fair number of uses of
map that would probably be faster with loop/recur.

Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Pure-functional N-body benchmark implementation

[Andy Fingerhut]

OK, I've got a new Clojure program for the n-body benchmark, and it is
significantly faster than my previous one -- down from 138 x Java run
time, to 37 x Java run time.  Still room for improvement somewhere
there, I'm sure, including perhaps using Java arrays instead of
Clojure vectors.

http://github.com/jafingerhut/clojure-benchmarks/blob/2423c5ba7bcded349bc21cfa4204bb840fd75bfa/n-body/nbody.clj-6.clj

The main changes from the slower version are:

+ make separate vectors for each "attribute" of a body in motion, i.e.
a separate vector of positions, a vector of velocities, etc., instead
of using maps.
+ Use loop/recur almost everywhere it makes sense.  I still have a few
map calls in the function 'energy' and the functions it calls, and
maybe in the init code, but that is only done twice and once during
the whole execution, versus advance which is called 50,000,000 times
in the long version of the benchmark.
+ It uses the new transient/assoc!/conj!/persistent! functions for
Clojure vectors, so you need a pretty new copy of Clojure if you want
to run it yourself.

Updates summary of results is here:

http://github.com/jafingerhut/clojure-benchmarks/blob/14c5df3ee28d418a938de584f6675860043dd0f5/RESULTS

And, as usual, improvements to the performance of these Clojure
programs are welcome.

Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Pure-functional N-body benchmark implementation

[Andy Fingerhut]

On Aug 10, 5:57 pm, Mark Engelberg  wrote:
> Andy,
>
> My understanding is that any double that gets stored in a vector or
> map is boxed, and therefore, the vast majority of your double
> conversions aren't really doing anything, because when you pull them
> out of the vector or map, they'll just be Double objects again.
>
> I believe that the biggest reason for the performance difference
> between Clojure and Java on this benchmark is that if you use a
> Clojure data structure (e.g., a map) to represent each "body" object,
> you get a lot of boxing and unboxing of the doubles.
>
> So I hypothesize that the biggest bang for the buck is to find a way
> in Clojure to create body objects with primitive double fields.  Can
> the new new do that?  If not, your best bet may be to represent a body
> object as a Java array of several doubles, rather than using a Clojure
> map.  The first double could represent the mass, the second could
> represent vx, the second vy, and so on.  Make macros for the accessors
> that extract these values using aget.
>
> It's speculation on my part, but I think this will get you much closer
> to Java speed than your current approach.

I've tried an approach like you suggest, using mutable Java arrays of
doubles, macros using aget / aset-double for reading and writing these
arrays, and loop/recur everywhere iteration is needed in the inner
loop.  It is here:

http://github.com/jafingerhut/clojure-benchmarks/blob/173e68a52ca2e180f9d5f3ac4c81a737358d1491/n-body/nbody.clj-7.clj

I was surprised to find that it is significantly slower (about 3x)
than my best Clojure one so far, which uses the new transients, and
vectors of doubles to store the state of the bodies -- the one I
posted earlier today.

http://github.com/jafingerhut/clojure-benchmarks/blob/173e68a52ca2e180f9d5f3ac4c81a737358d1491/n-body/nbody.clj-6.clj

The updated summary of results is here:

http://github.com/jafingerhut/clojure-benchmarks/blob/173e68a52ca2e180f9d5f3ac4c81a737358d1491/RESULTS

I suspect I'm doing something wrong in my mutable Java array
implementation, but I don't see what it could be.

Thanks,
Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Pure-functional N-body benchmark implementation

[Andy Fingerhut]

On Aug 10, 11:33 pm, Mark Engelberg  wrote:
> On Mon, Aug 10, 2009 at 11:15 PM, Andy
>
> Fingerhut wrote:
> > I suspect I'm doing something wrong in my mutable Java array
> > implementation, but I don't see what it could be.
>
> There still seems to be a lot of boxing and unboxing going on.  For example, 
> in:
> (let [[momx momy momz] (offset-momentum bodies)
> the call to offset-momentum returns a vector of the three doubles, so
> they would be boxed, so momx momy momz are not primitives.

That code is only run once during initialization, then never again.
It is well under 0.1% of the entire run time of the long benchmark.

The bulk of the computation is this call tree, where advance! is
called 50,000,000 times sequentially in the long benchmark:

advance!
bodies-update-velocities!
bodies-update-positions!

Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Pure-functional N-body benchmark implementation

[Andy Fingerhut]

On Aug 10, 11:50 pm, Christophe Grand  wrote:
> Hi Andy,
>
> On Tue, Aug 11, 2009 at 8:15 AM, Andy Fingerhut <
>
> andy_finger...@alum.wustl.edu> wrote:
> > I've tried an approach like you suggest, using mutable Java arrays of
> > doubles, macros using aget / aset-double for reading and writing these
> > arrays, and loop/recur everywhere iteration is needed in the inner
> > loop.  It is here:
>
> aget-* and aset-* are slow, just use aget and aset with type hints.
>
> Christophe

Wow, you ain't kiddin.  I changed about 10 lines from my last version,
to avoid using aset-double, using aset and type hints until the
reflection warnings went away, and it sped up by a factor of 10.  I'm
leaving the previous version's source and results there just so I can
have a future reference to point to at the difference it makes.  New
results here:

http://github.com/jafingerhut/clojure-benchmarks/blob/9dc56d8ff53f0b8d363f213317587432bd8793de/RESULTS

Still almost 11 times slower than the Java version, but a lot more
respectable than my earlier attempts.

Should there be a similar warning like *warn-on-reflection* that tells
you to avoid using aset-* if you want better performance?  Or at least
put a loud warning in the doc strings for those functions about their
poor performance?  Or can they just be removed?

Thanks!

Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Transient Data Structures

[Andy Fingerhut]

On Aug 10, 11:15 am, Christophe Grand  wrote:
> Hi Andy,
>
> On Thu, Aug 6, 2009 at 7:40 PM, Andy Fingerhut <
>
>
>
> andy_finger...@alum.wustl.edu> wrote:
> > Thank you, Christophe!  I've been wanting to try those out.
>
> > I made changes to 3 lines of my Clojure program for the k-nucleotide
> > benchmark, which spends most of its time in a function tally-dna-subs-
> > with-len that creates a hash map counting the number of times that
> > each of a bunch of length k strings occurs in a long string.  Source
> > here, if you're curious:
>
> >http://github.com/jafingerhut/clojure-benchmarks/blob/38e1f592ca3befe...
>
> > It went from about 19 minutes down to about 12 minutes.  Excellent
> > improvement for a small change to the code.  That brings it down to
> > about 7.7 times the run time of the Java version from the language
> > shootout web site.
>
> Could you try my "leafless" 
> branch;http://github.com/cgrand/clojure/tree/leafless?
>
> Thanks,
>
> Christophe

And Christophe's latest improvements to transient support for maps
improves the running time of my k-nucleotide benchmark program from
about 12 minutes to about 9.5 minutes.  Very nice stuff.  It isn't in
Rich's clojure repository yet, but you can get the changes from
Christophe's github repo if you want to try it out.

Thanks,
Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Pure-functional N-body benchmark implementation

[Andy Fingerhut]

In case it matters to anyone, my intent in creating these Clojure
programs to compare their speed to others isn't to try to rip into
Clojure, or start arguments.  It is for me to get my feet wet with
Clojure, and perhaps produce some examples that others can learn from
on what performs well in Clojure and what doesn't.  So far, it has
even had the unexpected benefit of providing a ready-made test for
Christophe Grand to do a quick test of some improvements to his
implement of transients for Clojure maps.  Cool beans.

Also, while it may appear I do benchmarks for a living, I don't :-)
I'd personally like to know before investing more time with Clojure
what kind of performance I can attain with it.  Most of my commercial
software development to date has been in assembly and C, and the
expressiveness of Lisp is a breath of fresh air that helps me remember
why I love programming.  Clojure's concurrency features, and keeping
around the power of Lisp macros, are a huge pull for me.

But there are those times that you want performance, and I'm curious
how much I can get out of Clojure code, vs. implementing certain inner
loops in Java, C, or what-have-you.  Just because some part of your
code might lead you, for desire of better performance, to implement it
by hand in Java or C doesn't deter me from Clojure at all -- but I'd
like to know how likely that is.

OK, with that said, I've now got the n-body benchmark within 3.2 times
Java's run time on the same problem.  The only change from my previous
version that sped things up was to replace 3-operand arithmetic
operations with 2-operand operations.  Currently, this enables Clojure
to use primitive ops, instead of something slower.

Updated results here:

http://github.com/jafingerhut/clojure-benchmarks/blob/0f7fdd358086cb95c898d7c2656215408a8d0689/RESULTS

As always, suggestions or improved versions are welcome.

Thanks,
Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Can Clojure be as fast as Java?

[Andy Fingerhut]

On Aug 11, 4:27 pm, Stuart Halloway  wrote:
> If I am reading the README correctly, clj-8 is 10.8 times slower than  
> Java, but clj-9 does better: 3.2 times slower.
>
> Stu

You are reading that correctly.  My clj-9 wasn't created until after
fft1976's message starting this thread.

Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Pure-functional N-body benchmark implementation

[Andy Fingerhut]

On Aug 11, 2:36 pm, Aaron Cohen  wrote:
> At that point is it possible you're just paying the price of
> PersistentVector for the "bodies" vector?  Does it improve much if you
> change bodies to an array?

About 7% faster changing bodies to a Java array of java.lang.Object's,
each of which happens to be a Java array of primitive doubles, as
before.  Now about 3.0 x Java run time.

http://github.com/jafingerhut/clojure-benchmarks/blob/43ed2f42e6c1485541532e51eacc66488949c658/RESULTS

Thanks,
Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Clojure, Java JIT, and inlining

[Andy Fingerhut]

My apologies for the noise if this is well known in the Clojure
community, but I'll ask anyway.

One of the tweaks to my Clojure benchmarks that people have suggested
for improving performance, and that does help, is changing some
function definitions to macros.  This is in effect inlining those
functions at the source level, so the Clojure compiler has a shot at
it.

Is there some reason that the Java JIT is not doing this, with the
original code using defn, as fast as it works when using defmacro?

Perhaps some JITs do inlining, but cannot do it as well as a defn ->
defmacro change permits?

Is it because of some kind of function call/return overhead that the
JIT cannot eliminate?

Thanks,
Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Newbie with problems building clojure-contrib

[Andy Fingerhut]

I've got a Mac and I've set it up to run jvm 1.6.0 instead of 1.5.0,
but I think I've done it with 1.5.0 before, too.  You should be fine
there.  Also I have ant version 1.7.1, but again, probably not a show
stopper difference.

I put my clojure-contrib in a directory "beside" my clojure directory,
instead of within it, and I gave a full absolute path name (e.g. /
Users/andy/clojure/clojure-1.0.0.jar) instead of a relative path name
when I built clojure-contrib.  I'd recommend trying that and see if it
makes any difference for you.

Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Newbie with problems building clojure-contrib

[Andy Fingerhut]

Oh, one more thing.  If you have latest git clojure-contrib, I'd
recommend trying it with latest git clojure, too.  Latest clojure-
contrib might not work with clojure 1.0.0.

Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Can dosync transaction result computation be parallelized over multiple threads?

[Andy Fingerhut]

I know that if you have a dosync call in some function executed by a
thread, and then that function calls other functions (which might have
their own dosyncs, which get bundled together with the original
transaction), then everything is fine.  It seems common that all of
that work would be done sequentially, in a single thread.

But what if that thread that started the dosync did a pmap call, say,
and the function called on each of the elements of the collection
accessed Refs, and perhaps computed new values for some of them, and
those were not the same Refs that the original thread was accessing.
Do all of those get bundled together with the original transaction,
too?

If so, did it require anything special in the implementation to make
that work?

Thanks,
Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Transcript of Rich's "Clojure for Lispers" talk uploaded

[Andy Fingerhut]

This is the same Clojure for Lispers talk with audio, and video of
slides, available on clojure.blip.tv, among others, from the September
2008 Boston Lisp meeting.

It has been uploaded to the files section of the group with this name:

clojure-for-lispers-transcript.txt

I've added a very few references to books and articles that Rich
mentions during the talk, but other than that, it is just the text of
what he says, and a fraction of what the audience asks (some of them
are much more difficult to understand in the audio).

People are welcome to put this in some more easily accessible place,
if you know how and have the access to it.  For example, if a link to
it could be put on clojure.blip.tv, that might be cool.

Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Memory leak in vec (and maybe persistent!)?

[Andy Fingerhut]

If there is a better place to report things like this, let me know.

I've done some looking at how much memory various kinds of Clojure
data structures use, and come across some behavior that I think might
be a memory leak, in the sense that perhaps a reference to a
java.util.concurrent.atomic.AtomicReference object is being held onto
after a call to vec, even though it isn't needed any more.

It might still be needed for reasons that I'm not aware of.  I don't
know the implementation well enough to say.

Steps to reproduce the behavior (should work on at least OS X and
Linux -- tested on OS X with java 1.6.0_13, and latest clojure and
clojure-contrib from github):

Get these two files:

clj-run.sh
memuse.clj

You can either pull my whole clojure-benchmarks git repo (only a few
megabytes), or just get those two files here:

http://github.com/jafingerhut/clojure-benchmarks/tree/450da3b949550087e6b97333ba065c1fc875564a/memuse

Edit clj-run.sh to set the location of your Clojure jar files.  Then
run:

./clj-run.sh 50 5 1

When it prints "Done.  Going to sleep now.  Good night." then either
put that process in the background, or in a separate shell window,
find out the process ID of the java process created above, and run:

jmap -histo:live   > jmap.txt

My belief is that the ":live" part means that only live objects with
references to them, that cannot be garbage collected, will be reported
in the output.  I've confirmed on my system that if I use "-histo"
instead of "histo:live" that I get a lot more objects reported, which
are presumably garbage.

When I did this, one of the first several lines of output included a
little more than 1 AtomicReference object per calls to vec.

 num #instances #bytes  class name
--
   6:500022   12000528
java.util.concurrent.atomic.AtomicReference

I think that this AtomicReference object is fairly new in the Clojure
Java source code, with the addition of transients.  It used to be an
AtomicBoolean before that.

Would it be correct to nullify the reference to the persistent object
returned by a call to persistent! on a transient object?

Thanks,
Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Memory leak in vec (and maybe persistent!)?

[Andy Fingerhut]

On Aug 15, 5:44 am, Christophe Grand  wrote:
> I wouldn't call this a memory leak: vectors have a bigger memory overhead
> than before (additional PersistentVector$Node and AtomicReference).
>
> On Sat, Aug 15, 2009 at 2:13 AM, Andy Fingerhut <
>
> andy_finger...@alum.wustl.edu> wrote:
> > When I did this, one of the first several lines of output included a
> > little more than 1 AtomicReference object per calls to vec.
>
> >  num     #instances         #bytes  class name
> > --
> >   6:        500022       12000528
> > java.util.concurrent.atomic.AtomicReference
>
> > Would it be correct to nullify the reference to the persistent object
> > returned by a call to persistent! on a transient object?
>
> For a single persistent object (built with transients) there are potentially
> several references to the same AtomicReference. I think that you can't
> nullify all references and have persistent! still be O(1).
>
> One AtomicRefrence is created for each pair of transient/persistent! calls
> and, in worst case, you can't have more than one AtomicReference per
> PersistentVector$Node. So, for a given vector it won't grow indefinitely.
>
> OTH,
>
> Christophe

Understood.

Not to say this ought to be done, but would it be possible to have no
'edit' field *at all* for persistent data structures (not even a null
reference), and only an edit field at the "root" data structure of a
transient (none of the children would have one, only the root), and
preserve O(1) transient/persistent! calls?

Even if that is possible, I guess the down side is two versions of all
the data structures for transients, one with edit and one without.

But even if edit remains in all of them, I'm wondering whether it
could be maintained that only the root node has a non-null edit field,
while that data structure is transient.

Thanks,
Andy

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: How to have a fast access to a java array of non-primitives?

[Andy Fingerhut]

On Aug 15, 9:45 am, Nicolas Oury  wrote:
> Thank you very much for your answers.
> The syntax was difficult to guess.
> Is there a general rule for forming the internal name of types?
>
> It seems after a few tests that is is slightly slower (15-20%) than
> making a final static function:
>
> static final A arrayAccess (A [] array, int i)
> { return array[i]}
>
> and calling it.
>
> and similar in speed to:
> static final Object arrayAccess (Object [] array, int i)
> { return array[i]}
>
> Am I wrong if I deduce that this is not specialized on the type?
>
> I use it in  a tight loop that does not much inside each loop, so the
> type checks do not seem to be negligible.
>
> I guess that's the price of trying to write solutions to not very
> interesting micro-benchmark :).
>
> Best regards,
>
> Nicolas.
>
> On Sat, 2009-08-15 at 12:06 -0400, David Nolen wrote:
> > You need to specify the array type. For example the following produces
> > a primitive array of n javax.vecmath.Vector2d elements.
>
> > (defn #^"[Ljavax.vecmath.Vector2d;" point-array [n]
> >   (make-array javax.vecmath.Vector2d n))
>
> > It works. If you want to see it in action take a look at:
>
> >http://github.com/swannodette/convex-hull/tree/master
>
> > On Sat, Aug 15, 2009 at 11:41 AM, Nicolas Oury
> >  wrote:
>
> >         Dear all,
>
> >         I try to write a program where I access a java array of non
> >          primitive
> >         and realize aget is very slow.
> >         (6x slower than the same program with clojure vectors instead
> >         of java
> >         arrays access)
>
> >         I tried a few combinations of type hints but can't manage to
> >         prevent mty
> >         program to spend most of its time there:
>
> >         99.4%     0  +  8804    java.lang.reflect.Array.get
>
> >         Does anyone know how to speed that up?
> >         Is it written somewhere in the java interop documentation?
>
> >         Best,
>
> >         Nicolas.

I don't know if this works for all classes, but you can try this:

user> (class (make-array java.lang.Object 5))
[Ljava.lang.Object;

Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Memory leak in vec (and maybe persistent!)?

[Andy Fingerhut]

On Aug 17, 3:51 am, Christophe Grand  wrote:
> On Sat, Aug 15, 2009 at 4:23 PM, Andy Fingerhut <
>
> andy_finger...@alum.wustl.edu> wrote:
> > Not to say this ought to be done, but would it be possible to have no
> > 'edit' field *at all* for persistent data structures (not even a null
> > reference), and only an edit field at the "root" data structure of a
> > transient (none of the children would have one, only the root), and
> > preserve O(1) transient/persistent! calls?
>
> The problem is when you have several calls to transient/persistent! You have
> to be able to tell transient nodes owned by the current transient processing
> from older transient nodes left in place by previous calls to persistent!.
> (and these transient nodes can be shared!)
> One could have a set of nodes owned by the current transient but it would
> incur some bookkeeping to not retain a reference to a reclaimable node.
> I don't know if there's another way to do that.

But if there is only a single non-null edit reference in the "root
node", then the non-root nodes don't really need a reference to the
thread that is currently mutating the data structure, right?

And after a call to peristent!, a different thread could later call
transient on it, and want to make its own modified version.  So any
non-null edit references anywhere in the persistent data structure are
useless, aren't they?

>
> But even if edit remains in all of them, I'm wondering whether it
>
> > could be maintained that only the root node has a non-null edit field,
> > while that data structure is transient.
>
> Why do you focus on these AtomicReferences? If you contrast pre-transient
> vectors and actual vectors you'll see that the overhead due to instance of
> PersistentVector$Node is far more important.
>
> Christophe

I only focus my questions on AtomicReferences because I noticed them.
I haven't yet taken the time to read through all the source code
related to the implementation of transient data structures.

It just seems to me that a persistent data structure doesn't _need_ a
reference to the thread that created it.  It is now immutable, and it
and all of its "sub-parts" always will be, until and unless they are
deallocated by garbage collection.

Thanks,
Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Memory leak in vec (and maybe persistent!)?

[Andy Fingerhut]

On Aug 19, 2:38 am, Christophe Grand  wrote:
> Imagine a persistent data structure S1 with a root node A and two child
> nodes B and C.
> On this data structure you call transient, make some updates and call
> persistent! which yields an updated persistent data structure S2 with the
> root node A' and child nodes B and C' (B isn't updated and is thus shared
> with S1). A' and C' are "transiently created" nodes, B is a "regular" node.
> Now you call transient again on S2. How can one know that A' and C' must be
> cloned before modifying them in place? If one can't tell and modify B' the
> modification is propagated to S2: S2 is no longer persistent :-(
> That's why each node must carry an identifier: to be able to know if a node
> can be updated in place or must be cloned. Actually, once persistent!
> called, this identifier is a nulled AtomicReference. (Granted a simple
> Object would save us a handful of bytes.)
>
> The alternative impls I can think of (a set of owned nodes or performing a
> cleanup on persistent!) would trade the memory overhead for a runtime one.
>
> Christophe

OK, I think that explanation broke through the fog in my brain.  I had
been mistakenly thinking that there ought to be a straightforward way
to keep some kind of reference to the mutating thread of a transient
at the root only, forgetting that while it is transient we need a way
to distinguish which nodes we are allowed to mutate in place, and
which we are not.

I also do not see a way that we can do that, and still have an O(1)
persistent! call.

It does seem we could have a persistent! call that is O(# of
temporarily mutable nodes created while the data structure was
transient), by walking the data structure from the root node and
stopping whenever we hit a persistent node that hasn't been mutated,
but in general that could be all of the nodes.

Thanks!
Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Dynamic enforcement of no side effects in transactions?

[Andy Fingerhut]

During the Clojure for Lispers talk in Boston (see clojure.blip.tv if
you want to watch it), someone asked Rich about enforcing the no-side-
effect rule in transactions.  Rich answered that he couldn't do that,
because "I'm a dynamic language."  (audience got a chuckle out of that
way of putting it :-)

I've been thinking a bit about ways to automatically check for side
effects in transactions, for people that want to catch them and
eliminate them.  I know that they can be found by hand via various
types of testing/debugging, but I'm curious how we might be able to do
better.

I was toying with a code walker that tried to check for whether some
code had side effects or not, and now I see more clearly several of
the difficulties of trying to statically check for such things.  There
are easy cases of determining that a function is pure, or has side
effects, but there are lots of cases where it is tricky (and I'm sure
in general undecidable, in the complexity theoretic sense of the
word).

Someone asked Rich about dynamic enforcement during the talk, and he
answered that it might be possible.  Of course, this won't guarantee
that you've caught all possible side effects in transactions without
"sufficient testing", but at least it will quickly catch them if you
try to do one.

I know there is an io! macro meant to flag an error if it is wrapped
around code that you try to run during a transaction.  That's cool.
But currently almost nothing in the core libraries use it.  It would
certainly have some performance impact if all I/O routines used it.

A couple of other ideas, although I don't know whether they have holes
in them, or what their performance impact might be.

(1) At the time of entering a transaction, dynamically bind all
functions known to possibly cause side effects to a version that
throws an exception.

One down side is coming up with, and maintaining across changes to
Clojure, that list of functions.  Another is that such dynamic
bindings do not propagate from thread to thread, so a different thread
could call one of those side-effecting functions without causing any
problems.  Also, I'm not sure, but it might be possible for a "non
cooperative" function to restore the original side-effecting version
of one of those rebound functions.

(2) Use Java sandboxing features to try to catch side effects.

I suspect it might be possible to configure sandboxing to catch
attempted I/O, but I'd be surprised if it can catch all kinds of side
effects.  I was hoping someone on the group might have more knowledge
of Java sandboxing features and could comment.  I also don't know what
the performance impact of creating such a sandbox might be.


Thanks,
Andy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Tight loop performance

[Andy Fingerhut]

Is there any reason to keep aset-* functions around in Clojure?  I
guess backwards compatibility?

It definitely seems worth flagging them when *warn-on-reflection* is
true, in a similar way to other warnings it enables.  Perhaps that
might be overloading the meaning of *warn-on-reflection*, but I think
there is definitely value in having _one_ flag you can turn on that
gives you as many kinds of performance warnings as we know how to
give.

Andy

On Sep 6, 7:49 am, Nicolas Oury  wrote:
> Hello,
>
> >         (aset-byte dst dst-offset (byte sample))
>
> Most of the time, aset is faster than aset-TYPE. Especially when, like here, 
> types are known.
> If you profile your code, you should see most of the time is in 
> Reflect.Array.set or something similar, and that should disappear
> if you want your code to perform well.
>
> Best,
>
> Nicolas.
>
> >         (aset-byte dst (inc dst-offset) (byte (bit-shift-right sample
> > 8
> >       (recur (inc src-offset) (unchecked-add 2 dst-offset)
>
> > Adding type coercions helped a bit but it's still too slow. In Java
> > I wrote this method:
>
> >  public static void shortsToBytes(short[] src, byte[] dst, int len)
> >  {
> >    int idx = 0;
> >    short s;
>
> >    while (len-- > 0) {
> >      s = src[idx];
> >      dst[idx*2] = (byte)s;
> >      dst[idx*2+1] = (byte)(s>>>8);
> >      idx++;
> >    }
> >  }
>
> > Then I timed them under both Clojure 1.0 and 1.1.0-SNAPSHOT and got
> > similar results.
>
> > [...]
> > (doseq [n (range 5)]
> >  (time (shorts-to-bytes a b 512000))
> >  (time (ArrayConverter/shortsToBytes a b 512000))
> >  (println))
>
> > "Elapsed time: 516.527512 msecs"
> > "Elapsed time: 32.316904 msecs"
>
> > "Elapsed time: 472.034037 msecs"
> > "Elapsed time: 5.096593 msecs"
>
> > "Elapsed time: 437.755411 msecs"
> > "Elapsed time: 4.10872 msecs"
>
> > "Elapsed time: 535.864767 msecs"
> > "Elapsed time: 3.106442 msecs"
>
> > "Elapsed time: 880.127444 msecs"
> > "Elapsed time: 3.124339 msecs"
>
> > So Java outperforms Clojure by two orders of magnitude.
>
> > Is there anything wrong with my code? I've been fine-tuning this code
> > for a couple of days now, to no avail.
>
> > Thank you and keep up!
> > Matt.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---

Re: Second Lisp to Learn

[Andy Fingerhut]

I don't have broad experience with various Lisps, but I have also done
some programming in Scheme and Common Lisp.

You can program in a functional style in both, or in an imperative
style in both.  In Scheme, functional style is a bit more idiomatic,
so you will find more examples of functional style code written in
Scheme than you will in Common Lisp, on average.  That makes it a bit
closer in idiomatic style to Clojure than Common Lisp is.

I can't speak to the awesomeness of the Scheme community, but they do
have lots of on line documentation, and in the last 5-8 years or so
they have taken the small "core" of scheme defined by the revised^5
report on Scheme (R5RS) and also version 6, and extended it with
"SRFIs", Scheme Requests For Implementation.  These are often
libraries of useful functionality.

http://srfi.schemers.org

There are many different Scheme implementations.  PLT Scheme might be
a good one to start with, in terms of the amount of example code and
documentation that comes with it.  I believe it implements many of the
SRFIs, as well as extensions of its own.

Andy


On Dec 20, 12:31 pm, Sean Devlin  wrote:
> Hi everyone,
> After hacking Clojure for a while, I've come to the conclusion that
> studying a second Lisp would help.  So, what do the people here
> think?  What is a good Lisp to study?  Are there particular dialects &
> distributions that are interesting? The things that are important to
> me are:
>
> A community at least 1/10th as awesome as this one.  Seriously.
> Libs in Lisp - I want to see if there are ideas worth stealing.
> Available documentation - I have to be able to read about it, and
> teach myself online.
>
> Thanks,
> Sean

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: performance improvments for the following code

[Andy Fingerhut]

On Jan 27, 8:21 pm, Scott  wrote:
> wondering if I can please get some advice on how to improve the
> performance of this piece of code
>
> (defn select-n-tournament
>   [popu fit-fn n]
>   (let [k 7]
>     (take n (repeatedly #(first (sort-by fit-fn > (take k (shuffle
> popu))
>   )
> )

I don't know about using map, but here is a function inspired by one
called 'most' in Paul Graham's On Lisp.  You could use (most fit-fn
(take k (shuffle popu))) in place of your (first ...) subexpression
above, and it would avoid sorting elements that you would otherwise
simply be throwing away anyway.

(defn most [fit-fn c]
  "Return the element e of collection c that has the largest numerical
value of (fit-fn e)."
  (let [c (seq c)]
(when c
  (loop [winner (first c)
 winner-fitness (fit-fn winner)
 c (rest c)]
(if (seq c)
  (let [e (first c)
e-fitness (fit-fn e)]
(if (> e-fitness winner-fitness)
  (recur e e-fitness (rest c))
  (recur winner winner-fitness (rest c
  winner)

Andy

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: ClassNotfound Exception while loading JAR files in Clojure 1.2

[Andy Fingerhut]

I'll be more blunt than Sean was :-)

Is there a reason why you *must* use Clojure 1.2?  If so, what is it?

If there isn't such a reason, you will likely get much better support from
other Clojure users if you use Clojure 1.4 or 1.5.1 (1.5.1 was released a
couple of months ago, so many are still using Clojure 1.4, and relatively
few are using 1.3).

Andy


On Wed, May 22, 2013 at 8:20 AM,  wrote:

> Hi Sean,
>
>Thanks for your reply. I use Clojure.Jar version 1.2.0 and the
> contrib.jar is also the same version.
>
>Copied below is the stack trace of the Error:
>
> Caused by: java.lang.NoClassDefFoundError:
> com/gargoylesoftware/htmlunit/html/BaseFrameElement (agent.clj:1)
> at clojure.lang.Compiler.eval(Compiler.java:5440)
> at clojure.lang.Compiler.eval(Compiler.java:5415)
> at clojure.lang.Compiler.load(Compiler.java:5857)
> at clojure.lang.RT.loadResourceScript(RT.java:340)
> at clojure.lang.RT.loadResourceScript(RT.java:331)
> at clojure.lang.RT.load(RT.java:409)
> at clojure.lang.RT.load(RT.java:381)
> at clojure.core$load$fn__4511.invoke(core.clj:4905)
> at clojure.core$load.doInvoke(core.clj:4904)
> at clojure.lang.RestFn.invoke(RestFn.java:409)
> at clojure.core$load_one.invoke(core.clj:4729)
> at clojure.core$load_lib.doInvoke(core.clj:4766)
> at clojure.lang.RestFn.applyTo(RestFn.java:143)
> at clojure.core$apply.invoke(core.clj:542)
> at clojure.core$load_libs.doInvoke(core.clj:4800)
> at clojure.lang.RestFn.applyTo(RestFn.java:138)
> at clojure.core$apply.invoke(core.clj:544)
> at clojure.core$use.doInvoke(core.clj:4880)
> at clojure.lang.RestFn.invoke(RestFn.java:458)
> at
> nlplabs.webfetch.lib.dgeneral$eval780$loading__4410__auto781.invoke(dgeneral.clj:1)
> at nlplabs.webfetch.lib.dgeneral$eval780.invoke(dgeneral.clj:1)
> at clojure.lang.Compiler.eval(Compiler.java:5424)
> at clojure.lang.Compiler.eval(Compiler.java:5415)
> at clojure.lang.Compiler.load(Compiler.java:5857)
> at clojure.lang.RT.loadResourceScript(RT.java:340)
> at clojure.lang.RT.loadResourceScript(RT.java:331)
> at clojure.lang.RT.load(RT.java:409)
> at clojure.lang.RT.load(RT.java:381)
> at clojure.core$load$fn__4511.invoke(core.clj:4905)
> at clojure.core$load.doInvoke(core.clj:4904)
> at clojure.lang.RestFn.invoke(RestFn.java:409)
> at clojure.core$load_one.invoke(core.clj:4729)
> at clojure.core$load_lib.doInvoke(core.clj:4766)
> at clojure.lang.RestFn.applyTo(RestFn.java:143)
> at clojure.core$apply.invoke(core.clj:542)
> at clojure.core$load_libs.doInvoke(core.clj:4804)
> at clojure.lang.RestFn.applyTo(RestFn.java:138)
> at clojure.core$apply.invoke(core.clj:542)
> at clojure.core$require.doInvoke(core.clj:4869)
> at clojure.lang.RestFn.invoke(RestFn.java:409)
> at nlplabs.webfetch.lib$eval776.invoke(lib.clj:35)
> at clojure.lang.Compiler.eval(Compiler.java:5424)
> at clojure.lang.Compiler.eval(Compiler.java:5414)
> at clojure.lang.Compiler.load(Compiler.java:5857)
> at com.nlplabs.util.GClojure.load(GClojure.java:41)
> at com.nlplabs.util.GClojure.loadExtractorLib(GClojure.java:32)
> at com.nlplabs.util.GClojure$loadExtractorLib.call(Unknown Source)
> at
> com.nlplabs.lf.protocols.WEBProtocol.(WEBProtocol.groovy:19)
> ... 27 more
> Caused by: java.lang.NoClassDefFoundError:
> com/gargoylesoftware/htmlunit/html/BaseFrameElement
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at
> nlplabs.webfetch.agent$eval786$loading__4410__auto787.invoke(agent.clj:1)
> at nlplabs.webfetch.agent$eval786.invoke(agent.clj:1)
> at clojure.lang.Compiler.eval(Compiler.java:5424)
> ... 74 more
> Caused by: java.lang.ClassNotFoundException:
> com.gargoylesoftware.htmlunit.html.BaseFrameElement
> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
> ... 79 more
> Server running. Browse to http://localhost:8080/jlfr
>
>   Copied below is the code snippet of agent.clj which has the
> import statement. ***
> (ns nlplabs.webfetch.agent
>   (:refer-clojure)
>   (:use nlplabs.lib nlplabs.webfetch.scheduler nlplabs.webfetch.util
> clojure.contrib.str-utils)
>
>
>   (:import (com.gargoyles

Re: why is this behaving badly?

[Andy Fingerhut]

Try adding (flush) after the print call.


On Fri, May 24, 2013 at 12:35 PM, Jim - FooBar(); wrote:

> Hi everyone,
>
> I am trying to do something very simple like asking the user whether he
> wants to continue or not (a-la bash). However, sometimes the print
> statement is printed sometimes it isn't! IN other words sometimes I get the
> prompt ,sometimes it looks like it's hanging (while in fact it's waiting
> for the user to type \y or ) and it's printing the warning altogether with
> what the user typed and the rest of the output...weird stuff!
>
>
> (print " \u001B[31mYOU ARE PERFORMING A POTENTIALLY ILLEGAL
> OPERATION...\n\t PROCEED AT YOUR OWN RISK!!!\u001B[m Proceed? (y/n):")
>  (when  (-> *in*
>   (java.util.Scanner.)
>   .next
>   (.charAt 0)
>   (= \y))
>
> any ideas?
>
> Jim
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscribe@**googlegroups.com
> For more options, visit this group at
> http://groups.google.com/**group/clojure?hl=en
> --- You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to 
> clojure+unsubscribe@**googlegroups.com
> .
> For more options, visit 
> https://groups.google.com/**groups/opt_out
> .
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: filter on sets ... [reward: me face palming]

[Andy Fingerhut]

If (map find-records query-parts) is returning this expression:

(({:a2p-id "1", :dh-uuid "abc-def-ghi-klm"} {:a2p-id "2", :dh-uuid
"def-ghi-klm-opq"} {:a2p-id "3", :dh-uuid nil}) ({:a2p-id "1", :dh-uuid
"abc-def-ghi-klm"} {:a2p-id "2", :dh-uuid "def-ghi-klm-opq"} {:a2p-id "3",
:dh-uuid nil}))

then especially note the double parentheses.  That is a list (or sequence),
whose first element is a list (or sequence) of maps.  You can use (first
(map find-records query-parts)) to get the inner list.

Andy


On Sat, May 25, 2013 at 4:21 AM, Mond Ray  wrote:

> I am missing something obvious... I get a list of maps back from a
> function and I want to find the elements with nil
>
> (({:a2p-id "1", :dh-uuid "abc-def-ghi-klm"} {:a2p-id "2", :dh-uuid
> "def-ghi-klm-opq"} {:a2p-id "3", :dh-uuid nil}) ({:a2p-id "1", :dh-uuid
> "abc-def-ghi-klm"} {:a2p-id "2", :dh-uuid "def-ghi-klm-opq"} {:a2p-id "3",
> :dh-uuid nil}))
>
> I try the select function but it has no effect ... same list
>
> (set/select #(not (:dh-uuid %)) (map find-records query-parts))
>
> also tried the previously working example... same list
>
> (filter #(not (:dh-uuid %)) (map find-records query-parts))
>
> I am assuming that I am not indexing into each of the maps but I cannot
> remember or find out how to do this ... all examples only show one map
>
> Thanks
>
> Ray
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: filter on sets ... [reward: me face palming]

[Andy Fingerhut]

Woops, and here I wasn't being careful enough...  You have a list (or
sequence) of two elements, both of which are lists (or sequences) of maps.
You can use (apply concat (map find-records query-parts)) to return a
single list containing nothing but maps.

Andy


On Sat, May 25, 2013 at 10:27 AM, Andy Fingerhut
wrote:

> If (map find-records query-parts) is returning this expression:
>
>
> (({:a2p-id "1", :dh-uuid "abc-def-ghi-klm"} {:a2p-id "2", :dh-uuid
> "def-ghi-klm-opq"} {:a2p-id "3", :dh-uuid nil}) ({:a2p-id "1", :dh-uuid
> "abc-def-ghi-klm"} {:a2p-id "2", :dh-uuid "def-ghi-klm-opq"} {:a2p-id "3",
> :dh-uuid nil}))
>
> then especially note the double parentheses.  That is a list (or
> sequence), whose first element is a list (or sequence) of maps.  You can
> use (first (map find-records query-parts)) to get the inner list.
>
> Andy
>
>
> On Sat, May 25, 2013 at 4:21 AM, Mond Ray  wrote:
>
>> I am missing something obvious... I get a list of maps back from a
>> function and I want to find the elements with nil
>>
>> (({:a2p-id "1", :dh-uuid "abc-def-ghi-klm"} {:a2p-id "2", :dh-uuid
>> "def-ghi-klm-opq"} {:a2p-id "3", :dh-uuid nil}) ({:a2p-id "1", :dh-uuid
>> "abc-def-ghi-klm"} {:a2p-id "2", :dh-uuid "def-ghi-klm-opq"} {:a2p-id "3",
>> :dh-uuid nil}))
>>
>> I try the select function but it has no effect ... same list
>>
>> (set/select #(not (:dh-uuid %)) (map find-records query-parts))
>>
>> also tried the previously working example... same list
>>
>> (filter #(not (:dh-uuid %)) (map find-records query-parts))
>>
>> I am assuming that I am not indexing into each of the maps but I cannot
>> remember or find out how to do this ... all examples only show one map
>>
>> Thanks
>>
>> Ray
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: java.lang.NoSuchMethodError: clojure.lang.RT.mapUniqueKeys

[Andy Fingerhut]

clojure.lang.RT.mapUniqueKeys was added to Clojure 1.5, and did not appear
in 1.4 or earlier.  It was added to Clojure 1.5 as part of this change:


https://github.com/clojure/clojure/blob/master/changes.md#210-set-and-map-constructor-functions-allow-duplicates

But yes, as Michael said, it means that you have some code compiled with
Clojure 1.5 or later that you are attempting to run with an earlier version
of Clojure.

Andy


On Mon, May 27, 2013 at 9:15 AM, Michael Klishin <
michael.s.klis...@gmail.com> wrote:

> 2013/5/27 ru 
>
>> Exception in thread "AWT-EventQueue-0" java.lang.NoSuchMethodError:
>> clojure.lang.RT.mapUniqueKeys([Ljava/lang/Object;)Lclojure/lang/IPersistentMap;
>>
>
> You have code compiled against 1.4 (or later) that's running against
> Clojure 1.3.
> --
> MK
>
> http://github.com/michaelklishin
> http://twitter.com/michaelklishin
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: sorted-map-by issue?

[Andy Fingerhut]

Your comparator #(if (= %1 %2) 0 1) may happen to give the correct answers
for your example sorted-maps, but it is also a bad comparator that will
fail for larger examples:

user=> (:a (sorted-map-by #(if (= %1 %2) 0 1) :z -26 :b 1 :a 2 :c 3 :m 13
:h 8))
nil
user=> (:z (sorted-map-by #(if (= %1 %2) 0 1) :z -26 :b 1 :a 2 :c 3 :m 13
:h 8))
nil

That is because if two items are not =, by returning 1 you are telling the
caller "the first argument should come after the second argument".  Thus if
at some time the comparator is called as (cmp :a :z), and later it is
called as (cmp :z :a), it returns the inconsistent results that :a should
come after :z, and later that :z should come after :a.  No sorted tree can
hope to return correct results given such an inconsistent comparator.

More examples and discussion at the link below, if you are interested:


https://github.com/jafingerhut/thalia/blob/master/doc/other-topics/comparators.md

Andy


On Thu, Jun 6, 2013 at 4:19 AM, dennis zhuang  wrote:

> Sorry, it's my mistake.
> Because treep map use the comparator to compare keys, and if the
> comparator returns 1 constantly,it can not find the item that equals the
> key.
>
> So i can modified the example,and it works:
>
> user=> (sorted-map-by #(if (= %1 %2) 0 1) :b 1 :a 2)
> {:b 1, :a 2}
> user=> (:a (sorted-map-by #(if (= %1 %2) 0 1) :b 1 :a 2))
> 2
> user=> (:b (sorted-map-by #(if (= %1 %2) 0 1) :b 1 :a 2))
> 1
>
>
> 2013/6/6 dennis zhuang 
>
>> user=> (sorted-map-by (constantly 1) :b 1 :a 2)
>> {:b 1, :a 2}
>> user=> (:a (sorted-map-by (constantly 1) :b 1 :a 2))
>> nil
>> user=> (keys (sorted-map-by (constantly 1) :b 1 :a 2))
>> (:b :a)
>> user=> (count (sorted-map-by (constantly 1) :b 1 :a 2))
>> 2
>> user=> (:a (sorted-map-by (constantly 1) :b 1 :a 2))
>> nil
>>
>> It looks so strange.The result map has keys :a and :b,but i can't get
>> their values.
>> Why? I try to hack the code,but i can't find the reason.
>>
>>
>> --
>> 庄晓丹
>> Email:killme2...@gmail.com xzhu...@avos.com
>> Site:   http://fnil.net
>> Twitter:  @killme2008
>>
>>
>>
>
>
> --
> 庄晓丹
> Email:killme2...@gmail.com xzhu...@avos.com
> Site:   http://fnil.net
> Twitter:  @killme2008
>
>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: sorted-map-by issue?

[Andy Fingerhut]

A few people, I believe primarily Alan Malloy and Anthony Grimes, have
created a Clojure library for what they call ordered sets and maps that do
exactly this.  They are implemented not as you tried to do, but by
remembering a number for each element (for ordered sets) or key (for
ordered maps) that is the relative order that it was added in.

https://github.com/flatland/ordered

Andy


On Thu, Jun 6, 2013 at 7:56 AM, dennis zhuang  wrote:

> Thanks,you are right.I want to creat a map which keeps elements in
> insertion order, but clojure doesn‘t have.
> 在 2013-6-6 下午10:02，"Andy Fingerhut" 写道：
>
> Your comparator #(if (= %1 %2) 0 1) may happen to give the correct answers
>> for your example sorted-maps, but it is also a bad comparator that will
>> fail for larger examples:
>>
>> user=> (:a (sorted-map-by #(if (= %1 %2) 0 1) :z -26 :b 1 :a 2 :c 3 :m 13
>> :h 8))
>> nil
>> user=> (:z (sorted-map-by #(if (= %1 %2) 0 1) :z -26 :b 1 :a 2 :c 3 :m 13
>> :h 8))
>> nil
>>
>> That is because if two items are not =, by returning 1 you are telling
>> the caller "the first argument should come after the second argument".
>> Thus if at some time the comparator is called as (cmp :a :z), and later it
>> is called as (cmp :z :a), it returns the inconsistent results that :a
>> should come after :z, and later that :z should come after :a.  No sorted
>> tree can hope to return correct results given such an inconsistent
>> comparator.
>>
>> More examples and discussion at the link below, if you are interested:
>>
>>
>> https://github.com/jafingerhut/thalia/blob/master/doc/other-topics/comparators.md
>>
>> Andy
>>
>>
>> On Thu, Jun 6, 2013 at 4:19 AM, dennis zhuang wrote:
>>
>>> Sorry, it's my mistake.
>>> Because treep map use the comparator to compare keys, and if the
>>> comparator returns 1 constantly,it can not find the item that equals the
>>> key.
>>>
>>> So i can modified the example,and it works:
>>>
>>> user=> (sorted-map-by #(if (= %1 %2) 0 1) :b 1 :a 2)
>>> {:b 1, :a 2}
>>> user=> (:a (sorted-map-by #(if (= %1 %2) 0 1) :b 1 :a 2))
>>> 2
>>> user=> (:b (sorted-map-by #(if (= %1 %2) 0 1) :b 1 :a 2))
>>> 1
>>>
>>>
>>> 2013/6/6 dennis zhuang 
>>>
>>>> user=> (sorted-map-by (constantly 1) :b 1 :a 2)
>>>> {:b 1, :a 2}
>>>> user=> (:a (sorted-map-by (constantly 1) :b 1 :a 2))
>>>> nil
>>>> user=> (keys (sorted-map-by (constantly 1) :b 1 :a 2))
>>>> (:b :a)
>>>> user=> (count (sorted-map-by (constantly 1) :b 1 :a 2))
>>>> 2
>>>> user=> (:a (sorted-map-by (constantly 1) :b 1 :a 2))
>>>> nil
>>>>
>>>> It looks so strange.The result map has keys :a and :b,but i can't get
>>>> their values.
>>>> Why? I try to hack the code,but i can't find the reason.
>>>>
>>>>
>>>> --
>>>> 庄晓丹
>>>> Email:killme2...@gmail.com xzhu...@avos.com
>>>> Site:   http://fnil.net
>>>> Twitter:  @killme2008
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> 庄晓丹
>>> Email:killme2...@gmail.com xzhu...@avos.com
>>> Site:   http://fnil.net
>>> Twitter:  @killme2008
>>>
>>>
>>>  --
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>>
>>>
>>
>>  --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that post

Re: Why aim for repeatability of specification / generative testing?

[Andy Fingerhut]

I've worked on hardware logic design, where the time and effort required to
create good tests that find subtle bugs rivals the complexity of the
hardware being designed itself.  In this context, much of the testing has
often been generated using pseudo-random streams similar to test.generative
and the simulation approach being advocated for software.  I think that is
a great idea for improving software quality.

In hardware testing via simulation, reproducible tests are important
because most of the testing cycles are run with a low amount of logging
enabled -- just enough logging to look for signs of incorrect behavior.
When a hardware designer wants to debug the problem, the test can be re-run
with very detailed logs enabled (basically amounting to recording the logic
0/1 value of every wire in the hardware being designed at every instant in
simulated time).  These are much slower and require a lot more temporary
disk space to record the logs.

So you end up using a large variety of different seed values to increase
test coverage, and if any of them fails, you can turn on the extra logging
and re-run it, knowing that you will hit the same problem as before.

Andy


On Thu, Jun 6, 2013 at 12:30 PM, Chas Emerick  wrote:

> Well, if *that's* all it is, I'll feel like quite the heel for putting so
> much thought into it! ;-)
>
> Assuming failures are rarer, then starting with a just-previously-failed
> seed would be better as an explicit action, rather than defaulting to a
> constant?
>
> - Chas
>
> On Jun 6, 2013, at 3:21 PM, Raoul Duke wrote:
>
> > i always thought it was basically solely for letting you re-run the
> > test that just/previously failed, nothing more weird or silly than
> > that.
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Instaparse 1.1.0

[Andy Fingerhut]

Mark,

I had not heard about Java changing the substring operation from O(1) to
O(n).  Do you have a link to any further info about this change?

I'm guessing that the implementation is changing from the O(1) "create a
new String object that is a reference to a portion of the original String
object" to O(n) "create a new String object that is a copy of a portion of
the original String object"?

If so, note that it is undesirable in some cases that the O(1) substring
operation works as it does, because it prevents the longer string from
being GC'ed if it would become garbage, except for the fact that one has
created and keeps substrings of it around.  One can explicitly call the
String constructor to copy the substrings if you want to avoid that
behavior, of course, so the O(1) with the O(n) fallback is more flexible in
that it lets the developer aware of this behavior choose between them.

I haven't done anything but think about it during commute time yet, but I
had been wondering in how many situations it might be useful to have a
string type that was something like Relaxed Radix Binary trees, in that
they can be concatenated and substring'd in O(log n) worst case time, and
yet substrings would only hold onto references of the parts that they
explicitly refer to, and perhaps a little bit more (but not the entire
original string).

Andy


On Tue, Jun 11, 2013 at 3:09 PM, Mark Engelberg wrote:

> Honestly I hadn't yet given it any thought.  Thanks for the interest in
> having it on Clojurescript.  Here are a few issues that come to mind:
>
> 1.  To achieve performance, I've spent time coding custom data structures
> that implement various Clojure and Java interfaces.  I haven't done much
> with Clojurescript, but my impression is that the ecosystem of protocols is
> completely different, so I imagine that could be a pain to transfer over.
>
> 2.  I don't know much about the performance of Javascript/Clojurescript's
> underlying data structures.  For example, in Java, the substring operation
> used to be O(1), but recently, much to my dismay, they changed it to O(n).
> That change was annoying, and it means I'm going to have to rework some
> code to deal with that, but at least I heard about it and can take it into
> account.  But in Javascript I don't even *know* the performance
> characteristics of its strings and substring operation.  What other
> Javascript performance gotchas don't I know about?
>
> 3. I'm assuming that due to the above, it won't be as simple as just
> recompiling the code for Clojurescript.  I have no idea what's involved
> with maintaining a code base for the two target languages simultaneously.
> Sounds non-trivial, although maybe it won't seem so intimidating once
> things have settled down and I'm not making quite so many performance
> tweaks and feature enhancements to the code.
>
> In any case, please add your request as a github issue, so I don't forget
> about it.
>
>
> On Tue, Jun 11, 2013 at 8:06 AM, JeremyS  wrote:
>
>> Hi Puzzler,
>>
>> I was wondering if you planned to port Instaparse to ClojureScript. I
>> know it's asking a lot, but I am one of those who would love
>> to be able to run it in a browser or in node.js...
>>
>> Cheers,
>>
>> Jeremys.
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Instaparse 1.1.0

[Andy Fingerhut]

And to answer my own question about a reference for this change in
behavior, which appears to have been made in Java 7:


http://stackoverflow.com/questions/16123446/java-7-string-substring-complexity

Andy


On Tue, Jun 11, 2013 at 3:33 PM, Andy Fingerhut wrote:

> Mark,
>
> I had not heard about Java changing the substring operation from O(1) to
> O(n).  Do you have a link to any further info about this change?
>
> I'm guessing that the implementation is changing from the O(1) "create a
> new String object that is a reference to a portion of the original String
> object" to O(n) "create a new String object that is a copy of a portion of
> the original String object"?
>
> If so, note that it is undesirable in some cases that the O(1) substring
> operation works as it does, because it prevents the longer string from
> being GC'ed if it would become garbage, except for the fact that one has
> created and keeps substrings of it around.  One can explicitly call the
> String constructor to copy the substrings if you want to avoid that
> behavior, of course, so the O(1) with the O(n) fallback is more flexible in
> that it lets the developer aware of this behavior choose between them.
>
> I haven't done anything but think about it during commute time yet, but I
> had been wondering in how many situations it might be useful to have a
> string type that was something like Relaxed Radix Binary trees, in that
> they can be concatenated and substring'd in O(log n) worst case time, and
> yet substrings would only hold onto references of the parts that they
> explicitly refer to, and perhaps a little bit more (but not the entire
> original string).
>
> Andy
>
>
> On Tue, Jun 11, 2013 at 3:09 PM, Mark Engelberg 
> wrote:
>
>> Honestly I hadn't yet given it any thought.  Thanks for the interest in
>> having it on Clojurescript.  Here are a few issues that come to mind:
>>
>> 1.  To achieve performance, I've spent time coding custom data structures
>> that implement various Clojure and Java interfaces.  I haven't done much
>> with Clojurescript, but my impression is that the ecosystem of protocols is
>> completely different, so I imagine that could be a pain to transfer over.
>>
>> 2.  I don't know much about the performance of Javascript/Clojurescript's
>> underlying data structures.  For example, in Java, the substring operation
>> used to be O(1), but recently, much to my dismay, they changed it to O(n).
>> That change was annoying, and it means I'm going to have to rework some
>> code to deal with that, but at least I heard about it and can take it into
>> account.  But in Javascript I don't even *know* the performance
>> characteristics of its strings and substring operation.  What other
>> Javascript performance gotchas don't I know about?
>>
>> 3. I'm assuming that due to the above, it won't be as simple as just
>> recompiling the code for Clojurescript.  I have no idea what's involved
>> with maintaining a code base for the two target languages simultaneously.
>> Sounds non-trivial, although maybe it won't seem so intimidating once
>> things have settled down and I'm not making quite so many performance
>> tweaks and feature enhancements to the code.
>>
>> In any case, please add your request as a github issue, so I don't forget
>> about it.
>>
>>
>> On Tue, Jun 11, 2013 at 8:06 AM, JeremyS  wrote:
>>
>>> Hi Puzzler,
>>>
>>> I was wondering if you planned to port Instaparse to ClojureScript. I
>>> know it's asking a lot, but I am one of those who would love
>>> to be able to run it in a browser or in node.js...
>>>
>>> Cheers,
>>>
>>> Jeremys.
>>>
>>  --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>

Re: (clojure 1.5.1) Weird performance results when using let versus def for variable

[Andy Fingerhut]

:jvm-opts and that ticket for Leiningen only affect the options passed to
the JVM if you let Leiningen invoke the JVM for you, e.g. via "lein run ..."

Colin showed pretty clearly in his email that he was using "lein uberjar"
followed by running the JVM explicitly with his own command line, so
Leiningen has no way to affect the JVM command line options in that case.

Andy


On Fri, Jun 21, 2013 at 6:59 AM, Jim - FooBar(); wrote:

>  Did you read the entire thread?
> both Jason and Leon (who originally posted) admit that this was the
> problem...Stuart even opened this issue:
> https://github.com/technomancy/leiningen/pull/1230
>
> the very last post reads:
>
> I should follow up on this and clarify that core.matrix's esum is in fact
> as fast as Java -- I apologize for the false statement (I was unaware that
> new versions of leiningen disable advanced JIT optimizations by default,
> which lead to the numbers I reported).
>
> Jim
>
>
>
>
> On 21/06/13 14:54, Michael Klishin wrote:
>
>  2013/6/21 Jim - FooBar(); 
>
>> If you're using leiningen, add this entry to your project.clj and rerun
>> your benchmarks.
>>
>> :jvm-opts ^replace []
>>
>
> Original post suggests the code is executed by building an uberjar running
> java -jar target/…
> so Leiningen default JVM options are not relevant.
> --
> MK
>
> http://github.com/michaelklishin
> http://twitter.com/michaelklishin
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: clojure 1.5 cheatsheet?

[Andy Fingerhut]

That link is also available near the top of
http://clojure.org/cheatsheet("Download other versions with
tooltips").

There are a few things new in 1.5 on the latest version, but I haven't yet
included any of the new reducers functions there yet.  If anyone has a
better suggestion for categorizing them, other than simply putting them all
together into one category together, I am open to suggestions.

Andy


On Mon, Jun 24, 2013 at 3:18 AM, terjesb  wrote:

> http://jafingerhut.github.io
>
> kl. 11:38:41 UTC+2 mandag 24. juni 2013 skrev Las følgende:
>
>> Hi,
>>
>> is there a clojure 1.5 cheatsheet somewhere, the last I could find was
>> 1.4.
>>
>> Thanks,
>>
>> --
>> László Török
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Unnecessary boxing with 1.5?

[Andy Fingerhut]

You have what is likely an undesired right paren at the end of the let
line, ending the scope of the let.

Andy


On Mon, Jun 24, 2013 at 4:32 PM, Cedric Greevey  wrote:

> (defn foo [x y z]
>   (let [x (long x) y (long y) z (long z)])
> (loop [a false b (long 0)]
>   (if a b (recur true (+ x (+ y z))
> NO_SOURCE_PATH:1 recur arg for primitive local: b is not matching
> primitive, had: Object, needed: long
> Auto-boxing loop arg: b
>
> What the fuh? Binary + with primitive args is supposed to produce
> primitive output. The auto-promoting +' and ternary-plus + are the ones
> that are supposed to produce Object. What gives?
>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Build fails with any IBM JDK above 6.0.x

[Andy Fingerhut]

Sherif:

The last time I tried a few months ago, Clojure 1.5.1 plus the patch
clj-967-disable-failing-io-copy-tests-on-ibm-jdk-16-patch1.txt built and
passed all tests with IBM JDK 1.6.

http://dev.clojure.org/jira/browse/CLJ-967

The main change in that patch is to disable a couple of unit tests for
clojure.java.io/copy that work on other JDKs but fail with IBM JDK 1.6,
related to how it handles UTF-16 encoding a bit differently.

Let me know if that works for you.

Andy


On Sun, Jun 30, 2013 at 10:22 AM, Sherif Ali  wrote:

> Hello all,
>
> I am having a problem building any Clojure version (1.1, 1.2, 1.3, 1.5)
> with any IBM JDK over 6.0.x. My main interest is for version 1.1 but I will
> take anything above as well. Is there a fix, certain setup, configuration,
> pre-built version, etc. that can get me running on IBM JDK 626 or above.
> Kindly take into consideration that if this is not possible then Clojure
> can't run on WebSphere Application Server 8.0 or above.
>
> Just to put your mind to ease, I was able to build all these version with
> SUN JDK normally.
>
> Your help is greatly appreciated. Thanks in advance.
> Sherif M Ali
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Overflowing (count ...)

[Andy Fingerhut]

Making count return a correct long or BigInt value in such cases would be a
significant change, given that Java's int is used for the return value for
many count() and countFrom() methods sprinkled around Clojure's source code.

I've created a ticket CLJ-1229 and attached a patch that causes count to
throw an ArithmeticException if the sequence contains more than
Integer/MAX_VALUE elements.  I don't know if such a patch is acceptable to
the Clojure/core team or not, but we may find out.

http://dev.clojure.org/jira/browse/CLJ-1229

Andy


On Sun, Jul 7, 2013 at 5:13 PM, John Jacobsen  wrote:

> Was unsure whether to post to clojure or clojure-dev, but here goes.
>
> I was surprised to learn today that count silently overflows to negative
> numbers:
>
> jenome.core> (time (count (range (*' 1000 1000 1000 3
> "Elapsed time: 375225.663 msecs"
> -1294967296
> jenome.core>
>
> I can easily get around this with my own (non-overflowing) count function,
> but would it make sense for count throw an ArithmeticException on overflow,
> with *unchecked-math* set to false (as it is in my case)?
>
> Curiously,
> John
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Group words by their first letter and sum their frequencies

[Andy Fingerhut]

There is a link "Download other versions with tooltips" near the top of the
Clojure Cheat Sheet page (http://clojure.org/cheatsheet) that links to
versions that are a bit more recent, and they include tooltips of the doc
strings when you hover your cursor over the symbol.

The version at clojure.org/cheatsheet gets updated less frequently, simply
because I have to ask someone else to do it who has the necessary
permissions.  I'll ask again now.

Andy


On Thu, Jul 11, 2013 at 6:50 AM, James Trunk  wrote:

> Hello Laurent,
>
> Thank you so much for your detailed and helpful reply.
>
> > You could use the new as-> macro in clojure 1.5
> Is it possible to get as-> to work with thread-last?
>
> > depends :-) since your 2 fns are helper fns rather than
> > generically-reusable fns, I would make them private to enforce that :
> > (defn- ...)
> Good point, and I'll definitely try to remember that in the future.
>
> > May I suggest alternative implementations ?
> > 
> I really like both of your alternative solutions, and have learned several
> new ideas from them. Thank you.
>
> By the way, I'm using the Clojure Cheat Sheet (
> http://clojure.org/cheatsheet) as my guide to the core functions, but it
> doesn't seem to be up-to-date with all these fancy 1.5 functions that
> you're using. Is there a new website that has superseded CCS? Do you know
> why CCS isn't kept up-to-date?
>
> Regards,
> James
>
> On Thursday, July 11, 2013 2:29:38 PM UTC+2, Laurent PETIT wrote:
>
>> Hello James,
>>
>> 2013/7/11 James Trunk :
>> > Hi everyone, I'm new to Clojure and trying to learn my way around the
>> > language.
>> >
>> > I've written a function that sums the frequencies of words starting
>> with the
>> > same letter, however I'm not fully satisfied with the result. I'm
>> striving
>> > for readability and idomaticity, but I fear that my rather limited
>> grasp of
>> > Clojure's core functions and lack of experience with functional
>> programming
>> > are letting me down on both counts.
>> >
>> > (def text (slurp 
>> > "http://www.ccel.org/ccel/**bible/kjv.txt"))
>>
>> >
>> > (defn sum-last-elements [coll]
>> >   (reduce + (map last coll)))
>> >
>> > (defn group-first-elements [coll]
>> >   (map first coll))
>> >
>> > (->> text
>> >  (re-seq
>> > #"(?:(?:\bm[a|e]n\b)|(?:\bwom[**a|e]n\b)|(?:\bchild\b)|(?:\**bchildren\b))")
>>
>> >  frequencies
>> >  (group-by ffirst)
>> >  vals
>> >  (#(zipmap (map group-first-elements %) (map sum-last-elements
>> %
>> >
>> > Which on that file gives the result:
>> >
>> > {("woman" "women") 663, ("children" "child") 2274, ("men" "man") 5314}
>> >
>> > My questions:
>> >
>> > Is there a way to avoid using ?: everywhere in the regex?
>> > (group-by ffirst) vals - is there a more readable/declarative/idiomatic
>> way
>> > to group by first letter?
>>
>> you could first (group-by first), then create the final map by calling
>> (distinct) and count
>>
>> > Is there a trick to avoid the ugly looking anonymous function (which
>> I'm
>> > only using to reorder the thread-last argument)?
>>
>> You could use the new as-> macro in clojure 1.5
>>
>> > Is it idiomatic to extract small functions to give them names (with an
>> eye
>> > to aiding readability) or would most Clojure programmers prefer these
>> to be
>> > anonymous and in-line?
>>
>> depends :-) since your 2 fns are helper fns rather than
>> generically-reusable fns, I would make them private to enforce that :
>> (defn- ...)
>>
>> > Is thread-last the right approach when dealing with nested
>> data-structures
>> > or is list comprehension, or some other approach preferred?
>>
>> The thread-last as you employed it is fine by me.
>>
>> May I suggest alternative implementations ?
>>
>> (def text (slurp 
>> "http://www.ccel.org/ccel/**bible/kjv.txt"))
>>
>> (let [words (re-seq
>> #"(?:(?:\bm[a|e]n\b)|(?:\bwom[**a|e]n\b)|(?:\bchild\b)|(?:\**bchildren\b))"
>>
>> text)
>>words-per-initial (vals (group-by first words))]
>>   (zipmap
>> (map distinct words-per-initial)
>> (map count words-per-initial)))
>>
>> ;; => {("woman" "women") 663, ("children" "child") 2274, ("men" "man")
>> 5314}
>>
>>
>>
>> or to avoid the intermediary creation of seqs of distincts and seqs of
>> counts:
>>
>> (defn- reduce-initial [m initial initial-words]
>>   (assoc m (distinct initial-words) (count initial-words)))
>>
>> (let [words (re-seq
>> #"(?:(?:\bm[a|e]n\b)|(?:\bwom[**a|e]n\b)|(?:\bchild\b)|(?:\**bchildren\b))"
>>
>> text)
>>words-per-initial (group-by first words)]
>>   (reduce-kv reduce-initial {} words-per-initial))
>>
>> ;; => {("woman" "women") 663, ("children" "child") 2274, ("men" "man")
>> 5314}
>>
>>
>> HTH,
>>
>> --
>> Laurent
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members 

Re: S-s-s-serious performance bug with `map` laziness

[Andy Fingerhut]

I believe that what you are seeing, with up to 32 more elements being
evaluated than is necessary to execute your code, is due precisely to
chunked sequences, as returned by (range n), preserved by map, and I
believe also for, doseq, and other Clojure functions and macros.

Chunked sequences can cause evaluation of things that you do not need.  In
the cases where you do want those things evaluated eventually anyway,
chunked sequences can save memory and time in producing those results.

They can also cause larger memory use than you want, and forcing full
laziness back into a chunked sequence can sometimes help, e.g. see the
issue description and my proposed pach here:
http://dev.clojure.org/jira/browse/MCOMB-2

Andy



On Sat, Jul 13, 2013 at 6:52 PM, Daniel Dinnyes  wrote:

> Seems these guys below have the same problem, and everyone thinks it is
> `apply` or `concat` to blame. One of the answerer even explains how apply
> is the issue, but in the reimplemented version of `mapcat` beside throwing
> out `apply` he does not use `map` either:
>
>
> http://stackoverflow.com/questions/4290665/does-concat-break-the-laziness-of-line-seq
>
> http://stackoverflow.com/questions/16194841/clojure-lazy-sequences-in-math-combinatorics-results-in-outofmemory-oom-error/16270113#16270113
>
>
> On Sunday, July 14, 2013 2:33:56 AM UTC+1, Daniel Dinnyes wrote:
>>
>> Hiya, check this code out guys:
>> 
>>
>> (defn point [x y]
>>   (println "x:" x "y:" y)
>>   [x y])
>>
>> (defn gen-data [n m]
>>   (for [i (range n)]
>> (for [j (range m)]
>>   (point i j
>>
>> (def data (apply concat (gen-data 100 100)))
>>
>> (nth data 5)
>>
>> "The output was the following:"
>>
>> "x: 0 y: 0
>>  x: 0 y: 1
>>  x: 0 y: 2
>>  x: 0 y: 3
>>  x: 0 y: 4
>>  x: 0 y: 5
>>  x: 0 y: 6
>>  x: 0 y: 7
>>  x: 0 y: 8
>>  x: 0 y: 9
>>  x: 0 y: 10
>>  x: 0 y: 11
>>  x: 0 y: 12
>>  x: 0 y: 13
>>  x: 0 y: 14
>>  x: 0 y: 15
>>  x: 0 y: 16
>>  x: 0 y: 17
>>  x: 0 y: 18
>>  x: 0 y: 19
>>  x: 0 y: 20
>>  x: 0 y: 21
>>  x: 0 y: 22
>>  x: 0 y: 23
>>  x: 0 y: 24
>>  x: 0 y: 25
>>  x: 0 y: 26
>>  x: 0 y: 27
>>  x: 0 y: 28
>>  x: 0 y: 29
>>  x: 0 y: 30
>>  x: 0 y: 31
>>  [0 5]"
>>
>> "Seems like other people have similar problems but the issue was
>> mis-attributed,
>> as they thought it has to do with `apply` and/or `concat` (read further
>> to find out why not):
>> https://groups.google.com/**forum/#!topic/clojure/**vzhFmpGkWTo
>> http://clojurian.blogspot.co.**uk/2012/11/beware-of-mapcat.**html
>> "
>>
>> "First I too was suspicious about `concat` and `apply`, so I wrote a
>> version of concat which was not using varargs."
>>
>> (defn concat2 [coll]
>>   (lazy-seq
>>(if-let [s (seq coll)]
>>  (if-let [ss (seq (first s))]
>>(cons (first ss) (concat2 (cons (rest ss) (rest s
>>(concat2 (rest s)))
>>  nil)))
>>
>> (def data (apply concat (gen-data 100 100)))
>>
>> (nth data 5)
>>
>> "The issue was still there unfortunately, exactly the same printout like
>> the with the first example"
>>
>> "So next i became suspicious of `for`. Maybe it has to do with the way it
>> is evaluated. So I rewrote `gen-data` using `map`"
>>
>> (defn gen-data [n m]
>>   (map (fn [x]
>>  (map (fn [y] (point x y))
>>   (range m)))
>>(range n)))
>>
>> "Even with `map` the issue was still present. Maybe both `map` and `for`
>> has the same problem? Let's rewrite `map` then"
>>
>> (defn map2 [f coll]
>>   (lazy-seq
>>(if-let [s (seq coll)]
>>  (cons (f (first s))
>>(map2 f (rest s)))
>>  nil)))
>>
>> (defn gen-data [n m]
>>   (map2 (fn [x]
>>  (map2 (fn [y] (point x y))
>>   (range m)))
>>(range n)))
>>
>> (def data (apply concat (gen-data 100 100)))
>>
>> (nth data 5)
>>
>> "x: 0 y: 0
>>  x: 0 y: 1
>>  x: 0 y: 2
>>  x: 0 y: 3
>>  x: 0 y: 4
>>  x: 0 y: 5
>>  [0 5]"
>>
>> "GOTCHA!!! WORKS CORRECTLY!!! BUG FOUND!!!"
>>
>> "...seems like both `map` and `for` are affected, possibly because `for`
>> depends on `map` (just assumption sorry, I was L.A.Z.Y. to check)"
>>
>> "Also, the bug was present while testing with both Clojure versions 1.4
>> and 1.5.1"
>>
>> "Finally, just to emphasize how serious the issue is try the same with
>> ONLY three levels of nested \"mapcatting\":"
>>
>> (defn point3d [x y z]
>> (println "x:" x "y:" y "z:" z)
>> [x y z])
>>
>> (defn gen-data3d [n m k]
>>   (mapcat (fn [x]
>>  (mapcat (fn [y]
>>(mapcat (fn [z] [(point3d x y z)])
>>(range k)))
>>  (range m)))
>>(range n)))
>>
>> (def data3d (gen-data3d 100 100 100))
>>
>> (nth data3d 5)
>>
>> "Not gonna copy the output here... seems like if the number of nested
>> level of mapcats is `n` then the total number of elements evaluating
>> non-lazily in one go is 32^n."
>>
>> ---
>> BTW, the `map

Re: is intellij idea a good ide for clojure development?

[Andy Fingerhut]

There are many who agree with Richard Stallman that it is unethical to 
distribute software without the source code.

There are many who disagree with him, myself included.  I think it is 100% 
ethical to sell proprietary software, sans source code.

I only mention this in hopes that people interested in discussing the topic at 
length will find an appropriate forum to do so, which I hope the Clojure Google 
group is not.

Andy

On Jul 26, 2013, at 12:07 PM, Cedric Greevey  wrote:

> On Fri, Jul 26, 2013 at 10:29 AM, Charlie Griefer  
> wrote:
>> On Jul 25, 2013, at 8:15 PM, Cedric Greevey  wrote:
>> 
>>> Someone makes free software plugins for nonfree software?!
>>> 
>>> 
>>> On Thu, Jul 25, 2013 at 11:04 PM, Greg  wrote:
> You submit patches to nonfree software?!
>> 
>> 
>> I may regret asking this… but don't people deserve to get paid for their 
>> work?
> 
> There's a difference between "getting paid for your work" and "getting paid 
> over and over again for work you did once, years ago". There's also a 
> difference between "getting paid for your work" and "getting free 
> improvements contributed to your code but not making your code, in turn, 
> freely available to the community". And that's leaving aside the matter of 
> the "free" in "(non)free software" being "free as in speech, not as in beer".
> 
> -- 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>  
>  

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: BigDecimal and ==, with a proposed fix

[Andy Fingerhut]

I agree, which is why I wrote a patch for the ticket
http://dev.clojure.org/jira/browse/CLJ-1118

(Depending on when you read this, dev.clojure.org may not be accessible due
to a recent DNS update made by the Clojure folks.  Wait a while and try
again.)

It makes not only the change you suggest, but also another one to how hash
is calculated for BigDecimal values, so that hash is still consistent with
=.  Take a look at it and see if you think it is correct.

Andy


On Thu, Aug 1, 2013 at 9:28 PM, CGAT  wrote:

>   My understanding of == is that it is intended to establish numerical
> equivalence
>   across types. But I think that basic contract fails with BigDecimal. For
> instance,
>
>   (== 1M 1.0M) ; => false
>
>   because the scale properties of these numbers are different. So then of
> course:
>
>   [(== 1 1N 1.0) (== 1 1N 1.0 1M) (== 1 1N 1.0 1.0M) (== 1 1.0 1N 1.0M)]
> ; => [true true true false]
>   and
>   [(== 1.0M 1.0) (== 1.0 1) (== 1 1N) (== 1N 1.0M)]
> ; => [true true true false]
>
>   I find your lack of transitivity (and commutativity) ... disturbing.
>
>   The issue is that there are two notions of equality for BigDecimal,
>   equals and compareTo, where equals compares value *and* scale while
>   compareTo compares numerically.
>
>   The other numeric types use equals for equivalence, quite reasonably.
>   But in class BigDecimalOps in clojure/lang/Numbers.java, I propose
>   that
>
> public boolean equiv(Number x, Number y){
>   return toBigDecimal(x).equals(toBigDecimal(y));
> }
>
>   should be
>
> public boolean equiv(Number x, Number y){
>   return toBigDecimal(x).compareTo(toBigDecimal(y)) == 0;
> }
>
>   to give the proper sense of equivalence.
>
>   I haven't had a chance yet to recompile with this change to test it,
>   but we do have
>
>(zero? (. 1.0M (compareTo 1M)))  ; => true
>(zero? (. 1.M (compareTo 1M)))   ; => true
>(zero? (. 1.000M (compareTo 1.0M)))  ; => true
>
>   as desired.
>
>   Reactions?
>
>   Best,
>
>Chris
>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: BigDecimal and ==, with a proposed fix

[Andy Fingerhut]

You can also see the patch at this link, in case it is a while before JIRA
is back up:

https://github.com/jafingerhut/clj-prescreen/blob/master/eval-results/2013-08-01/ticket-info/CLJ-1118-attachments/clj-1118-make-double-equals-true-for-more-bigdecimals-patch-v3.txt


On Thu, Aug 1, 2013 at 10:16 PM, Andy Fingerhut wrote:

> I agree, which is why I wrote a patch for the ticket
> http://dev.clojure.org/jira/browse/CLJ-1118
>
> (Depending on when you read this, dev.clojure.org may not be accessible
> due to a recent DNS update made by the Clojure folks.  Wait a while and try
> again.)
>
> It makes not only the change you suggest, but also another one to how hash
> is calculated for BigDecimal values, so that hash is still consistent with
> =.  Take a look at it and see if you think it is correct.
>
> Andy
>
>
> On Thu, Aug 1, 2013 at 9:28 PM, CGAT  wrote:
>
>>   My understanding of == is that it is intended to establish numerical
>> equivalence
>>   across types. But I think that basic contract fails with BigDecimal.
>> For instance,
>>
>>   (== 1M 1.0M) ; => false
>>
>>   because the scale properties of these numbers are different. So then of
>> course:
>>
>>   [(== 1 1N 1.0) (== 1 1N 1.0 1M) (== 1 1N 1.0 1.0M) (== 1 1.0 1N 1.0M)]
>> ; => [true true true false]
>>   and
>>   [(== 1.0M 1.0) (== 1.0 1) (== 1 1N) (== 1N 1.0M)]
>> ; => [true true true false]
>>
>>   I find your lack of transitivity (and commutativity) ... disturbing.
>>
>>   The issue is that there are two notions of equality for BigDecimal,
>>   equals and compareTo, where equals compares value *and* scale while
>>   compareTo compares numerically.
>>
>>   The other numeric types use equals for equivalence, quite reasonably.
>>   But in class BigDecimalOps in clojure/lang/Numbers.java, I propose
>>   that
>>
>> public boolean equiv(Number x, Number y){
>>   return toBigDecimal(x).equals(toBigDecimal(y));
>> }
>>
>>   should be
>>
>> public boolean equiv(Number x, Number y){
>>   return toBigDecimal(x).compareTo(toBigDecimal(y)) == 0;
>> }
>>
>>   to give the proper sense of equivalence.
>>
>>   I haven't had a chance yet to recompile with this change to test it,
>>   but we do have
>>
>>(zero? (. 1.0M (compareTo 1M)))  ; => true
>>(zero? (. 1.M (compareTo 1M)))   ; => true
>>(zero? (. 1.000M (compareTo 1.0M)))  ; => true
>>
>>   as desired.
>>
>>   Reactions?
>>
>>   Best,
>>
>>Chris
>>
>>  --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: should contains? throw an exception on sorted maps?

[Andy Fingerhut]

It throws because it calls the compare function on the value you are
checking and one or more values in the ordered set, and compare throws when
called with 2 values that are of different enough types.


https://github.com/jafingerhut/thalia/blob/master/doc/project-docs/clojure.core-1.5.1/clojure.core/compare.md

Nicola Mometto recently created ticket
http://dev.clojure.org/jira/browse/CLJ-1242 and wrote a patch for it that
would catch such exceptions in some cases, perhaps for contains? calls,
too, and cause them to return false instead of throwing.  It hasn't been
Vetted, so it remains to be seen what the Clojure developers answer to your
"should" question is.

Andy


On Sat, Aug 3, 2013 at 6:47 AM, Jay Fields  wrote:

> This: (contains? (sorted-map 1 2 3 4) :a)
> Results in this: ClassCastException java.lang.Long cannot be cast to
> clojure.lang.Keyword  clojure.lang.Keyword.compareTo (Keyword.java:102)
>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Clojure cheatsheets with several flavors of tooltips

[Andy Fingerhut]

It is relatively easy (with help from the right person with permission to
update clojure.org/cheatsheet) to update the non-tooltip version of the
cheatsheet there.

When last they checked for me some months ago, it was less easy to enable
the tooltip version of the cheatsheet at clojure.org/cheatsheet.

On the plus side, the link to the other versions of the cheatsheet is now
at the top of the page rather than the bottom, so it should be easier for
people to find.

Andy


On Thu, Aug 8, 2013 at 1:46 AM, Jakub Holy  wrote:

> Hi Andy,
>
> This cheatsheet of yours is wonderful!
>
> Are there any chances of getting it to clojure.org/cheatsheet? It is a
> shame that the cheatsheet at clojure.org is only for Clj 1.4 and doesn't
> have the beautiful tooltips.
>
> Thank you, Jakub
>
>
> On Monday, April 23, 2012 8:35:12 PM UTC+2, Andy Fingerhut wrote:
>>
>> The tooltip version of the Clojure/Java cheatsheet is not published at
>> [1] just yet, but hopefully we can figure out how to make that happen in a
>> while:
>>
>> [1] http://clojure.org/cheatsheet
>>
>> There is an updated link at the bottom of that page called "Download
>> other versions" that leads to [2]:
>>
>> [2] http://jafingerhut.github.com
>>
>> Page [2] has links to 5 different variations of the cheatsheet, and I
>> plan for it to be the long term home for where I publish cheatsheets (in
>> addition to [1]).  They differ only in whether they have tooltips, how the
>> tooltip is implemented, and whether the tooltip includes a 1-line summary
>> of what ClojureDocs.org contained at a recent snapshot time.  It does not
>> query ClojureDocs.org every time it displays a tooltip.  That would slow
>> down the tooltip display, and likely be too much load on ClojureDocs.org.
>>
>> There are also links on that page to the Github repository containing the
>> source that generated the cheatsheets, and to the PDF versions.
>>
>> Note: It is free and quick to create an account on ClojureDocs.org, and
>> you don't even need to create an account if you have an OpenID account on
>> Google, Yahoo, or myOpenId.  Just click "log in" in the upper right corner
>> of [3], and you can edit examples and see alsos, too.  Don't be dismayed
>> thinking that you must write special markup to create examples.  In most
>> cases, all it takes is copying and pasting a REPL session, with some
>> editing to add comments if it helps understanding what is going on.  The
>> color highlighting is automatically added by the web server.
>>
>> [3] http://clojuredocs.org
>>
>> Andy
>>
>>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Clojure Cheatsheet for Emacs

[Andy Fingerhut]

Very nice, Kris!

In case anyone wants to use the Clojure cheatsheet offline using a web
browser, in a form similar to the on-line one that includes the tooltips,
there is a git command documented at the link below to create a full local
copy of it on your computer:

http://jafingerhut.github.io

Then just open the index.html file with your browser.  The links to the
different variants of the cheatsheet all work off-line.  Some of the links
later in that page require being on-line (e.g. to get the PDF version of
the cheatsheet, or to the actual Github version of the repo).

Andy


On Fri, Aug 9, 2013 at 1:59 PM, Simone Mosciatti  wrote:

> Thank you so much...
>
> Actually was kinda funny, I typed something like "clojure cheatsheet
> offline" and your github link was one of the first, and I was very
> surprised... "Wow, it is possible that I didn't know of it ?"
> Well now I understand why I didn't know of it yet, it is only few hours
> old ;)
>
>
> On Friday, August 9, 2013 10:30:46 PM UTC+2, Kris Jenkins wrote:
>>
>> Hi,
>>
>> I find the Clojure Cheatsheet  really
>> useful, but since I often need at those times I don't have wifi, I've
>> packaged it up into an Emacs plugin:
>>
>>
>> 
>> In the hope that someone else finds it useful too, the project's here on
>> Github , along with
>> installation instructions. Any feedback is welcomed. :-)
>>
>> Cheers,
>> Kris
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: apply inc

[Andy Fingerhut]

(apply str [2 3]) does the same thing as (str 2 3), which is to attempt to
convert each of its args to a string, then concatenate them all.

(apply inc [4 5]) does the same thing as (inc 4 5), which is to throw an
exception because inc takes exactly one argument and returns that value
plus 1.

This site has more examples of apply, and many other Clojure functions, too:

http://clojuredocs.org/clojure_core/clojure.core/apply

Andy


On Sat, Aug 10, 2013 at 7:21 AM, drclj  wrote:

> Hi there:
>
> Why does
>
> (apply str [2 3]) work
>
> and not
>
> (apply inc [4 5])
>
> though
>
> (apply inc [4])
>
> does work?
>
>
> Thanks.
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Should ` ` be trimmed using `clojure.string/trim`? EOM

[Andy Fingerhut]

Clojure's clojure.string/trim uses Java's String/trim, but
clojure.string/triml and trimr use Java's Character/isWhitespace to
determine which characters are white space to remove.  CLJ-935 has a
suggested patch to make them all use Character/isWhitespace:

http://dev.clojure.org/jira/browse/CLJ-935

Character/isWhitespace doesn't consider Unicode code point 0x00A0 a
whitespace character, either, though.  Java's Character/isSpaceChar does,
but neither of those Java methods recognize a set of whitespace characters
that is a superset of the other.  Fun, eh?

I'd recommend writing your own trim that gets rid of exactly what you
want.  Start by copying from the existing triml or the version of trim in
the patch for CLJ-935 and tailoring the condition for whitespace characters
to your heart's desire.

Andy


On Sun, Aug 11, 2013 at 5:54 PM, Tim Visher  wrote:

> Sorry, I should have been more clear. In the following the space at
> the end of the string is a no break space and the first execution is
> under clojurescript, the second under clojure.
>
> user> (clojure.string/trim "54 ")
> "54"
> bible-plan.mcheyne> :cljs/quit
> :cljs/quit
> bible-plan.piggieback-rhino> (clojure.string/trim "54 ")
> "54 "
>
> So under clojurescript, a no break space is considered whitespace
> (certainly what I would expect!) but on the jvm it is not.
>
> That's the discrepancy I'm talking about. The unicode character NO
> BREAK SPACE (http://www.fileformat.info/info/unicode/char/a0/index.htm)
> seems like it should be considered whitespace.
>
> Of course I know that JVM land String.trim does not do this but I
> would basically consider that a bug.
>
> Further thoughts?
>
> --
>
> In Christ,
>
> Timmy V.
>
> http://blog.twonegatives.com/
> http://five.sentenc.es/ -- Spend less time on mail
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Clojure Cheatsheet for Emacs

[Andy Fingerhut]

I have added links to Kris's Clojure Emacs cheatsheet, and Michael's
ClojureScript cheatsheets, here:

http://jafingerhut.github.io

That page has had links to tooltip versions of the Clojure/JVM HTML & PDF
cheatsheets for some time now, and is linked from the top of
http://clojure.org/cheatsheet

I'd be happy to add links to other similar things there.  Suggest
additions/changes in email, pull requests, etc.

Andy



On Mon, Aug 12, 2013 at 7:48 AM, Michael Fogus  wrote:

> The number of cheatsheets is growing (this is a good thing IMO) and I
> wonder if it would be worth aggregating them all under one location?
>
> I have my own ClojureScript cheatsheet (
> https://github.com/readevalprintlove/clojurescript-cheatsheet) and the
> CLJS synonyms page (http://himera.herokuapp.com/synonym.html) that I
> would be willing to add to the mix.
>
>
> On Fri, Aug 9, 2013 at 4:30 PM, Kris Jenkins wrote:
>
>> Hi,
>>
>> I find the Clojure Cheatsheet  really
>> useful, but since I often need at those times I don't have wifi, I've
>> packaged it up into an Emacs plugin:
>>
>>
>> 
>> In the hope that someone else finds it useful too, the project's here on
>> Github , along with
>> installation instructions. Any feedback is welcomed. :-)
>>
>> Cheers,
>> Kris
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>
>
> --
> -- http://blog.fogus.me
> -- http://github.com/fogus
> --
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Problem Downloading Leiningen

[Andy Fingerhut]

Without more details, it would be difficult to say.

What OS and JDK are installed on the computer where you are having a
problem?

What error messages did you see when you tried "lein repl", "lein help", or
both?


On Thu, Aug 15, 2013 at 3:44 PM, MR  wrote:

> When I downloaded leiningen I couldn't get  lein help to open nor could I
> get Clojure REPL to open. I downloaded leiningen on my other computer and
> they work fine, but on this computer they don't. What seems to be the
> problem?
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: BigDecimal and ==, with a proposed fix

[Andy Fingerhut]

Thanks for the experience report.  I've copied it into the ticket's
description along with a link to this discussion.

If you have an account on JIRA, you can vote for the ticket.  More votes
might get the ticket some attention earlier than fewer votes.  If you don't
already have one, you can create a JIRA account by clicking on the "Log In"
link near the top right of the following page, and then click the "Sign up"
link.

http://dev.clojure.org/jira/browse/CLJ

To vote, find the JIRA ticket you want and click the "Vote" link.  Here is
a link to this particular ticket.

http://dev.clojure.org/jira/browse/CLJ-1118

The earliest such a change would get into an official Clojure release would
be Clojure 1.6.0, but it could make it into a 1.6 alpha or beta release
sooner.

You are of course welcome to apply the patch locally to your own copy of
the Clojure source code and build a jar file from that.  How easy or
difficult that would be to deploy in your local setup depends on various
factors, e.g. whether you have a local Maven repo that your projects use,
and your willlingness to update the Clojure version in all of your
Leiningen/Maven/whatever config files.

Andy


On Tue, Sep 3, 2013 at 3:49 PM, Korny Sietsma  wrote:

> It'd be great if this got fixed - we met an ugly bug yesterday due to this
> on our project.
>
> (Our system validates that the sum of two monetary fields A and B equals
> the sum of two monetary fields C and D.  We parse the fields via Cheshire
> with conversion to bigdecimal turned on - but any fields with no fractional
> part get parsed as Integers anyway.  So if A=$1.50, B=$1.50, C=$2 and D=$1,
> (== (+ A B) (+ C D)) is false... )
>
> - Korny
>
>
> On 2 August 2013 15:25, CGAT  wrote:
>
>> Terrific, Andy, thanks. I will definitely take a look.
>>
>>
>> On Friday, August 2, 2013 1:16:14 AM UTC-4, Andy Fingerhut wrote:
>>
>>> I agree, which is why I wrote a patch for the ticket
>>> http://dev.clojure.org/jira/**browse/CLJ-1118<http://dev.clojure.org/jira/browse/CLJ-1118>
>>>
>>> (Depending on when you read this, dev.clojure.org may not be accessible
>>> due to a recent DNS update made by the Clojure folks.  Wait a while and try
>>> again.)
>>>
>>> It makes not only the change you suggest, but also another one to how
>>> hash is calculated for BigDecimal values, so that hash is still consistent
>>> with =.  Take a look at it and see if you think it is correct.
>>>
>>> Andy
>>>
>>>
>>> On Thu, Aug 1, 2013 at 9:28 PM, CGAT  wrote:
>>>
>>>>   My understanding of == is that it is intended to establish numerical
>>>> equivalence
>>>>   across types. But I think that basic contract fails with BigDecimal.
>>>> For instance,
>>>>
>>>>   (== 1M 1.0M) ; => false
>>>>
>>>>   because the scale properties of these numbers are different. So then
>>>> of course:
>>>>
>>>>   [(== 1 1N 1.0) (== 1 1N 1.0 1M) (== 1 1N 1.0 1.0M) (== 1 1.0 1N 1.0M)]
>>>> ; => [true true true false]
>>>>   and
>>>>   [(== 1.0M 1.0) (== 1.0 1) (== 1 1N) (== 1N 1.0M)]
>>>> ; => [true true true false]
>>>>
>>>>   I find your lack of transitivity (and commutativity) ... disturbing.
>>>>
>>>>   The issue is that there are two notions of equality for BigDecimal,
>>>>   equals and compareTo, where equals compares value *and* scale while
>>>>   compareTo compares numerically.
>>>>
>>>>   The other numeric types use equals for equivalence, quite reasonably.
>>>>   But in class BigDecimalOps in clojure/lang/Numbers.java, I propose
>>>>   that
>>>>
>>>> public boolean equiv(Number x, Number y){
>>>>   return toBigDecimal(x).equals(**toBigDecimal(y));
>>>> }
>>>>
>>>>   should be
>>>>
>>>> public boolean equiv(Number x, Number y){
>>>>   return toBigDecimal(x).compareTo(**toBigDecimal(y)) == 0;
>>>> }
>>>>
>>>>   to give the proper sense of equivalence.
>>>>
>>>>   I haven't had a chance yet to recompile with this change to test it,
>>>>   but we do have
>>>>
>>>>(zero? (. 1.0M (compareTo 1M)))  ; => true
>>>>(zero? (. 1.M (compareTo 1M)))   ; => true
>>>>(zero? (. 1.000M (compareTo 1.0M)))  ; => true
>>>>
>>>>   as desired.
>>>>
>

Re: BigDecimal and ==, with a proposed fix

[Andy Fingerhut]

Mike, the equiv method that was shown was what Clojure does if it compares
a BigDecimal value with any other numeric type, except if the BigDecimal
value is compared with a Double.  Between a BigDecimal and a Double it
converts the BigDecimal to a Double before comparing.

Check out Numbers.java in the Clojure source code for yourself, but it is
probably already doing what you are hoping, or at least close to it.

Andy


On Tue, Sep 3, 2013 at 10:19 PM, Mikera wrote:

> On Friday, 2 August 2013 12:28:51 UTC+8, CGAT wrote:
>
>>   My understanding of == is that it is intended to establish numerical
>> equivalence
>>   across types. But I think that basic contract fails with BigDecimal.
>> For instance,
>>
>>   (== 1M 1.0M) ; => false
>>
>>   because the scale properties of these numbers are different. So then of
>> course:
>>
>>   [(== 1 1N 1.0) (== 1 1N 1.0 1M) (== 1 1N 1.0 1.0M) (== 1 1.0 1N 1.0M)]
>> ; => [true true true false]
>>   and
>>   [(== 1.0M 1.0) (== 1.0 1) (== 1 1N) (== 1N 1.0M)]
>> ; => [true true true false]
>>
>>   I find your lack of transitivity (and commutativity) ... disturbing.
>>
>>   The issue is that there are two notions of equality for BigDecimal,
>>   equals and compareTo, where equals compares value *and* scale while
>>   compareTo compares numerically.
>>
>>   The other numeric types use equals for equivalence, quite reasonably.
>>   But in class BigDecimalOps in clojure/lang/Numbers.java, I propose
>>   that
>>
>> public boolean equiv(Number x, Number y){
>>   return toBigDecimal(x).equals(**toBigDecimal(y));
>> }
>>
>>   should be
>>
>> public boolean equiv(Number x, Number y){
>>   return toBigDecimal(x).compareTo(**toBigDecimal(y)) == 0;
>> }
>
>
> Ouch, is that really what Numbers.equiv is doing?
>
> No wonder some of our idiomatic numerical code is so slow - a coercion to
> BigDecimal is really expensive!
>
> It really needs instanceof checks and branches to handle the different
> cases efficiently. I'll have a go at a ticket/patch that addresses this.
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

ClojureScript Google group

[Andy Fingerhut]

The ClojureScript Google group was created in January 2013, and is almost
up to 400 members.  I plan to occasionally announce its existence on the
Clojure group, for anyone here who may be interested.

You should be able to join the new group at this link:

https://groups.google.com/forum/?fromgroups#!forum/clojurescript

If you have any trouble with that link, just go to
http://groups.google.comand search for
ClojureScript.


Why a separate group for ClojureScript?

There are people who are more focused on using ClojureScript specifically,
and less interested in Clojure on other platforms such as the JVM, CLR,
etc.  The tools and development workflow are enough different that
discussing ClojureScript-specific issues in its own group makes sense.
 There are discussions appropriate for the Clojure group, e.g. bugs or
performance issues related to JVMs or parallelism, that are not of interest
to developers focused on ClojureScript.

There are also certainly many topics relevant to both Clojure and
ClojureScript groups.  Many people will want to join and participate in
both.  It is perfectly fine to post messages to the ClojureScript group
that would be relevant on the Clojure group.  For example, "I tried this
code snippet and it doesn't work for me", even though the code in question
would work the same way in both Clojure/JVM and ClojureScript.

In summary:

If your question is specific to ClojureScript, it is welcome on the
ClojureScript group.

If it is *specific* to Clojure/JVM, Clojure/CLR, etc., then it is not
welcome on the ClojureScript group.  Please send it to the Clojure group
instead.

If it is not specific to any one flavor of Clojure, but you prefer to send
it to the ClojureScript group, go right ahead, but realize that your
audience may be smaller than if you send it to the Clojure group.

Andy

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Clojure newbie code review

[Andy Fingerhut]

Regarding your question about join: The clojure.string namespace contains
the join function recommended to you.  All functions in the clojure.string
namespace, and several other namespaces (see below for a complete list) are
part of the Clojure JAR file, and in that sense come at "no additional
charge" over and above that Clojure JAR file size.  Whether you regard that
Clojure JAR file bloated or not is your judgement call (Clojure 1.5.1 is
~3.5 Mbytes).

$ java -cp $HOME/.m2/repository/org/clojure/clojure/1.5.1/clojure-1.5.1.jar
clojure.main
user=> (use 'clojure.pprint)
nil
user=> (clojure.pprint/pprint (sort (map str (all-ns
("clojure.core"
 "clojure.core.protocols"
 "clojure.instant"
 "clojure.java.browse"
 "clojure.java.io"
 "clojure.java.javadoc"
 "clojure.java.shell"
 "clojure.main"
 "clojure.pprint"
 "clojure.repl"
 "clojure.string"
 "clojure.uuid"
 "clojure.walk"
 "user")
nil

If you add other libraries with their own separate JAR files, you can look
at each one to decide whether you think its size is worth it.  Many that
are written purely in Clojure tend to be source-code only, and thus quite
small.  For example, Ring + Compojure + Hiccup JAR files total about 40
Kbytes.

Andy



On Tue, Sep 10, 2013 at 10:46 AM, Igor Demura  wrote:

> Thank you Philipp, this is really helpful!
>
> Marshall's guess where *read-eval* from is right. I didn't googled what
> does it mean, just left as-is from the Leiningen template.
>
> Talking on replacement to join. For this function I'll link additional
> library. My question how this works: in C++ world, linker will eliminate
> all unused symbols (classes in this case?) from output binary. Is the same
> true for JVM/Clojure world? Will it bloat binary?
> Why do you recommend update-in instead of assoc? First time I did this,
> than I saw using assoc in similar situation on ClojureDocs and  changed
> my mind.
>
> I see your point about architecture. You suggest to make it more
> sequential (composition like: (f (g x))), instead of injecting actions
> like this:
> (defn action [subaction]  ;; subaction is a function to pass control to
>   (do ... (subaction ...)))
>
> Thank you again.
>
> On Monday, September 9, 2013 4:24:10 PM UTC-7, Igor Demura wrote:
>>
>> Hi Clojure community,
>>
>> (I tried codereview.stackaxchange.com before, but no responses where)
>> I'm Clojure newbie, and feel very excited about it and functional
>> programming in general. I wrote tiny app (59 lines of code) which renders a
>> directory tree to the terminal, filtering with a regex. I'm sure that my
>> code is not perfect and something could be done much better. I'll
>> appreciate if you could spent sometime to review it: https://github.com/*
>> *idemura/incub/tree/master/tree
>> **. Don't know where to get help with this.
>>
>> Igor.
>>
>>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: another game of exploding heap, via clojure.string/split

[Andy Fingerhut]

Clojure's substr, and many other functions that return substrings of a
larger one (e.g. re-find, re-seq, etc) are based on Java's
java.lang.String/substring() method behavior.

Before Java version 7u6 or thereabouts, this was implemented in O(1) time
by creating a String object that referred to an offset and length within
the original String object, thus retaining a reference to it as long as the
substrings were referenced.

Around Java version 7u6, Java's substring() method behavior changed to copy
the desired substring into a new String object, so no references are kept
to the original.


http://www.javaadvent.com/2012/12/changes-to-stringsubstring-in-java-7.html

Fun, eh?  And no, this was not obvious to me until I ran across the issue
some time back.  Mark Engelberg encountered this issue while doing
performance tuning on his Instaparse library:
https://github.com/Engelberg/instaparse

If you know you are deploying on a Java version that is earlier than 7u6,
you can using the String constructor, e.g. (String. s) from Clojure to
force the copying of the string.  You could even get fancier and write code
that depends upon the Java version you are running upon, if that interests
you.

Andy


On Thu, Sep 12, 2013 at 9:08 AM, Brian Craft  wrote:

> After working around the seq + closure = death problem, I still had a
> severe memory leak in my code, which took many hours to find.
>
> Holding a reference to a string returned by clojure.string/split is
> somehow retaining a reference to the original string. In my case I needed
> to hold the first column of each row in a tsv file that was 4G in size.
> This resulted in holding the entire 4G in memory.
>
> Here's a demo. Function "data" returns a seq of lines that are about 1000
> bytes. The first column, however, is just a few bytes, and 10k of them
> should easily fit in 10M of heap space. But, no:
>
> $ LEIN_JVM_OPTS=-Xmx10M lein repl
> REPL started; server listening on localhost port 34955
> user=> (defn data [] (for [i (range)] (str "row " i "\t"
> (clojure.string/join "" (repeat 1000 "x")
> #'user/data
> user=> (def x (vec (take 1 (map #(first (clojure.string/split %
> #"\t")) (data)
> java.lang.OutOfMemoryError: Java heap space (NO_SOURCE_FILE:4)
> user=>
>
> If I copy the returned string with the String constructor, it's fine:
>
> $ LEIN_JVM_OPTS=-Xmx10M lein repl
> REPL started; server listening on localhost port 20587
> user=> (defn data [] (for [i (range)] (str "row " i "\t"
> (clojure.string/join "" (repeat 1000 "x")
> #'user/data
> user=> (def x (vec (take 1 (map #(String. (first (clojure.string/split
> % #"\t"))) (data)
> #'user/x
> user=> (x 10)
> "row 10"
> user=>
>
> Two observations about this.
>
> First, this behavior is very unexpected to me. I don't understand if it is
> a property of strings, collections, or string/split specifically that is
> causing it. Is there something in the docs that I overlooked, that would
> have warned of this?
>
> Second, for tracking down problems like this, the available tooling is
> pathetic, to put it as politely as possible. jhat would not trace the the
> leaked strings. It consistently froze up when tracing them to GC roots.
> visualvm traced it back to CacheLRU, as in the screenshot I posted in the
> other thread, which was perfectly uninformative.
>
> Without any usable tooling, the only workflow I found to narrow the
> problem was to iteratively stub out portions of code and re-run the program
> for several minutes to determine if the leak was active. Obviously, this is
> incredibly painful, slow, and tedious.
>
> I'm hoping someone can tell me there's a better way.
>
> Note that the leak did not appear in when exercising subsystems
> independently, because in that case no references were retained from one
> subsystem to the other. So, "try it in the repl" was not an effective
> strategy.
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http:

Re: another game of exploding heap, via clojure.string/split

[Andy Fingerhut]

I have just added some discussion of this on ClojureDocs.org for the
function clojure.core/subs, and references to that discussion for several
other Clojure functions that I am pretty sure are affected, e.g. re-find,
re-seq, re-matches, clojure.string/split, replace, replace-first

http://clojuredocs.org/clojure_core/clojure.core/subs

Andy


On Thu, Sep 12, 2013 at 9:59 AM, Brian Craft  wrote:

> Ouch. Thanks for the explanation.
>
>
> On Thursday, September 12, 2013 9:46:47 AM UTC-7, Andy Fingerhut wrote:
>
>> Clojure's substr, and many other functions that return substrings of a
>> larger one (e.g. re-find, re-seq, etc) are based on Java's
>> java.lang.String/substring() method behavior.
>>
>> Before Java version 7u6 or thereabouts, this was implemented in O(1) time
>> by creating a String object that referred to an offset and length within
>> the original String object, thus retaining a reference to it as long as the
>> substrings were referenced.
>>
>> Around Java version 7u6, Java's substring() method behavior changed to
>> copy the desired substring into a new String object, so no references are
>> kept to the original.
>>
>> http://www.javaadvent.com/**2012/12/changes-to-**
>> stringsubstring-in-java-7.html<http://www.javaadvent.com/2012/12/changes-to-stringsubstring-in-java-7.html>
>>
>> Fun, eh?  And no, this was not obvious to me until I ran across the issue
>> some time back.  Mark Engelberg encountered this issue while doing
>> performance tuning on his Instaparse library:
>> https://github.com/Engelberg/**instaparse<https://github.com/Engelberg/instaparse>
>>
>> If you know you are deploying on a Java version that is earlier than 7u6,
>> you can using the String constructor, e.g. (String. s) from Clojure to
>> force the copying of the string.  You could even get fancier and write code
>> that depends upon the Java version you are running upon, if that interests
>> you.
>>
>> Andy
>>
>>
>> On Thu, Sep 12, 2013 at 9:08 AM, Brian Craft  wrote:
>>
>>> After working around the seq + closure = death problem, I still had a
>>> severe memory leak in my code, which took many hours to find.
>>>
>>> Holding a reference to a string returned by clojure.string/split is
>>> somehow retaining a reference to the original string. In my case I needed
>>> to hold the first column of each row in a tsv file that was 4G in size.
>>> This resulted in holding the entire 4G in memory.
>>>
>>> Here's a demo. Function "data" returns a seq of lines that are about
>>> 1000 bytes. The first column, however, is just a few bytes, and 10k of them
>>> should easily fit in 10M of heap space. But, no:
>>>
>>> $ LEIN_JVM_OPTS=-Xmx10M lein repl
>>> REPL started; server listening on localhost port 34955
>>> user=> (defn data [] (for [i (range)] (str "row " i "\t"
>>> (clojure.string/join "" (repeat 1000 "x")
>>> #'user/data
>>> user=> (def x (vec (take 1 (map #(first (clojure.string/split %
>>> #"\t")) (data)
>>> java.lang.OutOfMemoryError: Java heap space (NO_SOURCE_FILE:4)
>>> user=>
>>>
>>> If I copy the returned string with the String constructor, it's fine:
>>>
>>> $ LEIN_JVM_OPTS=-Xmx10M lein repl
>>> REPL started; server listening on localhost port 20587
>>> user=> (defn data [] (for [i (range)] (str "row " i "\t"
>>> (clojure.string/join "" (repeat 1000 "x")
>>> #'user/data
>>> user=> (def x (vec (take 1 (map #(String. (first
>>> (clojure.string/split % #"\t"))) (data)
>>> #'user/x
>>> user=> (x 10)
>>> "row 10"
>>> user=>
>>>
>>> Two observations about this.
>>>
>>> First, this behavior is very unexpected to me. I don't understand if it
>>> is a property of strings, collections, or string/split specifically that is
>>> causing it. Is there something in the docs that I overlooked, that would
>>> have warned of this?
>>>
>>> Second, for tracking down problems like this, the available tooling is
>>> pathetic, to put it as politely as possible. jhat would not trace the the
>>> leaked strings. It consistently froze up when tracing them to GC roots.
>>> visualvm traced it back to CacheLRU, as in the screenshot I posted in the
>>> other thread, which was perfectly uninformati

Re: [ANN] Clojure 1.7.0-alpha5 now available

[Andy Fingerhut]

I can reproduce this.  I have also determined that the change in behavior
is due to the patch applied for ticket CLJ-1544 [1] [2].  There doesn't
appear to be an actual dependency cycle in the manifold lib, but I've only
looked at this for a few mins.

Andy

[1] http://dev.clojure.org/jira/browse/CLJ-1544
[2]
https://github.com/clojure/clojure/commit/e5a104e894ed82f244d69513918d570cee5df67d

On Wed, Jan 14, 2015 at 6:29 AM, Janne Lemmetti  wrote:

> I noticed that with alpha5, I can no longer build an uberjar with {:aot
> :all} if I depend on [gloss "0.2.4"]:
>
> $ lein uberjar
> Compiling alpha5-test.core
> java.lang.Exception: Cyclic load dependency: [ /manifold/stream
> ]->/manifold/stream/graph->[ /manifold/stream
> ]->/byte_streams->/gloss/core/formats->/gloss/data/bytes->/gloss/core/codecs->/gloss/io->/alpha5_test/core,
> compiling:(manifold/stream/graph.clj:1:1)
> at clojure.core$throw_if.doInvoke(core.clj:5612)
> at clojure.lang.RestFn.invoke(RestFn.java:442)
> at clojure.core$check_cyclic_dependency.invoke(core.clj:5763)
> at clojure.core$load.doInvoke(core.clj:5860)
> ...
>
> Minimal project to reproduce with:
>
> project.clj:
>
> (defproject alpha5-test "0.1.0-SNAPSHOT"
> :dependencies [[org.clojure/clojure "1.7.0-alpha5"] [gloss "0.2.4"]]
> :profiles {:uberjar {:aot :all}})
>
> src/alpha5_test/core.clj:
>
> (ns alpha5-test.core (:require [gloss.io :refer [decode-all]]))
>
>
> Br, Janne
>
>
> On Sunday, 11 January 2015 16:34:07 UTC+2, Alex Miller wrote:
>>
>> I would greatly appreciate hearing any feedback about this (or any other)
>> alpha, even if it's just: everything looks ok.
>>
>> We've had a couple of regressions reported and that is hugely helpful as
>> we can quickly turn around fixes for the next one.
>>
>> Interested particularly in: regressions, performance +/-, and for this
>> alpha, AOT.
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Clojure 1.7.0-alpha5 now available

[Andy Fingerhut]

Nicola, who is patient enough to remind me of things we have talked about
already before.  Now I recall running across this weirdness in the Manifold
library before, since it caused Eastwood to fail when run on Manifold.

Created a Clojure ticket to record anything discovered about this issue, or
any decisions made about it going forward:

http://dev.clojure.org/jira/browse/CLJ-1641

Andy


On Wed, Jan 14, 2015 at 7:13 AM, Nicola Mometto  wrote:

>
> There actually is a dependency cycle in manifold, it's just not in the
> ns macro:
>
> manifold.stream.graph depends on manifold.graph here:
>
> https://github.com/ztellman/manifold/blob/master/src/manifold/stream/graph.clj#L5
> maniphold.stream depends on manifold.stream.graph here:
>
> https://github.com/ztellman/manifold/blob/master/src/manifold/stream.clj#L410
>
> Whether or not this will be considered a regression or not, it's not up
> to me to decide. I suggest opening a ticket in JIRA to discuss this
> further.
>
> Nicola
>
> Andy Fingerhut writes:
>
> > I can reproduce this.  I have also determined that the change in behavior
> > is due to the patch applied for ticket CLJ-1544 [1] [2].  There doesn't
> > appear to be an actual dependency cycle in the manifold lib, but I've
> only
> > looked at this for a few mins.
> >
> > Andy
> >
> > [1] http://dev.clojure.org/jira/browse/CLJ-1544
> > [2]
> >
> https://github.com/clojure/clojure/commit/e5a104e894ed82f244d69513918d570cee5df67d
> >
> > On Wed, Jan 14, 2015 at 6:29 AM, Janne Lemmetti 
> wrote:
> >
> >> I noticed that with alpha5, I can no longer build an uberjar with {:aot
> >> :all} if I depend on [gloss "0.2.4"]:
> >>
> >> $ lein uberjar
> >> Compiling alpha5-test.core
> >> java.lang.Exception: Cyclic load dependency: [ /manifold/stream
> >> ]->/manifold/stream/graph->[ /manifold/stream
> >>
> ]->/byte_streams->/gloss/core/formats->/gloss/data/bytes->/gloss/core/codecs->/gloss/io->/alpha5_test/core,
> >> compiling:(manifold/stream/graph.clj:1:1)
> >> at clojure.core$throw_if.doInvoke(core.clj:5612)
> >> at clojure.lang.RestFn.invoke(RestFn.java:442)
> >> at clojure.core$check_cyclic_dependency.invoke(core.clj:5763)
> >> at clojure.core$load.doInvoke(core.clj:5860)
> >> ...
> >>
> >> Minimal project to reproduce with:
> >>
> >> project.clj:
> >>
> >> (defproject alpha5-test "0.1.0-SNAPSHOT"
> >> :dependencies [[org.clojure/clojure "1.7.0-alpha5"] [gloss "0.2.4"]]
> >> :profiles {:uberjar {:aot :all}})
> >>
> >> src/alpha5_test/core.clj:
> >>
> >> (ns alpha5-test.core (:require [gloss.io :refer [decode-all]]))
> >>
> >>
> >> Br, Janne
> >>
> >>
> >> On Sunday, 11 January 2015 16:34:07 UTC+2, Alex Miller wrote:
> >>>
> >>> I would greatly appreciate hearing any feedback about this (or any
> other)
> >>> alpha, even if it's just: everything looks ok.
> >>>
> >>> We've had a couple of regressions reported and that is hugely helpful
> as
> >>> we can quickly turn around fixes for the next one.
> >>>
> >>> Interested particularly in: regressions, performance +/-, and for this
> >>> alpha, AOT.
> >>
> >>  --
> >> You received this message because you are subscribed to the Google
> >> Groups "Clojure" group.
> >> To post to this group, send email to clojure@googlegroups.com
> >> Note that posts from new members are moderated - please be patient with
> >> your first post.
> >> To unsubscribe from this group, send email to
> >> clojure+unsubscr...@googlegroups.com
> >> For more options, visit this group at
> >> http://groups.google.com/group/clojure?hl=en
> >> ---
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Clojure" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an
> >> email to clojure+unsubscr...@googlegroups.com.
> >> For more options, visit https://groups.google.com/d/optout.
> >>
>
> --
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this gr

Re: Properly parse clojure source code?

[Andy Fingerhut]

I agree that tools.reader is probably the best way to go.

The following note is off the top of my head without double-checking the
details, so please take it with a bit of "needs verification of details"
thought.

Even with tools.reader, I believe you either need to eval ns forms, or do
your own parsing of ns forms to be able to read things like ::om/pass, with
namespace aliases in the keywords.  e.g. see documentation for *alias-map*
http://clojure.github.io/tools.reader/#clojure.tools.reader/*alias-map*

Andy


On Thu, Jan 15, 2015 at 3:21 PM, Thomas Heller  wrote:

> https://github.com/clojure/tools.reader is probably your best bet.
>
>
> On Friday, January 16, 2015 at 12:13:22 AM UTC+1, zirkonit wrote:
>>
>> I'm thoroughly confused. If I want to parse clojure code from string
>> without evaluating or caring a lot about its context, I'm out of luck.
>>
>> EDN tools choke on reader macros ( #(blah % blah) is not valid EDN ), yet
>> more context-aware tools like read-string with *read-eval* set to false
>> choke on namespace-specific tokens like ::om/pass.
>>
>> What would be a best choice to parse Clojure code into analyzable data
>> structure while both parsing all of it (so, not just clojure.edn/read),
>> _and_ not going the entire namespaces/shadowing/etc dance?
>>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Set equality bug?

[Andy Fingerhut]

"It is out of scope for Clojure to fix this for Java types Float/Double"
 -- comments in CLJ-1036: http://dev.clojure.org/jira/browse/CLJ-1036

Andy

On Thu, Jan 22, 2015 at 5:53 AM, Nicola Mometto  wrote:

>
> Not sure if this is intended behaviour:
> user=> (= (float 1.6) (double 1.6))
> false
> user=> (= (float 1.5) (double 1.5))
> true
>
> I.e. = (and ==) will return true when comparing floats with doubles IFF
> the float's .doubleValue roundtrips to the same double it's comparing to.
>
> user=> (.doubleValue (float 1.6))
> 1.60023841858
> user=> (.doubleValue (float 1.5))
> 1.5
>
> I'm not sure why floats are not handled specially in Numbers.java so
> that hash/comparision is handled consistently with doubles (e.g. by
> converting the floats to the fitting double via Double.parseDouble),
> the current behaviour seems odd to me.
>
> Jozef Wagner writes:
>
> > More on this behavior http://dev.clojure.org/jira/browse/CLJ-1036
> >
> > On Thu, Jan 22, 2015 at 2:00 PM, Nicola Mometto 
> wrote:
> >
> >>
> >> Looking at the PHM impl, this looks like it's caused by (float 0.5) and
> >> (double 0.5) hashing differently.
> >>
> >> user=> (= (float 0.5) (double 0.5))
> >> true
> >> user=> (map hash [(float 0.5) (double 0.5)])
> >> (1056964608 1071644672)
> >>
> >> Nicola Mometto writes:
> >>
> >> > Looks like it's a bug in PersistentHashMap:
> >> > user=> (contains? (hash-map {:a (float 0.5)} 1) {:a (double 0.5)})
> >> > false
> >> > user=> (contains? (array-map {:a (float 0.5)} 1) {:a (double 0.5)})
> >> > true
> >> >
> >> > Immo Heikkinen writes:
> >> >
> >> >> (= (float 0.5) (double 0.5))
> >> >> => true
> >> >> (= #{(float 0.5)} #{(double 0.5)})
> >> >> => true
> >> >> (= {:a (float 0.5)} {:a (double 0.5)})
> >> >> => true
> >> >> (= #{{:a (float 0.5)}} #{{:a (double 0.5)}})
> >> >> => false
> >> >>
> >> >>
> >> >> Tested with both 1.6.0 and 1.7.0-alpha5.
> >>
> >> --
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "Clojure" group.
> >> To post to this group, send email to clojure@googlegroups.com
> >> Note that posts from new members are moderated - please be patient with
> >> your first post.
> >> To unsubscribe from this group, send email to
> >> clojure+unsubscr...@googlegroups.com
> >> For more options, visit this group at
> >> http://groups.google.com/group/clojure?hl=en
> >> ---
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Clojure" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an
> >> email to clojure+unsubscr...@googlegroups.com.
> >> For more options, visit https://groups.google.com/d/optout.
> >>
>
> --
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Set equality bug?

[Andy Fingerhut]

On the side of positive recommendations to avoid problems like this, here
is some advice:

+ Don't mix floats and doubles in the same Clojure program.  Given that
Clojure uses double arithmetic by default, it is much easier to use doubles
everywhere than it is to try to use floats everywhere.

+ Don't use NaN in collections.  They aren't even equal to themselves, and
that inequality propagates to any collections they are in.

+ Don't use mutable collections as elements of Clojure's immutable
collections.  They are equal to each other in straightforward equality
tests, but are not consistent in their hashes are, much as float/double
values are not.  See http://dev.clojure.org/jira/browse/CLJ-1372 for one
example.

If you keep those things out of Clojure collections, I am not aware of any
other gotchas there.

Andy

On Thu, Jan 22, 2015 at 6:44 AM, Plínio Balduino 
wrote:

> My one cent:
>
> But I think (and it's just my humble opinion) is in the scope of Clojure
> keep its consistency, am I right?
>
> I mean, if doubles and floats are different, and they are, I think we
> should always get equality test as false. Or always as true, if they're
> nominally the same value.
>
> Regards
>
> Plínio
>
> On Thu, Jan 22, 2015 at 12:18 PM, Andy Fingerhut  > wrote:
>
>> "It is out of scope for Clojure to fix this for Java types Float/Double"
>>  -- comments in CLJ-1036: http://dev.clojure.org/jira/browse/CLJ-1036
>>
>> Andy
>>
>> On Thu, Jan 22, 2015 at 5:53 AM, Nicola Mometto 
>> wrote:
>>
>>>
>>> Not sure if this is intended behaviour:
>>> user=> (= (float 1.6) (double 1.6))
>>> false
>>> user=> (= (float 1.5) (double 1.5))
>>> true
>>>
>>> I.e. = (and ==) will return true when comparing floats with doubles IFF
>>> the float's .doubleValue roundtrips to the same double it's comparing to.
>>>
>>> user=> (.doubleValue (float 1.6))
>>> 1.60023841858
>>> user=> (.doubleValue (float 1.5))
>>> 1.5
>>>
>>> I'm not sure why floats are not handled specially in Numbers.java so
>>> that hash/comparision is handled consistently with doubles (e.g. by
>>> converting the floats to the fitting double via Double.parseDouble),
>>> the current behaviour seems odd to me.
>>>
>>> Jozef Wagner writes:
>>>
>>> > More on this behavior http://dev.clojure.org/jira/browse/CLJ-1036
>>> >
>>> > On Thu, Jan 22, 2015 at 2:00 PM, Nicola Mometto 
>>> wrote:
>>> >
>>> >>
>>> >> Looking at the PHM impl, this looks like it's caused by (float 0.5)
>>> and
>>> >> (double 0.5) hashing differently.
>>> >>
>>> >> user=> (= (float 0.5) (double 0.5))
>>> >> true
>>> >> user=> (map hash [(float 0.5) (double 0.5)])
>>> >> (1056964608 1071644672)
>>> >>
>>> >> Nicola Mometto writes:
>>> >>
>>> >> > Looks like it's a bug in PersistentHashMap:
>>> >> > user=> (contains? (hash-map {:a (float 0.5)} 1) {:a (double 0.5)})
>>> >> > false
>>> >> > user=> (contains? (array-map {:a (float 0.5)} 1) {:a (double 0.5)})
>>> >> > true
>>> >> >
>>> >> > Immo Heikkinen writes:
>>> >> >
>>> >> >> (= (float 0.5) (double 0.5))
>>> >> >> => true
>>> >> >> (= #{(float 0.5)} #{(double 0.5)})
>>> >> >> => true
>>> >> >> (= {:a (float 0.5)} {:a (double 0.5)})
>>> >> >> => true
>>> >> >> (= #{{:a (float 0.5)}} #{{:a (double 0.5)}})
>>> >> >> => false
>>> >> >>
>>> >> >>
>>> >> >> Tested with both 1.6.0 and 1.7.0-alpha5.
>>> >>
>>> >> --
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google
>>> >> Groups "Clojure" group.
>>> >> To post to this group, send email to clojure@googlegroups.com
>>> >> Note that posts from new members are moderated - please be patient
>>> with
>>> >> your first post.
>>> >> To unsubscribe from this group, send email to
>>> >> clojure+unsubscr...@googlegroups.com
>>> >> For more options, visit this group at
>>> >> http://

Re: Set equality bug?

[Andy Fingerhut]

Part of what you wish were true is already true: Using = to compare a
double to a BigDecimal always returns false in Clojure, as does comparing a
float to a BigDecimal.

It is only (= float-value double-value) that can return true in Clojure,
even though hash does not guarantee the usual hash consistency property of
"(= a b) implies (= (hash a) (hash b))".  Hence my recommendation earlier
in the thread to avoid mixing floats and doubles in the same Clojure
program.

Andy

On Thu, Jan 22, 2015 at 2:04 PM, Fluid Dynamics  wrote:

> On Thursday, January 22, 2015 at 2:12:52 PM UTC-5, Jozef Wagner wrote:
>>
>> As seen in CLJ-1372, this issue will probably cause some strong opinions.
>> I think this is an example of a leaky abstraction and the current approach
>> is to leave it as is, as there are some serious trade-offs and are there is
>> no rationale or real practical reason to fix this.
>>
>> Current implementation for double and float hashes uses Java's hash
>> algorithm, which
>> uses IEEE 754 bit representation of floats, that is a fast native
>> function [1], but is different for doubles and floats. While Murmur3 does
>> provide hashing for integer numbers, there is no hash function for floating
>> point ones.
>>
>> And note that there are big decimals too...
>>
>> > (hash 1.5)
>> 1073217536
>> > (hash 1.5M)
>> 466
>> > (hash (float 1.5))
>> 1069547520
>>
>
> Things that don't have the same hash shouldn't compare equal, at least so
> long as one avoids mutable java.util collections.
>
> If floats and doubles can't generally have the same hashes (let alone
> floats and BigDecimals) then = should return false for any comparison of
> two different of these three types (and numerical code that wants to check
> for numerical equality across types, to the extent that ever makes sense
> with FP types, should use ==).
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Set equality bug?

[Andy Fingerhut]

Michael:

You can try creating a JIRA ticket suggesting that Clojure's = should
return false when comparing floats and doubles to each other.  I have no
idea if it would go anywhere, but alternative (2) trying to get hashes to
be equal between = float/doubles has been suggested and declined.  Without
some new convincing argument in favor of (2), it seems unlikely to change.

Another alternative might be an option to throw an exception if any floats
creep into Clojure territory from Java, if that boundary line is even
drawable.

Andy

On Fri, Jan 23, 2015 at 4:45 AM, Michael Gardner 
wrote:

> I'm sure we are all aware of the various issues with floating point math
> (particularly equality comparisons); however none of that is relevant to
> this discussion. The only issue here is an inconsistency between hashing
> and equality testing in Clojure.
>
> My claim is that the property "any two objects can be equal only if their
> hashes are equal" is a fundamental one that should not be violated under
> any circumstances. Whether that means modifying Clojure's hashing algorithm
> or its implementation of = doesn't much matter to me, so long as that
> property is maintained.
>
> > On Jan 23, 2015, at 5:06 AM, Luc Préfontaine <
> lprefonta...@softaddicts.ca> wrote:
> >
> > Well if it breaks elsewhere than in your code because you mix
> representations and leak
> > them to some library in Java that you do not control I see my comments
> as absolutely relevant.
> >
> > It's not technical, it's failsafe. But that might not be of any interest
> to your problem scope.
> > However it does interest me. Hiding odd behaviors when consistency can
> be guaranteed,
> > I have no problem with that. Here to me its very clear that there is no
> failsafe approach.
> >
> > Clojure is hosted on a platform that does not provide the kind if
> consistency you want.
> > Anything you can build to hide this in Clojure is like building a tower
> on moving sand.
> >
> > Mark has a valid point about type safety which helps a bit in Java but
> you can dig
> > with google and you will find people that got bitten in Java by
> float/double mixing
> > even with type 'safety'. So long for type systems, it cannot help you
> with values that
> > 'look' the same superficially but are internally different.
> >
> > For some values, floats and doubles are equal because their internal
> representations are the
> > same. For many other values it's not working.
> >
> > I suggest some readings;
> >
> > http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
> > http://floating-point-gui.de/errors/comparison/
> >
> > Then maybe we can bring this thread to an end.
> >
> >
> >> If there's a technical reason why Clojure can't return false for all =
> comparisons between floats and doubles, I'd like to hear it. Otherwise, I
> don't see how your response is relevant.
> >>
> >>> On Jan 23, 2015, at 3:10 AM, Luc Prefontaine <
> lprefonta...@softaddicts.ca> wrote:
> >>>
> >>> Agree, it's broken... in java...
> >>> Has it has been broken in the past in several architectures...
> >>>
> >>> I understand your frustration but this is not something new. It's been
> a problem for at least 30 years.
> >>>
> >>> It is kind of a basic programming issue:
> >>>
> >>> - Never compare floats with different representations.
> >>>
> >>> - Never mix different representations in computations
> >>>
> >>> - Convert representations as early as possible to a common format
> >>>
> >>> These are the rules to follow to avoid running into trouble.
> >>>
> >>> Now if you think you can overcome this persistent (ah ! ah !) problem
> with some David Copperfield trick, fine.
> >>>
> >>> But that's a trick nothing else. The problem will resurface in some
> form in another. Better cope with reality...
> >>>
> >>> Luc P.
> >>>
> >>>
>  On Jan 23, 2015, at 1:33 AM, Immo Heikkinen 
> wrote:
> >
> > I actually ran into this while comparing nested data structures from
> two different sources and spent a good part of my day figuring out what's
> happening. While it is a good advice to avoid mixing floats and doubles, it
> is inevitable that Clojure users will get bitten by this once in a while
> and hours will be wasted.
> >
> > It is also very disturbing to realise that "(= a b)" doesn't always
> imply "(= (hash a) (hash b))" or "(= #{a} #{b})" as you would think.
> 
>  (inc)
> 
>  This is fundamentally broken behavior. Telling people to just learn
> to avoid it is not good, IMO. If the hashes must be unequal, then = should
> return false.
> 
>  As for backwards compatibility, note that if such a change were made
> to =, it wouldn't affect anyone who was already following Andy's advice to
> avoid mixing doubles and floats. IOW, it should only affect those who are
> doing something you're not "supposed" to do anyway.
> 
>  --
>  You received this message because you are subscribed to the Google
>  Groups "Clojure" group.
>  To post t

Re: Set equality bug?

[Andy Fingerhut]

Not sure which properties you prefer to be true in programming languages
you use.  Hash consistency is certainly nice, but if Clojure were changed
such that (= float-val double-val) were always false, and no other changes
were made, it would lead to this situation:

user=> (<= (float 1.5) (double 1.5))
true
user=> (>= (float 1.5) (double 1.5))
true
user=> (= (float 1.5) (double 1.5))
false

That certainly flies in the face of what most people learned in math class.

Andy

On Fri, Jan 23, 2015 at 9:29 AM, Michael Gardner 
wrote:

> On Jan 23, 2015, at 8:23 AM, Andy Fingerhut 
> wrote:
> > You can try creating a JIRA ticket suggesting that Clojure's = should
> return false when comparing floats and doubles to each other.
>
> CLJ-1649, for anyone interested.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: bool with strange behaviour

[Andy Fingerhut]

Eduardo:

This is due to the weirdness in Java, and Java's own documentation
recommends against using the Boolean class constructor.  See the discussion
at this link for more details:

http://clojuredocs.org/clojure.core/if

Andy

On Tue, Feb 3, 2015 at 3:16 AM, Eduardo Aquiles Affonso Radanovitsck <
eduardoaquiles...@gmail.com> wrote:

> Hello,
> we stepped into this on my team and we couldn't figure it out why the
> different behaviour of the first line. Could someone with deeper knowledge
> of the language explain us?
>
> (if (Boolean. false) 1 2) => 1
> (if false 1 2) => 2
> (if (Boolean/FALSE) 1 2) => 2
> (if (= false (Boolean. false))
> ​​
> 1 2) =>
> ​
> ​
> 1
> ​ ​
> ​
>
> ​
>
>
> *--*
> *Eduardo Aquiles Radanovitsck*
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Updates to cheat sheet at clojure.org/cheatsheet

[Andy Fingerhut]

Thanks to Alex Miller, who has updated the cheat sheet version published at
http://clojure.org/cheatsheet

It was updated from v13 to v21.  If you are curious what changes have been
made between those versions, you can read the change log here:
https://github.com/jafingerhut/clojure-cheatsheets/blob/master/src/clj-jvm/CHANGELOG.txt

As always, the most recent version, plus tooltips containing doc strings
and a search box, can be found by clicking on the "Download other versions
with tooltips" link near the top of that page.

Andy

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Updates to cheat sheet at clojure.org/cheatsheet

[Andy Fingerhut]

I'm happy to host an updated ClojureScript cheat sheet, and give technical
help and advice to anyone wanting to update it.

As I don't use ClojureScript myself (at least not yet), I don't have the
motivation to update it myself.

Andy

On Thu, Feb 5, 2015 at 10:22 AM, gvim  wrote:

> On 05/02/2015 17:04, Andy Fingerhut wrote:
>
>> Thanks to Alex Miller, who has updated the cheat sheet version published
>> at http://clojure.org/cheatsheet
>>
>> It was updated from v13 to v21.  If you are curious what changes have
>> been made between those versions, you can read the change log here:
>> https://github.com/jafingerhut/clojure-cheatsheets/blob/master/src/
>> clj-jvm/CHANGELOG.txt
>>
>> As always, the most recent version, plus tooltips containing doc strings
>> and a search box, can be found by clicking on the "Download other
>> versions with tooltips" link near the top of that page.
>>
>> Andy
>>
>>
> Contrasts sharply with cljs-cheatsheet which is now 3 years old:
>
> https://github.com/readevalprintlove/clojurescript-cheatsheet
>
> gvim
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Updates to cheat sheet at clojure.org/cheatsheet

[Andy Fingerhut]

Nice!  I've added a link to that newer ClojureScript cheat sheet to my page
of similar links, here, so hopefully more people can find it in the future:

http://jafingerhut.github.io

That is the page you get to when clicking on the "Download other versions
with tooltips" near the top of http://clojure.org/cheatsheet

Andy

On Thu, Feb 5, 2015 at 3:20 PM, Bruce Wang  wrote:

> There are http://cljs.info/cheatsheet/
>
> from the commit logs on https://github.com/oakmac/cljs.info  it's less
> than 4 days old
>
> On Fri, Feb 6, 2015 at 5:22 AM, gvim  wrote:
>
>> On 05/02/2015 17:04, Andy Fingerhut wrote:
>>
>>> Thanks to Alex Miller, who has updated the cheat sheet version published
>>> at http://clojure.org/cheatsheet
>>>
>>> It was updated from v13 to v21.  If you are curious what changes have
>>> been made between those versions, you can read the change log here:
>>> https://github.com/jafingerhut/clojure-cheatsheets/blob/master/src/
>>> clj-jvm/CHANGELOG.txt
>>>
>>> As always, the most recent version, plus tooltips containing doc strings
>>> and a search box, can be found by clicking on the "Download other
>>> versions with tooltips" link near the top of that page.
>>>
>>> Andy
>>>
>>>
>> Contrasts sharply with cljs-cheatsheet which is now 3 years old:
>>
>> https://github.com/readevalprintlove/clojurescript-cheatsheet
>>
>> gvim
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> simple is good
> http://brucewang.net
> http://twitter.com/number5
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Implementation of min-key inefficient?

[Andy Fingerhut]

Probably it was simplicity of coding, but that is just my guess.

There is a ticket open for Clojure suggesting reducing the number of calls
to the function.  You, and anyone else who would like to see this changed
in the future, are welcome to vote on this ticket.

http://dev.clojure.org/jira/browse/CLJ-99

If you do not already have a JIRA account, you can quickly create one here:
http://dev.clojure.org/jira/secure/Signup!default.jspa

Andy

On Mon, Feb 9, 2015 at 8:43 AM, Frank Schmidt 
wrote:

> Hi,
>
> the min-key function is implemented recursively. Therefore in
>
> (min-key #(.length ^String %) "a" "aa" "aaa")
>
> the .length function is evaluated 4 times instead of 3 times.
>
> What are the reasons for this nice but inefficient implementation?
>
> Best wishes,
> Frank Schmidt
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Anyone know why cheat sheet links sometimes don't keep anchors?

[Andy Fingerhut]

There are multiple links in the Clojure cheat sheet here:


http://jafingerhut.github.io/cheatsheet/clojuredocs/cheatsheet-tiptip-cdocs-summary.html

that go to clojure.org/somewhere#some-anchor

For example, in the "Special Forms" section, and the sub-section "Binding
Forms / Destructuring", there is a link "(examples)" that (tries to) go to
the URL: http://clojure.org/special_forms#binding-forms

Often when I click on it for the first time in a while, it will lose the
anchor and instead go to:

http://clojure.org/special_forms

If I hit back in my browser and click on the same link again, this time it
keeps the whole URL, including the anchor.   I have tested Safari on Mac OS
X 10.9.5, which definitely does this, sometimes.  It does not happen all of
the time, so I am not yet sure whether it happens with other browsers like
Firefox 35.0.1 and recent Chrome, too.

Other links that sometimes exhibit this behavior:

4 links "literals" in the Primitives/Other section, all of which should go
to here: http://clojure.org/reader#The%20Reader--Reader%20forms  but
sometimes I see it go to http://clojure.org/reader instead.

5 links in "Reader Macros" section -- not the ones to quote, var, or deref,
but the ones appearing first in the descriptions.

Anyone else see this behavior?  More importantly, can anyone explain why it
happens, and/or how the behavior can be prevented?  If it is simply a bug
in Safari, then I won't worry too much about it.

Thanks,
Andy

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.7.0-alpha5 clobber warning error?

[Andy Fingerhut]

Sam:

I believe if you proceed to do the following in your REPL:

(in-ns 'monger.collection)
(doc update)

You will see that the name 'update' _from within the namespace
monger.collection_ refers to monger.collection/update, not
clojure.core/update.

That is what the message is intended to convey.

In the user namespace where you are making your doc calls, the name update
still refers to clojure.core/update.

Andy

On Wed, Feb 11, 2015 at 5:53 PM, Sam Raker  wrote:

> I just added monger as a dependency to my Clojure 1.7.0-alpha5 project,
> and saw the following in my repl:
>
> user=> (require '(monger [core :as mg] [collection :as mc]))
> WARNING: update already refers to: #'clojure.core/update in namespace:
> monger.collection, being replaced by: #'monger.collection/update
>
> That warning seems to be in error, though, because `(doc update)` and
> `(doc mc/update)` print different (and correct) things.
>
> Is this a bug? Should I file something somewhere?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [newbie] strange behaviour in self-referential primes lazy-seq attempt

[Andy Fingerhut]

Or perhaps 1.7.0-alpha5 has a change vs. 1.6.0 that makes this code work by
accident.

Using lazy sequences to define later elements in terms of the prefix of the
sequence up to that point has come up multiple times, usually with the
example of generating primes (but different code each time).

I don't have time to dig for those examples and the explanation right now,
but I believe one potential issue is that lazy sequences are sometimes
'chunked' (the return value of range is) into groups of 32 consecutive
elements, and some functions operating on sequences preserve those chunks,
and operate on those batches of 32 elements all at once.  Also there are
some Clojure functions that are 'more eager than necessary', evaluating a
few elements of the lazy sequence past the minimum needed.

Andy

On Thu, Feb 12, 2015 at 12:06 PM, Jorge Marques Pelizzoni <
jorge.pelizz...@gmail.com> wrote:

> Well, that's a bug then :) And seems to have been fixed. Thanks!
>
> Em quinta-feira, 12 de fevereiro de 2015 17:51:13 UTC-2, Michael Blume
> escreveu:
>>
>> Oh, well this is fun -- with bleeding edge clojure I get the right
>> answer, but with 1.6.0 I see the same results you did.
>>
>>
>>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [newbie] strange behaviour in self-referential primes lazy-seq attempt

[Andy Fingerhut]

Laziness is pervasive in Haskell, for all computation (unless you force it
off with special constructs).

Laziness in Clojure is as pervasive as sequences, but it is not for all
computation like in Haskell, and sequences have the previously-mentioned
features of sometimes-more-eager-than-the-minimum required.

Self-referential lazy sequences are something I would recommend actively
avoiding in Clojure, unless you want to live with buggy code, or dig into
the implementation and convince yourself that you are within its behavior
(which can change from version to version).

Andy

On Thu, Feb 12, 2015 at 4:21 PM, Jorge Marques Pelizzoni <
jorge.pelizz...@gmail.com> wrote:

> Neither did delay help:
>
> (defn primes [] (let [primes' (atom nil)]
>   (reset! primes' (delay (cons 2 (filter #(prime? (force (
> deref primes')) %) (drop 3 (range
>
>
> Serious this is not a bug?
>
> Em quinta-feira, 12 de fevereiro de 2015 22:14:46 UTC-2, Jorge Marques
> Pelizzoni escreveu:
>>
>> Thanks, Michael, for your analysis and explanation. However not even this
>> worked:
>>
>> (defn primes [] (let [primes' (atom nil)]
>>   (lazy-seq (reset! primes' (lazy-seq (cons 2 (lazy-seq
>> (filter #(prime? (deref primes') %) (drop 3 (range))
>>
>>
   --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: calling functions that accepts keyword arguments with a map

[Andy Fingerhut]

There is a significant advantage to Francis's version when keys or values
in the map are collections that flatten flattens:

;; Most likely not the behavior you want:

user=> (flatten (seq {:colors ["red" "blue"] :cursor-set #{"pointer"
"hand"}}))
(:colors "red" "blue" :cursor-set #{"hand" "pointer"})

;; Better:

user=> (apply concat {:colors ["red" "blue"] :cursor-set #{"pointer"
"hand"}})
(:colors ["red" "blue"] :cursor-set #{"hand" "pointer"})

Andy

On Fri, Feb 13, 2015 at 9:44 AM, Francis Avila  wrote:

> Not fundamentally different from your approach:
>
> (apply set-style (apply concat {:color "red" :cursor "pointer"}))
>
>
> On Friday, February 13, 2015 at 11:30:44 AM UTC-6, Wilker wrote:
>>
>> Hi guys,
>>
>> I'm trying to find the best way to call a function that accepts keyword
>> arguments (in my case it's the set-style! on Enfocus library) with a map.
>>
>> So, an example of the regular call:
>>
>> (set-style :color "red" :cursor "pointer")
>>
>> I would like to call with a map (because that way I can manage the map
>> data before using it) but I found no very easy way to do it... The best way
>> I could found is like this:
>>
>> (apply set-style (flatten (seq {:color "red" :cursor "pointer"})))
>>
>> But that's a bit overwhelming, I wonder if there is a simpler way to
>> handle this situation.
>>
>> Thanks.
>>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Accessing branches in PersistentTreeMap / sorted-map / CLJ-1008

[Andy Fingerhut]

I haven't checked carefully, but from at least a quick look it appears that
implementing the NavigableMap and NavigableSet interfaces could be done by
using the existing subseq and rsubseq functions in clojure.core?

It doesn't give you access to subtree nodes directly, but perhaps it is
sufficient for your purposes?

Andy

On Wed, Feb 18, 2015 at 6:04 PM, David James  wrote:

> Summary: I'd like to find a public API to work with the underlying tree of
> a sorted-map.
>
> For example:
>
> (def t (sorted-map 1 :a 2 :b 3 :c 4 :d 5 :e 6 :f))
>
> The underlying implementation of sorted-map uses a PersistentTreeMap,
> which can be accessed with `tree`:
>
> (.tree t) ;=> [2 :b]
>
> I have not found a way to access the left and right branches, since
> calling `left` fails:
>
> (.left (.tree t))
>
> IllegalArgumentException Can't call public method of non-public class:
> public clojure.lang.PersistentTreeMap$Node
> clojure.lang.PersistentTreeMap$BlackBranch.left()
> clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:88)
>
> Perhaps it would be reasonable to make such calls possible, at least from
> the Java API (without reflection)?
>
> CLJ-1008 (http://dev.clojure.org/jira/browse/CLJ-1008) offers one
> possible way to support a public API. It was created in 2012. Perhaps it
> could use another look. Thoughts?
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Accessing branches in PersistentTreeMap / sorted-map / CLJ-1008

[Andy Fingerhut]

David:

I see why you would want that.

If you want this soon, I would recommend using data.avl as suggested by Michał,
or copy the parts of the sorted-set/map implementation from Clojure and
implement it yourself.

If you don't want it soon, you could try persuading a few dozen people to
vote on CLJ-1008, and hope the Clojure developers also want this change.
No guarantees there.

Andy

On Thu, Feb 19, 2015 at 12:18 PM, David James  wrote:

> Andy, to take advantage of the Red-Black Tree, I'm looking for public API
> access to the branches. (I'm not looking for a work-around.)
>
> More discussion on this at SO:
> http://stackoverflow.com/questions/1981859/finding-keys-closest-to-a-given-value-for-clojure-sorted-maps
>
> Thanks to both Michał Marczyk and Andy for commenting on CLJ-1008 (
> http://dev.clojure.org/jira/browse/CLJ-1008).
>
> On Thursday, February 19, 2015 at 1:02:53 PM UTC-5, Andy Fingerhut wrote:
>>
>> I haven't checked carefully, but from at least a quick look it appears
>> that implementing the NavigableMap and NavigableSet interfaces could be
>> done by using the existing subseq and rsubseq functions in clojure.core?
>>
>> It doesn't give you access to subtree nodes directly, but perhaps it is
>> sufficient for your purposes?
>>
>> Andy
>>
>> On Wed, Feb 18, 2015 at 6:04 PM, David James  wrote:
>>
>>> Summary: I'd like to find a public API to work with the underlying tree
>>> of a sorted-map.
>>>
>>> For example:
>>>
>>> (def t (sorted-map 1 :a 2 :b 3 :c 4 :d 5 :e 6 :f))
>>>
>>> The underlying implementation of sorted-map uses a PersistentTreeMap,
>>> which can be accessed with `tree`:
>>>
>>> (.tree t) ;=> [2 :b]
>>>
>>> I have not found a way to access the left and right branches, since
>>> calling `left` fails:
>>>
>>> (.left (.tree t))
>>>
>>> IllegalArgumentException Can't call public method of non-public class:
>>> public clojure.lang.PersistentTreeMap$Node clojure.lang.
>>> PersistentTreeMap$BlackBranch.left()  
>>> clojure.lang.Reflector.invokeMatchingMethod
>>> (Reflector.java:88)
>>>
>>> Perhaps it would be reasonable to make such calls possible, at least
>>> from the Java API (without reflection)?
>>>
>>> CLJ-1008 (http://dev.clojure.org/jira/browse/CLJ-1008) offers one
>>> possible way to support a public API. It was created in 2012. Perhaps it
>>> could use another look. Thoughts?
>>>
>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Workaround for CLJ-1604?

[Andy Fingerhut]

You could build your own patched version of Clojure, with the proposed
patch from the ticket, and use that.  If you try that, it would be good to
add a comment to the ticket of whether it fixed the problem for you, too.

Note: I fully understand if you'd prefer not to do this.  Simply giving
another alternative to the ones you mentioned.

Andy

On Thu, Feb 19, 2015 at 8:10 PM, Adam Krieg  wrote:

> I'm running into what appears to be the same issue described in CLJ-1604
> , where a library that I'm
> using (Korma) has a function that clashes with a new function in Clojure
> core with 1.7, update.
>
> At the point of invocation, I get the compilation exception:
> Exception in thread "main" java.lang.RuntimeException: No such var:
> korma.core/update,
>
> Other than ripping out Korma or downgrading to 1.6, is there anything I
> can do to avoid this issue?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [BUG] : clojure.data/diff handles nested empty collections incorrectly

[Andy Fingerhut]

Timothy, I probably haven't thought about this carefully enough yet, but
why do you expect the return value to be ({:x {:y 1}} nil nil) in your test
case?

Why not ({:x {:y 1}} {:x {}} nil)  ?

Andy

On Fri, Feb 20, 2015 at 4:13 PM, Timothy Pratley 
wrote:

> Alex/Rich, if a patch is welcome for this I am happy to raise a ticket and
> submit.
>
> Test case:
> (clojure.data/diff {:x {:y 1}} {:x {}})
>
> Expected:
> ({:x {:y 1}} nil nil)
>
> Actual:
> ;; => ({:x {:y 1}} {:x nil} nil)
>
> Problem:
> There is nothing only in the second input, so the second result should be
> nil.
> {:x nil} implies that there is a nil in the second input, which there is
> not.
> Furthermore the result cannot distinguish from
> (clojure.data/diff {:x {:y 1}} {:x nil})
> Which should and does correctly return {:x nil} as the second result.
>
> Affects:
> Empty collections in first or second argument
>
> Cause:
> clojure.data/diff-associative-key
> [(when (and in-a (or (not (nil? a*)) (not same))) {k a*})
>  (when (and in-b (or (not (nil? b*)) (not same))) {k b*})
>  (when same {k ab})]
> should be
> [(when (and in-a (or (not (nil? a*)) (and (not same) (nil? va {k
> a*})
>  (when (and in-b (or (not (nil? b*)) (and (not same) (nil? vb {k
> b*})
>  (when same {k ab})]))
>
> Why do I care?:
> I use and maintain a modest library to send state changes from Clojure to
> ClojureScript
> https://github.com/timothypratley/patchin
> In this context, handling empty collections/nils is desirable so the data
> can be treated abstractly.
> I have a workaround, so am not affected by the issue, just providing a
> motivating example where knowing the difference is useful.
>
>
>
> Regards,
> Timothy
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [BUG] : clojure.data/diff handles nested empty collections incorrectly

[Andy Fingerhut]

The goal with your library is to transmit deltas between two systems, and
then apply those deltas to the other system to make them match, yes?

How would you use the diff return value ({:x {:y 1}} nil nil) to cause a
system that currently has {:x {:y 1}} to change to {:x {}}, as opposed to
some other state?  That is, how would it know it needs to change the value
associated with :x to {} as opposed to removing the key :x and its value
entirely?

Andy

On Fri, Feb 20, 2015 at 5:34 PM, Timothy Pratley 
wrote:

> On Fri, Feb 20, 2015 at 4:40 PM, Andy Fingerhut 
> wrote:
>
>> Why not ({:x {:y 1}} {:x {}} nil)  ?
>>
>
> Hi Andy,
>
>
> Great point! Both solutions convey accurately the same meaning? I have a
> subjective preference to nil as the absence of things in b that are not in
> a. Conversely {:x {}} implies that :x is replaced by an empty map, which is
> indeed equivalent, but to my mind the map was always there. It does call
> out the fact that the collection is now empty, which might be useful if one
> was interested in identifying that condition.
>
> Do you have any thoughts on why {:x {}} might be better?
>
>
> Regards,
> Timothy
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [BUG] : clojure.data/diff handles nested empty collections incorrectly

[Andy Fingerhut]

Sorry if you've already explained this and I'm missing it, but for example,
suppose the current state on both sides is {:x {:y 1}}, and the 'sender'
wants to change to one of these states:

(a) {:x nil}
(b) {:x {}}
(b) {:x []}
(c) {:x {:y nil}}

Do you plan to support sending a different 'diff' in each of these cases,
such that the 'receiver' can end up in the same final state, for each of
these case?

Or perhaps the idea is that some of those are not supported?

Thanks,
Andy

On Fri, Feb 20, 2015 at 8:13 PM, Timothy Pratley 
wrote:

> Hi Andy,
>
>
> On Fri, Feb 20, 2015 at 5:51 PM, Andy Fingerhut 
> wrote:
>
>> The goal with your library is to transmit deltas between two systems, and
>> then apply those deltas to the other system to make them match, yes?
>>
>
> Correct, it works well for what I need it to do. :)
>
>
>> How would you use the diff return value ({:x {:y 1}} nil nil) to cause a
>> system that currently has {:x {:y 1}} to change to {:x {}}, as opposed to
>> some other state?  That is, how would it know it needs to change the value
>> associated with :x to {} as opposed to removing the key :x and its value
>> entirely?
>>
>
> The short answer is (update-in state [:x] dissoc :y).
> Which is identical in result to (assoc state :x {}).
> So either representation could be used to accomplish the goal, and can be
> interchanged.
>
> As such I think either solution would be great! (and am happy to provide a
> patch either way).
>
>
> Below digresses from clojure.core/diff concerns, but I'm including it to
> more fully answer your question.
>
> I'll expand a little on the way patchin currently works. The second
> clojure.data/diff return value is used as replacements. The first item
> returned from clojure.data/diff is used as the keys to dissoc, excluding
> things that would be replaced anyway. If the patch size is greater than the
> target, just the target is sent. So in the end it constructs the same patch
> you described (as it is smaller). Valid patches look like either:
> [x]
> [diss replace]
> So the patch sent is
> [{:x {}}]
> Which means "replace the state with {:x {}}
> but for {:x {:y "foo", z "bar", w "baz"}} -> {:x {:z "bar", w "baz", q
> "bonza"}} the patch sent is
> [{:x {:y 1}} {:x {:q "bonza"}}]
> Which means "replace in state [:x :q] bonza, and strip out [:x :y].
> Plus some minor tricks to handle sets, and treat sequences as values.
> There are good resources for sequence diff compression but haven't thought
> of a good way to represent them in the same patch as a structural diff, and
> haven't had the need.
>
> I do not think clojure.data/diff need concern itself with optimization, as
> it currently serves the purpose of identifying differences very well, and
> optimization comes with calculation/size tradeoffs specific to the usage.
>
>
> Regards,
> Timothy
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

To the person who recently tried to post a message about using vert.x library

[Andy Fingerhut]

I may have the name of that library incorrect.  I was going through
messages to the Clojure group from new senders.  I saw this message, and
was intending to approve it, but accidentally hit the 'report as spam and
block the sender' button.  Ugh.  My apologies.  If you were the sender, and
you are reading this, please send me a private message so I can try to
figure out how to undo this.

Thanks,
Andy

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Border cases for doseq

[Andy Fingerhut]

doseq does not have anything precisely like C/Java/Perl/etc.'s 'break' or
'continue'.  The closest thing might be the :while keyword.  You can see
some one example of its use near the end of the examples on this page:
http://clojuredocs.org/clojure.core/doseq

The best way I know to get such behavior is to use Clojure's loop, where
if/when/whatever-conditional-statements-you-wish can used to control
explicitly to do another loop iteration using recur, or not.

Andy

On Wed, Feb 25, 2015 at 8:59 AM, Cecil Westerhof 
wrote:

> At the moment I have the following function:
> (defn do-show-table
>   [table]
>   (doseq [{:keys [table_name]} (show-tables db-spec)]
>  (when (= (lower-case table) (lower-case table_name))
>(let [format "%-20s %-30s %-5s %-5s %-20s\n"]
>  (printf format "Field" "Type" "Null?" "Key" "Default")
>  (doseq [{:keys [field type null key default]}
>   (jdbc/query db-spec [(str "SHOW COLUMNS FROM "
> table)])]
> (printf format field type null key default))
>
> In this case it is not very important, because the outer sequence will not
> be very big. But I would like to leave the doseq at the moment that the
> when is executed. It will be done 0 or 1 times, so after it is done, there
> is no use in continuing the sequence walk.
>
> The second part is that I would like to do an action if it is done 0 times
> after the doseq. Is this possible?
>
> --
> Cecil Westerhof
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.