Re: Memory issues processing large lazy sequences

2012-06-08 Thread Chris Perkins
On Thursday, June 7, 2012 1:53:30 PM UTC-4, Tom Hume wrote: Hi I'm trying to generate a sequence which corresponds to a breadth-first search of a very wide, deep tree... and I'm hitting memory problems when I go too far along the sequence. Having asked around on the IRC channel and

Re: Memory issues processing large lazy sequences

2012-06-08 Thread David Powell
(dorun (take 2000 (add-layer))) take holds on to the head, because that is what it returns. Try changing take to drop. Take is lazy though, and dorun should drop the head, so that shouldn't be a problem. The problem here is not an holding onto the head issue. Lots of memory is being

Re: Memory issues processing large lazy sequences

2012-06-08 Thread Tom Hume
Thanks David. Could you recommend any other types of search, or a different algorithm entirely, which might give me the same output in a sequence? On Fri, Jun 8, 2012 at 1:29 PM, David Powell djpow...@djpowell.net wrote: (dorun (take 2000 (add-layer))) take holds on to the head, because

Re: Memory issues processing large lazy sequences

2012-06-08 Thread Tom Hume
Nope, that doesn't make any difference; if something's keeping hold of the head, it isn't the take: = (first (drop 1999 (add-ch))) OutOfMemoryError Java heap space clojure.lang.RT.cons (RT.java:552) = (first (drop 1999 (add-layer))) OutOfMemoryError Java heap space

Re: Memory issues processing large lazy sequences

2012-06-08 Thread Tom Hume
Are you able to go to billions of items with the first version? I'm going to be using this to work with truly vast sequences, so if there's any sort of memory issue with the approach, I would expect it to be exposed at some point - given that I can only throw finite amounts of memory at this.

Memory issues processing large lazy sequences

2012-06-07 Thread Tom Hume
Hi I'm trying to generate a sequence which corresponds to a breadth-first search of a very wide, deep tree... and I'm hitting memory problems when I go too far along the sequence. Having asked around on the IRC channel and looked here, the number 1 cause of such problems is inadvertently

Re: Memory issues processing large lazy sequences

2012-06-07 Thread Jorge Barrios
The first version completes without problems. Make sure your JVM has plenty of memory available to garbage collect. Look at the -Xmx setting (for instance with leiningen :jvm-opts [- Xmx2g -server]) REPL started; server listening on localhost port 18885 user= (def atoms '(a b c)) #'user/atoms