On Aug 7, 6:42 pm, Ben Mabey <b...@benmabey.com> wrote: > Hi all, > I've run into an issue with a lazy-seq either being prematurely realized > or having the head unwittingly retained. Reading chapter 5 in The Joy > of Clojure I realize that I am breaking one of the rules (page 150 in my > MEAP version): avoid binding your lazy sequences locally. That said, I > am using Clojure 1.2RC2, and the book states that as of 1.2 the compiler > is smarter about avoiding premature realization issues. I think I may > have stumbled upon a case where it is not handling (or is not able to). > Basically, the entire lazy-seq is being retained in memory when it is > bound locally and then realized in a different thread via a future. > > Here is an example of the problem I am facing: > > (deftype Foo []) > ; The entire lazy-seq is held in memory resulting in an OutOfMemoryError. > (let [foos (take 10000000 (repeatedly #(Foo.)))] > @(future (doseq [foo foos] foo))) > > I tried to reproduce this problem in different ways, but as far as I > could tell the above combination of local binding and realization in a > future is the only way the problem crops up. Here is a gist of some of > the other variants that did not cause the > problem:http://gist.github.com/513376 > > My co-worker, Tim Harper, also did some work with weak references > showing that the objects in the lazy-seq are not being > GCed:http://gist.github.com/510601 > > Could someone shed some light onto why I am seeing the entire lazy-seq > retained? FWIW, I ran across this issue when using fill-queue like so: > > (defn map-queue > "Like map, but populated eagerly via fill-queue > (map-queue odd? (range 1 10) :queue-size 5) ;; will always calculate > 5 ahead of the last used item from the lazy sequence" > [f coll & options] > (apply fill-queue (bound-fn [fill] (doseq [value coll] (fill (f > value)))) options)) > > So in my case coll is a lazy-seq that can't be retained in memory. Any > explanation of what I'm doing wrong and how to better go about my > problem would be greatly appreciated. > > Thanks! > > -Ben
As a follow up for anyone following this thread / discovering it, it was discussed in #clojure on Tuesday: http://clojure-log.n01se.net/date/2010-08-10.html (search for timcharper for the start of the discussion) I've distilled the issue down to a series of two tests: in one case, the head is properly released, and in the other, the head is retained even though it seems it shouldn't be. http://gist.github.com/510601 Tim -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
