I have a vector of values which compute large result sets, similar to a
vector of lazy seqs. It's a vector because it has to be sorted before the
values are evaluated, in order.
When evaluating each value in order of the vector, it's important that they
are garbage collected, because two won't fit in memory. Here's an
equivalent example:
(let [bigval (fn [] (map #(.getBytes %) (repeat (* 1000 1000) "blah")))]
(map #(do (println (count %)) (Thread/sleep (* 10 1000)))
[(bigval) (bigval)]))
If you watch this with jmap -histo:live, or whatever, both elements end up
in memory at the same time (2 million byte arrays). But this is some
property of the vector, or chunking, or something. If I change the
collection to this:
(into '() [(bigval) (bigval)])
the first bigval is garbage collected before the second is computed. Also,
if I use Stuart's "unchunk" from this post
http://stackoverflow.com/questions/3407876/how-do-i-avoid-clojures-chunking-behavior-for-lazy-seqs-that-i-want-to-short-ci
(unchunk [(bigval) (bigval)])
the first bigval is garbage collected before the second is computed.
Wondering if someone can explain what's happening in each example.
With the first, form [a b], the 'map' call returns a chunked sequence. The
'count' call realizes the first sequence, but it can't be garbage collected
because of a reference in the chunk?
With the second form, (into '() [a b]), the 'into' eagerly builds a
persistent list. The 'map' call does not chunk the list, because (seq
some-list) returns a list, not a chunked sequence. Not sure why lists are
different this way.
With the third form, (unchunk [a b]), unchunk returns a lazy seq. The 'map'
call does not chunk the seq. Not sure what's happening here. (seq (unchunk
[1 2])) is a Cons, but (seq (range 5)) is a ChunkedCons.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
