Hi all:
I think I encountered a problem in the clojure.core/take produces lazy
sequences, and I wanted to run it by some folks to confirm it was indeed a
bug.
In a nutshell, clojure.core/take produces lazy sequences in a way such that
(take
(count t) t) will always retain the head of its base sequence.
Here's some more detail using a variation of the example presented on pg.
117 in the Joy of Clojure. The following produces an out of memory
exception:
(let [[t d] (split-with (partial > 50) (range 1e8))]
[(doall (take 50 t)) (count d) (count t)])
But the following does not:
(let [[t d] (split-with (partial > 50) (range 1e8))]
[(doall (take 51 t)) (count d) (count t)])
This actually seems like a bug in clojure.core/take. Here is the current
implemenation:
(defn take
([n coll]
(lazy-seq
(when (pos? n)
(when-let [s (seq coll)]
(cons (first s) (take (dec n) (rest s))))))))
Defined as such, both t and (take 50 t) have 50 elements, and (doall (take
50 t)) realizes all fifty elements in both sequences. However, while each
lazy sequence only has 50 elements, there are actually 51 lazy-seq thunks.
The first 50 thunks produce the values of the lazy sequence, and the last
thunk produces a nil. When called, the first fifty thunks will realize
their corresponding thunk in coll, but the final nil thunk will fail the (pos?
n) predicate before it can call (seq coll).
What this means in our example is that despite having all 50 elements of
the lazy sequence t realized, the nil thunk of t will not be realized. It
will retain the head of (range 1e8), producing an out of memory exception.
Consider this alternate definition of take:
(defn take*
[n coll]
(lazy-seq
(when-let [s (seq coll)]
(when (pos? n)
(cons (first s) (take* (dec n) (rest s)))))))
Notice that unlike clojure.core/take, the final thunk produced by take*
will call (seq coll), thereby realizing the nil thunk of coll.
The following produces no exception:
(let [[t d] (split-with (partial > 50) (range 1e8))]
[(doall (take* 50 t)) (count d) (count t)])
And in fact, most of the other lazy-seq producing clojure.core functions
put the (seq coll) predicate first.
All this being said, I have two questions:
1. Does all this pass mustard?
2. If this is indeed a bug, is this the sort of thing that I would need to
produce a ticket/patch?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
