Hiya, check this code out guys:
----
(defn point [x y]
(println "x:" x "y:" y)
[x y])
(defn gen-data [n m]
(for [i (range n)]
(for [j (range m)]
(point i j))))
(def data (apply concat (gen-data 100 100)))
(nth data 5)
"The output was the following:"
"x: 0 y: 0
x: 0 y: 1
x: 0 y: 2
x: 0 y: 3
x: 0 y: 4
x: 0 y: 5
x: 0 y: 6
x: 0 y: 7
x: 0 y: 8
x: 0 y: 9
x: 0 y: 10
x: 0 y: 11
x: 0 y: 12
x: 0 y: 13
x: 0 y: 14
x: 0 y: 15
x: 0 y: 16
x: 0 y: 17
x: 0 y: 18
x: 0 y: 19
x: 0 y: 20
x: 0 y: 21
x: 0 y: 22
x: 0 y: 23
x: 0 y: 24
x: 0 y: 25
x: 0 y: 26
x: 0 y: 27
x: 0 y: 28
x: 0 y: 29
x: 0 y: 30
x: 0 y: 31
[0 5]"
"Seems like other people have similar problems but the issue was
mis-attributed,
as they thought it has to do with `apply` and/or `concat` (read further to
find out why not):
https://groups.google.com/forum/#!topic/clojure/vzhFmpGkWTo
http://clojurian.blogspot.co.uk/2012/11/beware-of-mapcat.html";
"First I too was suspicious about `concat` and `apply`, so I wrote a
version of concat which was not using varargs."
(defn concat2 [coll]
(lazy-seq
(if-let [s (seq coll)]
(if-let [ss (seq (first s))]
(cons (first ss) (concat2 (cons (rest ss) (rest s))))
(concat2 (rest s)))
nil)))
(def data (apply concat (gen-data 100 100)))
(nth data 5)
"The issue was still there unfortunately, exactly the same printout like
the with the first example"
"So next i became suspicious of `for`. Maybe it has to do with the way it
is evaluated. So I rewrote `gen-data` using `map`"
(defn gen-data [n m]
(map (fn [x]
(map (fn [y] (point x y))
(range m)))
(range n)))
"Even with `map` the issue was still present. Maybe both `map` and `for`
has the same problem? Let's rewrite `map` then"
(defn map2 [f coll]
(lazy-seq
(if-let [s (seq coll)]
(cons (f (first s))
(map2 f (rest s)))
nil)))
(defn gen-data [n m]
(map2 (fn [x]
(map2 (fn [y] (point x y))
(range m)))
(range n)))
(def data (apply concat (gen-data 100 100)))
(nth data 5)
"x: 0 y: 0
x: 0 y: 1
x: 0 y: 2
x: 0 y: 3
x: 0 y: 4
x: 0 y: 5
[0 5]"
"GOTCHA!!! WORKS CORRECTLY!!! BUG FOUND!!!"
"...seems like both `map` and `for` are affected, possibly because `for`
depends on `map` (just assumption sorry, I was L.A.Z.Y. to check)"
"Also, the bug was present while testing with both Clojure versions 1.4 and
1.5.1"
"Finally, just to emphasize how serious the issue is try the same with ONLY
three levels of nested \"mapcatting\":"
(defn point3d [x y z]
(println "x:" x "y:" y "z:" z)
[x y z])
(defn gen-data3d [n m k]
(mapcat (fn [x]
(mapcat (fn [y]
(mapcat (fn [z] [(point3d x y z)])
(range k)))
(range m)))
(range n)))
(def data3d (gen-data3d 100 100 100))
(nth data3d 5)
"Not gonna copy the output here... seems like if the number of nested level
of mapcats is `n` then the total number of elements evaluating non-lazily
in one go is 32^n."
---
BTW, the `map2` implementation above is the quite matching
clojure.core/map, except the core version has special handling for
ChunkedSeqs. Most probably the issue comes from there. Can someone look
into this please?
Thanks,
Daniel
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
