(running clojure 1.2 snapshot)
Q1: Why does pmap use the number of available processors + 2? I would
have thought it would just use the number of avail processors...
Q2: Could someone clear up my misunderstanding of pmap w/ respect to the
code snippets below? Pmap doesn't seem to be limiting the number of
threads to the number of processors + 2...
I've created an anon func that does't return, so I wouldn't expect the
pmap step function to advance beyond 4 (I have 2 processors):
#1: Limit pmap to # of processors
----------------------------------
user=> (pmap #(while true (do (println "Thread: " (.getId
(Thread/currentThread)) "item: " %)(Thread/sleep 500))) (range
(.availableProcessors (Runtime/getRuntime))))
Thread: 12 item: 0
(Thread: 13 item: 1
Thread: 12 item: 0
Thread: 13 item: 1
Thread: 12 item: 0
Thread: 13 item: 1
Thread: 12 item: 0
Thread: 13 item: 1
Thread: 12 item: 0
Thread: 13 item: 1
Thread: 12 item: 0
Thread: 13 item: 1
Thread: 12 item: 0
Thread: 13 item: 1
Thread: 12 item: 0
Thread: 13 item: 1
Thread: 12 item: 0
--> just two threads running, as expected
#2: Limit pmap to # of processors * 10
--------------------------------------
user=> (pmap #(while true (do (println "Thread: " (.getId
(Thread/currentThread)) "item: " %)(Thread/sleep 500))) (range (* 10
(.availableProcessors (Runtime/getRuntime)))))
Thread: Thread: 12 item: 0
(Thread: 25 item: 13
Thread: 24 item: 12
Thread: 23 item: 11
Thread: 22 item: 10
Thread: 21 item: 9
Thread: 20 item: 8
Thread: 19 item: 7
Thread: 18 item: 6
Thread: 17 item: 5
Thread: 16 item: 4
Thread: 15 item: 3
--> In this short snippet, you can see > 4 threads running...expected?
toddg=> (source pmap)
(defn pmap
"Like map, except f is applied in parallel. Semi-lazy in that the
parallel computation stays ahead of the consumption, but doesn't
realize the entire result unless required. Only useful for
computationally intensive functions where the time of f dominates
the coordination overhead."
{:added "1.0"}
([f coll]
(let [n (+ 2 (.. Runtime getRuntime availableProcessors))
rets (map #(future (f %)) coll)
step (fn step [[x & xs :as vs] fs]
(lazy-seq
(if-let [s (seq fs)]
(cons (deref x) (step xs (rest s)))
(map deref vs))))]
(step rets (drop n rets))))
<snip>
-Todd
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
