On Aug 4, 2010, at 12:12 PM, Laurent PETIT wrote: > Lee, I don't have the general answer, but as a related note, I think there > may be a problem with the "with futures" version: > > a. you quickly "bootstrap" all futures in the inner call to map > b. you collect the results of the futures in parallel in the outer call to > pmap > > as a reminder: > (defn burn-via-futures [n] > (print n " burns via futures: ") > (time (doall (pmap deref (map (fn [_] (future (burn))) > (range n)))))) > > problem #1: since map is lazy, the bootstrapping of the futures will follow > the consumption of the seq by pmap (modulo chunked seq behavior). So to > quickly bootstrap all your futures before passing the seq to pmap, you should > wrap the (map) inside a doall > problem #2: maybe deref is a quick enough operation that using pmap with > deref does not make sense (or would make sense if the number of cores were > realllly big, e.g. if the coll would be of size 1.000.000 and the number of > core of the same magnitude order.
I had thought that "(doall (pmap" would force the consumption of a lot of the lazy sequence created by map at the start -- at least as many items as there are cores -- and then that the calls to deref, which are happening concurrently, would force the futures (concurrently). I may not have that quite right, and I can imagine that another doall, around the call to map, might be a good idea -- I find myself sprinkling those around liberally... HOWEVER, the cases with futures are never the anomalies in any of my results, so I don't think this could explain the odd things that I'm seeing. It seems like my futures code must be doing the right thing in the cases in which concurrency wins more generally, e.g. with agents, and that whatever's going wrong in other cases isn't isolated to my futures code... Thanks, -Lee -- Lee Spector, Professor of Computer Science School of Cognitive Science, Hampshire College 893 West Street, Amherst, MA 01002-3359 lspec...@hampshire.edu, http://hampshire.edu/lspector/ Phone: 413-559-5352, Fax: 413-559-5438 Check out Genetic Programming and Evolvable Machines: http://www.springer.com/10710 - http://gpemjournal.blogspot.com/ -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
