On Thursday, 17 October 2013 10:34:18 UTC+8, Pradeep Gollakota wrote: > Hi All, > > I’m (very) new to clojure (and loving it)… and I’m trying to wrap my head > around how to correctly choose doseq vs dorun for my particular use case. > I’ve read this earlier post > https://groups.google.com/forum/#!msg/clojure/8ebJsllH8UY/mXtixH3CRRsJand I > had a clarifying question. > > From what I gathered in the above post, it’s more efficient to use doseq > instead of dorun since map creates another seq. However, if the fn you want > to apply on the seq can be parallelized, doseq wouldn’t give you the > ability to parallelize. With dorun you can use pmap instead of map and get > parallelization. > > (doseq [i some-lazy-seq] side-effect-fn) > (dorun (pmap side-effect-fn some-lazy-seq)) > > What is the idiomatic way of parallelizing a computation on a lazy seq? > I don't think there is a single idiomatic way. It depends on lots of things, e.g.: - How expensive is each side-effect-fn? If it is cheap, then the ovehead of making things parallel may not be worth it - Do you want to constrain the thread pool or have a separate thread for each element? For the later, futures are an option - Where is the actual bottleneck? If an external resource is constrained, CPU parallelization may not help you at all..... - How is the lazy sequence being produced? Is it already realised, or being computed on the fly? - Is there any concern about ordering / concurrent access to resources / race conditions?
Assuming that side-effect-fn is relatively CPU-expensive and that the runtimes of each call to it are reasonably similar, then I'd say that your (dorun (pmap .....)) version is a decent choice. Otherwise you make want to take a look at the "reducers" library - the Fork/Join capabilities are very impressive and should do what you need. -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
