On Oct 11, 2011, at 7:56 AM, Tassilo Horn wrote: > So indeed, one starts with the number of available processors + 2, and > one single longer running task will wipe out any parallelism. :-( > > IMO, that's not what 99% of the users would expect nor want when calling > (pmap foo coll). I'd vote for making pmap eager in the sense that it > should always try to keep n threads working as long as more tasks are > available. Clearly, the current implementation is useful when your > tasks are equally expensive, i.e., the costs don't depend on the > argument.
I can imagine cases in which one *would* want the current behavior of pmap, especially with infinite lazy sequences, so I wouldn't go quite as far as you are going here. But I agree that many users will often want an eager version that maximizes CPU utilization, and that they may expect pmap to do that. In fact that was what I wanted (and will usually want), and it's what I assumed pmap would do (even though, now that I look afresh at the doc string, it pretty clearly says that it *doesn't* do this). So my hope wouldn't be that we change pmap but rather that we provide something else that is just as simple to use but provides eager, "use all available cores to get the job done as fast as possible" behavior with a simple pmap-like interface. My agent-based pmapall probably isn't the best approach, since calls to it can't be nested. Perhaps j-g-faustus's pmap-pool approach is the way to go, but I do not understand it well enough to know. In any event I would hope that we could provide something like this, because I do think it will fill a common need. -Lee -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
