Hi > > The problem I'm trying to solve is running system commands in > > parallel.
> "system commands" means execution of external commands or just system > calls inside Haskell? Calls to System.Cmd.system, i.e. running external console processes. It's a make system I'm writing, so virtually all the time is spent in calls to ghc etc. To Bulat: I should have been clearer with the spec. The idea is that multiple calls to paralell_ can execute, and a function executing inside parallel_ can itself call parallel_. For this reason I need one top-level thread pool, which requires unsafePerformIO. If I create a thread pool every time, I end up with more threads than I want. > Your parallel_ does not return until all operations are finished. > >> parallel_ (x:xs) = do >> ys <- mapM idempotent xs >> mapM_ addParallel ys >> sequence_ $ x : reverse ys > > By the way, there is no obvious reason to insert "reverse" there. There is a reason :-) Imagine I do parallel_ [a,b,c] That's roughly doing (if b' is idempotent b): enqueue b' enqueue c' a b' c' If while executing a the thread pool starts on b', then after I've finished a, I end up with both threads waiting for b', and nothing doing c'. If I do a reverse, then the thread pool and me are starting at different ends, so if we lock then I know it's something important to me that the thread pool started first. It's still not idea, but it happens less often. > What I meant was something like: >> >> para [] = return () >> para [x] = x >> para xs = do >> q <- newQSemN 0 >> let wrap x = finally x (signalQSemN q 1) >> go [y] n = wrap x >> waitQSemN q (succ n) >> go (y:ys) n = addParallel (wrap y) >> go ys $! succ n >> go xs 0 > > This is nearly identical to your code, and avoid creating the MVar for each > operation. I use "finally" to ensure the count is correct, but if a worker > threads dies then bas things will happen. You can replace finally with (>>) > if > speed is important. Consider a thread pool with 2 threads and the call parallel_ [parallel_ [b,c],a] You get the sequence: enqueue (parallel_ [b,c]) a wait on parallel_ [b,c] While you are executing a, a thread pool starts: enqueue b c wait for b Now you have all the threads waiting, and no one dealing with the thread pool. This results in deadlock. I guess the "nested calls to parallel_" bit is the part of the spec that makes everything much harder! Thanks Neil _______________________________________________ Haskell-Cafe mailing list [email protected] http://www.haskell.org/mailman/listinfo/haskell-cafe
