> I'm slightly surprised by this - in my experience parallel builds beat
> --make as long as the parallelism is a factor of 2 or more. Is your
> dependency graph very narrow, or do you have lots of very small modules?
I get full parallelism, 4 threads at once on a 2 core machine * 2
hyperthread/whatever core i5, and SSD. Maybe I should try with just 2
threads. I only ever get 200% CPU at most, so it seems like the
hyperthreads are not really much like a whole core.
The modules are usually around 150-250 lines. Here are the timings
for an older run:
from scratch (191 modules):
runghc Shake/Shakefile.hs build/debug/seq 128.43s user 20.04s system 178%
cpu 1:23.01 total
no link: runghc Shake/Shakefile.hs build/debug/seq 118.92s user 19.21s sys
tem 249% cpu 55.383 total
make -j3 build/seq 68.81s user 9.98s system 98% cpu 1:19.60 total
modify nothing:
runghc Shake/Shakefile.hs build/debug/seq 0.65s user 0.10s system 96% cpu
0.780 total
make -j3 build/seq 6.05s user 1.21s system 85% cpu 8.492 total
modify one file:
runghc Shake/Shakefile.hs build/debug/seq 19.50s user 2.37s system 94% cpu
23.166 total
make -j3 build/seq 12.81s user 1.85s system 94% cpu 15.586 total
>From scratch, --make (that's what 'make -j3' winds up calling) wins
slightly. --make loses handily at detecting than nothing need be done
:) And as expected, modifying one file is all about the linking,
though it's odd how --make was faster.
> I like the idea! And it should be possible to build this without modifying
> GHC at all, on top of the GHC API. As you say, you'll need a server
> process, which accepts command lines, executes them, and sends back the
> results. A local socket should be fine (and will work on both Unix and
> Windows).
>
> The server process can either do the compilation itself, or have several
> workers. Unfortunately the workers would have to be separate processes,
> because the GHC API is single threaded.
>
> When a worker gets too large, just kill it and start a new one.
A benefit of real processes, I'm pretty confident all the memory will
be GCed after the whole process is killed :)
I'll start looking into the ghc api. I have no experience with it,
but I assume I can look at what GhcMake.hs is doing and learn from
that.
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
