Well, it's time for an update on how things are going in make-land. The initial pass at getting make -j in shape was enough to get it to work in most cases. As usual, when you start down an unused code path, it also uncovered a lot of issues, some of them fairly small, others really big.
If you read through src-change, after the initial switch to building stuff correctly one command at a time through -j, and getting the pipes to do sensible things, quite a few things happened: - I started keeping track of items very carefully, so that I could figure out when make failed, and where. - I started trying to give out decent error messages. That part is not perfect yet, the messages are often fragmented as they happened `long' after the initial error has happened. - there was some fiddling with variables to allow further changes. - we got to have Suff_FindDeps be lazy, to give it a better change to work with files that just got created (and this involved decoupling wildcard expansion from Suff_FindDeps). Along with other details, and one or two nasty bugs along the way. At the same time, we made great progress in getting make build to work with -j. We're to the point where it proceeds successfully most of the time. There are still a few races that show up, always in the same directories... afs and perl are most likely failure points. There are still oddities in binutils and gcc, and I've uncovered a new one in lynx... All in all, I can build my src/ on my bi-pro in 2/3 of the time it used to take. But some times it fails half-way through. So far, xenocara builds flawlessly. But it it not that useful, since it spends a lot of time in linear configury... the machine that benefits the most would probably be miod's mvme88k, since the compile speed itself makes a huge difference. There are still things to do. Some times, parallel make stops because it can't figure out how to build a given target, whereas sequential make does not. It's because parallel make is greedy and explores its graph of targets as deep as it can. I could try to make it lazier, but in this case, it's the wrong approach: make does not know (yet) how to map its targets to the file system, so if you're in WRKSRC/bin, and you try to build ../bin/autom4te, it does not realize it is the same as autom4te. (Quite a few ports actually build fine in parallel mode already, and the infrastructure proper is fully parallel-capable, but then this was redesigned specifically that way when I started working on it, so it is not a big surprise.) This is a problem I've been aware of for a very long time actually (probably at least 3 or 4 years). It is a complicated issue, but it is the issue we need to fix now. I could possibly try the `lazyness' fix first (if we don't know how to build the target, build everything else, and then come back to it), but this is just postponing the fix... and making it harder, because test cases would occur much less often.
