On 07/09/2021 18:02, Stefan Esser wrote:
Wouldn't this break META_MODE?
I have never managed to get META_MODE to work but my understanding is that META_MODE is addressing a problem that doesn't really exist in any other build system that I've used: that dependencies are not properly tracked.
When I do a build of LLVM with the upstream build system with no changes, it takes Ninja approximately a tenth of a second to stat all of the relevant files and tell me that I have no work to do. META_MODE apparently lets the FreeBSD build system extract these dependencies and do something similar, but it's not enabled by default and it's difficult to make work.
I'd rather be able to continue building the world within a few minutes (generally much less than 10 minutes, as long as there is no major LLVM upgrade) than have a faster LLVM build and then a slower build of the world ...
The rest of this thread has determined that building LLVM accounts for half of the build time in a clean FreeBSD build. LLVM's CMake is not a great example: it has been incrementally improved since CMake 2.8 and doesn't yet use any of the modern CMake features that allow encapsulating targets and providing import / export configurations.
In spite of that, it generates a ninja file that compiles *significantly* faster than the bmake-based system in FreeBSD. In other projects that I've worked on with a similar-sized codebase to FreeBSD that use CMake + Ninja, I've never had the same problems with build speed that I have with FreeBSD.
Working on LLVM, I generally spend well under 10% of my time either waiting for builds or fighting the build system. Working on FreeBSD, I generally spend over 90% of my time waiting for builds or fighting the build system. This means that my productivity contributing to FreeBSD is almost zero.
For reference, changes to LLVM typically build for me in under 30 seconds with Ninja, unless I've changed a header that everything
In particular, building FreeBSD on a 10-24 core machine has very long periods where a number of the cores are completely idle.
Ninja also has a few other nice features that improve performance relative to bmake:
- It lets you put jobs in different pools. In LLVM this is used to put link and compile jobs in different pools because linking with LLD uses multiple threads and a lot more memory than compilation, so a 10-core machine may want to do 12 compile jobs in parallel but only 2 link jobs. This makes it much easier to completely saturate the machine. - Ninja provides each parallel build task with a separate pipe for stdout and stderr, and does not print their output unless a build step fails (or unless you build with -v). With bmake, if a parallel build fails I have to rerun the build without -j, because the output is interleaved with succeeding jobs and it's difficult to see what actually failed. With ninja, the output is from each failed job, with no interleaving.
David
