After I did a recent "fink update-all" that pulled in a bunch of updates to
a whole slew of Gnome packages, I rediscovered that fink isn't able to take
advantage of the fact that I've got lots of processors (or lots of cores,
at least) on this machine by parallelizing the builds. Spending most of a
day compiling fink packages, when I've got 7 other cores hanging around
without much to do seems a waste.
Would it be possible to add support to fink for this sort of thing?
I hasten to add: I'm *not* asking that fink build multiple packages at the
same time. I suspect that inter-package dependencies would be too hard to
reason about. But it would be nice if fink could provide the option of
using make's "-j" flag so that building each individual package would take
less time.
Here's one possible way this could look, from the perspective of the users
and the package maintainer. (Disclaimer: while I have many years of
programming, OS X, and Linux experience, I've never created a Fink or .deb
package, so I may be unaware of some constraints imposed by the packaging
system.)
- An option in /sw/etc/fink.conf that sets the maximum number of jobs to
run at once (i.e., -j's argument). Default to 1.
- An abbreviation (call it %j), similar to %v, %n, %i etc, for package
authors to use in .info files:
CompileScript: <<
configure
make %j ...other args as necessary
<<
%j would expand to "-j N", where N is set in /sw/etc/fink.conf, or to
nothing at all, if the user didn't explicitly turn on parallel builds.
- It'd also be nice if fink took a command-line argument that allowed the
user to override the setting in /sw/etc/fink.conf as necessary, but
this is less important.
The big drawback to this proposal is that it requires package maintainers
to make a change, however small, to all the .info files. It'd be nice to
have fink just automatically specify the -j flag, but I don't see how to do
that and get around the following problems:
1) Fink would have to recognize the "make" commands in the CompileScript
and insert the -j flag in the right place. This recognition would be
very difficult in the general case, I suspect.
2) I have encountered some software packages that did not build correctly
when I used the -j argument to make, probably as a result of a fault
somewhere in the package or its makefiles. With %j optional, fink
maintainers can work around this problem if they don't have the time
or ability to fix the core issue. Also, they can continue to release
packages if they don't have access to a multi-processor machine to
test the build.
3) Worse, the bugs mentioned in #2 can be hard if not impossible to
reproduce (as is often the case for concurrency bugs). As a result, I
think it's essential to use sequential (and thus deterministic) builds
unless the user *explicitly* asks for parallelism. And it's essential
to allow the user to drop back to sequential builds as necessary.
The proposal above was inspired by Debian's "kernel-package" package,
which has a configuration option to support building the kernel with the -j
switch. In particular, it defaults to a single-threaded build; the user
has to explicitly ask for parallel builds.
I asked about some capacity for this on fink-users several years ago
(http://article.gmane.org/gmane.os.macosx.fink.user/18264), and Alexander
Hansen informed me that this functionality was not present as the result of
an explicit design decision. I wonder if it's time to revisit that
decision, especially as multi-processor and multi-core machines become more
and more widespread.
If it's an issue of developer resources, I'd be more than happy to help
out. Though I wouldn't be able to start working on this for a couple of
months; I really have to finish my dissertation first.
Thoughts? Or is there
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Fink-users mailing list
Fink-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fink-users
