On Sun, 1 May 2011 01:36:38 +0200, Andreas Barth wrote:
Sometimes we have a few packages we don't want to build on a certain
buildds. Sometimes this is because this package needs lots of ram. Or
it takes quite long and would waste the parallel building a machine
supports. Or whatever else. Of course a package could be in more than
one category.
Yes, you're facing basically the same problem I tried to address in
2000/2001 when doing my renderserver and later for what Multibuild was
intended to do as well. ;-)
Now, what I would like to do is to write that down in a central file
with categories.
I would recommend to use a database, really.
That is, to mark packages as "builds only with more than one gigabyte
of ram". And to mark buildds as "has 6 cores", "only ... ram" - so
that I don't need to copy entries from buildd to buildd, but just say
"that new machine is the same class as ...", and that's it.
Another category would be "fast disk/raid". There are some packages
with lots of disk accesses. When you can schedule those packages to a
buildd that has faster disk access like in having multiple spindles for
faster seeks, you can minimize build times as well. We faced that
problem on m68k particularly on IDE vs SCSI disks on Amigas, as IDE was
dog slow. Another example there was the faster disks on Amigas vs slower
SCSI disks in Apple machines.
Now my question is just: How to do that efficient? I.e. how would
such
a configuration file look like, and how the code to distribute the
package on the most fitting buildd(s)? (I.e. it's better to waste 5
out of 6 cores than to not build a package at all, but a package
needing at least 1g ram can't build on a buildd with only 512mb - but
no package should starve in the end.)
Ideas? Suggestions? Code?
Look at my update-buildd.net from Buildd.net, which I used to collect
data from the buildds such as RAM, kernel, uptime, used swap and such
(http://buildd.net/cgi/hostpackages.cgi?unstable_arch=m68k&searchtype=arrakis).
I store this information into the database and also the build times of
the packages. With this dataset it should be possible to have the
wanna-buildd schedule packages in such a way to minimize the build times
because you can decide which buildd is the most suitable buildd for the
next package.
--
Ciao... // Fon: 0381-2744150
Ingo \X/ http://blog.windfluechter.net
gpg pubkey: http://www.juergensmann.de/ij_public_key.
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/f423f24d7f17a3abe30510a870357...@muaddib.hro.localnet
