On Fri, May 07, 2010 at 05:31:16AM +0200, Martin Bähr wrote: > On Thu, May 06, 2010 at 06:43:04PM -0400, Michael K. Johnson wrote: > > Here's a configuration that uses fewer larger machines: > > > > 16GB, dual-quad-core systems with decent disk: > > 1 Postgresql commit repository, conary repository, Yum repository (RAID1) > > 1 rmake postgresql (ephemeral) repository, rmake job database > > Mirrorball front end, rMake head node (RAID1) > > could these be combined into one even larger machine? > (say a dual-quad-core with 32GB or ram (or even more) or is that not > enough cpu then?
Not easily. You get significant contention between the two heavy-use postgresql databases here -- the postgresql commit repository database, and the postgresql rmake (ephemeral) repository database -- because the I/O load between them is not random; they tend to peak together, so running them together is going to tend to slow it all down. It's not so much the CPU; the limiting factor is probably disk IO. If you configured the system with enough disks and put the two databases on entirely separate sets of spindles so that the heads move entirely independently, it is possible that this configuration could work OK. However, I want to make it clear that we have not tested this configuration and so we can't tell you that it would perform well. Others might be able to point out reasons I haven't thought of why it would have trouble. If I had an unused and idle 32GB system with 8-12 cores and 6 fast spindles, I'd probably try this configuration just as an experiment to see how it would work... :) But it seems to me that recommending it without having tried something similar is an invitation to possibly waste time, and I don't want to go there. > > 16GB, dual-quad-core systems with decent disk: > > 2-3 rMake build nodes (RAID0) > > 8GB, quad-core systems with decent disk: > > 3-4 rMake build nodes (RAID0) > > 4GB, dual-core systems: > > 4-8 rMake build nodes (RAID0) > > it looks like build nodes would scale independently of the servers, weak > build nodes could be scaled by just having more of them. (convert > desktops overnight :-) It doesn't scale linearly. As you add build nodes, you can trip over various performance cliffs. For example, adding build nodes can saturate the DB, at which point adding more rMake build nodes would just make the builds go slower, not faster -- a machine corollary of Brooks' law that "adding more programmers to a late software project makes it later"... (If you compare http://hg.rpath.com/rmake-2.0/ to trunk development at http://hg.rpath.com/rmake/ you'll see massive changes in trunk -- one of several significant motivations for the redesign is to make rMake scale better when given more build nodes.) A problem with adding desktops overnight is that current rMake does not have a good way to idle a build node; the whole build process can recover if an rMake build node goes offline, but some builds will have to be restarted. Another is that, as I've mentioned before, all the systems have to be in close network proximity to each other for this to work well; this is not a s...@home style workload. > do buildnodes need to run a conary OS or would it be enough to run them > chroot style inside any linux OS? We have not tried to run them in chroots. I can't tell you off the top of my head what might or might not work there. We would provide appliances to install on systems if we're going to help with this so that we don't waste time dealing with config. _______________________________________________ Foresight-devel mailing list Foresight-devel@lists.rpath.org http://lists.rpath.org/mailman/listinfo/foresight-devel
