I'm currenlty upgrading build.webkit.org from 0.8.3 to 0.8.5. It should be back online shortly. Lucas
On Feb 29, 2012, at 12:38 PM, Ryosuke Niwa wrote: > On Wed, Feb 29, 2012 at 11:32 AM, Lucas Forschler <[email protected]> > wrote: > I am looking into improving performance of build.webkit.org. After doing > some initial research I feel that buildbot isn't the most efficient in terms > of processing a large number of requests. I believe requests are processed > serially, and not in parallel. The buildbot documentation mentions a limit > to the number of slaves that can be supported, but doesn't give any specific > numbers. As webkit has grown, and the number of builds and slaves we are > supporting has increased, it seems the server is showing signs of reaching > this capacity. We may have reached a limit on the number of slaves we can > handle with our current configuration. > > On Chromium side, buildbot appears to leak a great amount of memory. e.g. we > end up running out of memory at some point because buildbot keeps leaking > some memory over time :( Do you see the same issue on your buildbot? > > -Upgrades to hardware. It's possible faster hardware could alleviate some > problems. It is unclear if this is really the limiting factor, or how much > additional capacity this would provide. > > My understanding is that we use a really fast server but still experience a > similar problem. > > -Updating the way files are transferred > (https://bugs.webkit.org/show_bug.cgi?id=73484) > > Would it make sense to have a separate apache server, etc... for receiving > these files? twistd doesn't appear to be a cut out for uploading/downloading > megabytes of data. > > -Upgrade the version of buildbot. (0.8.3 -> 0.8.5). > Version 0.8.4 and higher supposedly has a threading improvement, but > it's unclear how much it would benefit us since it looks to be mostly > database related. > > I've been told that upgrading buildbot didn't improve the performance. But > upgrading it to 0.8.5 is probably a good nonetheless since it fixes various > bugs I've encountered while maintaining Chromium bots and also when I ran > buildbot locally. > > -Investigating some kind of apache caching layer on top of buildbot > > MUST DO THIS. This will greatly reduce the latency for most people. > > -Adding a second master to the buildbot system. > buildbot supports multi-master mode. We could potentially split out > build and test onto two separate masters. > build.webkit.org and test.webkit.org ? > > build.chromium.org takes this approach. But then we won't be able to see both > build and test step at once though... Would it make more sense to split per > port? e.g. Apple Mac & Apple Win, Qt & Gtk, Chromium & misc or something > along that line? > > - Ryosuke >
_______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
