Right now, /results/ is served from the new storage and is receiving test results data since a day or two ago. For anything older, you will get redirected to /old-results/ which is on the old storage. This probably breaks your code if you are trying to load /results/ and walk backwards in revisions. We should probably look at adding some sort of map to the /json/builders/ data instead.
On a side note, Chromium test results account for 75% of the 700GB of result data, SnowLeopard is 11%, then everyone else. I assume Chromium generating so much more data than everyone else is expected and desired? -Bill On Oct 18, 2010, at 5:04 PM, Eric Seidel wrote: > The most frequent consumer of the historical data is webkit-patch, > which uses it to map from revisions to builds: > http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/webkitpy/common/net/buildbot.py#L109 > > It's used when we're walking back through revisions trying to find > when the build broke, or when the user passes us a revision and > expects us to know build information about such. > > It's possible we could move off that map with some re-design. > > > One thing which would *hugely* speed up webkit-patch failure-reason > (and sherriff-bot, and other commands which use the > build_to_revision_map) is if we could make the results/ pages > paginated. :) > > > I would be nice to keep all the build data for forever. Even if after > some date in the past its on a slower server. > > -eric > > > On Sat, Oct 16, 2010 at 12:38 AM, William Siegrist <[email protected]> > wrote: >> On Oct 14, 2010, at 10:13 AM, William Siegrist wrote: >> >>> On Oct 14, 2010, at 9:27 AM, William Siegrist wrote: >>> >>>> I am in the process of moving buildbot onto faster storage which should >>>> help with performance. However, during the move, performance will be even >>>> worse due to the extra i/o. There will be a downtime period in the next >>>> few days to do the final switchover, but I won't know when that will be >>>> until the preliminary copying is done. I am trying not to kill the master >>>> completely, but there have been some slave disconnects due to the load >>>> already this morning. I'll let everyone know when the downtime will be >>>> once I know. >>>> >>> >>> >>> The copying of data will take days at the rate we're going, and the server >>> is exhibiting some strange memory paging in the process. I am going to >>> reboot the server and try copying with the buildbot master down. The master >>> will be down for about 15m, if I can't get the copy done in that time I >>> will schedule a longer downtime at a better time. Sorry for the churn. >>> >> >> >> Most of build.webkit.org is now running on the newer/faster storage. >> However, the results data[1] is hundreds of gigabytes, going back 6 months, >> and the new storage is not big enough. Does anyone have any opinion on how >> much data to keep in results? Does anyone ever look back more than a month >> or two? For now, the results will still come up a slowly, but hopefully the >> rest of buildbot is a little more responsive. We're still planning to move >> all of webkit.org to better hardware soon, but we hit some delays in that >> process. >> >> [1] http://build.webkit.org/results/ >> >> Thanks >> -Bill >> >> _______________________________________________ >> webkit-dev mailing list >> [email protected] >> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >> _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
