On Thu, Dec 15, 2011 at 6:43 PM, Brad Roberts <[email protected]> wrote:
> On Thu, 15 Dec 2011, Andrew Wiley wrote: > > > On Thu, Dec 15, 2011 at 6:04 PM, Robert Clipsham > > <[email protected]> wrote: > > > I really think github needs built in review tools (more advanced than > just > > > pull requests) to allow things like the auto-tester to be run, or > algorithms > > > like this to be used for manual review and so on. > > > > Well, Github does have an API to allow that sort of thing to happen. > > In theory, a bot could examine a pull request, merge it and run tests, > > and post back the results. > > I'm 90% done with adding pull testing to the existing auto-tester fleet. > It's based on the work that Daniel did. > > The basic overview: > > server: > > every 10 minutes, check github for changes to pull requests > (if they had notifications for pull changes, I'd use it instead) > > every time a trunk commit notification is received, check github for > changes to pull requests > > > client: > > forever { > check for trunk build > if yes { build trunk; continue } > check for pull build > if yes { build pull; continue } > sleep 1m > } > > > Left to do: > > 1) deploy changes to the tester hosts (it's on 2 already) > > 2) finish the ui > > 3) trigger pull rebuilds when trunk is updated > > 4) add back in support for related pull requests (ie, two pulls that > separately fail but together succeed) > > 5) consider adding updating the pull request on github with tester > results. This one needs to be very carefully done to not spam the > report every time a build fails or succeeds. > > 6) update the auto-tester grease monkey script to integrate the pull > tester results with github's ui. > > I'll hopefully finish 1 and 2 tonight. I can do 3 manually until it's > automated. I'm not sure about the ordering of 4-6. They're nice to haves > rather than must haves. > > All these extra builds are going to cost a lot of time. There's about 100 > open pull requests right now. The fastest runs are on the order of 10 > minutes. That's 6 per hour or roughly 17 hours. The slowest are closer > to an hour. So, obviously there's some growing pains to deal with. I'll > probably add a way for github committers to prioritize pull requests so > they build first. > > Luckily this stuff is trivial to throw hardware at.. it's super > parallelizeable. Also the hardware I have for those long runs is super > old. I think the freebsd/32 box is p4 era box. The win/32 is my asus eee > atom based netbook. > > If anyone wants to volunteer build hardware, particularly for the > non-linux platforms, please contact me via email (let's not clutter up the > newsgroup with that chatter). > > Requirements: > I need to be able to access it remotely. > > It needs to be have reliable connectivity (bandwidth is pretty much a > non-issue.. it doesn't need much at all). > > It needs to be hardware you're willing to have hammered fairly hard at > random times. > > I'll almost certainly write some code during the holiday to fire up and > use ec2 nodes for windows and linux builds. With the application of just > a little money, all of those runs could be done fully parallel.. which > would just be sweet to see. Ok, I admit it.. I love working at amazon on > ec2.. and I'm happy to finally have a project that could actually use it. > > Later, > Brad > > With the application of just a little money Out of curiosity, how much is a little?
