Howdy! I've been operating a shellci for a while now and overall it is very smooth.
The main new feature now is to automatically retrigger events that neither definitely succeed (exit status 100) nor definitely fail (exit status 101). In this case the CI will vote "0" with the logs and then automatically schedule a new test, up to a max of 5 tests, until it can conclude +1 or -1. This seems to work well. On my setup I often see tests fail due to network problems (e.g. timeout during a Git checkout) and automatic retries filer away this class of error very neatly, ultimately casting the +1/-1 votes based purely on the relatively reliable contents of tempest.log. (Thanks Jim Gray for pointing out that detectable intermittent bugs are really easy to deal with http://www.hpl.hp.com/techreports/tandem/TR-85.7.pdf). [The network problems themselves are a little suspicious but for now I see them as a blessing that helps me make shellci robust and I don't worry too much about the root cause.] I'm running 5 parallel builds at the moment. I've tried ramping that up to 10 or 20 but then I start to see Vagrant/VirtualBox startup errors. Again, I'm using this as a test case for the automatic retries, and not digging too deep yet. I'd like to be able to track the results via RSS feeds. Can I do this via openstack.org or should I support that directly in shellci? disk space will become an issue soon. Currently I'm running around 500 builds per day and each one generates around 8MB of logs. Compression and redundancy-avoidance may go a long way towards reducing his problem, however it seems a pity to have a "scarcity" mentality when it comes to logging (I'd prefer to do more rather than less). Any bright ideas for conveniently archiving some terabytes of logs? Overall I'd say that shellci is quite reliable now. The main reason I don't recommend it to others yet is that I'm a bit too busy to provide support with setting it up this week. Cheers, -Luke
_______________________________________________ OpenStack-dev mailing list OpenStackfirstname.lastname@example.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev