So we are close to being able to start doing this. The current whitelist is here https://github.com/openstack/tempest/blob/master/etc/whitelist.yaml. I have a "find-errors" script that watches for successful builds and pulls out the non-whitelisted errors. For the past few weeks I have been doing the following:

1. Run find-errors
2. File bugs on any new errors
3. Add to whitelist
4. Repeat

There are still some very flaky cases. I will do one more iteration of this. Right now this script https://github.com/openstack/tempest/blob/master/tools/check_logs.py dumps non-whitelisted errors to the console log but always returns success. The question now is how long should all jobs run with no new errors showing, before changing check_logs.py to fail if there are any new errors?

 -David

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to