So we are close to being able to start doing this. The current whitelist
is here
https://github.com/openstack/tempest/blob/master/etc/whitelist.yaml. I
have a "find-errors" script that watches for successful builds and pulls
out the non-whitelisted errors. For the past few weeks I have been doing
the following:
1. Run find-errors
2. File bugs on any new errors
3. Add to whitelist
4. Repeat
There are still some very flaky cases. I will do one more iteration of
this. Right now this script
https://github.com/openstack/tempest/blob/master/tools/check_logs.py
dumps non-whitelisted errors to the console log but
always returns success. The question now is how long should all jobs run
with no new errors showing, before changing check_logs.py to fail if
there are any new errors?
-David
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev