tl;dr Soon, perhaps next week, tempest gate jobs will start failing if
there are any ERROR lines in the logs that are not matched by an entry
in https://github.com/openstack/tempest/blob/master/etc/whitelist.yaml.
There is an exception for neutron because
more work needs to be done there for this to be feasible.
The whitelist file contains a lot of entries that look more like
substantial bugs than incorrectly logging an ERROR due to bad
client data. I have been tracking this for a while and new things show
up frequently. But since the tests pass no one looks at the logs and
notices the bug indicators. We need to stop these bugs from getting
merged. I have filed individual bugs for many of the items in the
whitelist, but by no means all of them. The neutron team is taking on
the task of getting rid of their errors and there are so many it is too
much work to keep an up-to-date whitelist for neutron. So for the time
being, neutron runs will dump all errors to the console but not fail.
In addition to the fact that these log errors indicate bugs, they make
it more difficult to diagnose a problem when builds actually fail in the
tempest tests because it can be hard to tell which log errors are
"known" and which might be causing the failure. Hopefully some priority
will be given to fixing these bugs and removing entries from the
whitelist until it is driven to zero.
If any one has any comments or suggestions to improve this process,
please speak up.
-David
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
