On Thu, Mar 06, 2014 at 07:53:03AM -0500, Sean Dague wrote: > We're at Freeze, so I want to pick up and understand where we currently > stand with both Neutron and Heat actually getting tested fully in the gate. > > First Neutron - > https://blueprints.launchpad.net/tempest/+spec/fix-gate-tempest-devstack-vm-quantum-full > > > We know that this is *close* as the full job is running non voting > everywhere, and typically passing. How close are we? Or should we be > defering this until Juno (which would be unfortunate).
Can you please clarify - does FF apply to tempest *tests*? My assumption was that we could move from feature development to testing during the FF/RC phase of the cycle, the natural by-product of which will be additional tempest testcases.. > Second Heat - > https://blueprints.launchpad.net/tempest/+spec/tempest-heat-integration > > The Heat tests that are in a normal Tempest job are relatively trivial > surface verification, and in no way actually make sure that Heat is > operating at a real level. This fact is a contributing factor to why > Heat was broken in i2. Yeah, we have a list of wishlist bugs so we can track the additional less-trivial tests we'd like to implement: https://bugs.launchpad.net/heat/+bugs?field.tag=tempest I was hoping we'd get patches at least proposed for most of these before Icehouse ships, but we'll have to see how it goes.. :) > The first real verification for Heat is in the Heat slow job (which we > created to give Heat a separate time budget, because doing work that > requires real guests takes time). > > The heat slow job looks like it is finally passing much of the time - > http://logstash.openstack.org/#eyJzZWFyY2giOiIobWVzc2FnZTpcIkZpbmlzaGVkOiBTVUNDRVNTXCIgT1IgbWVzc2FnZTpcIkZpbmlzaGVkOiBGQUlMVVJFXCIpIEFORCBidWlsZF9uYW1lOmNoZWNrLXRlbXBlc3QtZHN2bS1uZXV0cm9uLWhlYXQtc2xvdyIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDEwOTg3NDQ4OH0= > > It's seeing a 78% pass rate in check. Can anyone in the Heat team > confirm that the Failures in this job are actually real failures on > patches that should have been blocked? > > I'd like to get that turned on (and on all the projects) as soon as the > Heat team is confident on it so that Heat actually participates in the > tempest/devstack gate in a material way and we can prevent future issues > where a keystone, nova, neutron or whatever change would break Heat in git. Steve Baker can make the final decision, but I am +2 on turning it on - during the process of getting my instance-users patches through the gate, this test consistently found real issues with no false positives (other than those caused by other known bugs). Steve _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev