Dan Smith <d...@danplanet.com> wrote on 10/10/2013 08:26:14 PM: > From: Dan Smith <d...@danplanet.com> > To: OpenStack Development Mailing List <openstack-dev@lists.openstack.org>, > Date: 10/10/2013 08:31 PM > Subject: Re: [openstack-dev] [nova][powervm] my notes from the > meeting on powervm CI > > > 4. What is the max amount of time for us to report test results? Dan > > didn't seem to think 48 hours would fly. :) > > Honestly, I think that 12 hours during peak times is the upper limit of > what could be considered useful. If it's longer than that, many patches > could go into the tree without a vote, which defeats the point.
Yeah, I was just joking about the 48 hour thing, 12 hours seems excessive but I guess that has happened when things are super backed up with gate issues and rechecks. Right now things take about 4 hours, with Tempest being around 1.5 hours of that. The rest of the time is setup and install, which includes heat and ceilometer. So I guess that raises another question, if we're really setting this up right now because of nova, do we need to have heat and ceilometer installed and configured in the initial delivery of this if we're not going to run tempest tests against them (we don't right now)? I think some aspect of the slow setup time is related to DB2 and how the migrations perform with some of that, but the overall time is not considerably different from when we were running this with MySQL so I'm reluctant to blame it all on DB2. I think some of our topology could have something to do with it too since the IVM hypervisor is running on a separate system and we are gated on how it's performing at any given time. I think that will be our biggest challenge for the scale issues with community CI. > > > 5. What are the minimum tests that need to run (excluding APIs that the > > powervm driver doesn't currently support)? > > - smoke/gate/negative/whitebox/scenario/cli? Right now we have > > 1152 tempest tests running, those are only within api/scenario/cli and > > we don't run everything. > > I think that "a full run of tempest" should be required. That said, if > there are things that the driver legitimately doesn't support, it makes > sense to exclude those from the tempest run, otherwise it's not useful. > > I think you should publish the tempest config (or config script, or > patch, or whatever) that you're using so that we can see what it means > in terms of the coverage you're providing. Just to clarify, do you mean publish what we are using now or publish once it's all working? I can certainly attach our nose.cfg and latest x-unit results xml file. > > > 6. Network service? We're running with openvswitch 1.10 today so we > > probably want to continue with that if possible. > > Hmm, so that means neutron? AFAIK, not much of tempest runs with > Nova/Neutron. > > I kinda think that since nova-network is our default right now (for > better or worse) that the run should include that mode, especially if > using neutron excludes a large portion of the tests. > > I think you said you're actually running a bunch of tempest right now, > which conflicts with my understanding of neutron workiness. Can you clarify? Correct, we're running with neutron using the ovs plugin. We basically have the same issues that the neutron gate jobs have, which is related to concurrency issues and tenant isolation (we're doing the same as devstack with neutron in that we don't run tempest with tenant isolation). We are running most of the nova and most of the neutron API tests though (we don't have all of the neutron-dependent scenario tests working though, probably more due to incompetence in setting up neutron than anything else). > > > 7. Cinder backend? We're running with the storwize driver but we do we > > do about the remote v7000? > > Is there any reason not to just run with a local LVM setup like we do in > the real gate? I mean, additional coverage for the v7000 driver is > great, but if it breaks and causes you to not have any coverage at all, > that seems, like, bad to me :) Yeah, I think we'd just run with a local LVM setup, that's what we do for x86_64 and s390x tempest runs. For whatever reason we thought we'd do storwize for our ppc64 runs, probably just to have a matrix of coverage. > > > Again, just getting some thoughts out there to help us figure out our > > goals for this, especially around 4 and 5. > > Yeah, thanks for starting this discussion! > > --Dan > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev