On Thu, Oct 10, 2013 at 7:28 PM, Matt Riedemann <[email protected]> wrote:
> > > > > Dan Smith <[email protected]> wrote on 10/10/2013 08:26:14 PM: > > > From: Dan Smith <[email protected]> > > To: OpenStack Development Mailing List < > [email protected]>, > > Date: 10/10/2013 08:31 PM > > Subject: Re: [openstack-dev] [nova][powervm] my notes from the > > meeting on powervm CI > > > > > 4. What is the max amount of time for us to report test results? Dan > > > didn't seem to think 48 hours would fly. :) > > > > Honestly, I think that 12 hours during peak times is the upper limit of > > what could be considered useful. If it's longer than that, many patches > > could go into the tree without a vote, which defeats the point. > > Yeah, I was just joking about the 48 hour thing, 12 hours seems excessive > but I guess that has happened when things are super backed up with gate > issues and rechecks. > > Right now things take about 4 hours, with Tempest being around 1.5 hours > of that. The rest of the time is setup and install, which includes heat > and ceilometer. So I guess that raises another question, if we're really > setting this up right now because of nova, do we need to have heat and > ceilometer installed and configured in the initial delivery of this if > we're not going to run tempest tests against them (we don't right now)? > In general the faster the better, and if things get to slow enough that we have to wait for powervm CI to report back, I think its reasonable to go ahead and approve things without hearing back. In reality if you can report back in under 12 hours this will rarely happen (I think). > > I think some aspect of the slow setup time is related to DB2 and how > the migrations perform with some of that, but the overall time is not > considerably different from when we were running this with MySQL so > I'm reluctant to blame it all on DB2. I think some of our topology > could have something to do with it too since the IVM hypervisor is running > on a separate system and we are gated on how it's performing at any > given time. I think that will be our biggest challenge for the scale > issues with community CI. > > > > > > 5. What are the minimum tests that need to run (excluding APIs that the > > > powervm driver doesn't currently support)? > > > - smoke/gate/negative/whitebox/scenario/cli? Right now we have > > > 1152 tempest tests running, those are only within api/scenario/cli and > > > we don't run everything. > > > > I think that "a full run of tempest" should be required. That said, if > > there are things that the driver legitimately doesn't support, it makes > > sense to exclude those from the tempest run, otherwise it's not useful. > ++ > > > > I think you should publish the tempest config (or config script, or > > patch, or whatever) that you're using so that we can see what it means > > in terms of the coverage you're providing. > > Just to clarify, do you mean publish what we are using now or publish > once it's all working? I can certainly attach our nose.cfg and > latest x-unit results xml file. > We should publish all logs, similar to what we do for upstream ( http://logs.openstack.org/96/48196/8/gate/gate-tempest-devstack-vm-full/70ae562/ ). > > > > > > 6. Network service? We're running with openvswitch 1.10 today so we > > > probably want to continue with that if possible. > > > > Hmm, so that means neutron? AFAIK, not much of tempest runs with > > Nova/Neutron. > > > > I kinda think that since nova-network is our default right now (for > > better or worse) that the run should include that mode, especially if > > using neutron excludes a large portion of the tests. > > > > I think you said you're actually running a bunch of tempest right now, > > which conflicts with my understanding of neutron workiness. Can you > clarify? > > Correct, we're running with neutron using the ovs plugin. We basically have > the same issues that the neutron gate jobs have, which is related to > concurrency > issues and tenant isolation (we're doing the same as devstack with neutron > in that we don't run tempest with tenant isolation). We are running most > of the nova and most of the neutron API tests though (we don't have all > of the neutron-dependent scenario tests working though, probably more due > to incompetence in setting up neutron than anything else). > > > > > > 7. Cinder backend? We're running with the storwize driver but we do we > > > do about the remote v7000? > > > > Is there any reason not to just run with a local LVM setup like we do in > > the real gate? I mean, additional coverage for the v7000 driver is > > great, but if it breaks and causes you to not have any coverage at all, > > that seems, like, bad to me :) > > Yeah, I think we'd just run with a local LVM setup, that's what we do for > x86_64 and s390x tempest runs. For whatever reason we thought we'd do > storwize for our ppc64 runs, probably just to have a matrix of coverage. > > > > > > Again, just getting some thoughts out there to help us figure out our > > > goals for this, especially around 4 and 5. > > > > Yeah, thanks for starting this discussion! > > > > --Dan > > > > _______________________________________________ > > OpenStack-dev mailing list > > [email protected] > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
