On 12/10/2013 08:41 PM, Devananda van der Veen wrote:
Tue, Dec 10, 2013 at 12:43 PM, David Kranz <dkr...@redhat.com
<mailto:dkr...@redhat.com>> wrote:
On 12/09/2013 01:37 PM, Devananda van der Veen wrote:
On Fri, Dec 6, 2013 at 2:13 PM, Clark Boylan
<clark.boy...@gmail.com <mailto:clark.boy...@gmail.com>> wrote:
On Fri, Dec 6, 2013 at 1:53 PM, David Kranz
<dkr...@redhat.com <mailto:dkr...@redhat.com>> wrote:
> It's great that tempest tests for ironic have been
submitted! I was
> reviewing https://review.openstack.org/#/c/48109/ and
noticed that the tests
> do not actually run. They are skipped because baremetal is
not enabled. This
> is not terribly surprising but we have had a policy in
tempest to only merge
> code that has demonstrated that it works. For services that
cannot run in
> the single-vm environment of the upstream gate we said
there could be a
> system running somewhere that would run them and report a
result to gerrit.
> Is there a plan for this, or to make an exception for ironic?
>
> -David
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
There is a change[0] to openstack-infra/config to add
experimental
tempest jobs to test ironic. I think that change is close to
being
ready, but I need to give it time for a proper review. Once
in that
will allow you to test 48109 (in theory, not sure if all the
bits will
just work). I don't think these tests fall under the cannot
run in a
single vm environment umbrella, we should be able to test the
baremetal code via the pxe booting of VMs within the single VM
environment.
[0] https://review.openstack.org/#/c/53917/
Clark
We can test the ironic services, database, and the driver
interfaces by using our "fake" driver within a single devstack VM
today (I'm not sure the exercises for all of this have been
written yet, but it's practical to test it). OTOH, I don't
believe we can test a PXE deploy within a single VM today, and
need to resume discussions with infra about this.
There are some other aspects of Ironic (IPMI, SOL access, any
vendor-specific drivers) which we'll need real hardware to test
because they can't effectively be virtualized. TripleO should
cover some (much?) of those needs, once they are able to switch
to using Ironic instead of nova-baremetal.
-Devananda
So it seems that the code in the submitted tempest tests can run
in a regular job if devstack is configured to enable ironic, but
that this cannot be the default. So I propose that we create a
regular devstack+ironic job that will run in the ironic and
tempest gates, and run just the ironic tests. When third-party
bare-metal results can be reported for ironic, tempest can then
accept tests that require bare-metal. Does any one have a problem
with this approach?
-David
As I understand it, the infra/config patch which Clark already linked
(https://review.openstack.org/#/c/53917), which has gone through
several iterations, should be enabling Ironic within devstack -- and
thus causing tempest to run the relevant tests -- within the Ironic
and Tempest check and gate pipelines. This will exercise Ironic's API
by performing CRUD actions on resources. It doesn't do any more than
that yet.
It looks like that patch is adding ironic jobs to the experimental queue
but I think we want them on check/gate.
David, I'm not sure what you mean by "when third-party bare-metal
results can be reported for ironic" -- I don't see any reason why we
couldn't accept third-party smoke tests right now, except that none of
the tempest tests are written... Am I missing something?
I was assuming there were some ironic tests that actually need bare
metal resources to run. Perhaps there are not. Either way, we just want
to make sure that when tests are submitted to tempest we have evidence
that they have successfully run. Sounds like the CRUD tests will just
work the same way as our existing tests once ironic is enabled in devstack.
In the longer term, we are planning to enable tempest testing of
deployment by ironic within devstack-gate as all the pieces come
together. This will take a fair bit more work / time, but I'm going to
start nudging resources in this direction very soon. In fact, we just
talked about this in #infra for a bit. Here's an attempt to summarize
what came of it w.r.t. Ironic's testing plans. We will need:
- some changes in devstack-gate to prepare a new environment by...
-- install sshd + firewall it to only allow connections from localhost
-- create a bunch of tiny qemu VMs (of configurable size and number)
- some changes in devstack to...
-- suck up a list of those VM's MAC addresses and enroll them in Ironic
-- configure nova to use ironic
-- configure ironic to use the pxe+ssh driver
- a new test job that turns all this on, thus allowing tempest to do
all its usual work against a "virtual baremetal" cloud
Also, it's worth mentioning, the above-described plan won't exercise
the CRUD of Ironic resources -- I think they need to be pre-enrolled
with Ironic before tempest starts (maybe not? maybe tempest can enroll
them, instead of devstack?). This is one reason why we have proposed
separate tempest tests for exercising Ironic's API. Another reason is,
testing our API is a valuable thing all by itself, and has a much
lower development cost than all the changes above.
-Devananda
Looking forward to it!
-David
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
