Re: [openstack-dev] [qa] Issue with tests of host admin api
nice idea, Chris! +1 for me. and very thankful for dkranz to bring this to maillist here. 2013/10/25 Christopher Yeoh > > > On 26/10/2013, at 12:01 AM, David Kranz wrote: > > > On 10/25/2013 09:10 AM, Sean Dague wrote: > >> On 10/25/2013 08:39 AM, David Kranz wrote: > >>> A patch was submitted with some new tests of this api > >>> https://review.openstack.org/#/c/49778/. I gave a -1 because if a > >>> negative test to shutdown a host fails, a compute node will be > shutdown. > >>> The author thinks this test should be part of tempest. My issue was > that > >>> we should not have tempest tests for apis that: > >>> > >>> 1. May corrupt the underlying system (that is part of the reason we > >>> moved whitebox out) > >> > >> I really felt the reason we moved out whitebox is that OpenStack > internals move way to fast to have them being validated by an external > system. We have defined surfaces (i.e. API) and that should be the focus. > > It was also because we were side-effecting the database out-of-band. > >> > >>> 2. Can have only negative tests because positive ones could prevent > >>> other tests from executing > >> > >> Honestly, trying to shut down the host with invalid credentials seems > like a fair test. Because if we fail, we're going to know really quick when > everything blackholes. > >> > >> In the gate this is slightly funnier because tempest is running on the > same place as the host, however it seems like a sane check to have in there. > >> > >>-Sean > > OK, I don't feel strongly about it. Just seemed like a potential > landmine. > > > > I think this is something we want to test in the gate. But perhaps there > could be a tag for these sorts of test cases that some people may not want > to risk running on their system so they can exclude them easily? > > Chris > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- *---* *Lingxian Kong* Huawei Technologies Co.,LTD. IT Product Line CloudOS PDU China, Xi'an Mobile: +86-18602962792 Email: konglingx...@huawei.com; anlin.k...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Issue with tests of host admin api
On 26/10/2013, at 12:01 AM, David Kranz wrote: > On 10/25/2013 09:10 AM, Sean Dague wrote: >> On 10/25/2013 08:39 AM, David Kranz wrote: >>> A patch was submitted with some new tests of this api >>> https://review.openstack.org/#/c/49778/. I gave a -1 because if a >>> negative test to shutdown a host fails, a compute node will be shutdown. >>> The author thinks this test should be part of tempest. My issue was that >>> we should not have tempest tests for apis that: >>> >>> 1. May corrupt the underlying system (that is part of the reason we >>> moved whitebox out) >> >> I really felt the reason we moved out whitebox is that OpenStack internals >> move way to fast to have them being validated by an external system. We have >> defined surfaces (i.e. API) and that should be the focus. > It was also because we were side-effecting the database out-of-band. >> >>> 2. Can have only negative tests because positive ones could prevent >>> other tests from executing >> >> Honestly, trying to shut down the host with invalid credentials seems like a >> fair test. Because if we fail, we're going to know really quick when >> everything blackholes. >> >> In the gate this is slightly funnier because tempest is running on the same >> place as the host, however it seems like a sane check to have in there. >> >>-Sean > OK, I don't feel strongly about it. Just seemed like a potential landmine. > I think this is something we want to test in the gate. But perhaps there could be a tag for these sorts of test cases that some people may not want to risk running on their system so they can exclude them easily? Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Issue with tests of host admin api
On 10/25/2013 09:10 AM, Sean Dague wrote: On 10/25/2013 08:39 AM, David Kranz wrote: A patch was submitted with some new tests of this api https://review.openstack.org/#/c/49778/. I gave a -1 because if a negative test to shutdown a host fails, a compute node will be shutdown. The author thinks this test should be part of tempest. My issue was that we should not have tempest tests for apis that: 1. May corrupt the underlying system (that is part of the reason we moved whitebox out) I really felt the reason we moved out whitebox is that OpenStack internals move way to fast to have them being validated by an external system. We have defined surfaces (i.e. API) and that should be the focus. It was also because we were side-effecting the database out-of-band. 2. Can have only negative tests because positive ones could prevent other tests from executing Honestly, trying to shut down the host with invalid credentials seems like a fair test. Because if we fail, we're going to know really quick when everything blackholes. In the gate this is slightly funnier because tempest is running on the same place as the host, however it seems like a sane check to have in there. -Sean OK, I don't feel strongly about it. Just seemed like a potential landmine. -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Issue with tests of host admin api
On 10/25/2013 08:39 AM, David Kranz wrote: A patch was submitted with some new tests of this api https://review.openstack.org/#/c/49778/. I gave a -1 because if a negative test to shutdown a host fails, a compute node will be shutdown. The author thinks this test should be part of tempest. My issue was that we should not have tempest tests for apis that: 1. May corrupt the underlying system (that is part of the reason we moved whitebox out) I really felt the reason we moved out whitebox is that OpenStack internals move way to fast to have them being validated by an external system. We have defined surfaces (i.e. API) and that should be the focus. 2. Can have only negative tests because positive ones could prevent other tests from executing Honestly, trying to shut down the host with invalid credentials seems like a fair test. Because if we fail, we're going to know really quick when everything blackholes. In the gate this is slightly funnier because tempest is running on the same place as the host, however it seems like a sane check to have in there. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [qa] Issue with tests of host admin api
A patch was submitted with some new tests of this api https://review.openstack.org/#/c/49778/. I gave a -1 because if a negative test to shutdown a host fails, a compute node will be shutdown. The author thinks this test should be part of tempest. My issue was that we should not have tempest tests for apis that: 1. May corrupt the underlying system (that is part of the reason we moved whitebox out) 2. Can have only negative tests because positive ones could prevent other tests from executing Thoughts? -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
