Re: [Openstack] running Tempest continuously in the openstack project?
On 02/22/2012 07:17 PM, Jay Pipes wrote: On 02/22/2012 10:49 AM, James E. Blair wrote: Indeed, as soon as someone says I have Tempest working on a system configured by devstack with a repeatable process, here's how I did it... we'll start running it in Jenkins. But so far I've heard from the Tempest developers that it's not quite ready yet. That would be correct. Still work needed to get things working more consistently. How can I help with this effort? Who should I contact? Right now I'm having trouble setting up running the tests against devstack (a configuration problem on my part I suppose, but I blame it on the lack of documentation). -Ionuț ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
On 02/22/2012 07:17 PM, Jay Pipes wrote: On 02/22/2012 10:49 AM, James E. Blair wrote: Indeed, as soon as someone says I have Tempest working on a system configured by devstack with a repeatable process, here's how I did it... we'll start running it in Jenkins. But so far I've heard from the Tempest developers that it's not quite ready yet. That would be correct. Still work needed to get things working more consistently. How can I help with this effort? Who should I contact? Right now I'm having trouble setting up running the tests against devstack (a configuration problem on my part I suppose, but I blame it on the lack of documentation). -Ionuț ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
There is still a bug in tempest and/or keystone. To run Tempest and devstack you have to: 1. Add catalog_name=compute to tempest.conf 2. Change name to type in rest_client.py -David On 2/27/2012 8:18 AM, Ionuț Arțăriși wrote: On 02/22/2012 07:17 PM, Jay Pipes wrote: On 02/22/2012 10:49 AM, James E. Blair wrote: Indeed, as soon as someone says I have Tempest working on a system configured by devstack with a repeatable process, here's how I did it... we'll start running it in Jenkins. But so far I've heard from the Tempest developers that it's not quite ready yet. That would be correct. Still work needed to get things working more consistently. How can I help with this effort? Who should I contact? Right now I'm having trouble setting up running the tests against devstack (a configuration problem on my part I suppose, but I blame it on the lack of documentation). -Ionuț ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
I'm actively looking into any issues. I have all these tests passing locally in my environment, so the issues seem to be focused around people using devstack. I've made some merge prop's that will take care of a few of the issues. One issue that will certainly come up is that if you have rate limiting enabled, most of the tests will certainly fail. I've also included the link to the Tempest documentation, which also should help make using a bit more clear. I'm working on updating this document as I'm using Devstack so that I can either smooth over or enumerate issues that may come up when running the tests. If you run into anything though, please make a report to the Tempest project and I'll have a look. http://qa.openstack.org/integration.html https://launchpad.net/tempest Daryl On Feb 27, 2012, at 7:18 AM, Ionuț Arțăriși wrote: On 02/22/2012 07:17 PM, Jay Pipes wrote: On 02/22/2012 10:49 AM, James E. Blair wrote: Indeed, as soon as someone says I have Tempest working on a system configured by devstack with a repeatable process, here's how I did it... we'll start running it in Jenkins. But so far I've heard from the Tempest developers that it's not quite ready yet. That would be correct. Still work needed to get things working more consistently. How can I help with this effort? Who should I contact? Right now I'm having trouble setting up running the tests against devstack (a configuration problem on my part I suppose, but I blame it on the lack of documentation). -Ionuț ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
On 02/27/2012 05:13 PM, Daryl Walleck wrote: I'm actively looking into any issues. I have all these tests passing locally in my environment, so the issues seem to be focused around people using devstack. I've made some merge prop's that will take care of a few of the issues. One issue that will certainly come up is that if you have rate limiting enabled, most of the tests will certainly fail. I've also included the link to the Tempest documentation, which also should help make using a bit more clear. I'm working on updating this document as I'm using Devstack so that I can either smooth over or enumerate issues that may come up when running the tests. If you run into anything though, please make a report to the Tempest project and I'll have a look. http://qa.openstack.org/integration.html https://launchpad.net/tempest Daryl Can you give more details about your environment? What operating system are you running the tests on? Are all the tested openstack components the latest versions from master? What specific configurations have you done? The documentation that you linked to does not explain how to actually set up the tests. There are currently three README files in the repository which say to rename the tempest.conf.sample and config.ini.sample files and then edit the variables to fit your test environment, but none of them explain what that actually means. Which values should be reset? Where would I get the information to be able to set them? I'm trying hard to understand how all of this works, but without a canonical working configuration I don't know which errors are mine and which are tempest bugs. -Ionuț ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
1. Add catalog_name=compute to tempest.conf 2. Change name to type in rest_client.py Yep, easiest to just apply this patch: git fetch https://review.openstack.org/p/openstack/tempest refs/changes/59/4259/1 git format-patch -1 --stdout FETCH_HEAD Cheers, Eoghan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
This isn't really a bug in either project, but a configuration issue. The name of the endpoint isn't static, so making it 'nova' or 'compute' may not always be correct. If you check the name of your Compute project that's returned in the Keystone auth request, you can find the name and configure Tempest by setting the catalog property with that. To avoid having to even configure that, I have a merge prop in queue to look at the type (which for Nova as far as I understand should always be 'compute') to have one less configuration for people to deal with. Daryl On Feb 27, 2012, at 9:49 AM, David Kranz wrote: There is still a bug in tempest and/or keystone. To run Tempest and devstack you have to: 1. Add catalog_name=compute to tempest.conf 2. Change name to type in rest_client.py -David On 2/27/2012 8:18 AM, Ionuț Arțăriși wrote: On 02/22/2012 07:17 PM, Jay Pipes wrote: On 02/22/2012 10:49 AM, James E. Blair wrote: Indeed, as soon as someone says I have Tempest working on a system configured by devstack with a repeatable process, here's how I did it... we'll start running it in Jenkins. But so far I've heard from the Tempest developers that it's not quite ready yet. That would be correct. Still work needed to get things working more consistently. How can I help with this effort? Who should I contact? Right now I'm having trouble setting up running the tests against devstack (a configuration problem on my part I suppose, but I blame it on the lack of documentation). -Ionuț ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
On 2/27/2012 3:02 PM, Dean Troyer wrote: On Mon, Feb 27, 2012 at 10:13 AM, Daryl Walleck daryl.wall...@rackspace.com wrote: I'm working on updating this document as I'm using Devstack so that I can either smooth over or enumerate issues that may come up when running the tests. If you run into anything though, please make a report to the Tempest project and I'll have a look. Daryl, devstack has a script (tools/configure_tempest.sh) that is meant to transfer the devstack settings to tempest.conf. It edits tempest.conf.sample so it should just need to know the attribute names for the settings. Let me know if there is more that devstack needs to do to set up tempest. dt It would be helpful to have a NORATELIMIT variable that could be set in localrc. -David ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
There is currently a bug https://bugs.launchpad.net/tempest/+bug/933845 that will prevent tempest from working with keystone. That ticket provides a workaround but still not all of the tests are working at the moment. I think the goal is for tempest to become actively run but we are not there yet... -David On 2/22/2012 9:12 AM, Ionuț Arțăriși wrote: Hello, I'm trying to get started with Tempest and I would really like to be able to see a running setup somewhere. I'm getting a lot of tests failing and I don't know if it's because of my setup or because of bugs in Tempest or incompatibilities with the current code in the other components. I looked at Jenkins, but it seems there are only two tasks related to Tempest there, testing for git-merge and pep8. Does the OpenStack project actively run the full Tempest test suite anywhere? It would be great if anyone can offer a working configuration or at least say if they have Tempest running successfully against which versions of nova, glance, swift etc. Thanks, Ionuț ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
We're running Tempest as part of our continuous build. For the bug you're referring to David, there is a configuration in Tempest where the name of the Nova/Compute project can be set, so I don't think its fair to say that Tempest is currently broken. However, I am working on a more comprehensive set of documentation and auto-configuration tools to allow people to work with Tempest more easily. Expect more very soon coming down the pipeline. Daryl From: openstack-bounces+daryl.walleck=rackspace@lists.launchpad.net [openstack-bounces+daryl.walleck=rackspace@lists.launchpad.net] on behalf of David Kranz [david.kr...@qrclab.com] Sent: Wednesday, February 22, 2012 8:24 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] running Tempest continuously in the openstack project? There is currently a bug https://bugs.launchpad.net/tempest/+bug/933845 that will prevent tempest from working with keystone. That ticket provides a workaround but still not all of the tests are working at the moment. I think the goal is for tempest to become actively run but we are not there yet... -David On 2/22/2012 9:12 AM, Ionuț Arțăriși wrote: Hello, I'm trying to get started with Tempest and I would really like to be able to see a running setup somewhere. I'm getting a lot of tests failing and I don't know if it's because of my setup or because of bugs in Tempest or incompatibilities with the current code in the other components. I looked at Jenkins, but it seems there are only two tasks related to Tempest there, testing for git-merge and pep8. Does the OpenStack project actively run the full Tempest test suite anywhere? It would be great if anyone can offer a working configuration or at least say if they have Tempest running successfully against which versions of nova, glance, swift etc. Thanks, Ionuț ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
Indeed, as soon as someone says I have Tempest working on a system configured by devstack with a repeatable process, here's how I did it... we'll start running it in Jenkins. But so far I've heard from the Tempest developers that it's not quite ready yet. -Jim David Kranz david.kr...@qrclab.com writes: There is currently a bug https://bugs.launchpad.net/tempest/+bug/933845 that will prevent tempest from working with keystone. That ticket provides a workaround but still not all of the tests are working at the moment. I think the goal is for tempest to become actively run but we are not there yet... -David On 2/22/2012 9:12 AM, Ionuț Arțăriși wrote: Hello, I'm trying to get started with Tempest and I would really like to be able to see a running setup somewhere. I'm getting a lot of tests failing and I don't know if it's because of my setup or because of bugs in Tempest or incompatibilities with the current code in the other components. I looked at Jenkins, but it seems there are only two tasks related to Tempest there, testing for git-merge and pep8. Does the OpenStack project actively run the full Tempest test suite anywhere? It would be great if anyone can offer a working configuration or at least say if they have Tempest running successfully against which versions of nova, glance, swift etc. Thanks, Ionuț ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] running Tempest continuously in the openstack project?
On 02/22/2012 10:49 AM, James E. Blair wrote: Indeed, as soon as someone says I have Tempest working on a system configured by devstack with a repeatable process, here's how I did it... we'll start running it in Jenkins. But so far I've heard from the Tempest developers that it's not quite ready yet. That would be correct. Still work needed to get things working more consistently. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
