Re: [openstack-dev] unable to run unit tests
fwiw, clarkb advised me to to update testrepository with the following - bzr clone lp:testrepository to get a more meaningful error log output. i have version testrepository=0.0.18 after i updated. thanks! daya From: Alex Leonhardt aleonhardt...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Sunday, August 24, 2014 2:39 AM Subject: Re: [openstack-dev] unable to run unit tests Thanks! So the real error is : UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 36: ordinal not in range(128) Any clues ? :\ Alex On 23 August 2014 21:46, Doug Wiegley do...@a10networks.com wrote: That’s testr’s friendly way of telling you that you have an import error. Run it with the py26 environment and you’ll get a useful error. No, don’t ask me why. Doug On 8/23/14, 2:33 PM, Alex Leonhardt aleonhardt...@gmail.com wrote: Hi All, after some fighting with installing all the bits - I can finally execute run_tests.sh but I still cannot seem to get it to work properly. This is what is returning : Non-zero exit code (2) from test listing. stdout='\xb3)\x01@B8nova.tests.test_matchers.TestDictListMatches.test__str __\xde|\xe2\x9e\xb3)\x01@P@Enova.tests.test_matchers.TestDictListMatches.t est_describe_difference\xfeL\xc1\xc4\xb3)\x01@I?nova.tests.test_matchers.T estDictListMatches.test_matches_match\x93\xfau\xb5\xb3)\x01@M@Bnova.tests. test_matchers.TestDictListMatches.test_mismatch_details\x18`_\xb3)\x01=4n ova.tests.test_matchers.TestDictMatches.test__str__\xca\x9bK\xb9\xb3)\x01@ L@Anova.tests.test_matchers.TestDictMatches.test_describe_difference\x94\x b6\x89\\\xb3)\x01@E;nova.tests.test_matchers.TestDictMatches.test_matches_ match^V\xc2v\xb3)\x01@Hnova.tests.test_matchers.TestDictMatches.test_mism atch_details\x1bV\x9e\xff\xb3)\x01=4nova.tests.test_matchers.TestIsSubDict Of.test__str__\xb8\x06h(\xb3)\x01@L@Anova.tests.test_matchers.TestIsSubDic tOf.test_describe_difference\xd2\xdd3\xea\xb3)\x01@E;nova.tests.test_match ers.TestIsSubDictOf.test_matches_match~qC\xd0\xb3)\x01@Hnova.tests.test_m atchers.TestIsSubDictOf.test_mismatch_detailsA1\xa3l\xb3)\x013nova.tests. test_matchers.TestXMLMatches.test__str__C\xd1E\x13\xb3)\x01@K@@nova.tests. test_matchers.TestXMLMatches.test_describe_difference\xe0\xa0\xb2\x80\xb3) \x01@D:nova.tests.test_matchers.TestXMLMatches.test_matches_matchDT\xe9h\x b3)\x01@G=nova.tests.test_matchers.TestXMLMatches.test_mismatch_detailse\x 16\xb1\xad\xb3 `\x80N\xdb\x17text/plain;charset=utf8\rimport errors\x80N\xa8nova.tests.api.ec2.test_api\nnova.tests.api.ec2.test_cinder _cloud\nnova.tests.api.ec2.test_cloud\nnova.tests.api.ec2.test_ec2_validat e\nnova.tests.api.ec2.test_ec2utils\nnova.tests.api.ec2.test_error_respons e\nnova.tests.api.ec2.test_faults\nnova.tests.api.ec2.test_middleware\nnov a.tests.api.openstack.compute.contrib.test_admin_actions\nnova.tests.api.o penstack.compute.contrib.test_agents\nnova.tests.api.openstack.compute.con trib.test_aggregates\nnova.tests.api.openstack.compute.contrib.test_attach _interfaces\nnova.tests.api.openstack.compute.contrib.test_availability_zo ne\nnova.tests.api.openstack.compute.contrib.test_baremetal_nodes\nnova.te sts.api.openstack.compute.contrib.test_cells\nnova.tests.api.openstack.com pute.contrib.test_certificates\nnova.tests.api.openstack.compute.contrib.t est_cloudpipe\nnova.tests.api.openstack.compute.contrib.test_cloudpipe_upd ate\nnova.tests.api.openstack.compute.contrib.test_config_drive\nnova.test s.api.openstack.compute.contrib.test_console_auth_tokens\nnova.tests.api.o penstack.compute.contrib.test_console_output\nnova.tests.api.openstack.com pute.contrib.test_consoles\nnova.tests.api.openstack.compute.contrib.test_ createserverext\nnova.tests.api.openstack.compute.contrib.test_deferred_de lete\nnova.tests.api.openstack.compute.contrib.test_disk_config\nnova.test s.api.openstack.compute.contrib.test_evacuate\nnova.tests.api.openstack.co mpute.contrib.test_extended_availability_zone\nnova.tests.api.openstack.co mpute.contrib.test_extended_evacuate_find_host\nnova.tests.api.openstack.c ompute.contrib.test_extended_hypervisors\nnova.tests.api.openstack.compute .contrib.test_extended_ips\nnova.tests.api.openstack.compute.contrib.test_ extended_ips_mac\nnova.tests.api.openstack.compute.contrib.test_extended_r escue_with_image\nnova.tests.api.openstack.compute.contrib.test_extended_s erver_attributes\nnova.tests.api.openstack.compute.contrib.test_extended_s tatus\nnova.tests.api.openstack.compute.contrib.test_extended_virtual_inte rfaces_net\nnova.tests.api.openstack.compute.contrib.test_extended_volumes \nnova.tests.api.openstack.compute.contrib.test_fixed_ips\nnova.tests.api. openstack.compute.contrib.test_flavor_access\nnova.tests.api.openstack.com pute.contrib.test_flavor_disabled\nnova.tests.api.openstack.compute.contri b.test_flavor_manage\nnova.tests.api.openstack.compute.contrib.test_flavor
[openstack-dev] [heat] heat.conf.sample is not up to date
What is going on with this? If I do a fresh clone of heat and run `tox -epep8` then I get that complaint. If I then run the recommended command to fix it, and then `tox -epep8` again, I get the same complaint again --- and with different differences exhibited! The email below carries a typescript showing this. What I really need to know is what to do when committing a change that really does require a change in the sample configuration file. Of course I tried running generate_sample.sh, but `tox -epep8` still complains. What is the right procedure to get a correct sample committed? BTW, I am doing the following admittedly risky thing: I run DevStack, and make my changes in /opt/stack/heat/. Thanks, Mike - Forwarded by Mike Spreitzer/Watson/IBM on 08/24/2014 03:03 AM - From: ubuntu@mjs-dstk-821a (Ubuntu) To: Mike Spreitzer/Watson/IBM@IBMUS, Date: 08/24/2014 02:55 AM Subject:fresh flake fail ubuntu@mjs-dstk-821a:~/code$ git clone git://git.openstack.org/openstack/heat.git Cloning into 'heat'... remote: Counting objects: 49690, done. remote: Compressing objects: 100% (19765/19765), done. remote: Total 49690 (delta 36660), reused 39014 (delta 26526) Receiving objects: 100% (49690/49690), 7.92 MiB | 7.29 MiB/s, done. Resolving deltas: 100% (36660/36660), done. Checking connectivity... done. ubuntu@mjs-dstk-821a:~/code$ cd heat ubuntu@mjs-dstk-821a:~/code/heat$ tox -epep8 pep8 create: /home/ubuntu/code/heat/.tox/pep8 pep8 installdeps: -r/home/ubuntu/code/heat/requirements.txt, -r/home/ubuntu/code/heat/test-requirements.txt pep8 develop-inst: /home/ubuntu/code/heat pep8 runtests: PYTHONHASHSEED='0' pep8 runtests: commands[0] | flake8 heat bin/heat-api bin/heat-api-cfn bin/heat-api-cloudwatch bin/heat-engine bin/heat-manage contrib pep8 runtests: commands[1] | /home/ubuntu/code/heat/tools/config/check_uptodate.sh --- /tmp/heat.ep2CBe/heat.conf.sample2014-08-24 06:52:54.16484 + +++ etc/heat/heat.conf.sample2014-08-24 06:48:13.66484 + @@ -164,7 +164,7 @@ #allowed_rpc_exception_modules=oslo.messaging.exceptions,nova.exception,cinder.exception,exceptions # Qpid broker hostname. (string value) -#qpid_hostname=heat +#qpid_hostname=localhost # Qpid broker port. (integer value) #qpid_port=5672 @@ -221,7 +221,7 @@ # The RabbitMQ broker address where a single node is used. # (string value) -#rabbit_host=heat +#rabbit_host=localhost # The RabbitMQ broker port where a single node is used. # (integer value) check_uptodate.sh: heat.conf.sample is not up to date. check_uptodate.sh: Please run /home/ubuntu/code/heat/tools/config/generate_sample.sh. ERROR: InvocationError: '/home/ubuntu/code/heat/tools/config/check_uptodate.sh' pep8 runtests: commands[2] | /home/ubuntu/code/heat/tools/requirements_style_check.sh requirements.txt test-requirements.txt pep8 runtests: commands[3] | bash -c find heat -type f -regex '.*\.pot?' -print0|xargs -0 -n 1 msgfmt --check-format -o /dev/null ___ summary ERROR: pep8: commands failed ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ tools/config/generate_sample.sh ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ tox -epep8 pep8 develop-inst-noop: /home/ubuntu/code/heat pep8 runtests: PYTHONHASHSEED='0' pep8 runtests: commands[0] | flake8 heat bin/heat-api bin/heat-api-cfn bin/heat-api-cloudwatch bin/heat-engine bin/heat-manage contrib pep8 runtests: commands[1] | /home/ubuntu/code/heat/tools/config/check_uptodate.sh --- /tmp/heat.DqIhK5/heat.conf.sample2014-08-24 06:54:34.62884 + +++ etc/heat/heat.conf.sample2014-08-24 06:53:51.54084 + @@ -159,10 +159,6 @@ # Size of RPC connection pool. (integer value) #rpc_conn_pool_size=30 -# Modules of exceptions that are permitted to be recreated -# upon receiving exception data from an rpc call. (list value) -#allowed_rpc_exception_modules=oslo.messaging.exceptions,nova.exception,cinder.exception,exceptions - # Qpid broker hostname. (string value) #qpid_hostname=heat @@ -301,15 +297,6 @@ # Heartbeat time-to-live. (integer value) #matchmaker_heartbeat_ttl=600 -# Host to locate redis. (string value) -#host=127.0.0.1 - -# Use this port to connect to redis host. (integer value) -#port=6379 - -# Password for Redis server (optional). (string value) -#password=None - # Size of RPC greenthread pool. (integer value) #rpc_thread_pool_size=64 @@ -1229,6 +1216,22 @@ #hash_algorithms=md5 +[matchmaker_redis] + +# +# Options defined in oslo.messaging +# + +# Host to locate redis. (string value) +#host=127.0.0.1 + +# Use this port to connect to redis host. (integer value) +#port=6379 + +# Password for Redis server (optional). (string value) +#password=None + + [matchmaker_ring] # check_uptodate.sh: heat.conf.sample is not up to
Re: [openstack-dev] [neutron][ml2] Openvswitch agent support for non promic mode adapters
Hi Andreas, Thank you for this initiative. We were looking on similar problem for mixing OVS and SR-IOV on same network adapter, which also requires mac addresses registration of OVS ports. Please let me know if you would like to collaborate on this effort. BR, Irena -Original Message- From: Andreas Scheuring [mailto:scheu...@linux.vnet.ibm.com] Sent: Friday, August 22, 2014 11:16 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [neutron][ml2] Openvswitch agent support for non promic mode adapters Thanks for your feedback. No, I do not yet have code for it. Just wanted to get a feeling if such a feature would get acceptance in the community. But if that helps I can sit down and start some prototyping while I'm preparing a blueprint spec in parallel. The main part of the implementation I wanted to do on my own to get more familiar with the code base and to get more in touch with the community. But of course advice and feedback of experienced neutron developers is essential! So I will proceed like this - Create a blueprint - Commit first pieces of code to get early feedback (e.g. ask via the mailing list or irc) - Upload a spec (as soon as the repo is available for K) Does that make sense for you? Thanks, Andreas On Thu, 2014-08-21 at 13:44 -0700, Kevin Benton wrote: I think this sounds reasonable. Do you have code for this already, or are you looking for a developer to help implement it? On Thu, Aug 21, 2014 at 8:45 AM, Andreas Scheuring scheu...@linux.vnet.ibm.com wrote: Hi, last week I started discussing an extension to the existing neutron openvswitch agent to support network adapters that are not in promiscuous mode. Now I would like to enhance the round to get feedback from a broader audience via the mailing list. The Problem When driving vlan or flat networking, openvswitch requires an network adapter in promiscuous mode. Why not having promiscuous mode in your adapter? - Admins like to have full control over their environment and which network packets enter the system. - The network adapter just does not have support for it. What to do? Linux net-dev driver offer an interface to manually register additional mac addresses (also called secondary unicast addresses). Exploiting this one can register additional mac addresses to the network adapter. This also works via a well known ip user space tool. `bridge fdb add aa:aa:aa:aa:aa:aa dev eth0` What to do in openstack? As neutron is aware of all the mac addresses that are in use it's the perfect candidate for doing the mac registrations. The idea is to modify the neutron openvswitch agent that it does the registration on port add and port remove via the bridge command. There would be a new optional configuration parameter, something like 'non-promisc-mode' that is by default set to false. Only when set to true, macs get manually registered. Otherwise the agent behaves like it does today. So I guess only very little changes to the agent code are required. From my current point of view we do not need any changes to the ml2 plug-in. Blueprint or a bug? I guess it's a blueprint. What's the timeframe? K would be great. I would be thankful for any feedback on this! Feel free to contact me anytime. Thanks in advance! Regards, Andreas (irc: scheuran) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] heat.conf.sample is not up to date
Guessing this is due to the new tox feature which randomizes python's hash seed. Excerpts from Mike Spreitzer's message of 2014-08-24 00:10:42 -0700: What is going on with this? If I do a fresh clone of heat and run `tox -epep8` then I get that complaint. If I then run the recommended command to fix it, and then `tox -epep8` again, I get the same complaint again --- and with different differences exhibited! The email below carries a typescript showing this. What I really need to know is what to do when committing a change that really does require a change in the sample configuration file. Of course I tried running generate_sample.sh, but `tox -epep8` still complains. What is the right procedure to get a correct sample committed? BTW, I am doing the following admittedly risky thing: I run DevStack, and make my changes in /opt/stack/heat/. Thanks, Mike - Forwarded by Mike Spreitzer/Watson/IBM on 08/24/2014 03:03 AM - From: ubuntu@mjs-dstk-821a (Ubuntu) To: Mike Spreitzer/Watson/IBM@IBMUS, Date: 08/24/2014 02:55 AM Subject:fresh flake fail ubuntu@mjs-dstk-821a:~/code$ git clone git://git.openstack.org/openstack/heat.git Cloning into 'heat'... remote: Counting objects: 49690, done. remote: Compressing objects: 100% (19765/19765), done. remote: Total 49690 (delta 36660), reused 39014 (delta 26526) Receiving objects: 100% (49690/49690), 7.92 MiB | 7.29 MiB/s, done. Resolving deltas: 100% (36660/36660), done. Checking connectivity... done. ubuntu@mjs-dstk-821a:~/code$ cd heat ubuntu@mjs-dstk-821a:~/code/heat$ tox -epep8 pep8 create: /home/ubuntu/code/heat/.tox/pep8 pep8 installdeps: -r/home/ubuntu/code/heat/requirements.txt, -r/home/ubuntu/code/heat/test-requirements.txt pep8 develop-inst: /home/ubuntu/code/heat pep8 runtests: PYTHONHASHSEED='0' pep8 runtests: commands[0] | flake8 heat bin/heat-api bin/heat-api-cfn bin/heat-api-cloudwatch bin/heat-engine bin/heat-manage contrib pep8 runtests: commands[1] | /home/ubuntu/code/heat/tools/config/check_uptodate.sh --- /tmp/heat.ep2CBe/heat.conf.sample2014-08-24 06:52:54.16484 + +++ etc/heat/heat.conf.sample2014-08-24 06:48:13.66484 + @@ -164,7 +164,7 @@ #allowed_rpc_exception_modules=oslo.messaging.exceptions,nova.exception,cinder.exception,exceptions # Qpid broker hostname. (string value) -#qpid_hostname=heat +#qpid_hostname=localhost # Qpid broker port. (integer value) #qpid_port=5672 @@ -221,7 +221,7 @@ # The RabbitMQ broker address where a single node is used. # (string value) -#rabbit_host=heat +#rabbit_host=localhost # The RabbitMQ broker port where a single node is used. # (integer value) check_uptodate.sh: heat.conf.sample is not up to date. check_uptodate.sh: Please run /home/ubuntu/code/heat/tools/config/generate_sample.sh. ERROR: InvocationError: '/home/ubuntu/code/heat/tools/config/check_uptodate.sh' pep8 runtests: commands[2] | /home/ubuntu/code/heat/tools/requirements_style_check.sh requirements.txt test-requirements.txt pep8 runtests: commands[3] | bash -c find heat -type f -regex '.*\.pot?' -print0|xargs -0 -n 1 msgfmt --check-format -o /dev/null ___ summary ERROR: pep8: commands failed ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ tools/config/generate_sample.sh ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ tox -epep8 pep8 develop-inst-noop: /home/ubuntu/code/heat pep8 runtests: PYTHONHASHSEED='0' pep8 runtests: commands[0] | flake8 heat bin/heat-api bin/heat-api-cfn bin/heat-api-cloudwatch bin/heat-engine bin/heat-manage contrib pep8 runtests: commands[1] | /home/ubuntu/code/heat/tools/config/check_uptodate.sh --- /tmp/heat.DqIhK5/heat.conf.sample2014-08-24 06:54:34.62884 + +++ etc/heat/heat.conf.sample2014-08-24 06:53:51.54084 + @@ -159,10 +159,6 @@ # Size of RPC connection pool. (integer value) #rpc_conn_pool_size=30 -# Modules of exceptions that are permitted to be recreated -# upon receiving exception data from an rpc call. (list value) -#allowed_rpc_exception_modules=oslo.messaging.exceptions,nova.exception,cinder.exception,exceptions - # Qpid broker hostname. (string value) #qpid_hostname=heat @@ -301,15 +297,6 @@ # Heartbeat time-to-live. (integer value) #matchmaker_heartbeat_ttl=600 -# Host to locate redis. (string value) -#host=127.0.0.1 - -# Use this port to connect to redis host. (integer value) -#port=6379 - -# Password for Redis server (optional). (string value) -#password=None - # Size of RPC greenthread pool. (integer value) #rpc_thread_pool_size=64 @@ -1229,6 +1216,22 @@ #hash_algorithms=md5 +[matchmaker_redis] + +# +# Options defined in oslo.messaging +# + +#
Re: [openstack-dev] [neutron] Incubator concerns from packaging perspective
On 21 August 2014 12:12, Ihar Hrachyshka ihrac...@redhat.com wrote: Let the ones that are primarily interested in good quality of that code (vendors) to drive development. And if some plugins become garbage, it's bad news for specific vendors; if neutron screws because of lack of concentration on core features and open source plugins, everyone is doomed. Completely agree with this sentiment. Is there a crisp distinction between a vendor plugin and an open source plugin though? The Snabb NFV (http://snabb.co/nfv.html) driver superficially looks like a vendor plugin but is actually completely open source. The development is driven by end-user organisations who want to make the standard upstream Neutron support their NFV use cases. We are looking for a good way to engage with the upstream community. In this cycle we have found kindred spirits in the NFV subteam., but we did not find a good way to engage with Neutron upstream (see https://review.openstack.org/#/c/116476/). It would be wonderful if there is a suitable process available for us to use in Kilo e.g. incubation. Cheers, -Luke ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs
On Fri, Aug 22, 2014 at 02:33:27PM +0200, Thierry Carrez wrote: [Snip some well articulated thoughts.] Enter the Czar system: each project should have a number of liaisons / official contacts / delegates that are fully responsible to cover one aspect of the project. We need to have Bugs czars, which are responsible for getting bugs under control. We need to have Oslo czars, which serve as liaisons for the Oslo program but also as active project-local oslo advocates. We need Security czars, which the VMT can go to to progress quickly on plugging vulnerabilities. We need release management czars, to handle the communication and process with that painful OpenStack release manager. We need Gate czars to serve as first-line-of-contact getting gate issues fixed... You get the idea. Just a gentle note on Gate: From my limited observation, I feel there's a chance that people might burn out early if they're only focused on Gate 100% of the time -- given the sheer number of moving parts and the cross-project coordination one has to be involved with. I recall Sean Dague mention elsewhere in a different thread[1] about feeling (understandably) frustrated trying to chase Gate issues and the lack of enough volunteers on that front. So, probably Gate czars might need to offset their work with some other activity, to retain a sense of sanity. Some people can be czars of multiple areas. This point should cover that. [1] http://lists.openstack.org/pipermail/openstack-dev/2014-June/037620.html -- /kashyap ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Neutron] [LBaaS] LBaaS v2 API syntax additions/changes
Hi, With the ongoing development of LBaaS v2, support for v2 of LBaaS in neutronclient is also being developed, as can be seen in [1]. The current implementation adds a new syntax for v2; Whereas the v1 syntax is 'neutron lb-object-command', the new v2 syntax is 'neutron lbaas-object-command'. We fear that this can lead to some level of confusion on the users' side. Currently, nothing prevents a user from trying to create a v1 pool and then trying to add v2 members. Of course the second command will fail, but the error message will be a non-intuitive one. This was discussed at the last weekly IRC meeting ([2]). Some bulletins: 1. As was discussed in the hackathon, there shouldn't be more than one version active at any given time - either v1 or v2. Also, using the same syntax for both v1 and v2 is not a good idea (db migration will be difficult when both version share syntax). We believe this should also be enforced in the server side. 2. Therefor, there should be some configuration to specifically enable either version (not both) in case LBaaS is needed. In this case, the other version is disabled (ie. a REST query for non-active version should return a not activated error). Additionally, adding a 'lb-version' command to return the version currently active seems like a good user-facing idea. We should see how this doesn't negatively effect the db migration process (for example, allowing read-only commands for both versions?) 3. Another decision that's needed to be made is the syntax for v2. As mentioned, the current new syntax is 'neutron lbaas-object-command' (against the old 'lb-object-action'), keeping in mind that once v1 is deprecated, a syntax like 'lbv2-object-action' would be probably unwanted. Is 'lbaas-object-command' okay with everyone? 4. If we are going for different API between versions, appropriate patches also need to be written for lbaas-related scripts and also Tempest, and their maintainers should probably be notified. Are there any issues with one of the points raised above? Does anyone see any other problems which we forgot to write down? Thanks a lot, John Schwarz, Yair Fried, Redhat. [1]: https://review.openstack.org/#/c/111475 [2]: http://eavesdrop.openstack.org/meetings/neutron_lbaas/2014/neutron_lbaas.2014-08-21-14.00.log.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] heat.conf.sample is not up to date
The differences do not look to me like ones that would follow from a difference in hash seed. Clint Byrum cl...@fewbar.com wrote on 08/24/2014 05:00:21 AM: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 08/24/2014 05:02 AM Subject: Re: [openstack-dev] [heat] heat.conf.sample is not up to date Guessing this is due to the new tox feature which randomizes python's hash seed. Excerpts from Mike Spreitzer's message of 2014-08-24 00:10:42 -0700: What is going on with this? If I do a fresh clone of heat and run `tox -epep8` then I get that complaint. If I then run the recommended command to fix it, and then `tox -epep8` again, I get the same complaint again --- and with different differences exhibited! The email below carries a typescript showing this. What I really need to know is what to do when committing a change that really does require a change in the sample configuration file. Of course I tried running generate_sample.sh, but `tox -epep8` still complains. What is the right procedure to get a correct sample committed? BTW, I am doing the following admittedly risky thing: I run DevStack, and make my changes in /opt/stack/heat/. Thanks, Mike - Forwarded by Mike Spreitzer/Watson/IBM on 08/24/2014 03:03 AM - From: ubuntu@mjs-dstk-821a (Ubuntu) To: Mike Spreitzer/Watson/IBM@IBMUS, Date: 08/24/2014 02:55 AM Subject:fresh flake fail ubuntu@mjs-dstk-821a:~/code$ git clone git://git.openstack.org/openstack/heat.git Cloning into 'heat'... remote: Counting objects: 49690, done. remote: Compressing objects: 100% (19765/19765), done. remote: Total 49690 (delta 36660), reused 39014 (delta 26526) Receiving objects: 100% (49690/49690), 7.92 MiB | 7.29 MiB/s, done. Resolving deltas: 100% (36660/36660), done. Checking connectivity... done. ubuntu@mjs-dstk-821a:~/code$ cd heat ubuntu@mjs-dstk-821a:~/code/heat$ tox -epep8 pep8 create: /home/ubuntu/code/heat/.tox/pep8 pep8 installdeps: -r/home/ubuntu/code/heat/requirements.txt, -r/home/ubuntu/code/heat/test-requirements.txt pep8 develop-inst: /home/ubuntu/code/heat pep8 runtests: PYTHONHASHSEED='0' pep8 runtests: commands[0] | flake8 heat bin/heat-api bin/heat-api-cfn bin/heat-api-cloudwatch bin/heat-engine bin/heat-manage contrib pep8 runtests: commands[1] | /home/ubuntu/code/heat/tools/config/check_uptodate.sh --- /tmp/heat.ep2CBe/heat.conf.sample2014-08-24 06:52:54.16484 + +++ etc/heat/heat.conf.sample2014-08-24 06:48:13.66484 + @@ -164,7 +164,7 @@ #allowed_rpc_exception_modules=oslo.messaging.exceptions,nova.exception,cinder.exception,exceptions # Qpid broker hostname. (string value) -#qpid_hostname=heat +#qpid_hostname=localhost # Qpid broker port. (integer value) #qpid_port=5672 @@ -221,7 +221,7 @@ # The RabbitMQ broker address where a single node is used. # (string value) -#rabbit_host=heat +#rabbit_host=localhost # The RabbitMQ broker port where a single node is used. # (integer value) check_uptodate.sh: heat.conf.sample is not up to date. check_uptodate.sh: Please run /home/ubuntu/code/heat/tools/config/generate_sample.sh. ERROR: InvocationError: '/home/ubuntu/code/heat/tools/config/check_uptodate.sh' pep8 runtests: commands[2] | /home/ubuntu/code/heat/tools/requirements_style_check.sh requirements.txt test-requirements.txt pep8 runtests: commands[3] | bash -c find heat -type f -regex '.*\.pot?' -print0|xargs -0 -n 1 msgfmt --check-format -o /dev/null ___ summary ERROR: pep8: commands failed ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ tools/config/generate_sample.sh ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ tox -epep8 pep8 develop-inst-noop: /home/ubuntu/code/heat pep8 runtests: PYTHONHASHSEED='0' pep8 runtests: commands[0] | flake8 heat bin/heat-api bin/heat-api-cfn bin/heat-api-cloudwatch bin/heat-engine bin/heat-manage contrib pep8 runtests: commands[1] | /home/ubuntu/code/heat/tools/config/check_uptodate.sh --- /tmp/heat.DqIhK5/heat.conf.sample2014-08-24 06:54:34.62884 + +++ etc/heat/heat.conf.sample2014-08-24 06:53:51.54084 + @@ -159,10 +159,6 @@ # Size of RPC connection pool. (integer value) #rpc_conn_pool_size=30 -# Modules of exceptions that are permitted to be recreated -# upon receiving exception data from an rpc call. (list value) - #allowed_rpc_exception_modules=oslo.messaging.exceptions,nova.exception,cinder.exception,exceptions - # Qpid broker hostname. (string value) #qpid_hostname=heat @@ -301,15 +297,6 @@ # Heartbeat time-to-live.
Re: [openstack-dev] [neutron] [third-party] What tests are required to be run
Hi Kevin: Thanks, this is a great idea! I may try just a slight variation of this concept. Maybe your idea could be the recommended way to create a 3rd party CI for plugins that are just being introduced and need to limit the scope of testing to a small set of plugin-related commits (or plugins blocked on a certain fix). Thanks, Dane From: Kevin Benton [mailto:blak...@gmail.com] Sent: Saturday, August 23, 2014 5:47 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to be run Can you disable posting of results directly from your Jenkins/Zuul setup and have a script that just checks the log file for special markers to determine if the vote should be FAILED/PASSED/SKIPPED? Another advantage of this approach is that it gives you an opportunity to detect when a job just failed to setup due to infrastructure reasons and trigger a recheck without ever first posting a failure to gerrit. On Fri, Aug 22, 2014 at 3:06 PM, Dane Leblanc (leblancd) lebla...@cisco.commailto:lebla...@cisco.com wrote: Thanks Edgar for updating the APIC status!!! Edgar and Kyle: *PLEASE NOTE** I need your understanding and advice on the following: We are still stuck with a problem stemming from a design limitation of Jenkins that prevents us from being compliant with Neutron 3rd Party CI requirements for our DFA CI. The issue is that Jenkins only allows our scripts to (programmatically) return either Success or Fail. There is no option to return Aborted, Not Tested, or Skipped. Why does this matter? The DFA plugin is just being introduced, and initial DFA-enabling change sets have not yet been merged. Therefore, all other change sets will fail our Tempest tests, since they are not DFA-enabled. Similarly, we were recently blocked in our APIC CI with a critical bug, causing all change sets without this fix to fail on our APIC testbed. In these cases, we would like to enter a throttled or partially blocked mode, where we would skip testing on change sets we know will fail, and (in an ideal world) signal this shortcoming to Gerrit e.g. by returning a Skipped status. Unfortunately, this option is not available in Jenkins scripts, as Jenkins is currently designed. The only options we have available is Success or all Fail, which are both misleading. We would also incorrectly report success or fail on one of the following test commits: https://review.openstack.org/#/c/114393/ https://review.openstack.org/#/c/40296/ I've brought this issue up on the openstack-infra IRC, and jeblair confirmed the Jenkins limitation, but asked me to get consensus from the Neutron community as to this being a problem/requirement. I've also sent out an e-mail on the Neutron ML trying to start a discussion on this problem (no traction). I plan on bringing this up in the 3rd Party CI IRC on Monday, assuming there is time permitted in the open discussion. I'm also investigating For the short term, I would like to propose the following: * We bring this up on the 3rd Party CI IRC on Monday to get a solution or workaround, if available. If a solution is available, let's consider including that as a hint when we come up with CI requirements for handling CIs bocked by some critical fix. * I'm also looking into using a REST API to cancel a Jenkins job programmatically. * If no solution or workaround is available, we work with infra team or with Jenkins team to create a solution. * Until a solution is available, for plugins which are blocked by a critical bug, we post a status/notes indicating the plugin's situation on our 3rd party CI status wiki, e.g.: Vendor Plugin/Driver Name Contact NameStatus Notes My Vendor Name My Plugin CIMy Contact Person T Throttled / Partially blocked / Awaiting Intial Commits The status/notes should be clear and understood by the Neutron team. The console logs for change sets where the tests were skipped should also contain a message that all testing is being skipped for that commit. Note that when the DFA initial commits are merged, then this issue would go away for the DFA CI. However, this problem will reappear every time a blocking critical bug shows up for a 3rd party CI setup, or a new plugin is introduced and the hardware-enabling commits are not yet merged. (That is, until we have a solution for the Jenkins limitation). Let me know what you think. Thanks, Dane -Original Message- From: Edgar Magana [mailto:edgar.mag...@workday.commailto:edgar.mag...@workday.com] Sent: Friday, August 22, 2014 1:57 PM To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to be run Sorry my bad but I just changed. Edgar On 8/21/14, 2:13 PM, Dane Leblanc (leblancd) lebla...@cisco.commailto:lebla...@cisco.com
Re: [openstack-dev] [neutron][cisco] Cisco Nexus requires patched ncclient
Ihar Hrachyshka ihrac...@redhat.com wrote: Now, maybe putting the module into requirements.txt is an overkill (though I doubt it). In that case, we could be interested in getting the info in some other centralized way. Maru is of the opinion that it is overkill. I feel the same way, but I am not involved much with deployment issues so my feelings should not sway anyone. Note that ncclient is not the only package used by vendor solutions that is not listed in requirements.txt. The ones I am aware of are: ncclient (cisco and brocade) heleosapi (embrane) a10_neutron_lbaas (a10networks) Maybe we should start exploring some other centralized way to list these type of dependencies? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] heat.conf.sample is not up to date
I'm following this as well since I have the exact same problem in a docstring patch for heat. On Sun, Aug 24, 2014 at 9:30 AM, Mike Spreitzer mspre...@us.ibm.com wrote: The differences do not look to me like ones that would follow from a difference in hash seed. Clint Byrum cl...@fewbar.com wrote on 08/24/2014 05:00:21 AM: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 08/24/2014 05:02 AM Subject: Re: [openstack-dev] [heat] heat.conf.sample is not up to date Guessing this is due to the new tox feature which randomizes python's hash seed. Excerpts from Mike Spreitzer's message of 2014-08-24 00:10:42 -0700: What is going on with this? If I do a fresh clone of heat and run `tox -epep8` then I get that complaint. If I then run the recommended command to fix it, and then `tox -epep8` again, I get the same complaint again --- and with different differences exhibited! The email below carries a typescript showing this. What I really need to know is what to do when committing a change that really does require a change in the sample configuration file. Of course I tried running generate_sample.sh, but `tox -epep8` still complains. What is the right procedure to get a correct sample committed? BTW, I am doing the following admittedly risky thing: I run DevStack, and make my changes in /opt/stack/heat/. Thanks, Mike - Forwarded by Mike Spreitzer/Watson/IBM on 08/24/2014 03:03 AM - From: ubuntu@mjs-dstk-821a (Ubuntu) To: Mike Spreitzer/Watson/IBM@IBMUS, Date: 08/24/2014 02:55 AM Subject:fresh flake fail ubuntu@mjs-dstk-821a:~/code$ git clone git://git.openstack.org/openstack/heat.git Cloning into 'heat'... remote: Counting objects: 49690, done. remote: Compressing objects: 100% (19765/19765), done. remote: Total 49690 (delta 36660), reused 39014 (delta 26526) Receiving objects: 100% (49690/49690), 7.92 MiB | 7.29 MiB/s, done. Resolving deltas: 100% (36660/36660), done. Checking connectivity... done. ubuntu@mjs-dstk-821a:~/code$ cd heat ubuntu@mjs-dstk-821a:~/code/heat$ tox -epep8 pep8 create: /home/ubuntu/code/heat/.tox/pep8 pep8 installdeps: -r/home/ubuntu/code/heat/requirements.txt, -r/home/ubuntu/code/heat/test-requirements.txt pep8 develop-inst: /home/ubuntu/code/heat pep8 runtests: PYTHONHASHSEED='0' pep8 runtests: commands[0] | flake8 heat bin/heat-api bin/heat-api-cfn bin/heat-api-cloudwatch bin/heat-engine bin/heat-manage contrib pep8 runtests: commands[1] | /home/ubuntu/code/heat/tools/config/check_uptodate.sh --- /tmp/heat.ep2CBe/heat.conf.sample2014-08-24 06:52:54.16484 + +++ etc/heat/heat.conf.sample2014-08-24 06:48:13.66484 + @@ -164,7 +164,7 @@ #allowed_rpc_exception_modules=oslo.messaging.exceptions,nova.exception,cinder.exception,exceptions # Qpid broker hostname. (string value) -#qpid_hostname=heat +#qpid_hostname=localhost # Qpid broker port. (integer value) #qpid_port=5672 @@ -221,7 +221,7 @@ # The RabbitMQ broker address where a single node is used. # (string value) -#rabbit_host=heat +#rabbit_host=localhost # The RabbitMQ broker port where a single node is used. # (integer value) check_uptodate.sh: heat.conf.sample is not up to date. check_uptodate.sh: Please run /home/ubuntu/code/heat/tools/config/generate_sample.sh. ERROR: InvocationError: '/home/ubuntu/code/heat/tools/config/check_uptodate.sh' pep8 runtests: commands[2] | /home/ubuntu/code/heat/tools/requirements_style_check.sh requirements.txt test-requirements.txt pep8 runtests: commands[3] | bash -c find heat -type f -regex '.*\.pot?' -print0|xargs -0 -n 1 msgfmt --check-format -o /dev/null ___ summary ERROR: pep8: commands failed ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ tools/config/generate_sample.sh ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ ubuntu@mjs-dstk-821a:~/code/heat$ tox -epep8 pep8 develop-inst-noop: /home/ubuntu/code/heat pep8 runtests: PYTHONHASHSEED='0' pep8 runtests: commands[0] | flake8 heat bin/heat-api bin/heat-api-cfn bin/heat-api-cloudwatch bin/heat-engine bin/heat-manage contrib pep8 runtests: commands[1] | /home/ubuntu/code/heat/tools/config/check_uptodate.sh --- /tmp/heat.DqIhK5/heat.conf.sample2014-08-24 06:54:34.62884 + +++ etc/heat/heat.conf.sample2014-08-24 06:53:51.54084 + @@ -159,10 +159,6 @@ # Size of RPC connection pool. (integer value) #rpc_conn_pool_size=30 -# Modules of exceptions that are permitted to be recreated -# upon receiving exception data from an rpc call. (list
Re: [openstack-dev] [heat] heat.conf.sample is not up to date
On Sunday, August 24, 2014, Anne Gentle a...@openstack.org wrote: I'm following this as well since I have the exact same problem in a docstring patch for heat. Keystone saw an oddity with the new sample config generator (changing how options are sorted and therefore changing the way the sample config is rendered). This could be a similar / related issue. Most of the projects stopped gating on up-to-date sample config a few reasons, the first is that with external library dependencies you never know when / if something upstream will break the test run (e.g. New Oslo.config or new keystonemiddleware). Now imagine that issue occurred and was blocking a gate-fixing bug (happened at least a couple times). In short, making sample config being up-to-date to merge code causes a lot if headaches. Different projects handle this differently. Nova doesn't have a sample config in tree, keystone updates on a semi-regular basis (sometimes as part of a patch, sometimes as a separate patch). Keystone team is looking at adding a simple non-voting gate job (if infra doesn't mind) that will tell us the config is out of date. While it is nice to always have an updated sample config, I think it is not worth the breakage / issues it adds to the gate. It might make sense to standardize how we handle sample config files across the projects or at least standardize on removing the gate block if the config is out of date. I know it was floated earlier that there would be a proposal bot job (like translations) for sample config files, but I don't remember the specifics of why it wasn't well liked. --Morgan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] heat.conf.sample is not up to date
Morgan Fainberg morgan.fainb...@gmail.com wrote on 08/24/2014 12:01:37 PM: ... Keystone saw an oddity with the new sample config generator (changing how options are sorted and therefore changing the way the sample config is rendered). This could be a similar / related issue. Most of the projects stopped gating on up-to-date sample config a few reasons, the first is that with external library dependencies you never know when / if something upstream will break the test run (e.g. New Oslo.config or new keystonemiddleware). Now imagine that issue occurred and was blocking a gate-fixing bug (happened at least a couple times). In short, making sample config being up-to-date to merge code causes a lot if headaches. Different projects handle this differently. Nova doesn't have a sample config in tree, keystone updates on a semi-regular basis (sometimes as part of a patch, sometimes as a separate patch). Keystone team is looking at adding a simple non-voting gate job (if infra doesn't mind) that will tell us the config is out of date. While it is nice to always have an updated sample config, I think it is not worth the breakage / issues it adds to the gate. It might make sense to standardize how we handle sample config files across the projects or at least standardize on removing the gate block if the config is out of date. I know it was floated earlier that there would be a proposal bot job (like translations) for sample config files, but I don't remember the specifics of why it wasn't well liked. Consistency would be nice. The must-have is just to document each project's procedure (accurately, of course). Thanks, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] heat.conf.sample is not up to date
On Sun, Aug 24, 2014 at 11:10 AM, Mike Spreitzer mspre...@us.ibm.com wrote: What I really need to know is what to do when committing a change that really does require a change in the sample configuration file. Of course I tried running generate_sample.sh, but `tox -epep8` still complains. What is the right procedure to get a correct sample committed? BTW, I am doing the following admittedly risky thing: I run DevStack, and make my changes in /opt/stack/heat/. Mike, It seems that you have oslo.messaging installed from master (default behavior of Devstack), while heat.sample.config is built for oslo.messaging v 1.3.1. As a quick workaround I'd suggest to downgrade oslo.messaging to 1.3.1 in pep8 virtual environment: $ cd /opt/stack/heat $ source .tox/pep8/bin/activate $ pip install oslo.messaging==1.3.1 $ deactivate $ tox -e pep8 # should pass now I'd also like to know what would be the procedure to update sample config when new version of oslo.messaging is released? Maybe we should pin requirements to specific version of oslo library and update that requirement along with the sample config once new version is released? -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Use public IP address as instance fixed IP
Would this work? We used to have warnings in Neutron docs indicating that instances should not be attached to external networks: It is important to understand that you should not attach the instance to Ext-Net directly. Instead, you must use a floating IP to make it accessible from the external network. In this particular case and with the OVS plugin, the traffic on the external network which now hosts tenant VMs (on OpenStack compute nodes) should get routed from the br-int to the external bridge br-ex using for example the appropriate vlan id (what if external network does not use vlan?) and then to the external network without doing the NATing. Would this traffic go through the veth pair connecting the br-int and br-ex? Mohammad From: Kevin Benton blak...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 08/23/2014 01:37 AM Subject:Re: [openstack-dev] [Neutron] Use public IP address as instance fixed IP Yes, you should be able to create a shared/external network within Neutron to accomplish this. On Fri, Aug 22, 2014 at 7:25 AM, Bao Wang bywan...@gmail.com wrote: Thank you for your response. Could this be done naturally with Openstack neutron or have to be done manually outside neutron ? As we are expecting to orchestrate hundreds of NFV with all similar network configuration, programmability is another key element. On Thu, Aug 21, 2014 at 3:52 PM, Kevin Benton blak...@gmail.com wrote: Have you tried making the external network shared as well? Instances that need a private IP with NAT attach to an internal network and go through the router like normal. Instances that need a public IP without NAT would just attach directly to the external network. On Thu, Aug 21, 2014 at 7:06 AM, Bao Wang bywan...@gmail.com wrote: I have a very complex Openstack deployment for NFV. It could not be deployed as Flat. It will have a lot of isolated private networks. Some interfaces of a group VM instances will need bridged network with their fixed IP addresses to communicate with outside world while other interfaces from the same set of VM should keep isolated with real private/fixed IP addresses. What happen if we use public IP addresses directly as fixed IP on those interfaces ? Will this work with Openstack neutron networking ? Will Openstack do NAT automatically on those ? Overall, the requirement is to use the fixed/public IP to communicate with outside directly on some interfaces of some VM instances while keeping others as private. The floating IP is not an option here ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][cisco] Cisco Nexus requires patched ncclient
On Aug 24, 2014, at 5:14 PM, Henry Gessau ges...@cisco.com wrote: Ihar Hrachyshka ihrac...@redhat.com wrote: Now, maybe putting the module into requirements.txt is an overkill (though I doubt it). In that case, we could be interested in getting the info in some other centralized way. Maru is of the opinion that it is overkill. I feel the same way, but I am not involved much with deployment issues so my feelings should not sway anyone. Note that ncclient is not the only package used by vendor solutions that is not listed in requirements.txt. The ones I am aware of are: ncclient (cisco and brocade) heleosapi (embrane) a10_neutron_lbaas (a10networks) Maybe we should start exploring some other centralized way to list these type of dependencies? I think each plugin should be able to have its own requirements.txt file to aid in manual deployment and in packaging of a given plugin. This suggests the need to maintain a global plugin requirements file (in the tree) to ensure use of only a single version of any dependencies used across more than one plugin. Given that 3rd party CI is likely having to install these dependencies anyway, I think it would be good to make this deployment reproducible while avoiding the need to add new dependencies to core Neutron. Maru ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [OpenStack-dev][all] Having os-auth-file as parameter for running command line client
When we deploy devstack, we get the files with authentication credentials created in devstack/accr directory. I was playing around the command line clients with one of my script which can use the auth files created to execute different auth credentials, when the idea about having os-auth-file as an argument to the command line client came to my mind, which can help to use all the required credentials in one go instead of specifying individually or sourcing the file. Can we consider this as some additional optional parameter we can include? Best Regards, Swapnil Kulkarni irc : coolsvap ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] [LBaaS] LBaaS v2 API syntax additions/changes
On Sun, Aug 24, 2014, at 06:37 AM, John Schwarz wrote: Hi, With the ongoing development of LBaaS v2, support for v2 of LBaaS in neutronclient is also being developed, as can be seen in [1]. The current implementation adds a new syntax for v2; Whereas the v1 syntax is 'neutron lb-object-command', the new v2 syntax is 'neutron lbaas-object-command'. We fear that this can lead to some level of confusion on the users' side. Currently, nothing prevents a user from trying to create a v1 pool and then trying to add v2 members. Of course the second command will fail, but the error message will be a non-intuitive one. This was discussed at the last weekly IRC meeting ([2]). Some bulletins: 1. As was discussed in the hackathon, there shouldn't be more than one version active at any given time - either v1 or v2. Also, using the same syntax for both v1 and v2 is not a good idea (db migration will be difficult when both version share syntax). We believe this should also be enforced in the server side. 2. Therefor, there should be some configuration to specifically enable either version (not both) in case LBaaS is needed. In this case, the other version is disabled (ie. a REST query for non-active version should return a not activated error). Additionally, adding a 'lb-version' command to return the version currently active seems like a good user-facing idea. We should see how this doesn't negatively effect the db migration process (for example, allowing read-only commands for both versions?) 3. Another decision that's needed to be made is the syntax for v2. As mentioned, the current new syntax is 'neutron lbaas-object-command' (against the old 'lb-object-action'), keeping in mind that once v1 is deprecated, a syntax like 'lbv2-object-action' would be probably unwanted. Is 'lbaas-object-command' okay with everyone? 4. If we are going for different API between versions, appropriate patches also need to be written for lbaas-related scripts and also Tempest, and their maintainers should probably be notified. Are there any issues with one of the points raised above? Does anyone see any other problems which we forgot to write down? Thanks a lot, John Schwarz, Yair Fried, Redhat. [1]: https://review.openstack.org/#/c/111475 [2]: http://eavesdrop.openstack.org/meetings/neutron_lbaas/2014/neutron_lbaas.2014-08-21-14.00.log.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev As a user of both the neutron API and python-neutronclient it would be much better to have the client use a common set of commands for both v1 and v2 where the specific version used is determined by the keystone catalog or other overriding information. I don't want to have to remember two different sets of commands with potentially two different outputs. Client libraries exist so that users don't have to think about this stuff. This should prevent confusion as users will use a common version unless they specifically provide an override of some sort. Clark ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Neutron][IPv6] Allowing Plugins in tree that explicitly do not support IPv6
Hi, The amount of tests that we are having to add skips to for the One Convergence plugin has grown quite a bit lately. http://git.openstack.org/cgit/openstack/neutron/tree/neutron/tests/unit/oneconvergence/test_nvsd_plugin.py#n58 This is the only plugin that inherits from the NeutronDbPluginV2 class and has completely disabled creating Subnets that have an ip_version of 6. http://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/oneconvergence/plugin.py#n228 I hate to single out this plugin. Before this development cycle, open source plugins would allow the creation of Subnets with ip_version set to 6, and then have lots of bugs which would prevent any instances from actually obtaining IPv6 addresses and routing. But we didn't have to explicitly skip tests every time someone wanted to fix bugs - like the following patch. https://review.openstack.org/#/c/116317/ Also, I want to highlight some meeting minutes, where it was discussed that in the K release, all plugins will be required to support IPv6. I hope we can discuss this, build consensus, and ensure that the K release requirement is reasonable. http://eavesdrop.openstack.org/meetings/networking/2014/networking.2014-07-14-21.02.html Sean M. Collins ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [congress] specs update
So I have the spec changes all fixed up and pending reviews https://review.openstack.org/#/c/113638/ https://review.openstack.org/#/c/116530/ https://review.openstack.org/#/c/116531/ https://review.openstack.org/#/c/116532/ Once these are merged, i will start pinging people on blueprints that are missing specs. There is a process for holding over non-implemented specs into the next release. I will update the wiki. ~ sean ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs
On 08/23/2014 06:35 PM, Clint Byrum wrote: I agree as well. PTL is a servant of the community, as any good leader is. If the PTL feels they have to drop the hammer, or if an impass is reached where they are asked to, it is because they have failed to get everyone communicating effectively, not because that's their job. The problem isn't really that teams are not communicating effectively, nor is the problem to do with some deficit of a PTL in either putting the hammer down or failing to figure out common ground. The issue in my opinion and my experience is that there are multiple valid ways of doing something (say, deployment or metering or making toast) and the TC and our governing structure has decided to pick winners in spaces instead of having a big tent and welcoming different solutions and projects into the OpenStack fold. We pick winners and by doing so, we are exclusionary, and this exclusivity does not benefit our user community, but rather just gives it fewer options. IMHO, the TC should become an advisory team that recommends to interested project teams ways in which they can design and architect their projects to integrate well with other projects in the OpenStack community, and design their projects for the scale, stability and requirements (such as multi-tenancy) that an open cloud software ecosystem demands. Just my two cents, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [congress] policy summit
I am looking for policy overview/impact covering heat, neutron, nova, olso, mistral, and rubric. Are there any others that want to join us and discuss how policy impacts their project? I would welcome some discussion around Opendaylight policy as well. We have 21 people signed up to join us so far here https://www.eventbrite.com/e/openstack-policy-summit-tickets-12642081807. It will be much better to join us in person, but I will setup a remote, so Kyle and others that can join in. Go ahead and add yourself as remote to the event. Those people that have signed up, update the etherpad https://etherpad.openstack.org/p/juno-midcycle-policy-summit. The proposed agenda is Thursday covers various project policy overview and/or impacts. Friday covers various cross-project policy use cases including security. More information on the etherpad sooner, will allow us to pre-debate content and we will get a lot more out of the two days. ~ sean ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] new testtools breaking gate
On 23 August 2014 00:55, Ihar Hrachyshka ihrac...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi all, this week is quite bumpy for unit testing in gate. First, it was upgrade to new 'tox' version that broke quite some branches. And today new testtools 0.9.36 were released and were caught by gate, which resulted in the following unit test failures in multiple projects: TestCase.setUp was already called. Do not explicitly call setUp from your tests. In your own setUp, use super to call the base setUp. All branches are affected: havana, icehouse, and master. This is because the following check was released with the new version of the library: https://github.com/testing-cabal/testtools/commit/5c3b92d90a64efaecdc4010a98002bfe8b888517 And the temporary fix is to merge the version pin patch in global requirements, backport it to stable branches, and merge the updates from Openstack Proposal Bot to all affected projects. The patch for master requirements is: https://review.openstack.org/#/c/116267/ In the meantime, projects will need to fix their tests not to call setUp() and tearDown() twice. This will be the requirement to unpin the version of the library. So, please review, backport, and make sure it lands in project requirements files. In hindsight I should have introduced a warning for a period of time then the actual error. Sorry! -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs
On 08/22/2014 09:02 PM, Anne Gentle wrote: /flame-on Let's call spades, spades here. Czar is not only overkill, but the wrong metaphor. /flame-off I'm with Rocky on the anti-czar-as-a-word camp. We all like clever names to shed the corporate stigma but this word ain't it. Liaison or lead? Also wanted to point to https://wiki.openstack.org/wiki/PTLguide as it's quite nice. In the Linux Kernel world they are called Lieutenants, but they are per sub system. I'm guessing that is more like the PTL role today. THe etymology is good, to, as a Lieutenant is one who stands in Lieu of the commander. I think that some projects probably need to divvy up the roles along different line. For example, in Keystone, we have KVS, SQL and LDAP backends. Morgan's been the KVS guy due to rewriting things for Dogpile, and I've been kind of the LDAP guy since I was originally responsible for that. I suspect the same is true of other projects. Many of the roles you listed (Bug, etc) are straight PTL responsibilities in my book. What is lacking is a structure over delegating out some subset of that responsibility: no regular bug triage meeting, etc. I'd rather not throw all bug-triage onto one persons shoulders, as it really should be a team effort. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone][Marconi][Heat] Creating accounts in Keystone
On 08/23/2014 02:01 AM, Clint Byrum wrote: I don't know how Zaqar does its magic, but I'd love to see simple signed URLs rather than users/passwords. This would work for Heat as well. That way we only have to pass in a single predictably formatted string. Excerpts from Zane Bitter's message of 2014-08-22 14:35:38 -0700: Here's an interesting fact about Zaqar (the project formerly known as Marconi) that I hadn't thought about before this week: it's probably the first OpenStack project where a major part of the API primarily faces Nah, this is the direction we are headed. Service users (out of LDAP!) are going to be the norm with a recent feature add to Keytone: http://adam.younglogic.com/2014/08/getting-service-users-out-of-ldap/ software running in the cloud rather than facing the user. That is to say, nobody is going to be sending themselves messages on their laptop, from their laptop, via a cloud. At least one end of any given queue is likely to be on a VM in the cloud. That makes me wonder: how does Zaqar authenticate users who are sending and receiving messages (as opposed to setting up the queues in the first place)? Presumably using Keystone, in which case it will run into a problem we've been struggling with in Heat since the very early days. Keystone is generally a front end for an identity store with a 1:1 correspondence between users and actual natural persons. Only the operator can add or remove accounts. This breaks down as soon as you need to authenticate automated services running in the cloud - in particular, you never ever want to store the credentials belonging to an actual natural person in a server in the cloud. Heat has managed to work around this to some extent (for those running the Keystone v3 API) by creating users in a separate domain and more or less doing our own authorisation for them. However, this requires action on the part of the operator, and isn't an option for the end user. I guess Zaqar could do something similar and pass out sets of credentials good only for reading and writing to queues (respectively), but it seems like it would be better if the user could create the keystone accounts and set their own access control rules on the queues. On AWS the very first thing a user does is create a bunch of IAM accounts so that they virtually never have to use the credentials associated with their natural person ever again. There are both user accounts and service accounts - the latter IIUC have automatically-rotating keys. Is there anything like this planned in Keystone? Zaqar is likely only the first (I guess second, if you count Heat) of many services that will need it. I have this irrational fear that somebody is going to tell me that this issue is the reason for the hierarchical-multitenancy idea - fear because that both sounds like it requires intrusive changes in every OpenStack project and fails to solve the problem. I hope somebody will disabuse me of that notion in 3... 2... 1... cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] heat.conf.sample is not up to date
Ruslan Kamaldinov rkamaldi...@mirantis.com wrote on 08/24/2014 12:30:04 PM: From: Ruslan Kamaldinov rkamaldi...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 08/24/2014 12:32 PM Subject: Re: [openstack-dev] [heat] heat.conf.sample is not up to date On Sun, Aug 24, 2014 at 11:10 AM, Mike Spreitzer mspre...@us.ibm.com wrote: What I really need to know is what to do when committing a change that really does require a change in the sample configuration file. Of course I tried running generate_sample.sh, but `tox -epep8` still complains. What is the right procedure to get a correct sample committed? BTW, I am doing the following admittedly risky thing: I run DevStack, and make my changes in /opt/stack/heat/. Mike, It seems that you have oslo.messaging installed from master (default behavior of Devstack), while heat.sample.config is built for oslo.messaging v 1.3.1. As a quick workaround I'd suggest to downgrade oslo.messaging to 1.3.1 in pep8 virtual environment: $ cd /opt/stack/heat $ source .tox/pep8/bin/activate $ pip install oslo.messaging==1.3.1 $ deactivate $ tox -e pep8 # should pass now In that same VM in which I ran DevStack, I also made an independent clone of heat in ~/code/heat; my original email quoted a failure from that clone, hoping that it might be easier to debug (being a simpler scenario). Below is a typescript showing me try again there, including a trial of what Mathieu Gagné suggested (which has some kind of command line syntax error, and did nothing). You will see that I investigated and found that in this case tox's venv contained oslo.messaging version 1.3.1, so no adjustment about that was needed. Yet it still failed. ubuntu@mjs-dstk-821a:~/code/heat$ rm -rf .tox ubuntu@mjs-dstk-821a:~/code/heat$ tox -evenv bash ./tools/config/generate_sample.sh -b . -p heat -o etc/heat usage: tox [-h] [--version] [-v] [--showconfig] [-l] [-c CONFIGFILE] [-e envlist] [--notest] [--sdistonly] [--installpkg PATH] [--develop] [--set-home] [-i URL] [-r] [--result-json PATH] [--hashseed SEED] [--force-dep REQ] [--sitepackages] [--skip-missing-interpreters] [args [args ...]] tox: error: unrecognized arguments: -b . -p heat -o etc/heat ubuntu@mjs-dstk-821a:~/code/heat$ tox -epep8 pep8 create: /home/ubuntu/code/heat/.tox/pep8 pep8 installdeps: -r/home/ubuntu/code/heat/requirements.txt, -r/home/ubuntu/code/heat/test-requirements.txt pep8 develop-inst: /home/ubuntu/code/heat pep8 runtests: PYTHONHASHSEED='0' pep8 runtests: commands[0] | flake8 heat bin/heat-api bin/heat-api-cfn bin/heat-api-cloudwatch bin/heat-engine bin/heat-manage contrib pep8 runtests: commands[1] | /home/ubuntu/code/heat/tools/config/check_uptodate.sh --- /tmp/heat.UnHAZD/heat.conf.sample 2014-08-25 04:06:07.64884 + +++ etc/heat/heat.conf.sample 2014-08-24 06:53:51.54084 + @@ -159,10 +159,6 @@ # Size of RPC connection pool. (integer value) #rpc_conn_pool_size=30 -# Modules of exceptions that are permitted to be recreated -# upon receiving exception data from an rpc call. (list value) -#allowed_rpc_exception_modules=oslo.messaging.exceptions,nova.exception,cinder.exception,exceptions - # Qpid broker hostname. (string value) #qpid_hostname=heat @@ -301,15 +297,6 @@ # Heartbeat time-to-live. (integer value) #matchmaker_heartbeat_ttl=600 -# Host to locate redis. (string value) -#host=127.0.0.1 - -# Use this port to connect to redis host. (integer value) -#port=6379 - -# Password for Redis server (optional). (string value) -#password=None - # Size of RPC greenthread pool. (integer value) #rpc_thread_pool_size=64 @@ -1229,6 +1216,22 @@ #hash_algorithms=md5 +[matchmaker_redis] + +# +# Options defined in oslo.messaging +# + +# Host to locate redis. (string value) +#host=127.0.0.1 + +# Use this port to connect to redis host. (integer value) +#port=6379 + +# Password for Redis server (optional). (string value) +#password=None + + [matchmaker_ring] # check_uptodate.sh: heat.conf.sample is not up to date. check_uptodate.sh: Please run /home/ubuntu/code/heat/tools/config/generate_sample.sh. ERROR: InvocationError: '/home/ubuntu/code/heat/tools/config/check_uptodate.sh' pep8 runtests: commands[2] | /home/ubuntu/code/heat/tools/requirements_style_check.sh requirements.txt test-requirements.txt pep8 runtests: commands[3] | bash -c find heat -type f -regex '.*\.pot?' -print0|xargs -0 -n 1 msgfmt --check-format -o /dev/null _ __ summary _ ___ ERROR: pep8: commands failed ubuntu@mjs-dstk-821a:~/code/heat$ . .tox/pep8/bin/activate (pep8)ubuntu@mjs-dstk-821a:~/code/heat$ python Python 2.7.6 (default, Mar 22 2014, 22:59:56) [GCC 4.8.2] on linux2 Type help, copyright, credits or license for more information. system Traceback (most recent call last): File
