[openstack-dev] [Savanna] Savanna meeting minutes
Thanks everyone who have joined today's Savanna meeting. Here are the logs from the last meeting (July, 11): Minutes: http://eavesdrop.openstack.org/meetings/savanna/2013/savanna.2013-07-11-18.05.html Minutes (text): http://eavesdrop.openstack.org/meetings/savanna/2013/savanna.2013-07-11-18.05.txt Log: http://eavesdrop.openstack.org/meetings/savanna/2013/savanna.2013-07-11-18.05.log.html Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Bug #1194026
Hi folks I think I found possible cause of this problems. so we expected all RPC call is executed serialized way on l3-agent However it is executed in random order. http://paste.openstack.org/show/40156/ line starts from Get is RPC message log. line starts from [ Process is when l3-agent start processing rpc messages. (I added rpc message number for debugging) https://bugs.launchpad.net/neutron/+bug/1194026 Here is my proposal for fixing code. - Server side simply notifies when something updated. - Client will update updated flag in the client when it get updated - some looping call will check the flag, if the flag is true, it will full sync with servers If this is OK, I'll start write it. Thanks Nachi 2013/7/11 Salvatore Orlando sorla...@nicira.com: Adding openstack-dev to this discussion thread. Looks like the test is going to be skipped at the moment, but we probably need to consider raising the priority of this issue and assign our cores with more experience with tempest/gating on this. salvatore On 9 July 2013 22:48, Maru Newby ma...@redhat.com wrote: My suggestion is that the quantum exercise script be disabled for now if that will allow the tempest test to run, since the tempest test is more useful (it does an ssh check to ensure that the metadata service has configured the VM). Doing so would allow useful gating while we look at resolving the timing bug. m. On Jul 9, 2013, at 5:42 PM, Nachi Ueno na...@ntti3.com wrote: Hi Maru The gating test will not fail everytime. Sometimes, both of tests works, sometimes not. In this test, execise.sh works and tempest don't works. I'm still not sure is there any dependencies in this failure or not. So I'm assuming this is kind of timing issue.. hmm this kind of bug is hard to fix.. 2013/7/9 Maru Newby ma...@redhat.com: If there is a conflict between the exercise test and the tempest test, does the tempest test pass if the exercise script isn't run beforehand? m. On Jul 9, 2013, at 5:20 PM, Nachi Ueno na...@ntti3.com wrote: Hi I checked briefly, and it looks some timing bug of l3-agent. I added note on the bug report. https://bugs.launchpad.net/neutron/+bug/1194026 2013/7/9 Salvatore Orlando sorla...@nicira.com: Sean Dague singled it out as the biggest cause for gate failures, and invited us to have a look at it. I've raised its importance to high, but I don't have the cycles to look at it in the short term. It would be really if somebody from the core team finds some time to triage it. Salvatore ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer
Monty Taylor wrote: This brings us to the most important question: Who wants to be on the core team? That's the important question indeed. Accepting it (permanently or temporarily) under stackforge is an easy decision. But it's useless unless we have a set of people sufficiently interested in it not bitrotting to volunteer to maintain it... -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Nova service(s) prolem when using Mysql behind HAProxy
Hi, when using Mysql behind Haproxy, i have a problem on reboot when some
nova services start after Haproxy service, but before Mysql service.
These service failed: (i re-checked for sure in /var/log/boot.log)
* Starting Nova cert
[fail]
* Starting Nova conductor
[fail]
* Starting Nova scheduler
[fail]
* Starting Cinder scheduler server
[fail]
I must login to server and re-start these services manually.
When check log of Nova-cert, I saw:
*2013-07-12 15:20:05.020 2490 CRITICAL nova [-] (OperationalError) (2013,
Lost connection to MySQL server at 'reading initial communic
ation packet', system error: 0) None None*
2013-07-12 15:20:05.020 2490 TRACE nova Traceback (most recent call last):
2013-07-12 15:20:05.020 2490 TRACE nova File /usr/bin/nova-cert, line
51, in module
2013-07-12 15:20:05.020 2490 TRACE nova service.wait()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/service.py, line 689, in wait
2013-07-12 15:20:05.020 2490 TRACE nova _launcher.wait()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/service.py, line 209, in wait
2013-07-12 15:20:05.020 2490 TRACE nova super(ServiceLauncher,
self).wait()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/service.py, line 179, in wait
2013-07-12 15:20:05.020 2490 TRACE nova service.wait()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/eventlet/greenthread.py, line 168, in
wait
2013-07-12 15:20:05.020 2490 TRACE nova return self._exit_event.wait()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/eventlet/event.py, line 116, in wait
2013-07-12 15:20:05.020 2490 TRACE nova return hubs.get_hub().switch()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py, line 187, in switch
2013-07-12 15:20:05.020 2490 TRACE nova return self.greenlet.switch()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/eventlet/greenthread.py, line 194, in
main
2013-07-12 15:20:05.020 2490 TRACE nova result = function(*args,
**kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/service.py, line 147, in run_server
2013-07-12 15:20:05.020 2490 TRACE nova server.start()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/service.py, line 434, in start
2013-07-12 15:20:05.020 2490 TRACE nova self.host, self.binary)
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/conductor/api.py, line 261, in
service_get_by_a
rgs
2013-07-12 15:20:05.020 2490 TRACE nova binary=binary)
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/utils.py, line 1348, in wrapper
2013-07-12 15:20:05.020 2490 TRACE nova return func(*args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py,
line 424, in in
ner
2013-07-12 15:20:05.020 2490 TRACE nova return
catch_client_exception(exceptions, func, *args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py,
line 407, in ca
tch_client_exception
2013-07-12 15:20:05.020 2490 TRACE nova return func(*args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/conductor/manager.py, line 325, in
service_get_
all_by
2013-07-12 15:20:05.020 2490 TRACE nova result =
self.db.service_get_by_args(context, host, binary)
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/db/api.py, line 155, in
service_get_by_args
2013-07-12 15:20:05.020 2490 TRACE nova return
IMPL.service_get_by_args(context, host, binary)
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py, line 96, in
wrapper
2013-07-12 15:20:05.020 2490 TRACE nova return f(*args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py, line 409, in
service_get_by_args
2013-07-12 15:20:05.020 2490 TRACE nova result = model_query(context,
models.Service).\
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py, line 177, in
model_query
2013-07-12 15:20:05.020 2490 TRACE nova session = kwargs.get('session')
or get_session()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py,
line 325, in get_session
2013-07-12 15:20:05.020 2490 TRACE nova engine = get_engine()
2013-07-12 15:20:05.020 2490 TRACE nova File
/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py,
line 446, in get_engine
2013-07-12 15:20:05.020 2490 TRACE nova _ENGINE =
create_engine(CONF.sql_connection)
Re: [openstack-dev] [Openstack] Improve inject network configuration
Brian Lamar wrote: Honestly, I think network injection is evil and I'd rather remove it completely. I'm certainly not too interested in trying to add more features to it. Can you elaborate on this a little more? Do you not like file injection or dynamic network allocation? It's an old discussion... in summary: Nova inserting stuff pre-booting into the VM it runs = evil, brittle and the source of countless past vulnerabilities VMs auto-configuring at boot-time using cloud-init based on data provided through generic input channels (config drive, metadata servers...) = good So this is not about disliking the ability to insert files or specify network parameters for a VM, it's about who is in charge of actually creating files and network configurations. Nova shouldn't have to learn about the specificities of the VM image it runs, nor should it have to mount VM filesystems before booting them. The VM itself should take care of the translation based on standardized input (if it wants to). Can you provide alternative strategies that could be applied to solve the issue of dynamically brining up interfaces or do you think this is out of the project scope (controlling the internals of VMs)? Config-drive should pass that config to the VM, and cloud-init on the VM should pick it up. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack] Improve inject network configuration
On 12 July 2013 20:43, Thierry Carrez thie...@openstack.org wrote: Brian Lamar wrote: Honestly, I think network injection is evil and I'd rather remove it completely. I'm certainly not too interested in trying to add more features to it. Can you elaborate on this a little more? Do you not like file injection or dynamic network allocation? It's an old discussion... in summary: Nova inserting stuff pre-booting into the VM it runs = evil, brittle and the source of countless past vulnerabilities VMs auto-configuring at boot-time using cloud-init based on data provided through generic input channels (config drive, metadata servers...) = good So this is not about disliking the ability to insert files or specify network parameters for a VM, it's about who is in charge of actually creating files and network configurations. Nova shouldn't have to learn about the specificities of the VM image it runs, nor should it have to mount VM filesystems before booting them. The VM itself should take care of the translation based on standardized input (if it wants to). Can you provide alternative strategies that could be applied to solve the issue of dynamically brining up interfaces or do you think this is out of the project scope (controlling the internals of VMs)? Config-drive should pass that config to the VM, and cloud-init on the VM should pick it up. Or the instance should just auto-adjust. Chris Jones wrote some code for that for tripleo, but we can't use it until we can disable file injection... and I can't find where we stashed it. Chris? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Savanna-all] installation problem
It seems that you're using the latest code from Savanna v0.2, but following instructions from Savanna v0.1. Please follow these docs: https://savanna.readthedocs.org/en/latest/ Ruslan On Friday, July 12, 2013 at 1:19 PM, Arindam Choudhury wrote: Hi, While installing savanna I am getting this error: [arindam@sl-3 savanna]$ tox -evenv -- bin/savanna-db-manage --config-file etc/savanna/savanna.conf reset-db --with-gen-templates GLOB sdist-make: /home/arindam/savanna/setup.py venv inst-nodeps: /home/arindam/savanna/.tox/dist/savanna-0.2.a12.gf85d74f.zip venv runtests: commands[0] | bin/savanna-db-manage --config-file etc/savanna/savanna.conf reset-db --with-gen-templates ERROR: InvocationError: could not find executable 'bin/savanna-db-manage' _ summary __ ERROR: venv: commands failed Regards, Arindam -- Mailing list: https://launchpad.net/~savanna-all Post to : savanna-...@lists.launchpad.net (mailto:savanna-...@lists.launchpad.net) Unsubscribe : https://launchpad.net/~savanna-all More help : https://help.launchpad.net/ListHelp ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Campus Network Blueprint
Hi Filipe: I disagree your ml2-external-port BP It is unsuitable to connect multiple l2 networks directly, there may be ip conflict, dhcp conflict and other problems. although neutron dhcp agent won't respond dhcp request from unknown source, an external dhcp may respond vm dhcp request. If we move an external port form a network to another network, how can we ensure that the arp cache is cleared. And it will aslo make l2-population bp ( https://blueprints.launchpad.net/quantum/+spec/l2-population ) more difficault. Our l2 forwarding works better if the device knows the whole topology of the network, but the external part is totally unknown. So, I suggest a layer-3 solution, where the out world connects to vms via l3 agent. Thank you for sharing the idea Regards ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Savanna-all] installation problem
That's ok. You need to specify tenant id and provide auth token in headers. You can find detailed how-tos here https://savanna.readthedocs.org/en/latest/devref/quickstart.html btw, please use openstack-dev mail-list for Savanna-related questions. Just prefix mail subject with [Savanna]. Ruslan On Friday, July 12, 2013 at 1:59 PM, Arindam Choudhury wrote: Hi, The new document works great except this command $ sudo pip install savannadashboard. I changed the savanna config to: [DEFAULT] # REST API config port=8386 allow_cluster_ops=true # Address and credentials that will be used to check auth tokens os_auth_host=192.168.122.11 os_auth_port=35357 os_admin_username=admin os_admin_password=openstack os_admin_tenant_name=admin # (Optional) Name of log file to output to. If not set, # logging will go to stdout. (string value) log_file=/home/arindam/savanna.log [cluster_node] # An existing user on Hadoop image (string value) #username=root # User's password (string value) #password=swordfish # When set to false, Savanna uses only internal IP of VMs. # When set to true, Savanna expects OpenStack to auto-assign # floating IPs to cluster nodes. Internal IPs will be used for # inter-cluster communication, while floating ones will be # used by Savanna to configure nodes. Also floating IPs will # be exposed in service URLs (boolean value) use_floating_ips=true [sqlalchemy] # URL for sqlalchemy database (string value) database_uri=sqlite:tmp/savanna-server.db and I changed config of dashboard as specified. but I can not access the savanna dashboard: [arindam@sl-3 ~]$ curl http://localhost:8386/v1.0 html head title401 Unauthorized/title /head body h1401 Unauthorized/h1 This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required.br /br / Authentication required /body /html From: arin...@live.com (mailto:arin...@live.com) To: rkamaldi...@mirantis.com (mailto:rkamaldi...@mirantis.com) Subject: RE: [Savanna-all] installation problem Date: Fri, 12 Jul 2013 11:33:24 +0200 Thanks. Date: Fri, 12 Jul 2013 13:25:45 +0400 From: rkamaldi...@mirantis.com (mailto:rkamaldi...@mirantis.com) To: arin...@live.com (mailto:arin...@live.com) CC: openstack-dev@lists.openstack.org (mailto:openstack-dev@lists.openstack.org); savanna-...@lists.launchpad.net (mailto:savanna-...@lists.launchpad.net) Subject: Re: [Savanna-all] installation problem It seems that you're using the latest code from Savanna v0.2, but following instructions from Savanna v0.1. Please follow these docs: https://savanna.readthedocs.org/en/latest/ Ruslan On Friday, July 12, 2013 at 1:19 PM, Arindam Choudhury wrote: Hi, While installing savanna I am getting this error: [arindam@sl-3 savanna]$ tox -evenv -- bin/savanna-db-manage --config-file etc/savanna/savanna.conf reset-db --with-gen-templates GLOB sdist-make: /home/arindam/savanna/setup.py venv inst-nodeps: /home/arindam/savanna/.tox/dist/savanna-0.2.a12.gf85d74f.zip venv runtests: commands[0] | bin/savanna-db-manage --config-file etc/savanna/savanna.conf reset-db --with-gen-templates ERROR: InvocationError: could not find executable 'bin/savanna-db-manage' _ summary __ ERROR: venv: commands failed Regards, Arindam -- Mailing list: https://launchpad.net/~savanna-all Post to : savanna-...@lists.launchpad.net (mailto:savanna-...@lists.launchpad.net) Unsubscribe : https://launchpad.net/~savanna-all More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~savanna-all Post to : savanna-...@lists.launchpad.net (mailto:savanna-...@lists.launchpad.net) Unsubscribe : https://launchpad.net/~savanna-all More help : https://help.launchpad.net/ListHelp ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] mid-cycle sprint?
On Thu, Jul 11, 2013 at 9:43 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Robert Collins's message of 2013-07-10 20:54:26 -0700: Clint suggested we do a mid-cycle sprint at the weekly meeting a fortnight ago, but ETIME and stuff - so I'm following up. HP would be delighted to host a get-together of TripleO contributors [or 'I will be contributing soon, honest'] folk. We're proposing a late August / early Sept time - a couple weeks before H3, so we can be dealing with features that have landed // ensuring necessary features *do* land. That would give a start date of the 19th or 26th August. Probable venue of either Sunnyvale, CA or Seattle. Wife booked a trip out of town August 27 - Sep 2 so that time frame is unavailable to me. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev I'll be in DebConf13 (Switzerland) the week of August 11th-17th so I prefer the week of the 19th. Ghe Rivero -- Pinky: Gee, Brain, what do you want to do tonight? The Brain: The same thing we do every night, Pinky—try to take over the world! .''`. Pienso, Luego Incordio : :' : `. `' `-www.debian.orgwww.openstack.com GPG Key: 26F020F7 GPG fingerprint: 4986 39DA D152 050B 4699 9A71 66DB 5A36 26F0 20F7 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer
On 07/12/2013 04:29 AM, Thierry Carrez wrote: Monty Taylor wrote: This brings us to the most important question: Who wants to be on the core team? That's the important question indeed. Accepting it (permanently or temporarily) under stackforge is an easy decision. But it's useless unless we have a set of people sufficiently interested in it not bitrotting to volunteer to maintain it... I'd recommend the nova-db subteam folks, like: jog0, dripton, boris-42 as good people to be +2 on this. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer
Hi Sean, I agree to help with sqlalchemy-migrate until we remove it. But probably there should be one more person mikal Best regards, Boris Pavlovic On Fri, Jul 12, 2013 at 3:31 PM, Sean Dague s...@dague.net wrote: On 07/12/2013 04:29 AM, Thierry Carrez wrote: Monty Taylor wrote: This brings us to the most important question: Who wants to be on the core team? That's the important question indeed. Accepting it (permanently or temporarily) under stackforge is an easy decision. But it's useless unless we have a set of people sufficiently interested in it not bitrotting to volunteer to maintain it... I'd recommend the nova-db subteam folks, like: jog0, dripton, boris-42 as good people to be +2 on this. -Sean -- Sean Dague http://dague.net __**_ OpenStack-dev mailing list OpenStack-dev@lists.openstack.**org OpenStack-dev@lists.openstack.org http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-devhttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack] Improve inject network configuration
On 07/12/2013 01:10 AM, Brian Lamar wrote: Russell Bryant wrote: On 07/11/2013 08:53 PM, Jae Sang Lee wrote: Hi, stackers. When creating vm using multi nics, User should power up the second interface on the instance manually for use second IP. http://docs.openstack.org/trunk/openstack-compute/admin/content/using-multi-nics.html I intend to fix interfaces.template file, so It can be possible power up other interface during booting time automatically. I registered blueprint for this a month ago.(https://blueprints.launchpad.net/nova/+spec/better-network-injection) But not yet approved. If you have permission to approve who read this mail, please approve my blueprint. Honestly, I think network injection is evil and I'd rather remove it completely. I'm certainly not too interested in trying to add more features to it. Can you elaborate on this a little more? Do you not like file injection or dynamic network allocation? File injection in general. Can you provide alternative strategies that could be applied to solve the issue of dynamically brining up interfaces or do you think this is out of the project scope (controlling the internals of VMs)? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Revert Pass instance host-id to Quantum using port bindings extension.
On 07/11/2013 04:30 PM, Aaron Rosen wrote: Hi, I think we should revert this patch that was added here (https://review.openstack.org/#/c/29767/). What this patch does is when nova-compute calls into quantum to create the port it passes in the hostname on which the instance was booted on. The idea of the patch was that providing this information would allow hardware device vendors management stations to allow them to segment the network in a more precise manager (for example automatically trunk the vlan on the physical switch port connected to the compute node on which the vm instance was started). In my opinion I don't think this is the right approach. There are several other ways to get this information of where a specific port lives. For example, in the OVS plugin case the agent running on the nova-compute node can update the port in quantum to provide this information. Alternatively, quantum could query nova using the port.device_id to determine which server the instance is on. My motivation for removing this code is I now have the free cycles to work on https://blueprints.launchpad.net/nova/+spec/nova-api-quantum-create-port discussed here (http://lists.openstack.org/pipermail/openstack-dev/2013-May/009088.html) . This was about moving the quantum port creation from the nova-compute host to nova-api if a network-uuid is passed in. This will allow us to remove all the quantum logic from the nova-compute nodes and simplify orchestration. Thoughts? Aaron, The ml2-portbinding BP I am currently working on depends on nova setting the binding:host_id attribute on a port before accessing binding:vif_type. The ml2 plugin's MechanismDrivers will use the binding:host_id with the agents_db info to see what (if any) L2 agent is running on that host, or what other networking mechanisms might provide connectivity for that host. Based on this, the port's binding:vif_type will be set to the appropriate type for that agent/mechanism. When an L2 agent is involved, the associated ml2 MechanismDriver will use the agent's interface or bridge mapping info to determine whether the agent on that host can connect to any of the port's network's segments, and select the specific segment (network_type, physical_network, segmentation_id) to be used. If there is no connectivity possible on the host (due to either no L2 agent or other applicable mechanism, or no mapping for any of the network's segment's physical_networks), the ml2 plugin will set the binding:vif_type attribute to BINDING_FAILED. Nova will then be able to gracefully put the instance into an error state rather than have the instance boot without the required connectivity. I don't see any problem with nova creating the port before scheduling it to a specific host, but the binding:host_id needs to be set before the binding:vif_type attribute is accessed. Note that the host needs to be determined before the vif_type can be determined, so it is not possible to rely on the agent discovering the VIF, which can't be created until the vif_type is determined. Back when the port binding extension was originally being hashed out, I had suggested using an explicit bind() operation on port that took the host_id as a parameter and returned the vif_type as a result. But the current attribute-based approach was chosen instead. We could consider adding a bind() operation for the next neutron API revision, but I don't see any reason the current attribute-based binding approach cannot work for now. -Bob Best, Aaron ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack] Improve inject network configuration
On 07/12/2013 04:43 AM, Thierry Carrez wrote: Brian Lamar wrote: Honestly, I think network injection is evil and I'd rather remove it completely. I'm certainly not too interested in trying to add more features to it. Can you elaborate on this a little more? Do you not like file injection or dynamic network allocation? It's an old discussion... in summary: Nova inserting stuff pre-booting into the VM it runs = evil, brittle and the source of countless past vulnerabilities VMs auto-configuring at boot-time using cloud-init based on data provided through generic input channels (config drive, metadata servers...) = good So this is not about disliking the ability to insert files or specify network parameters for a VM, it's about who is in charge of actually creating files and network configurations. Nova shouldn't have to learn about the specificities of the VM image it runs, nor should it have to mount VM filesystems before booting them. The VM itself should take care of the translation based on standardized input (if it wants to). Thank you for the nice summary. :-) Can you provide alternative strategies that could be applied to solve the issue of dynamically brining up interfaces or do you think this is out of the project scope (controlling the internals of VMs)? Config-drive should pass that config to the VM, and cloud-init on the VM should pick it up. Or you can use the metadata server instead of config-drive. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer
On 07/12/2013 07:37 AM, Boris Pavlovic wrote: Hi Sean, I agree to help with sqlalchemy-migrate until we remove it. But probably there should be one more person mikal Done. https://github.com/stackforge/sqlalchemy-migrate This is up and active, and it will run unittests. Right now it only runs them on sqlite, but making them run against mysql should be easy (we just need to put the user/pass we use for nova unittests against mysql into the test_db.cfg file - same with postgres) The project is set up to publish docs to RTFD (although there is a bug in that setup right now) and to publish packages to PyPI on tags just like our other projects. Have fun everyone. Best regards, Boris Pavlovic On Fri, Jul 12, 2013 at 3:31 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 07/12/2013 04:29 AM, Thierry Carrez wrote: Monty Taylor wrote: This brings us to the most important question: Who wants to be on the core team? That's the important question indeed. Accepting it (permanently or temporarily) under stackforge is an easy decision. But it's useless unless we have a set of people sufficiently interested in it not bitrotting to volunteer to maintain it... I'd recommend the nova-db subteam folks, like: jog0, dripton, boris-42 as good people to be +2 on this. -Sean -- Sean Dague http://dague.net _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
I'd like top-post and hijack this thread for another exception related
thing:
a) Anyone writing code such as:
try:
blah()
except SomeException:
raise SomeOtherExceptionLeavingOutStackContextFromSomeException
should be mocked ruthlessly.
Ok, mock me ruthlessly then.
Mock. Mock. Mock. Mock.
Part of designing any API is specifying what predictable exceptions it
will raise. For any predictable error condition, you don't want callers
to have to catch random exceptions from the underlying libraries you
might be calling into.
Absolutely. But you don't want to go so overboard that you remove the
ability for a developer to debug it. More importantly, INSIDE of server
code, we're not designing fun apis for the consumption of people we
don't know. There is clearly an API, but we are the consumers of our own
API.
Say if I was designing an image downloading API, I'd do something like
this:
https://gist.github.com/markmc/5973868
Assume there's a tonne more stuff that the API would do. You don't want
callers to have to catch socket.error exceptions and whatever other
exceptions might be thrown.
That works out as:
Traceback (most recent call last):
File t.py, line 20, in module
download_image('localhost', 3366, 'foobar')
File t.py, line 18, in download_image
raise ImageDownloadFailure(host, port, path, e.strerror)
__main__.ImageDownloadFailure: Failed to download foobar from
localhost:3366: Connection refused
Which is a pretty clear exception.
Right, but what if the message from the exception does not give you
enough information to walk down the stack and see it...
Also, I have more than once seen:
class MyIOError(Exception):
pass
try:
s = socket.create_connection((host, port))
except socket.error:
raise MyIOError(something went wrong!)
Which is an extreme example of where my rage comes from, but not a made
up example. I have, actually, seen code exactly like that - in Nova.
BTW - I'd have download_image return None if it can't download the
image, and I'd have it either deal with the socket.error or not locally
at the source. But now we're nitpicking.
But I think what you're saying is missing is the stack trace from the
underlying exception.
YES - and I'll let David's response respond to the details of the rest...
But essentially, what I was really mocking earlier is throwing a new
exception in such a way that you lose the exception propagation path. So
if you throw an exception inside of an except block, you should be sure
that you're passing on all of the info, because if a developer gets an
exception, it's quite likely that they want to know how to fix it. :)
As I understood it, Python doesn't have a way of chaining exceptions
like this but e.g. Java does. A little bit more poking right now shows
up this:
http://www.python.org/dev/peps/pep-3134/
i.e. we can't do the right thing until Python 3, where we'd do:
def download_image(host, port, path):
try:
s = socket.create_connection((host, port))
except socket.error as e:
raise ImageDownloadFailure(host, port, path, e.strerror) from e
This will certainly be cleaner to write and read.
I haven't read the PEP in detail yet, though.
Cheers,
Mark.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] savanna version 0.3 - added UI mockups for EDP workflow
I have added some initial UI mockups for version 0.3. Any comments are appreciated. https://wiki.openstack.org/wiki/Savanna/UIMockups/JobCreation Thanks, Chad ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
On 07/11/2013 05:43 AM, Thomas Hervé wrote: On Thu, Jul 11, 2013 at 1:49 AM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: I'd like top-post and hijack this thread for another exception related thing: a) Anyone writing code such as: try: blah() except SomeException: raise SomeOtherExceptionLeavingOutStackContextFromSomeException should be mocked ruthlessly. i don't think mocking is a good way to convey your point. Losing tracebacks is not great, but having an API raising random exceptions is probably worse. Can you develop? Mocking is bad at many things, and email is a bad way to express generalizations through exaggeration. I would almost certainly not actually mock anyone. I doubt very seriously that any of us here working on OpenStack lack the ability to develop. In response to your question though- I find that when the process of debugging an error involves editing an installed library to remove an improperly done exception translation so that I can actually see the traceback and find the error, then someone has made the wrong choice in their implementation of the library. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
On 07/11/2013 05:43 AM, Thomas Hervé wrote: On Thu, Jul 11, 2013 at 1:49 AM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: I'd like top-post and hijack this thread for another exception related thing: a) Anyone writing code such as: try: blah() except SomeException: raise SomeOtherExceptionLeavingOutStackContextFromSomeException should be mocked ruthlessly. i don't think mocking is a good way to convey your point. Losing tracebacks is not great, but having an API raising random exceptions is probably worse. Can you develop? I have learned that I may have mis-read your last three words due to translation problems. You were not asking if I had the ability to write code, rather you were asking if I could elaborate. I will consider that I have learned something new today! Regards, -- Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
On 07/11/2013 06:22 AM, Sean Dague wrote: On 07/11/2013 05:43 AM, Thomas Hervé wrote: On Thu, Jul 11, 2013 at 1:49 AM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: I'd like top-post and hijack this thread for another exception related thing: a) Anyone writing code such as: try: blah() except SomeException: raise SomeOtherExceptionLeavingOutStackContextFromSomeException should be mocked ruthlessly. i don't think mocking is a good way to convey your point. Losing tracebacks is not great, but having an API raising random exceptions is probably worse. Can you develop? We have defined APIs. We have server projects that call other server projects through python-fooclients. We have python-fooclients that generate exceptions and stack traces on any non 20X return. So we have a general exception translation issue. Because unless we want nova-api returns to be completely fluid based on what keystone/neutron/glance/cinder clients decided their exceptions are this week, you have to translate. And the stack trace from a 404 isn't something that we really care about from the client, we care that it happened, because that affects nova logic, but it's an expected exception. It's an expected exception until it's not. And where would we even put the stack trace? take it to the user on their API call? Oh gosh no! I'm CERTAINLY not suggesting that we passthrough translated exceptions to our REST API consumers. I'm talking about actual exceptions in server code which generate actual error logging because they are actual errors but where you cannot find what the error is because of masking. fill up the logs with stack traces for normal events (thus hiding real errors)? (we actually have some blueprints openned now to remove these because a *normal* passing tempest run generates 30+ stack traces in nova logs, which aren't actually internal errors.) Yes. This is actually much worse, and I fully support you in all of these efforts! The bulk of these cases I've seen in Nova are exactly of that nature. We actually have a pretty standard pattern of 3 levels of exception translations for user API calls. Underlying client exception (be it DB / python-fooclient) - Nova internal exception - Webobj exception to create our 40x / 50x returns. I believe striping the underlying exception in the webobj translation is perfectly reasonable. The alternative would be madness. I honestly can't find a great example of this in our code this second (I think concrete examples are great) but I know that I've hit it enough to lodge an annoyance flag - so let me pull an example from the insides of our friend d2to1: try: attrs = cfg_to_args(path) except: e = sys.exc_info()[1] raise DistutilsSetupError( 'Error parsing %s: %s: %s' % (path, e.__class__.__name__, e.args[0])) This is an attempt to give the user a meaningful error in the case that they have a problem. So - first thing is, there's a bug, becuase e.args[0] is the error code, so you get: error in setup command: Error parsing setup.cfg: IOError: 2 Which is TOTALLY meaningless. If you change the string line to: +'Error parsing %s: %s: %s' % (path, e.__class__.__name__, e)) You at least get: error in setup command: Error parsing setup.cfg: IOError: No such file or directory: 'README' Which is much more helpful. However, that's a simple error, if you have a more complex error, such as your pbr hook threw a traceback while loading, it's a bit bonghits and to track it down you may have to expend some effort. BUT - for the general case, you have a typo in your setup.cfg file, IOError: No such file or directory: 'README' is probably sufficient and you don't probably need the traceback. That said - when I'm having problems using this code, I tend to just edit that file, remove that exception wrapper altogether, and let the exceptions fly- and can usually find the problem in about a second. Maybe I should add a debug flag which skips the wrapping... ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer
On 07/12/2013 07:31 AM, Sean Dague wrote: On 07/12/2013 04:29 AM, Thierry Carrez wrote: Monty Taylor wrote: This brings us to the most important question: Who wants to be on the core team? That's the important question indeed. Accepting it (permanently or temporarily) under stackforge is an easy decision. But it's useless unless we have a set of people sufficiently interested in it not bitrotting to volunteer to maintain it... I'd recommend the nova-db subteam folks, like: jog0, dripton, boris-42 as good people to be +2 on this. I'm happy to help, as long as there are at least 2 others. -- David Ripton Red Hat drip...@redhat.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
On Fri, Jul 12, 2013 at 5:33 PM, Monty Taylor mord...@inaugust.com wrote: On 07/11/2013 05:43 AM, Thomas Hervé wrote: On Thu, Jul 11, 2013 at 1:49 AM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: I'd like top-post and hijack this thread for another exception related thing: a) Anyone writing code such as: try: blah() except SomeException: raise SomeOtherExceptionLeavingOutStackContextFromSomeException should be mocked ruthlessly. i don't think mocking is a good way to convey your point. Losing tracebacks is not great, but having an API raising random exceptions is probably worse. Can you develop? I have learned that I may have mis-read your last three words due to translation problems. You were not asking if I had the ability to write code, rather you were asking if I could elaborate. Ah thanks, and sorry for the frenchism. I think I understand your point now, which is not so much about tracebacks but about context in the wide sense, ie enough information to understand what's going on. I've found that we're not necessarily doing a great job at testing the error messages: it's nice to know that FooError has been raised, but if the message is 'Error' it leads to what you're describing, where you need to look at the code and potentially change it to debug it. I believe more consistent tests may help that a bit. Cheers, -- Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
There're still keystone client conflict issues: https://review.openstack.org/#/c/36684/ It seems uncapping keystone client(remove upper bound) in some else project first is needed? On Fri, Jul 12, 2013 at 7:25 PM, Sean Dague s...@dague.net wrote: We are working towards uncapping all the clients, with the exception of neutron client, because they need to make some incompatible changes on their next major release. On 07/12/2013 12:12 AM, Gareth wrote: so, what's the final conclusion about this issue? On Fri, Jul 12, 2013 at 12:11 PM, Gareth academicgar...@gmail.com mailto:academicgareth@gmail.**com academicgar...@gmail.com wrote: Thanks, Monty but in my review https://review.openstack.org/#**/c/36684/https://review.openstack.org/#/c/36684/, Doug said we will go without upper bound with those python-*clients and in this one https://review.openstack.org/#**/c/36753/https://review.openstack.org/#/c/36753/, keystoneclient still keep '0.4' and requirements test doesn't fail in keystoneclient (https://jenkins.openstack.**org/job/gate-cinder-** requirements/96/consolehttps://jenkins.openstack.org/job/gate-cinder-requirements/96/console it failed on glanceclient) On Fri, Jul 12, 2013 at 11:54 AM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: On 07/11/2013 11:38 PM, Gareth wrote: I heard there's a talk about this issue in #openstack-infra last night (china standard time), what's the conclusion of that? BTW, how to find meeting log of #openstack-infra? I didn't find it in http://eavesdrop.openstack.**org/http://eavesdrop.openstack.org/ We don't log it currently. There is a wider conversation going on about which things we should log and which things we should not log ... but for the time being I've submitted this: https://review.openstack.org/**36773https://review.openstack.org/36773 to add -infra. I think we talk about enough things that have ramifications on everyone in there that we should really capture it. On Thu, Jul 11, 2013 at 11:35 PM, Dirk Müller d...@dmllr.de mailto:d...@dmllr.de mailto:d...@dmllr.de mailto:d...@dmllr.de wrote: See for example https://bugs.launchpad.net/**horizon/+bug/1196823https://bugs.launchpad.net/horizon/+bug/1196823 This is arguably a deficiency of mox, which (apparently?) doesn't let us mock properties automatically. I agree, but it is just one example. other test-only issues can happen as well. Similar problem: the *client packages are not self-contained, they have pretty strict dependencies on other packages. One case I already run into was a dependency on python-requests: newer python-*client packages (rightfully) require requests = 1.x. running those on a system that has OpenStack services from Grizzly or Folsom installed cause a conflict: there are one or two that require requests to be 1.0. When you run gating on this scenario, I think the same flipping would happen on e.g. requests as well, due to *client or the module being installed in varying order. Greetings, Dirk __**_ OpenStack-dev mailing list OpenStack-dev@lists.openstack.**orgOpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.**openstack.orgOpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.**openstack.orgOpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.**openstack.orgOpenStack-dev@lists.openstack.org http://lists.openstack.org/**cgi-bin/mailman/listinfo/** openstack-devhttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Gareth /Cloud Computing, OpenStack, Fitness, Basketball/ /OpenStack contributor/ /Company: UnitedStack http://www.ustack.com/ /My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me / /and I'll donate $1 or ￥1 to an open organization you specify./ __**_ OpenStack-dev mailing list OpenStack-dev@lists.openstack.**orgOpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.**openstack.orgOpenStack-dev@lists.openstack.org
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor mord...@inaugust.com wrote:
On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
I'd like top-post and hijack this thread for another exception related
thing:
a) Anyone writing code such as:
try:
blah()
except SomeException:
raise SomeOtherExceptionLeavingOutStackContextFromSomeException
should be mocked ruthlessly.
Ok, mock me ruthlessly then.
Mock. Mock. Mock. Mock.
Part of designing any API is specifying what predictable exceptions it
will raise. For any predictable error condition, you don't want callers
to have to catch random exceptions from the underlying libraries you
might be calling into.
Absolutely. But you don't want to go so overboard that you remove the
ability for a developer to debug it. More importantly, INSIDE of server
code, we're not designing fun apis for the consumption of people we
don't know. There is clearly an API, but we are the consumers of our own
API.
Lies! Write everything to be intuitive for new contributors or we won't
have any :(
Say if I was designing an image downloading API, I'd do something like
this:
https://gist.github.com/markmc/5973868
Assume there's a tonne more stuff that the API would do. You don't want
callers to have to catch socket.error exceptions and whatever other
exceptions might be thrown.
That works out as:
Traceback (most recent call last):
File t.py, line 20, in module
download_image('localhost', 3366, 'foobar')
File t.py, line 18, in download_image
raise ImageDownloadFailure(host, port, path, e.strerror)
__main__.ImageDownloadFailure: Failed to download foobar from
localhost:3366: Connection refused
Which is a pretty clear exception.
Right, but what if the message from the exception does not give you
enough information to walk down the stack and see it...
Also, I have more than once seen:
class MyIOError(Exception):
pass
try:
s = socket.create_connection((host, port))
except socket.error:
raise MyIOError(something went wrong!)
Which is an extreme example of where my rage comes from, but not a made
up example. I have, actually, seen code exactly like that - in Nova.
BTW - I'd have download_image return None if it can't download the
image, and I'd have it either deal with the socket.error or not locally
at the source. But now we're nitpicking.
But I think what you're saying is missing is the stack trace from the
underlying exception.
YES - and I'll let David's response respond to the details of the rest...
But essentially, what I was really mocking earlier is throwing a new
exception in such a way that you lose the exception propagation path. So
if you throw an exception inside of an except block, you should be sure
that you're passing on all of the info, because if a developer gets an
exception, it's quite likely that they want to know how to fix it. :)
As I understood it, Python doesn't have a way of chaining exceptions
like this but e.g. Java does. A little bit more poking right now shows
up this:
http://www.python.org/dev/peps/pep-3134/
i.e. we can't do the right thing until Python 3, where we'd do:
def download_image(host, port, path):
try:
s = socket.create_connection((host, port))
except socket.error as e:
raise ImageDownloadFailure(host, port, path, e.strerror) from e
This will certainly be cleaner to write and read.
I haven't read the PEP in detail yet, though.
Cheers,
Mark.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] mid-cycle sprint?
Hey I'm away from 5th-15th August and if possible I would prefer to be at home for 11th September. I'll start putting out feelers for family to come and stay to help with childcare while I'm out :) No preferences on the location. Cheers, Chris On 12 July 2013 17:02, Devananda van der Veen devananda@gmail.comwrote: Neither week is possible for me. There's that thing in the desert... So while I'd like to attend, I don't think my absence will affect the rest of you being productive :) Cheers, Devananda On Wed, Jul 10, 2013 at 8:54 PM, Robert Collins robe...@robertcollins.net wrote: Clint suggested we do a mid-cycle sprint at the weekly meeting a fortnight ago, but ETIME and stuff - so I'm following up. HP would be delighted to host a get-together of TripleO contributors [or 'I will be contributing soon, honest'] folk. We're proposing a late August / early Sept time - a couple weeks before H3, so we can be dealing with features that have landed // ensuring necessary features *do* land. That would give a start date of the 19th or 26th August. Probable venue of either Sunnyvale, CA or Seattle. I need a rough count of numbers to kick off the approval and final venue stuff w/in HP. I've cc'd some fairly obvious folk that should come :) So - who is interested and would come, and what constraints do you have? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
Hi folks
Monty
Thank you for your adding good topic for this.
I agree, hiding true cause by exception get hard to debug
For, example, Sqlalchemy gives me this error for general sql errors..
exceptions must be old-style classes or derived from BaseException, not str
(It should be hidden from REST API users though)
Also, I agree with you about log policies.
sometimes 404 get warn log..
IMO, The log more than warn level should help Operators.
also If the exception is truly, the exceptional case which needs help
of operators.
It should be logged as error level.
Thanks
Nachi
2013/7/12 Dolph Mathews dolph.math...@gmail.com:
On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor mord...@inaugust.com wrote:
On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
I'd like top-post and hijack this thread for another exception related
thing:
a) Anyone writing code such as:
try:
blah()
except SomeException:
raise SomeOtherExceptionLeavingOutStackContextFromSomeException
should be mocked ruthlessly.
Ok, mock me ruthlessly then.
Mock. Mock. Mock. Mock.
Part of designing any API is specifying what predictable exceptions it
will raise. For any predictable error condition, you don't want callers
to have to catch random exceptions from the underlying libraries you
might be calling into.
Absolutely. But you don't want to go so overboard that you remove the
ability for a developer to debug it. More importantly, INSIDE of server
code, we're not designing fun apis for the consumption of people we
don't know. There is clearly an API, but we are the consumers of our own
API.
Lies! Write everything to be intuitive for new contributors or we won't have
any :(
Say if I was designing an image downloading API, I'd do something like
this:
https://gist.github.com/markmc/5973868
Assume there's a tonne more stuff that the API would do. You don't want
callers to have to catch socket.error exceptions and whatever other
exceptions might be thrown.
That works out as:
Traceback (most recent call last):
File t.py, line 20, in module
download_image('localhost', 3366, 'foobar')
File t.py, line 18, in download_image
raise ImageDownloadFailure(host, port, path, e.strerror)
__main__.ImageDownloadFailure: Failed to download foobar from
localhost:3366: Connection refused
Which is a pretty clear exception.
Right, but what if the message from the exception does not give you
enough information to walk down the stack and see it...
Also, I have more than once seen:
class MyIOError(Exception):
pass
try:
s = socket.create_connection((host, port))
except socket.error:
raise MyIOError(something went wrong!)
Which is an extreme example of where my rage comes from, but not a made
up example. I have, actually, seen code exactly like that - in Nova.
BTW - I'd have download_image return None if it can't download the
image, and I'd have it either deal with the socket.error or not locally
at the source. But now we're nitpicking.
But I think what you're saying is missing is the stack trace from the
underlying exception.
YES - and I'll let David's response respond to the details of the rest...
But essentially, what I was really mocking earlier is throwing a new
exception in such a way that you lose the exception propagation path. So
if you throw an exception inside of an except block, you should be sure
that you're passing on all of the info, because if a developer gets an
exception, it's quite likely that they want to know how to fix it. :)
As I understood it, Python doesn't have a way of chaining exceptions
like this but e.g. Java does. A little bit more poking right now shows
up this:
http://www.python.org/dev/peps/pep-3134/
i.e. we can't do the right thing until Python 3, where we'd do:
def download_image(host, port, path):
try:
s = socket.create_connection((host, port))
except socket.error as e:
raise ImageDownloadFailure(host, port, path, e.strerror) from e
This will certainly be cleaner to write and read.
I haven't read the PEP in detail yet, though.
Cheers,
Mark.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
Re: [openstack-dev] [TripleO] mid-cycle sprint?
Excerpts from Devananda van der Veen's message of 2013-07-12 09:02:49 -0700: Neither week is possible for me. There's that thing in the desert... So while I'd like to attend, I don't think my absence will affect the rest of you being productive :) I see no reason to write off the Mojave desert in August as a potential sprint location. What could possibly go wrong? I am unavailable 8/26 - 9/5.. so the week before that would be better for me. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Tempest testing for optional middlewares
Hello everyone, I'm addressing this email to the dev list because I couldn't find a way to get in touch with the testing team. Hopefully someone here will have the answer to my question or can point me to the correct people to ask. I am writing Tempest tests that cover the behavior of some optional Swift middlewares (precisely account_quota and container_quota). It occurs to me that these middlewares are optional and may not be present in every Swift installation. In this case, I'd like Tempest to skip this test rather than fail it. What's the correct way of detecting the presence of the middleware before launching the test? Thank you, Joe H. Rahme ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Savanna-all] Cluster scaling discussion
A comment on how you go about this. I believe you've already run into issues w/ using the start/stop-*.sh scripts as a foundation for this feature. Long term I believe that an active cluster need not mean every instance is up and running. The core infrastructure must be (ambari + jt + nn), and some % of worker instances (jt + dn). For example, if I want to make a 500 instance cluster, I won't need to wait for all 500 instances before I can reasonably start using the cluster. In fact, I may never have 500 instances at any given time, 98% may be acceptable operating procedure. The start/stop-*.sh scripts are not good for that use case either. However you go about this, keep the 98% cluster use case in mind. Best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
Somewhat offtopic, but others might find it interesting. On another project
I used a different technique for exceptions, where the original exception
is enhanced with new fields as it propagates up the stack to where it's
handled. In this way all the information needed to handle the exception is
available (even if it's just to log it).
Often there's some information that would be useful for the exception
handler that isn't available at the place where the exception is raised.
The classic example is an error writing to a file where you'd like to know
the filename but all you've got is the file descriptor. An example from an
OpenStack REST API might be that you get an exception and you'd like to log
a message that includes the resource path, but the resource path isn't
known at this point.
So rather than logging the exception when it happens, enhance the
exception, re-raise it, and only once it's got all the information where
you can print out a useful log message you log it.
Here's a short example to illustrate. An exception is raised by f1(), but
at this point we don't know the status code that the server should respond
with or the request path which would be useful in a log message. These bits
of information are added as the exception propagates and then where it's
caught we've got all the information for a useful log message.
def f1():
raise Exception('something')
def f2():
try:
f1()
except Exception as e:
e.status_code = 403
raise
def do_tokens():
try:
f2()
except Exception as e:
e.req_url = '/v3/tokens'
raise
status_code = 200
try:
do_tokens()
except Exception as e:
status_code = getattr(e, 'status_code', 500)
req_url = getattr(e, 'req_url', 'unknown')
if status_code != 200:
print 'Got %s from %s' % (status_code, req_url)
# build an error document for the response.
On Fri, Jul 12, 2013 at 12:25 PM, Nachi Ueno na...@ntti3.com wrote:
Hi folks
Monty
Thank you for your adding good topic for this.
I agree, hiding true cause by exception get hard to debug
For, example, Sqlalchemy gives me this error for general sql errors..
exceptions must be old-style classes or derived from BaseException, not
str
(It should be hidden from REST API users though)
Also, I agree with you about log policies.
sometimes 404 get warn log..
IMO, The log more than warn level should help Operators.
also If the exception is truly, the exceptional case which needs help
of operators.
It should be logged as error level.
Thanks
Nachi
2013/7/12 Dolph Mathews dolph.math...@gmail.com:
On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor mord...@inaugust.com
wrote:
On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
I'd like top-post and hijack this thread for another exception
related
thing:
a) Anyone writing code such as:
try:
blah()
except SomeException:
raise SomeOtherExceptionLeavingOutStackContextFromSomeException
should be mocked ruthlessly.
Ok, mock me ruthlessly then.
Mock. Mock. Mock. Mock.
Part of designing any API is specifying what predictable exceptions it
will raise. For any predictable error condition, you don't want
callers
to have to catch random exceptions from the underlying libraries you
might be calling into.
Absolutely. But you don't want to go so overboard that you remove the
ability for a developer to debug it. More importantly, INSIDE of server
code, we're not designing fun apis for the consumption of people we
don't know. There is clearly an API, but we are the consumers of our own
API.
Lies! Write everything to be intuitive for new contributors or we won't
have
any :(
Say if I was designing an image downloading API, I'd do something like
this:
https://gist.github.com/markmc/5973868
Assume there's a tonne more stuff that the API would do. You don't
want
callers to have to catch socket.error exceptions and whatever other
exceptions might be thrown.
That works out as:
Traceback (most recent call last):
File t.py, line 20, in module
download_image('localhost', 3366, 'foobar')
File t.py, line 18, in download_image
raise ImageDownloadFailure(host, port, path, e.strerror)
__main__.ImageDownloadFailure: Failed to download foobar from
localhost:3366: Connection refused
Which is a pretty clear exception.
Right, but what if the message from the exception does not give you
enough information to walk down the stack and see it...
Also, I have more than once seen:
class MyIOError(Exception):
pass
try:
s = socket.create_connection((host, port))
except socket.error:
raise MyIOError(something went wrong!)
Which is an extreme example of where my rage comes from, but not a made
up example. I have, actually, seen code exactly like that - in Nova.
BTW - I'd have
[openstack-dev] Program Description for OpenStack QA
For general review (took us a little longer, we needed to elected a PTL first). Official Program Name: OpenStack Quality Assurance PTL: Sean Dague Mission Statement: Develop, maintain, and initiate tools and plans to ensure the upstream stability and quality of OpenStack, and its release readiness at any point during the release cycle. The OpenStack QA program starts with 2 git trees * tempest - https://github.com/openstack/tempest * grenade - https://github.com/openstack-dev/grenade We'll get more info up on the wiki over the coming weeks to fill out full description. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] savanna version 0.3 - added UI mockups for EDP workflow
On the tab with parameters, we see case for Hadoop streaming API. Could you please add more examples for parameters tab including cases for Hadoop jar, Pig and Hive scripts? Thanks, Alexander Kuznetsov. On Fri, Jul 12, 2013 at 7:14 PM, Chad Roberts crobe...@redhat.com wrote: I have added some initial UI mockups for version 0.3. Any comments are appreciated. https://wiki.openstack.org/wiki/Savanna/UIMockups/JobCreation Thanks, Chad ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Revert Pass instance host-id to Quantum using port bindings extension.
Hi, On Fri, Jul 12, 2013 at 6:47 AM, Robert Kukura rkuk...@redhat.com wrote: On 07/11/2013 04:30 PM, Aaron Rosen wrote: Hi, I think we should revert this patch that was added here (https://review.openstack.org/#/c/29767/). What this patch does is when nova-compute calls into quantum to create the port it passes in the hostname on which the instance was booted on. The idea of the patch was that providing this information would allow hardware device vendors management stations to allow them to segment the network in a more precise manager (for example automatically trunk the vlan on the physical switch port connected to the compute node on which the vm instance was started). In my opinion I don't think this is the right approach. There are several other ways to get this information of where a specific port lives. For example, in the OVS plugin case the agent running on the nova-compute node can update the port in quantum to provide this information. Alternatively, quantum could query nova using the port.device_id to determine which server the instance is on. My motivation for removing this code is I now have the free cycles to work on https://blueprints.launchpad.net/nova/+spec/nova-api-quantum-create-port discussed here (http://lists.openstack.org/pipermail/openstack-dev/2013-May/009088.html) . This was about moving the quantum port creation from the nova-compute host to nova-api if a network-uuid is passed in. This will allow us to remove all the quantum logic from the nova-compute nodes and simplify orchestration. Thoughts? Aaron, The ml2-portbinding BP I am currently working on depends on nova setting the binding:host_id attribute on a port before accessing binding:vif_type. The ml2 plugin's MechanismDrivers will use the binding:host_id with the agents_db info to see what (if any) L2 agent is running on that host, or what other networking mechanisms might provide connectivity for that host. Based on this, the port's binding:vif_type will be set to the appropriate type for that agent/mechanism. When an L2 agent is involved, the associated ml2 MechanismDriver will use the agent's interface or bridge mapping info to determine whether the agent on that host can connect to any of the port's network's segments, and select the specific segment (network_type, physical_network, segmentation_id) to be used. If there is no connectivity possible on the host (due to either no L2 agent or other applicable mechanism, or no mapping for any of the network's segment's physical_networks), the ml2 plugin will set the binding:vif_type attribute to BINDING_FAILED. Nova will then be able to gracefully put the instance into an error state rather than have the instance boot without the required connectivity. I don't see any problem with nova creating the port before scheduling it to a specific host, but the binding:host_id needs to be set before the binding:vif_type attribute is accessed. Note that the host needs to be determined before the vif_type can be determined, so it is not possible to rely on the agent discovering the VIF, which can't be created until the vif_type is determined. So what your saying is the current workflow is this: nova-compute creates a port in quantum passing in the host-id (which is the hostname of the compute host). Now quantum looks in the agent table in it's database to determine the VIF type that should be used based on the agent that is running on the nova-compute node? My question would be why the nova-compute node doesn't already know which VIF_TYPE it should be using? Back when the port binding extension was originally being hashed out, I had suggested using an explicit bind() operation on port that took the host_id as a parameter and returned the vif_type as a result. But the current attribute-based approach was chosen instead. We could consider adding a bind() operation for the next neutron API revision, but I don't see any reason the current attribute-based binding approach cannot work for now. -Bob Best, Aaron ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] sqlite doesn't support migrations
On 07/11/2013 01:12 PM, Dolph Mathews wrote: Just as a general statement, outside the scope of openstack, I don't think sqlite is intended to support schema evolution. From the sqlite docs [1]: SQLite supports a limited subset of ALTER TABLE. [...] It is not possible to rename a column, remove a column, or add or remove constraints from a table. We've been through hell trying to support migrations on sqlite, because we test against sqlite, and because we test our migrations... on sqlite. So, we've already shot ourselves in the foot. We're clearly moving towards gating against mysql + postgresql, so in the mean time, let's limit the amount of effort we put into further support sqlite migrations until we can safely rip it out altogether. [1]: http://www.sqlite.org/lang_altertable.html I agree. The reason to use sqlite in unitests and stuff is because it's easy and doesn't require users and system things and everything. If we're spending extra effort to maintain the simple thing, then it's probably not a simple thing. As an aside, (ignore the fact that I'm a former Drizzle core dev) it might be worthwhile taking 30 minutes one day and exploring a drizzle database test fixture. One of the things we did in drizzle was make it not need any bootstrapping and to work sanely with no config files ... so launching a drizzle on a spare port, running database tests against it and then deleting it should actually be super simple - and at the worst no harder than doing what glance does in their functional tests. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [OSLO] Comments/Questions on Messaging Wiki
Hi all, I've been reading through the Messaging Wiki and have some comments. Not criticisms, just comments and questions. I have found this to be a very useful document. Thanks. 1. There are multiple backend transport drivers which implement the API semantics using different messaging systems - e.g. RabbitMQ, Qpid, ZeroMQ. While both sides of a connection must use the same transport driver configured in the same way, the API avoids exposing details of transports so that code written using one transport should work with any other transport. The good news for AMQP 1.0 users is that technically boths sides of the connection do not have to use same transport driver. In pre-AMQP 1.0 days this was the case. But today interoperability between AMQP 1.0 implementations has been demonstrated. 2. I notice under the RPC concepts section that you mention Exchanges as a container in which topics are scoped. Is this exchange a pre AMQP 1.0 artifact or just a general term for oslo.messaging that is loosely based on the pre-AMQP 1.0 artifact called an Exchange? i.e. are you assuming that messaging implementations have something called an exchange? Or do you mean that messaging implementations can scope a topic and in oslo we call that scoping an exchange? 3. Some messaging nomenclature: The way the wiki describes RPC Invoke Method on One of Multiple Servers is more like a queue than a topic. In messaging a queue is something that multiple consumers can attach to and one of them gets and services a message/request. A topic is where 1+ consumers are connected and each receives a the message and each can service it as it sees fit. In pre-AMQP 1.0 terms what this seems to describe is a direct exchange. And a direct excahnge can have multiple consumers listening to a queue on that exchange. (Remember that fanout is just a generalization of topic in that all consumers get all fanout messages - there are no sub-topics etc.) In AMQP 1.0 the addressing doesn't care or know about exchanges but it can support this queue type behavior on an address or topic type behavior on an address. I know this isn't about AMQP specifically but therefore this is even more important. Topics are pub/sub with multiple consumer/services responding to a single message. Queues are next consumer up gets the next message. (BTW I've seen this kind of confusion also in early versions of MCollective in Puppet.) It might be better to change some of the references to topic to address. This would solve the problem. i.e. a use case where one of many servers listening on an address services a message/request. And later all of servers listening on an address service a message/request. Addressing also solves the one-to-one as the address is specific to the server (and the others don't have to receive and reject the message). Please feel free to respond and critique my comments/suggestions. Best, William ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OSLO] Comments/Questions on Messaging Wiki
Woops the wiki I am referring to is: https://wiki.openstack.org/wiki/Oslo/Messaging William - Original Message - Hi all, I've been reading through the Messaging Wiki and have some comments. Not criticisms, just comments and questions. I have found this to be a very useful document. Thanks. 1. There are multiple backend transport drivers which implement the API semantics using different messaging systems - e.g. RabbitMQ, Qpid, ZeroMQ. While both sides of a connection must use the same transport driver configured in the same way, the API avoids exposing details of transports so that code written using one transport should work with any other transport. The good news for AMQP 1.0 users is that technically boths sides of the connection do not have to use same transport driver. In pre-AMQP 1.0 days this was the case. But today interoperability between AMQP 1.0 implementations has been demonstrated. 2. I notice under the RPC concepts section that you mention Exchanges as a container in which topics are scoped. Is this exchange a pre AMQP 1.0 artifact or just a general term for oslo.messaging that is loosely based on the pre-AMQP 1.0 artifact called an Exchange? i.e. are you assuming that messaging implementations have something called an exchange? Or do you mean that messaging implementations can scope a topic and in oslo we call that scoping an exchange? 3. Some messaging nomenclature: The way the wiki describes RPC Invoke Method on One of Multiple Servers is more like a queue than a topic. In messaging a queue is something that multiple consumers can attach to and one of them gets and services a message/request. A topic is where 1+ consumers are connected and each receives a the message and each can service it as it sees fit. In pre-AMQP 1.0 terms what this seems to describe is a direct exchange. And a direct excahnge can have multiple consumers listening to a queue on that exchange. (Remember that fanout is just a generalization of topic in that all consumers get all fanout messages - there are no sub-topics etc.) In AMQP 1.0 the addressing doesn't care or know about exchanges but it can support this queue type behavior on an address or topic type behavior on an address. I know this isn't about AMQP specifically but therefore this is even more important. Topics are pub/sub with multiple consumer/services responding to a single message. Queues are next consumer up gets the next message. (BTW I've seen this kind of confusion also in early versions of MCollective in Puppet.) It might be better to change some of the references to topic to address. This would solve the problem. i.e. a use case where one of many servers listening on an address services a message/request. And later all of servers listening on an address service a message/request. Addressing also solves the one-to-one as the address is specific to the server (and the others don't have to receive and reject the message). Please feel free to respond and critique my comments/suggestions. Best, William ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] AMQP Version upgrade plans?
- Original Message - On 07/11/2013 12:06 PM, William Henry wrote: - Original Message - On 07/08/2013 10:51 AM, Ted Ross wrote: If someone from the Qpid community were to work on integrating the new AMQP 1.0 technology into OpenStack, where would be the right place to start? Would it be to add a new transport to oslo.messaging? I think so, yes. oslo.messaging is new, but it will deprecate the existing 'rpc' library in oslo-incubator. All projects will need to move to oslo.messaging, so for something new I would focus efforts there. I think that one of the important points that Ted brought up is that AMQP 1.0 doesn't have the concepts of broker artifacts like exchanges etc. A recent change I proposed to the existing impl_qpid.py which focuses more on addressing and not exchanges is a very important first step to solve several issues: a recent qpidd leak issue, transitioning to AMQP 1.0 (addressing), and possible HA solutions. This is an area I'd really like to continue to help out in. I'm back from some vacation and would like to get stuck in soon. Regarding the qpid exchange leak, that issue is mitigated largely by the fact that the only time we declare a direct exchange is for replies to an rpc. In previous versions there was a new one of these for *every* method call, which made this problem really bad. In the current code, we only create a single one. However, we're still left with a leak. The fact that RabbitMQ supports auto-delete on exchanges and Qpid doesn't is what got us into this spot, since this code works just like the kombu driver with respect to all of this. Right you still have the leak. Right auto-delete on exchanges in Qpid would mitigate. Qpid never supported this because, to my knowledge, auto-delete on exchanges didn't mean anything or have any value in Qpid. I'm not sure it was ever mentioned in the old pre AMQP 1.0 spec. (I'd need to check). And anyway exchanges don't mean anything in AMQP anymore. And don't mean anything in nonAMQP messaging - though AMQP implementations are not prohibited from implementing the queue/topic/fanout using exchange patterns. As for migration to AMQP 1.0, how do these changes help? Supporting AMQP 1.0 requires an entirely new driver that uses Qpid Proton, right? How does changing addressing in the current Qpid driver (that will never do 1.0) help? Actually Qpid project saw the pending AMQP 1.0 ramifications early on and created an addressing mechanism that supported AMQP 1.0 addressing in Qpid long before Qpid/Proton. So the changes I made recently would mean a simplified driver migration process. But you are correct, it would still need that migration to a new driver. It's my opinion that moving from a driver I recently submitted to the new driver would be very simple because of the way I've restructured the addressing. All great points you raise Russell. William I'm curious what you mean by possible HA solutions. Can you elaborate? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OSLO] Comments/Questions on Messaging Wiki
Sent from my iPhone On Jul 12, 2013, at 5:27 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Fri, Jul 12, 2013 at 5:40 PM, William Henry whe...@redhat.com wrote: Hi all, I've been reading through the Messaging Wiki and have some comments. Not criticisms, just comments and questions. I have found this to be a very useful document. Thanks. 1. There are multiple backend transport drivers which implement the API semantics using different messaging systems - e.g. RabbitMQ, Qpid, ZeroMQ. While both sides of a connection must use the same transport driver configured in the same way, the API avoids exposing details of transports so that code written using one transport should work with any other transport. The good news for AMQP 1.0 users is that technically boths sides of the connection do not have to use same transport driver. In pre-AMQP 1.0 days this was the case. But today interoperability between AMQP 1.0 implementations has been demonstrated. In this case I think we mean the Transport driver from within Oslo. So you could not connect a ZMQ Transport on one end to an AMQP Transport on the other. It will be an implementation detail of the AMQP Transport class to decide whether it supports more than one version of AMQP, or if the different versions are implemented as different Transports. 2. I notice under the RPC concepts section that you mention Exchanges as a container in which topics are scoped. Is this exchange a pre AMQP 1.0 artifact or just a general term for oslo.messaging that is loosely based on the pre-AMQP 1.0 artifact called an Exchange? i.e. are you assuming that messaging implementations have something called an exchange? Or do you mean that messaging implementations can scope a topic and in oslo we call that scoping an exchange? The latter. Ack. Good. Fits very well into AMQP 1.0 then ;-) 3. Some messaging nomenclature: The way the wiki describes RPC Invoke Method on One of Multiple Servers is more like a queue than a topic. In messaging a queue is something that multiple consumers can attach to and one of them gets and services a message/request. A topic is where 1+ consumers are connected and each receives a the message and each can service it as it sees fit. In pre-AMQP 1.0 terms what this seems to describe is a direct exchange. And a direct excahnge can have multiple consumers listening to a queue on that exchange. (Remember that fanout is just a generalization of topic in that all consumers get all fanout messages - there are no sub-topics etc.) In AMQP 1.0 the addressing doesn't care or know about exchanges but it can support this queue type behavior on an address or topic type behavior on an address. I know this isn't about AMQP specifically but therefore this is even more important. Topics are pub/sub with multiple consumer/services responding to a single message. Queues are next consumer up gets the next message. (BTW I've seen this kind of confusion also in early versions of MCollective in Puppet.) It might be better to change some of the references to topic to address. This would solve the problem. i.e. a use case where one of many servers listening on an address services a message/request. And later all of servers listening on an address service a message/request. Addressing also solves the one-to-one as the address is specific to the server (and the others don't have to receive and reject the message). Too many of these terms are overloaded. :-) Yep. But topic pup/sub is certainly different to a queue. ;-) I'm not sure of the details of how topic and address are different in AMQP 1.0. The word address implies to me that the message sender knows where the message receiver is in some concrete sense. We don't want those semantics in a lot of our use cases. If the address is abstract, then it sounds like it works much as a topic does. Maybe you can expand on the differences? Nope the address is essentially a namespace. The send knows not where it ends up. Hence in some applications it doesn't even know of its a topic or a queue an it could go to one or many depending. William Thanks, Doug Please feel free to respond and critique my comments/suggestions. Best, William ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Event Service
All, Does open stack have pub/sub event service? I would like to be notified of the event of VM creation/deletion/Migration etc. What is the best way to do this? Thanks, Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Review] Use of exception for non-exceptional cases
I'm curious if folks know about this?
import sys
def foo():
raise Exception('Foo Exception')
def bar():
try:
foo()
except Exception:
exc_info = sys.exc_info()
raise Exception('Bar Exception'), None, exc_info[2]
bar()
Traceback (most recent call last):
File test.py, line 15, in module
bar()
File test.py, line 9, in bar
foo()
File test.py, line 4, in foo
raise Exception('Foo Exception')
Exception: Bar Exception
On Fri, Jul 12, 2013 at 3:53 PM, Doug Hellmann
doug.hellm...@dreamhost.comwrote:
On Fri, Jul 12, 2013 at 2:40 PM, Brant Knudson b...@acm.org wrote:
Somewhat offtopic, but others might find it interesting. On another
project I used a different technique for exceptions, where the original
exception is enhanced with new fields as it propagates up the stack to
where it's handled. In this way all the information needed to handle the
exception is available (even if it's just to log it).
Often there's some information that would be useful for the exception
handler that isn't available at the place where the exception is raised.
The classic example is an error writing to a file where you'd like to know
the filename but all you've got is the file descriptor. An example from an
OpenStack REST API might be that you get an exception and you'd like to log
a message that includes the resource path, but the resource path isn't
known at this point.
So rather than logging the exception when it happens, enhance the
exception, re-raise it, and only once it's got all the information where
you can print out a useful log message you log it.
Here's a short example to illustrate. An exception is raised by f1(), but
at this point we don't know the status code that the server should respond
with or the request path which would be useful in a log message. These bits
of information are added as the exception propagates and then where it's
caught we've got all the information for a useful log message.
def f1():
raise Exception('something')
def f2():
try:
f1()
except Exception as e:
e.status_code = 403
raise
def do_tokens():
try:
f2()
except Exception as e:
e.req_url = '/v3/tokens'
raise
status_code = 200
try:
do_tokens()
except Exception as e:
status_code = getattr(e, 'status_code', 500)
req_url = getattr(e, 'req_url', 'unknown')
if status_code != 200:
print 'Got %s from %s' % (status_code, req_url)
# build an error document for the response.
One problem with this approach is it spreads knowledge of the error
construction up and down the stack through different layers of the
application, and that brings with it assumptions about the implementation
at the different layers. For example, should the application code above
know that do_tokens() is making web requests to URLs? Why does it need that
information?
SRP-ly,
Doug
On Fri, Jul 12, 2013 at 12:25 PM, Nachi Ueno na...@ntti3.com wrote:
Hi folks
Monty
Thank you for your adding good topic for this.
I agree, hiding true cause by exception get hard to debug
For, example, Sqlalchemy gives me this error for general sql errors..
exceptions must be old-style classes or derived from BaseException, not
str
(It should be hidden from REST API users though)
Also, I agree with you about log policies.
sometimes 404 get warn log..
IMO, The log more than warn level should help Operators.
also If the exception is truly, the exceptional case which needs help
of operators.
It should be logged as error level.
Thanks
Nachi
2013/7/12 Dolph Mathews dolph.math...@gmail.com:
On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor mord...@inaugust.com
wrote:
On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
I'd like top-post and hijack this thread for another exception
related
thing:
a) Anyone writing code such as:
try:
blah()
except SomeException:
raise SomeOtherExceptionLeavingOutStackContextFromSomeException
should be mocked ruthlessly.
Ok, mock me ruthlessly then.
Mock. Mock. Mock. Mock.
Part of designing any API is specifying what predictable exceptions
it
will raise. For any predictable error condition, you don't want
callers
to have to catch random exceptions from the underlying libraries you
might be calling into.
Absolutely. But you don't want to go so overboard that you remove the
ability for a developer to debug it. More importantly, INSIDE of
server
code, we're not designing fun apis for the consumption of people we
don't know. There is clearly an API, but we are the consumers of our
own
API.
Lies! Write everything to be intuitive for new contributors or we
won't have
any :(
Say if I was designing an image downloading API, I'd do something
like
this:
https://gist.github.com/markmc/5973868
Assume there's a
Re: [openstack-dev] Event Service
OpenStack has a system called notifications which does what you're looking for. I've never used it, but I am sure its documented. Cheers, Michael On Sat, Jul 13, 2013 at 10:12 AM, Qing He qing...@radisys.com wrote: All, Does open stack have pub/sub event service? I would like to be notified of the event of VM creation/deletion/Migration etc. What is the best way to do this? Thanks, Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Not good idea to specify host in EVACUATE API
Hello, I found it's not good idea to specify host in calling EVACUATE API for HA purpose. In my case, one application includes many VMs, and each of them should run on seperate host, eg. co-location of VMs is not allowed for application robust purpose. The VM is created with scheduler hint to avoiding co-location. If a host is specified in calling evacuate API, then the task to find a proper host (avoiding co-location) has to be done before calling, and the logic is similar with nova scheduler. The HA function must reuse the NOVA scheduler logic, but not to duplicate it outside nova. Your comment is welcome . Joehuang ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Event Service
Thanks Michael! I found it: https://wiki.openstack.org/wiki/NotificationEventExamples -Original Message- From: Michael Still [mailto:mi...@stillhq.com] Sent: Friday, July 12, 2013 6:38 PM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] Event Service OpenStack has a system called notifications which does what you're looking for. I've never used it, but I am sure its documented. Cheers, Michael On Sat, Jul 13, 2013 at 10:12 AM, Qing He qing...@radisys.com wrote: All, Does open stack have pub/sub event service? I would like to be notified of the event of VM creation/deletion/Migration etc. What is the best way to do this? Thanks, Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
