Re: [openstack-dev] [Neutron] Race condition between DB layer and plugin back-end implementation

2013-11-19 Thread Tomoe Sugihara 
Hi Edgar,

We had a similar issue and worked around by something like the following
(which I believe similar to what Aaron said):

@@ -45,6 +45,8 @@ SNAT_RULE_PROPERTY = {OS_TENANT_ROUTER_RULE_KEY: SNAT_RULE}
 class MidonetResourceNotFound(q_exc.NotFound):
 message = _('MidoNet %(resource_type)s %(id)s could not be found')

+from eventlet.semaphore import Semaphore
+PORT_ALLOC_SEM = Semaphore()

 class MidonetPluginV2(db_base_plugin_v2.QuantumDbPluginV2,
   l3_db.L3_NAT_db_mixin):
@@ -428,21 +430,31 @@ class MidonetPluginV2(db_base_plugin_v2.QuantumDbPluginV2,
 # set midonet port id to quantum port id and create a DB record.
 port_data['id'] = bridge_port.get_id()

-session = context.session
-with session.begin(subtransactions=True):
-qport = super(MidonetPluginV2, self).create_port(context, port)
-if is_compute_interface:
-# get ip and mac from DB record.
-fixed_ip = qport['fixed_ips'][0]['ip_address']
-mac = qport['mac_address']
-
+qport = None
+with PORT_ALLOC_SEM:
+session = context.session
+with session.begin(subtransactions=True):
+qport = super(MidonetPluginV2, self).create_port(context, port)
+if is_compute_interface:
+# get ip and mac from DB record.
+id = qport['id']
+fixed_ip = qport['fixed_ips'][0]['ip_address']
+mac = qport['mac_address']
+
+if qport and is_compute_interface:
+try:
 # create dhcp host entry under the bridge.
 dhcp_subnets = bridge.get_dhcp_subnets()
 if len(dhcp_subnets)  0:
 dhcp_subnets[0].add_dhcp_host().ip_addr(fixed_ip)\
.mac_addr(mac)\
.create()
-return qport
+return qport
+except Exception:
+self.delete_port(context, id)
+return None
+else:
+return qport

 def update_port(self, context, id, port):
 


We are also looking to fix this for upstream icehouce.

Also, I have just submitted a (regression) test for this in tempest:
https://review.openstack.org/#/c/57355

Hope the test makes sense.

Thanks,
Tomoe



On Tue, Nov 19, 2013 at 5:25 AM, Edgar Magana emag...@plumgrid.com wrote:

 Developers,

 This topic has been discussed before but I do not remember if we have a
 good solution or not.
 Basically, if concurrent API calls are sent to Neutron, all of them are
 sent to the plug-in level where two actions have to be made:

 1. DB transaction – No just for data persistence but also to collect the
 information needed for the next action
 2. Plug-in back-end implementation – In our case is a call to the python
 library than consequentially calls PLUMgrid REST GW (soon SAL)

 For instance:

 def create_port(self, context, port):
 with context.session.begin(subtransactions=True):
 # Plugin DB - Port Create and Return port
 port_db = super(NeutronPluginPLUMgridV2,
 self).create_port(context,

  port)
 device_id = port_db[device_id]
 if port_db[device_owner] == network:router_gateway:
 router_db = self._get_router(context, device_id)
 else:
 router_db = None
 try:
 LOG.debug(_(PLUMgrid Library: create_port() called))
 # Back-end implementation
 self._plumlib.create_port(port_db, router_db)
 except Exception:
 …

 The way we have implemented at the plugin-level in Havana (even in
 Grizzly) is that both action are wrapped in the same transaction which
 automatically rolls back any operation done to its original state
 protecting mostly the DB of having any inconsistency state or left over
 data if the back-end part fails.=.
 The problem that we are experiencing is when concurrent calls to the same
 API are sent, the number of operation at the plug-in back-end are long
 enough to make the next concurrent API call to get stuck at the DB
 transaction level, which creates a hung state for the Neutron Server to the
 point that all concurrent API calls will fail.

 This can be fixed if we include some locking system such as calling:

 from neutron.common import utile
 …

 @utils.synchronized('any-name', external=True)
 def create_port(self, context, port):
 …

 Obviously, this will create a serialization of all concurrent calls which
 will ends up in having a really bad performance. Does anyone has a better
 solution?

 Thanks,

 Edgar

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack][qa][Tempest][Network] Test for external connectivity

2013-11-19 Thread Tomoe Sugihara 
Hi Salvatore, et al,

On Mon, Nov 18, 2013 at 9:19 PM, Salvatore Orlando sorla...@nicira.comwrote:

 Hi Yair,

 I had in mind of doing something similar. I also registered a tempest
 blueprint for it.
 Basically I think we can assume test machines have access to the Internet,
 but the devstack deployment might not be able to route packets from VMs to
 the Internet.

 Being able to ping the external network gateway, which by default is
 172.24.4.225 is a valuable connectivity test IMHO (and that's your #1 item)
 For items #2 and #3 I'm not sure of your intentions; precisely so far I'm
 not sure if we're adding any coverage to Neutron. I assume you want to
 check servers such as www.google.com are reachable, but the routing from
 the external_gateway_ip to the final destination is beyond's Neutron
 control. DNS resolution might be interesting, but however I think
 resolution of external names is too beyond Neutron's control.

 Two more things to consider on external network connectivity tests:
 1) SNAT can be enabled or not. In this case we need a test that can tell
 us the SRC IP of the host connecting to the public external gateway,
 because I think that if SNAT kicks in, it should be an IP on the ext
 network, otherwise it should be an IP on the internal network. In this case
 we can use netcat to this aim, emulating a web server and use verbose
 output to print the source IP

2) When the connection happens from a port associated with a floating IP it
 is important that the SNAT happens with the floating IP address, and not
 with the default SNAT address. This is actually a test which would have
 avoided us a regression in the havana release cycle.



As far as I know from the code (I'm new to tempest and might be missing
something), test_network_basic_ops launches a single VM with a floating IP
associated and test is performed by accessing from the tempest host to the
guest VM using floating ip.

So, I have some questions:

- How can we test the internal network connectivity (when the tenant
networks are not accessible from the host, which I believe is the case for
most of the plugins)?

- For external connectivity, how can we test connectivity without floating
ip?
  - should we have another vm and control that from the access VM e.g. by
ssh remote command? or
  - spawn specific VMs which sends traffic upon boot (e.g. metadata server
+ userdata with cloud init installed VM, etc) to public and assert traffics
on the tempest host side?

Thanks,
Tomoe



 Regards,
 Salvatore


 On 18 November 2013 13:13, Giulio Fidente gfide...@redhat.com wrote:

 On 11/18/2013 11:41 AM, Yair Fried wrote:

 I'm editing tempest/scenario/test_network_basic_ops.py for external
 connectivity as the TODO listed in its docstring.

 the test cases are for pinging against external ip and url to test
 connectivity and dns respectivly.
 since default deployement (devstack gate) doesn't have external
 connectivity I was thinking on one or all of the following


 I think it's a nice thing to have!

   2. add fields in tempest.conf for
   * external connectivity = False/True
   * external ip to test against (ie 8.8.8.8)


 I like this option.

 One can easily disable it entirely OR pick a more relevant ip address
 if needed. Seems to me it would give the greatest flexibility.
 --
 Giulio Fidente
 GPG KEY: 08D733BA | IRC: giulivo

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Plugin and Driver Inclusion Requirements

2013-11-19 Thread Tomoe Sugihara 
Hello Neutron team,

From the Testing Requirements section, Tempest is mentioned as a
requirement.
Does that mean all the thirdparty vendors' system should run all the tests
in Tempest?

It might not make perfect sense to test image API for changes in the
neutron code.
So can we define some subset of tests that are relevant to neutron (e.g.
tempest.api.network, tempest.scenario.test_network_* ).

Also, some plugins may not implement full set of APIs defined by Neutron,
(for example. Vendor X hasn't supported floating IP API).
What should be the acceptance criteria for the API compatibility?

Thanks,
Tomoe

On Mon, Nov 18, 2013 at 10:59 PM, Russell Bryant rbry...@redhat.com wrote:

 On 11/18/2013 08:42 AM, Kyle Mestery (kmestery) wrote:
  Yong, if you read Mark's proposal closely, the third party tests will
 only run when the specific third party code is touched, or when the Jenkins
 user submits code.
 
  On Nov 17, 2013, at 11:50 PM, Yongsheng Gong gong...@unitedstack.com
 wrote:
 
  For third party testing, I am afraid these tests will make the patch
 merge process much longer since each patch, which even has nothing to do
 with the specific plugins, will triggers the unwanted third party testing
 jobs.

 And note that if it's not gating, we do not necessarily have to wait for
 every third party system to report in before merging the patch.

 --
 Russell Bryant

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron] Missing logs in Midokura CI Bot Inbox x

2014-07-16 Thread Tomoe Sugihara 
Hi there,

Just to apologize and inform that most of the links to the logs of Midokura
CI bot on gerrit are dead now. That is because I accidentally deleted all
the logs (instead of over a month old logs) today. Logs for the jobs after
the deletion are saved just fine.
We'll be more careful about handling the logs.

Best,
Tomoe
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] BPs for Juno-1

2014-05-29 Thread Tomoe Sugihara 
Hello Neutron core team,

We have three specs[1][2][3] submitted last week, for which have gotten +1s
from non core contributors.
It would be great if core devs could review them and give us some feedback.
(I noticed that I didn't include links to gerrit reviews in launchpad BPs
and just fixed them)

They are all our plugin specific changes, so it shouldn't affect the core
or the other parts of the Neutron.
Note that our 3rd party CI system is now running in a good shape, I believe
there's no major
issues with them going through the process.

If there are issues, we'd like to address them as soon as possible.
I look forward to getting your feedback.

Best Regards,
Tomoe

[1] Add blueprint for MidoNet quotas ext support:
https://review.openstack.org/#/c/94553/
[2] Add blueprint for MidoNet L3-ext-gw-modes support:
https://review.openstack.org/#/c/94785/
[3] Add spec for Update MidoNet plugin in Juno release:
https://review.openstack.org/#/c/95100/



On Thu, May 29, 2014 at 3:39 PM, mar...@redhat.com mandr...@redhat.com
wrote:

 On 28/05/14 17:57, Kyle Mestery wrote:
  On Wed, May 28, 2014 at 12:41 AM, mar...@redhat.com mandr...@redhat.com
 wrote:
  On 27/05/14 17:14, Kyle Mestery wrote:
  Hi Neutron developers:
 
  I've spent some time cleaning up the BPs for Juno-1, and they are
  documented at the link below [1]. There are a large number of BPs
  currently under review right now in neutron-specs. If we land some of
  those specs this week, it's possible some of these could make it into
  Juno-1, pending review cycles and such. But I just wanted to highlight
  that I removed a large number of BPs from targeting Juno-1 now which
  did not have specifications linked to them nor specifications which
  were actively under review in neutron-specs.
 
  Also, a gentle reminder that the process for submitting specifications
  to Neutron is documented here [2].
 
  Thanks, and please reach out to me if you have any questions!
 
 
  Hi Kyle,
 
  Can you please consider my PUT /subnets/subnet allocation_pools:{}
  review at [1] for Juno-1? Also, I see you have included a bug [1] and
  associated review [2] that I've worked on but the review is already
  pushed to master. Is it there for any pending backports?
 
  Done, I've added the bug referenced in [2] to Juno-1.

  Thanks!

 
  With regards to [3] below, are you saying you would like to submit
  that as a backport to stable?

 No I was more asking if that was the reason it was included (as it has
 already been merged) - though I can do that if you think it's a good idea,

 thanks, marios


 
  thanks! marios
 
  [1] https://review.openstack.org/#/c/62042/
  [2] https://bugs.launchpad.net/neutron/+bug/1255338
  [3] https://review.openstack.org/#/c/59212/
 
 
 
  Kyle
 
  [1] https://launchpad.net/neutron/+milestone/juno-1
  [2] https://wiki.openstack.org/wiki/Blueprints#Neutron
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] BPs for Juno-1

2014-05-29 Thread Tomoe Sugihara 
Thanks, Edgar and those who gave us feedback. We'll address comments.
And thanks for driving this big and important Neutron project.

Best,
Tomoe


On Fri, May 30, 2014 at 1:15 AM, Edgar Magana Perdomo (eperdomo) 
eperd...@cisco.com wrote:

  I will take them!

  Edgar

   From: Tomoe Sugihara to...@midokura.com

 Reply-To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Date: Thursday, May 29, 2014 at 7:57 AM

 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [neutron] BPs for Juno-1

   Hello Neutron core team,

  We have three specs[1][2][3] submitted last week, for which have gotten
 +1s from non core contributors.
 It would be great if core devs could review them and give us some feedback.
  (I noticed that I didn't include links to gerrit reviews in launchpad
 BPs and just fixed them)

  They are all our plugin specific changes, so it shouldn't affect the
 core or the other parts of the Neutron.
 Note that our 3rd party CI system is now running in a good shape, I
 believe there's no major
  issues with them going through the process.

  If there are issues, we'd like to address them as soon as possible.
 I look forward to getting your feedback.

  Best Regards,
 Tomoe

  [1] Add blueprint for MidoNet quotas ext support:
 https://review.openstack.org/#/c/94553/
 [2] Add blueprint for MidoNet L3-ext-gw-modes support:
 https://review.openstack.org/#/c/94785/
 [3] Add spec for Update MidoNet plugin in Juno release:
 https://review.openstack.org/#/c/95100/



 On Thu, May 29, 2014 at 3:39 PM, mar...@redhat.com mandr...@redhat.com
 wrote:

 On 28/05/14 17:57, Kyle Mestery wrote:
  On Wed, May 28, 2014 at 12:41 AM, mar...@redhat.com 
 mandr...@redhat.com wrote:
  On 27/05/14 17:14, Kyle Mestery wrote:
  Hi Neutron developers:
 
  I've spent some time cleaning up the BPs for Juno-1, and they are
  documented at the link below [1]. There are a large number of BPs
  currently under review right now in neutron-specs. If we land some of
  those specs this week, it's possible some of these could make it into
  Juno-1, pending review cycles and such. But I just wanted to highlight
  that I removed a large number of BPs from targeting Juno-1 now which
  did not have specifications linked to them nor specifications which
  were actively under review in neutron-specs.
 
  Also, a gentle reminder that the process for submitting specifications
  to Neutron is documented here [2].
 
  Thanks, and please reach out to me if you have any questions!
 
 
  Hi Kyle,
 
  Can you please consider my PUT /subnets/subnet allocation_pools:{}
  review at [1] for Juno-1? Also, I see you have included a bug [1] and
  associated review [2] that I've worked on but the review is already
  pushed to master. Is it there for any pending backports?
 
  Done, I've added the bug referenced in [2] to Juno-1.

   Thanks!

 
  With regards to [3] below, are you saying you would like to submit
  that as a backport to stable?

  No I was more asking if that was the reason it was included (as it has
 already been merged) - though I can do that if you think it's a good idea,

 thanks, marios


 
  thanks! marios
 
  [1] https://review.openstack.org/#/c/62042/
  [2] https://bugs.launchpad.net/neutron/+bug/1255338
  [3] https://review.openstack.org/#/c/59212/
 
 
 
  Kyle
 
  [1] https://launchpad.net/neutron/+milestone/juno-1
  [2] https://wiki.openstack.org/wiki/Blueprints#Neutron
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev