Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-25 Thread ZZelle
Hi everyone, A new change (https://review.openstack.org/101982) has been proposed to improve vxlan pool initiation with an improvement on delete of obsolete unallocated vnis using a unique delete SQL command. I've tested performance with the following (delete only) scenario: vxlan range is

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-09 Thread Eugene Nikanorov
Mike, Thanks a lot for your response! Some comments: There’s some in-Python filtering following it which does not seem necessary; the alloc.vxlan_vni not in vxlan_vnis” phrase could just as well be a SQL “NOT IN” expression. There we have to do specific set intersection between configured

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-08 Thread Mike Bayer
On Jun 7, 2014, at 4:38 PM, Eugene Nikanorov enikano...@mirantis.com wrote: Hi folks, There was a small discussion about the better way of doing sql operations for vni synchronization with the config. Initial proposal was to handle those in chunks. Carl also suggested to issue a single

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-07 Thread Eugene Nikanorov
Hi folks, There was a small discussion about the better way of doing sql operations for vni synchronization with the config. Initial proposal was to handle those in chunks. Carl also suggested to issue a single sql query. I've did some testing with my sql and postgress. I've tested the following

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-05 Thread Xurong Yang
great. I will do more test base on Eugene Nikanorov's modification. *Thanks,* 2014-06-05 11:01 GMT+08:00 Isaku Yamahata isaku.yamah...@gmail.com: Wow great. I think the same applies to gre type driver. so we should create similar one after vxlan case is resolved. thanks, On Thu, Jun

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-04 Thread Wang, Yalei
performance issue about VXLAN pool initiation Hi, Folks, When we configure VXLAN range [1,16M], neutron-server service costs long time and cpu rate is very high(100%) when initiation. One test base on postgresql has been verified: more than 1h when VXLAN range is [1, 1M]. So, any good

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-04 Thread Eugene Nikanorov
the result(1h)? /Yalei *From:* Xurong Yang [mailto:ido...@gmail.com] *Sent:* Thursday, May 29, 2014 6:01 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation Hi, Folks

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-04 Thread Carl Baldwin
Yes, memcached is a candidate that looks promising. First things first, though. I think we need the abstraction of an ipam interface merged. That will take some more discussion and work on its own. Carl On May 30, 2014 4:37 PM, Eugene Nikanorov enikano...@mirantis.com wrote: I was thinking

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-04 Thread Carl Baldwin
You are right. I did feel a bit bad about hijacking the thread. But, most of discussion was related closely enough that I never decided to fork in to a newer thread. I think I'm done now. I'll have a look at your review and we'll put IPAM to rest for now. :) Carl On Wed, Jun 4, 2014 at 2:36

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-04 Thread Wang, Yalei
: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation We hijacked the vxlan initialization performance thread with ipam! :) I've tried to address initial problem with some simple sqla stuff: https://review.openstack.org/97774 With sqlite it gives ~3x benefit over existing

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-06-04 Thread Isaku Yamahata
Wow great. I think the same applies to gre type driver. so we should create similar one after vxlan case is resolved. thanks, On Thu, Jun 05, 2014 at 12:36:54AM +0400, Eugene Nikanorov enikano...@mirantis.com wrote: We hijacked the vxlan initialization performance thread with ipam! :) I've

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-05-31 Thread Xurong Yang
Hi, i have reported a bug[1] [1]https://bugs.launchpad.net/neutron/+bug/1324875 but no better idea about this issue now, maybe need more discussion. any thoughts? :) Xurong Yang 2014-05-31 6:33 GMT+08:00 Eugene Nikanorov enikano...@mirantis.com: I was thinking it would be a separate

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-05-30 Thread Salvatore Orlando
It seems that method has some room for optimization, and I suspect the same logic has been used in other type drivers as well. If optimization is possible, it might be the case to open a bug for it. Salvatore On 30 May 2014 04:58, Xurong Yang ido...@gmail.com wrote: Hi, Thanks for your

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-05-30 Thread Kyle Mestery
I agree with Salvatore, I don't the optimization of that method (and possibly others) requires a BP, but rather a bug. Can you please file one Xurong? Thanks, Kyle On Fri, May 30, 2014 at 3:39 AM, Salvatore Orlando sorla...@nicira.com wrote: It seems that method has some room for

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-05-30 Thread Eugene Nikanorov
Hi Carl, The idea of in-memory storage was discussed for similar problem, but might not work for multiple server deployment. Some hybrid approach though may be used, I think. Thanks, Eugene. On Fri, May 30, 2014 at 8:53 PM, Carl Baldwin c...@ecbaldwin.net wrote: This is very similar to

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-05-30 Thread Carl Baldwin
Eugene, That was part of the whole new set of complications that I dismissively waved my hands at. :) I was thinking it would be a separate process that would communicate over the RPC channel or something. More complications come when you think about making this process HA, etc. It would mean

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-05-30 Thread Eugene Nikanorov
I was thinking it would be a separate process that would communicate over the RPC channel or something. memcached? Eugene. On Sat, May 31, 2014 at 2:27 AM, Carl Baldwin c...@ecbaldwin.net wrote: Eugene, That was part of the whole new set of complications that I dismissively waved my hands

[openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-05-29 Thread Xurong Yang
Hi, Folks, When we configure VXLAN range [1,16M], neutron-server service costs long time and cpu rate is very high(100%) when initiation. One test base on postgresql has been verified: more than 1h when VXLAN range is [1, 1M]. So, any good solution about this performance issue? Thanks, Xurong

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-05-29 Thread ZZelle
Hi, vxlan network are inserted/verified in DB one by one, which could explain the time required https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/type_vxlan.py#L138-L172 Cédric On Thu, May 29, 2014 at 12:01 PM, Xurong Yang ido...@gmail.com wrote: Hi, Folks,

Re: [openstack-dev] [Neutron] One performance issue about VXLAN pool initiation

2014-05-29 Thread Xurong Yang
Hi, Thanks for your response, yes, i get the reason, so, That's why i question that whether one good solution can have a high performance with a large vxlan range? if possible, a blueprint is deserved to consider. Tanks, Xurong Yang 2014-05-29 18:12 GMT+08:00 ZZelle zze...@gmail.com: Hi,