Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-22 Thread Joshua Harlow
And for another recent one that came out yesterday: Interesting to read for those who are using mongodb + openstack... https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-reads -Josh Joshua Harlow wrote: Joshua Harlow wrote: Kevin Benton wrote: Timestamps are just one way (and likely

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-17 Thread joehuang
questions) Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? - Original Message - From: joehuang joehu...@huawei.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Sunday, April 12, 2015 3:46:24 AM Subject

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-17 Thread Attila Fazekas
- Original Message - From: joehuang joehu...@huawei.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, April 17, 2015 9:46:12 AM Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Hi, Attila

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-16 Thread Attila Fazekas
- Original Message - From: joehuang joehu...@huawei.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Sunday, April 12, 2015 3:46:24 AM Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? As Kevin

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-16 Thread Neil Jerram
) -Original Message- From: Neil Jerram [mailto:neil.jer...@metaswitch.com] Sent: Wednesday, April 15, 2015 9:46 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Hi again Joe, (+ list) On 11/04/15 02:00, joehuang wrote: Hi, Neil, See

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-16 Thread joehuang
Huang ( Joe Huang ) -Original Message- From: Neil Jerram [mailto:neil.jer...@metaswitch.com] Sent: Thursday, April 16, 2015 5:15 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Thanks Joe, I really

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-15 Thread Neil Jerram
questions) Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Hi Joe, Many thanks for your reply! On 09/04/15 03:34, joehuang wrote: Hi, Neil, From theoretic, Neutron is like a broadcast domain, for example, enforcement of DVR and security group has to touch each regarding host where

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-15 Thread Joshua Harlow
(not for usage questions) Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Hi Joe, Many thanks for your reply! On 09/04/15 03:34, joehuang wrote: Hi, Neil, From theoretic, Neutron is like a broadcast domain, for example, enforcement of DVR and security group has to touch each

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-15 Thread joehuang
@lists.openstack.org Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Hi again Joe, (+ list) On 11/04/15 02:00, joehuang wrote: Hi, Neil, See inline comments. Best Regards Chaoyi Huang From: Neil Jerram [neil.jer...@metaswitch.com] Sent

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-15 Thread joehuang
: [openstack-dev] [neutron] Neutron scaling datapoints? __ __ Which periodic updates did you have in mind to eliminate? One of the few remaining ones I can think of is sync_routers but it would be great if you can

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-13 Thread joehuang
Huang ) -Original Message- From: Joshua Harlow [mailto:harlo...@outlook.com] Sent: Monday, April 13, 2015 11:11 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? joehuang wrote: Hi, Kevin and Joshua

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-13 Thread joehuang
-Original Message- From: Attila Fazekas [mailto:afaze...@redhat.com] Sent: Monday, April 13, 2015 3:19 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? - Original Message - From: joehuang

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-13 Thread Attila Fazekas
- Original Message - From: joehuang joehu...@huawei.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Sunday, April 12, 2015 1:20:48 PM Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Hi, Kevin

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-13 Thread Attila Fazekas
- Original Message - From: Kevin Benton blak...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Sunday, April 12, 2015 4:17:29 AM Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? So IIUC

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-12 Thread Joshua Harlow
Kevin Benton wrote: Timestamps are just one way (and likely the most primitive), using redis (or memcache) key/value and expiry are another (and letting memcache or redis expire using its own internal algorithms), using zookeeper ephemeral nodes[1] are another... The point being that its

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-12 Thread Joshua Harlow
:* Re: [openstack-dev] [neutron] Neutron scaling datapoints? Timestamps are just one way (and likely the most primitive), using redis (or memcache) key/value and expiry are another (and letting memcache or redis expire using its own internal algorithms), using zookeeper ephemeral nodes[1] are another

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-12 Thread joehuang
To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Timestamps are just one way (and likely the most primitive), using redis (or memcache) key/value and expiry are another (and letting memcache or redis expire using

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-12 Thread joehuang
:59 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? The TCP/IP stack keeps track of connections as a combination of IP + TCP port. The two byte port limit doesn't matter unless all of the agents are connecting

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-12 Thread Joshua Harlow
Kevin Benton wrote: So IIUC tooz would be handling the liveness detection for the agents. That would be nice to get ride of that logic in Neutron and just register callbacks for rescheduling the dead. Where does it store that state, does it persist timestamps to the DB like Neutron does? If so,

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-12 Thread Kevin Benton
Timestamps are just one way (and likely the most primitive), using redis (or memcache) key/value and expiry are another (and letting memcache or redis expire using its own internal algorithms), using zookeeper ephemeral nodes[1] are another... The point being that its backend specific and tooz

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-12 Thread Kevin Benton
(not for usage questions) *Subject:* Re: [openstack-dev] [neutron] Neutron scaling datapoints? The TCP/IP stack keeps track of connections as a combination of IP + TCP port. The two byte port limit doesn't matter unless all of the agents are connecting from the same IP address, which shouldn't

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-11 Thread joehuang
To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Sent: Thursday, April 9, 2015 5:01:45 PM Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Hi Joe, Many thanks for your reply! On 09/04/15

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-11 Thread Kevin Benton
) *Subject:* Re: [openstack-dev] [neutron] Neutron scaling datapoints? Which periodic updates did you have in mind to eliminate? One of the few remaining ones I can think of is sync_routers but it would be great if you can enumerate the ones you observed because eliminating overhead in agents

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-11 Thread Kevin Benton
So IIUC tooz would be handling the liveness detection for the agents. That would be nice to get ride of that logic in Neutron and just register callbacks for rescheduling the dead. Where does it store that state, does it persist timestamps to the DB like Neutron does? If so, how would that scale

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-10 Thread Kevin Benton
) openstack-dev@lists.openstack.org Sent: Thursday, April 9, 2015 5:01:45 PM Subject: Re: [openstack-dev] [neutron] Neutron scaling datapoints? Hi Joe, Many thanks for your reply! On 09/04/15 03:34, joehuang wrote: Hi, Neil, From theoretic, Neutron is like a broadcast domain

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-10 Thread joehuang
Hi, Neil, See inline comments. Best Regards Chaoyi Huang From: Neil Jerram [neil.jer...@metaswitch.com] Sent: 09 April 2015 23:01 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] Neutron scaling

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-09 Thread Neil Jerram
Hi Joe, Many thanks for your reply! On 09/04/15 03:34, joehuang wrote: Hi, Neil, From theoretic, Neutron is like a broadcast domain, for example, enforcement of DVR and security group has to touch each regarding host where there is VM of this project resides. Even using SDN controller, the

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-09 Thread Neil Jerram
Hi Mike, Many thanks for your reply! On 08/04/15 17:56, Mike Spreitzer wrote: Are you looking at scaling the numbers of tenants, Neutron routers, and tenant networks as you scale hosts and guests? I think this is a plausible way to grow. The compartmentalizations that comes with growing

[openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-08 Thread Neil Jerram
My team is working on experiments looking at how far the Neutron server will scale, with increasing numbers of compute hosts and VMs. Does anyone have any datapoints on this that they can share? Or any clever hints? I'm already aware of the following ones:

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-08 Thread Mike Spreitzer
neil.jer...@metaswitch.com To: openstack-dev@lists.openstack.org Date: 04/08/2015 12:29 PM Subject:[openstack-dev] [neutron] Neutron scaling datapoints? My team is working on experiments looking at how far the Neutron server will scale, with increasing numbers of compute hosts

Re: [openstack-dev] [neutron] Neutron scaling datapoints?

2015-04-08 Thread joehuang
Jerram [mailto:neil.jer...@metaswitch.com] Sent: Thursday, April 09, 2015 12:27 AM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [neutron] Neutron scaling datapoints? My team is working on experiments looking at how far the Neutron server will scale, with increasing numbers