Re: [openstack-dev] [Openstack] How to deploy OpenStack on thousands of nodes?

2013-06-26 Thread Brent Roskos
Kylin,

I think there is some confusion as to the term broadcast.  Many of the
Rabbit docs describe the delivery of a message from one publisher to
multiple subscribers as a 'broadcast'.  This is not to be confused with a
network broadcast where traffic is sent over the network broadcast address.
 Rabbit uses tcp and a publisher/subscriber model - even in more complex
configurations where there are multiple publishers (think cluster).

I have personally implemented large openstack compute clouds that had many
hypervisors, each on individual subnets and a rabbit server on yet another
subnet and all message traffic worked as expected.  There were no actual
network broadcasts to worry about.

In my previous message I had assumed that you were actually in the process
of implementation and were running into problems.  It now seems that is not
the case - you are in a review or planning period.  However - as I noted
above the openstack queues on rabbit will work in a distributed network
configuration as long as all of the subscribers can reach the rabbit server
on tcp/5672.  I've personally done it and not had an issue.

Brent


On Tue, Jun 25, 2013 at 9:40 PM, Sg Kylin kylin7...@gmail.com wrote:

 Hi Brent,

 Thanks for your reply! But we are afraid that Rabbitmq needs broadcast to
 work correctly and usually broadcast is not available in cross-subnets
 deployments. That is what we are worrying about...

 Best,

 Kylin CG




 2013/6/26 Brent Roskos brent.ros...@solinea.com

 By default rabbit uses tcp port 5672 for communication.. tcp can
 certainly cross subnet boundaries and be routed without issue.

   I suggest you do some network troubleshooting; ping your rabbit server
 then telnet to port 5672 on the rabbit server from hosts on the other
 subnets.

 Check your router acls and local host firewalls.  Check to make sure that
 your rabbit server has a route to get back to the other subnets with the
 reply.

 Dual homed hosts with one local connection and one Internet connection
 will need specific routes added to allow them to reach other local subnets
 since you wouldn't want that traffic to try to traverse the default route
 which points out to the Internet.  This is true even if you are using
 virtual interfaces with vlans instead of separate physical interfaces.

 Regards,
 Brent


 On Tue, Jun 25, 2013 at 6:10 AM, Sg Kylin kylin7...@gmail.com wrote:

 Hi All,

 We are currently trying to deploy OpenStack on thousands of nodes. We
 are using Grizzly stable version and Ubuntu 12.04.2. However, the big
 problem we meet now is the network topology. If we want to use HA
 (haproxy + keepalived) for the controller nodes on which *-apis are
 running as well as network nodes which are deployed across different
 VLANs (VLANs can reach each other by setting gateways), e.g
 10.1.0.0/16 and 10.2.0.0/16, HA would not work correctly. Also we
 found that rabbitmq could not work when nova-* services were deployed
 across different subnets.

 Thus, we want to know whether HA and rabbitmq can be used across
 subnets? If it not true, we can only deploy them in a single flat
 layer 2 net, which seems unfeasible in real-world because of
 broadcast storms...

 Best,

 Kylin CG

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openst...@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] How to deploy OpenStack on thousands of nodes?

2013-06-26 Thread Brent Roskos
I stand corrected.  Mostly confused since the keepalived didn't actually
need addresses in the multicast IP range.  It does use it - as I can see
with ifconfig.  We minimized the  impact of this by creating a small subnet
that just had the switch address, host addresses and vrrp address in it.
 All the chatter was contained within that block.

We avoided pacemaker in this particular instance because the keepalived
setup and configuration was so very simple - only a couple of lines in a
config file, and because we didn't need any of the other available HA
features.

Brent


On Wed, Jun 26, 2013 at 10:03 AM, Jesse Pretorius jesse.pretor...@gmail.com
 wrote:

 On 26 June 2013 15:42, Brent Roskos brent.ros...@solinea.com wrote:

 I've also used keepalived for services that did not scale laterally.  In
 this case I put two horizon servers behind an active/passive virtual IP.
  This was also pretty simple as there was no need to maintain state
 information in for active passive. That wouldn't work quite as well when
 capacity thresholds started to become a concern.

 Neither of the above required multicast support - which really helps with
 deployment options.


 *ahem* keepalived most definitely requires multicast support for its
 vrrp... and it's quite noisy. If there's a way to make it use unicast
 instead, I'd definitely like to know.

 corosync  pacemaker can do a virtual IP failover between as many nodes as
 you like using unicast instead of multicast.

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openst...@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev