Hello Rohit, I'm going to redeploy my management nodes and retest my load balacning confgiuration.
Also concerning the commands i used in order to deploy the management servers : " For the first node (192.168.0.10) i've done : cloudstack-setup-databases cloud:[email protected] <cloud%[email protected]> --deploy-as=root:mypassroot -e file -m mypassphrase -k mypassphrase -i 192.168.0.100 The second node (192.168.0.11) was deployed with cloudstack-setup-databases cloud:[email protected] <cloud%[email protected]> -e file -m mypassphrase -k mypassphrase -i 192.168.0.100 Is it alright, or is this what did a split brain scenario ? " Was i in the right way ? Concerning my split brain scenario, i had a management server whom detected (after being up for few minutes, deployed a zone, cluster, host and first virtual machine) there was split brain and so it killed itself, if i tried to reup this management server, i got full entries in the logs like : DEBUG [c.c.a.m. ClusteredAgentManagerImpl] (AgentManager-Handler-8:null) Seq 1-289341441: MgmtId 104425904713066: Resp: Routing to peer Fulling the logs files all the times Thanks for your responses 2014-10-27 10:15 GMT+01:00 Rohit Yadav <[email protected]>: > Hi Benoit, > > Can you describe your split brain issues you’re seeing? > > By setting agent.lb.enable to true, all requests to the management servers > will be internally balanced by the clustered management servers. In case of > split-brain, you would get different API results, for example listing of > resources such as users, hosts etc. > > Just make sure that you’re MySQL configuration works well with both the > management servers individually. A possible case could be that one of the > servers can reach the mysql server but the other one fails. > > > On 27-Oct-2014, at 2:30 pm, benoit lair <[email protected]> wrote: > > > > Hi Geoff, > > > > > > Thanks a lot for your explanations. > > > > So when testing with a 2 nodes management servers, i had split brain > > scenario :/ > > > > When installing the management server, did i do the rights things ? > > > > In my case 192.168.0.100 (VIP in netscaler, load balancing the ports you > > listed above) with a service group including the 2 management server > nodes > > (192.168.0.10 for the first and 192.168.0.11 for the secondary) and a > mysql > > server with 192.168.0.200 > > > > For the first node (192.168.0.10) i've done : > > > > cloudstack-setup-databases cloud:[email protected] > > <cloud%[email protected]> --deploy-as=root:mypassroot -e file -m > > mypassphrase -k mypassphrase -i 192.168.0.100 > > > > The second node (192.168.0.11) was deployed with > > cloudstack-setup-databases cloud:[email protected] > > <cloud%[email protected]> -e file -m mypassphrase -k mypassphrase > -i > > 192.168.0.100 > > > > Is it alright, or is this what did a split brain scenario ? > > > > > > Thanks a lot four your responses. > > > > Regards, Benoit. > > > > 2014-10-23 10:43 GMT+02:00 Geoff Higginbottom < > > [email protected]>: > > > >> Hi Benoit > >> > >> When running more than one CloudStack Management Server you need to put > a > >> load balancer in front of them. This load balancer (or ideally pair of > HA > >> load balancers) need to manage the following ports > >> > >> 8250 - This is the port the System VMs use when communicating with the > >> management servers, and they use the address specified in the 'host' > global > >> settings. > >> > >> 8080 - Access to the Web UI > >> > >> 8096 (or a port of your choosing) - Optional - only required is you are > >> using the unauthenticated API port > >> > >> You can look in the 'cloud.hosts' table in the DB and check out the > >> 'status' and 'mgmt_server_id' columns, the latter is the ID of the > >> management server which is responsible for managing the System VM or > >> Hypervisor. > >> > >> One last point, you need to ensure you have set global settings > >> agent.lb.enable to true to enable the load balancing of System VMs and > >> Hypervisors across multiple Management Servers. > >> > >> Regards > >> > >> Geoff Higginbottom > >> > >> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 > >> > >> [email protected] > >> > >> -----Original Message----- > >> From: benoit lair [mailto:[email protected]] > >> Sent: 23 October 2014 08:49 > >> To: [email protected] > >> Subject: Re: [ACS43][MGMT Server][Load Balancing] > >> > >> Any advice ? Any feedback ? Does somebody already did that ? > >> > >> Thanks for your responses. > >> > >> 2014-10-13 10:51 GMT+02:00 benoit lair <[email protected]>: > >> > >>> Hello, > >>> > >>> > >>> Is there somebody already accomplished some load balancing on the > >>> management app server ? > >>> > >>> > >>> > >>> Thanks for your responses. > >>> > >>> Regards. > >>> > >>> 2014-10-09 16:24 GMT+02:00 benoit lair <[email protected]>: > >>> > >>>> Here are the logs entries i have in the other mgmt node : > >>>> > >>>> DEBUG [c.c.a.m.ClusteredAgentManagerImpl] > >>>> (AgentManager-Handler-8:null) Seq 1-289341441: MgmtId > >>>> 104425904713066: Resp: Routing to peer > >>>> > >>>> 2014-10-09 16:21 GMT+02:00 benoit lair <[email protected]>: > >>>> > >>>>> What i was thinking of.. > >>>>> > >>>>> Here are the logs entries in one of my mgmt server : > >>>>> > >>>>> 2014-10-09 15:47:27,451 ERROR [c.c.c.ClusterManagerImpl] > >>>>> (Cluster-Heartbeat-1:cx-71073990) We have detected that at least one > >>>>> management server peer reports tat this management server is down, > >>>>> perform active fencing to avoid split-brain ituation > >>>>> > >>>>> So this one has been shutdown. How to recover my 2 mgmt nodes in > >>>>> normal state ? > >>>>> > >>>>> Thanks. > >>>>> Regards, Benoit > >>>>> > >>>>> 2014-10-09 16:20 GMT+02:00 benoit lair <[email protected]>: > >>>>> > >>>>>> Hello, > >>>>>> > >>>>>> > >>>>>> I would like to do some load balancing on the mgmt server, i would > >>>>>> like to know if i am on the good way : > >>>>>> > >>>>>> I installed Two management servers. > >>>>>> > >>>>>> The first with the "deploy-as" in the command > >>>>>> cloudstack-setup-databases with the netscalerlbip as the management > >>>>>> server > >>>>>> : > >>>>>> > >>>>>> cloudstack-setup-databases cloud:[email protected] > >>>>>> --deploy-as=root:mypassroot -e file -m mypassphrase -k mypassphrase > >>>>>> -i netscalerlbip > >>>>>> > >>>>>> The second node was deployed with > >>>>>> cloudstack-setup-databases cloud:[email protected] -e file -m > >>>>>> mypassphrase -k mypassphrase -i netscalerlbip > >>>>>> > >>>>>> So my netscaler is doing load balancing on the 2 nodes availables > >>>>>> accross netscalerlbip. > >>>>>> > >>>>>> I have confgured the "host" global parameter with the netscalerlbip > >>>>>> as ip and restarted the 2 mgmt servers. > >>>>>> > >>>>>> How to be sure that my mgmts servers are working correctly ? Am i > >>>>>> sure not to have split brain ? > >>>>>> > >>>>>> Thanks for your responses. > >>>>>> > >>>>>> Regards, > >>>>>> Benoit > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> Find out more about ShapeBlue and our range of CloudStack related > services > >> > >> IaaS Cloud Design & Build< > >> http://shapeblue.com/iaas-cloud-design-and-build//> > >> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/ > > > >> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> > >> CloudStack Infrastructure Support< > >> http://shapeblue.com/cloudstack-infrastructure-support/> > >> CloudStack Bootcamp Training Courses< > >> http://shapeblue.com/cloudstack-training/> > >> > >> This email and any attachments to it may be confidential and are > intended > >> solely for the use of the individual to whom it is addressed. Any views > or > >> opinions expressed are solely those of the author and do not necessarily > >> represent those of Shape Blue Ltd or related companies. If you are not > the > >> intended recipient of this email, you must neither take any action based > >> upon its contents, nor copy or show it to anyone. Please contact the > sender > >> if you believe you have received this email in error. Shape Blue Ltd is > a > >> company incorporated in England & Wales. ShapeBlue Services India LLP > is a > >> company incorporated in India and is operated under license from Shape > Blue > >> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in > Brasil > >> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd > is > >> a company registered by The Republic of South Africa and is traded under > >> license from Shape Blue Ltd. ShapeBlue is a registered trademark. > >> > > Regards, > Rohit Yadav > Software Architect, ShapeBlue > M. +91 88 262 30892 | [email protected] > Blog: bhaisaab.org | Twitter: @_bhaisaab > > > > Find out more about ShapeBlue and our range of CloudStack related services > > IaaS Cloud Design & Build< > http://shapeblue.com/iaas-cloud-design-and-build//> > CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> > CloudStack Infrastructure Support< > http://shapeblue.com/cloudstack-infrastructure-support/> > CloudStack Bootcamp Training Courses< > http://shapeblue.com/cloudstack-training/> > > This email and any attachments to it may be confidential and are intended > solely for the use of the individual to whom it is addressed. Any views or > opinions expressed are solely those of the author and do not necessarily > represent those of Shape Blue Ltd or related companies. If you are not the > intended recipient of this email, you must neither take any action based > upon its contents, nor copy or show it to anyone. Please contact the sender > if you believe you have received this email in error. Shape Blue Ltd is a > company incorporated in England & Wales. ShapeBlue Services India LLP is a > company incorporated in India and is operated under license from Shape Blue > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil > and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is > a company registered by The Republic of South Africa and is traded under > license from Shape Blue Ltd. ShapeBlue is a registered trademark. >
