Thanks a lot Mathi for the detailed instructions. It worked. I was missing the must condition: All the controllers and payloads defined in nodes.cfg were not running which was preventing from expanding cluster size. In my case standby controller node was down and hence the error I was getting.
Appreciate the help.. Greg On Thu, Dec 12, 2013 at 2:42 AM, Mathivanan Naickan Palanivelu < [email protected]> wrote: > Greg, > > Comments inline: > > > -----Original Message----- > > From: Greg Hurlman [mailto:[email protected]] > > Sent: Thursday, December 12, 2013 1:01 PM > > To: Mathivanan Naickan Palanivelu > > Cc: Nivrutti Kale; [email protected] > > Subject: Re: [users] Payload node addition > > > > Hi Mathi, > > > > In my case one node maps to a physical node/VM. Hence I would add a > > physical node to the AMF cluster at runtime. If I use immcfg then issues > > would be: > > > > 1. When both the controllers restarts, it may not persists this new node > > configuration vs when configured through nodes.cfg and immxml-modify- > > config /immxml-config script combination. > > > You could make the OpenSAF configuration changes to persist and survive > controller restarts by > using the OpenSAF IMM PBE(Persistent Back End) feature. This could be > enabled using ./configure --enable-imm-pbe enable the IMM > Persistent Backend Interface) > Check the osaf/services/saf/immsv/README for more info on that. > > However, if you do not wish to use the IMM PBE feature, you have to > regenerate > the imm.xml after running immxml-modify-config script. > > > > 2, By using immcfg with above two commands, node configuration got added > > but opensafd service in the new payload node/VM does not start. > > Commands "service opensafd start" never returns OK or failed etc. just > > hangs there forever. But it says that PL-5 joined the cluster. In this > case even > > I tried adding the new node host name (PL-5) in the active controller > > nodes.cfg file. > > > > I prefer to do using immxml-modify-configure if that adds this physical > node > > to the AMFcluster.(I am not using clm cluster in my environment). > > But not sure what are the parameters the modify script depends on. > > > > Anybody success on using the immxml-modify-configure script in expanding > > the existing cluster size?? > > > > I assume you would have done the following: > > 1) If you initially had generated the imm.xml for 3 nodes (two controllers > and 1 payload) > and *all the 3 nodes were started and all 3 are running now with slot id > 1, 2 and 3 respectively*. > your nodes.cfg looks like as below(for eg:-) > SC SC-1 SC-1 > SC SC-2 SC-2 > PL PL-3 PL-3 > > 2) Now, if you want to expand the cluster and add two more nodes to the > configuration, then you would edit > the nodes.cfg and it will look like as below > SC SC-1 SC-1 > SC SC-2 SC-2 > PL PL-3 PL-3 > PL PL-4 PL-4 > PL PL-5 PL-5 > > 3) Run the command immxml-modify-config > > 4) Run ./immxml-configure to generate the new imm.xml. > > 5) Now on the two new payloads 4 and 5, set /etc/opensaf/slot_id with a > value of say 4 and 5 and > And unique /etc/opensaf/node_name with the value of PL-4 and PL-5 > respectively, you would have to start > OpenSAF(obviously after editing nodeinit.conf*). > > The two nodes should join the cluster. > > Share the syslog of the ACTIVE controller and the payload if you are > struck at step (5) > > -Mathi. > > > Thanks for all the help. > > > > Greg > > > > > > > > > > On Wed, Dec 11, 2013 at 12:34 AM, Mathivanan Naickan Palanivelu > > <[email protected]> wrote: > > Hi, > > > > I'm not sure about your problem scope, But did you try running the > following > > commands? > > (adds a CLM node(say PL-5) and a logical AMF node(PL-5) mapped to that > > CLM node) > > > > immcfg -c SaClmNode safNode=PL-5,safCluster=myClmCluster > > immcfg -c SaAmfNode -a saAmfNodeSuFailOverProb=1200000000000 -a > > saAmfNodeSuFailoverMax=2 -a saAmfNodeClmNode=safNode=PL- > > 5,safCluster=myClmCluster safAmfNode=PL-5,safAmfCluster=myAmfCluster > > > > immcfg helps you to create/modify/delete changes to the IMM model. Try > > immcfg --help > > > > Regards, > > Mathi. > > > > > > > -----Original Message----- > > > From: Greg Hurlman [mailto:[email protected]] > > > Sent: Friday, December 06, 2013 12:42 PM > > > To: Nivrutti Kale > > > Cc: [email protected] > > > Subject: Re: [users] Payload node addition > > > > > > Yes, all the payload nodes were running while expanding the cluster > size > > > from 2 payload nodes to 3. Even I tried stopping the opensafd service > in > > both > > > payload1 and paylod2. > > > > > > The content of the nodes.cfg are: > > > > > > Existing Nodes: > > > SC SC-1 sc-active > > > SC SC-2 sc-standby > > > PL PL-1 payload1 > > > PL PL-2 payload2 > > > > > > New Node Configuration added: > > > PL PL-3 payload3 > > > > > > Where, payload1, payload2 and payload3 are the host names for all the > > > payload nodes and sc-active, sc-standby are the host names of the > > controller > > > nodes. > > > > > > Thanks, > > > Greg > > > > > > > > > > > > > > > > > > On Thu, Dec 5, 2013 at 9:28 PM, Nivrutti Kale <[email protected]> > wrote: > > > > > > > Hi Greg, > > > > > > > > Does all your existing nodes running when you are trying to expand > the > > > > cluster? > > > > Also share the contents of nodes.cfg mentioning current and new nodes > > > > entry. > > > > > > > > Thanks & Regards, > > > > Nivrutti > > > > > > > > > > > > On Fri, Dec 6, 2013 at 4:04 AM, Greg Hurlman > > > <[email protected]>wrote: > > > > > > > >> Hi, > > > >> > > > >> > > > >> I followed the above mentioned steps to add dynamically a new > payload > > > >> node while the existing payload nodes were running. And when I ran > > > >> the modify script after adding the entry in nodes.cfg, the below > > > >> error I am getting. Anybody faced similar issue before?? > > > >> > > > >> *[root@sc-active immxml]# ./immxml-modify-config* > > > >> *error: When expanding the cluster all existing nodes must be > started > > > >> (saAmfNodeOperState=1). Exiting!* > > > >> > > > >> It did not reboot the payload nodes, neither it got added to the > cluster. > > > >> Anything obvious I am missing here? > > > >> > > > >> Thanks, > > > >> Greg > > > >> > > > >> > > > >> On Wed, Dec 4, 2013 at 9:34 PM, Nivrutti Kale <[email protected]> > > > wrote: > > > >> > > > >>> Hi Greg, > > > >>> > > > >>> You can use the ./immxml-modify-config for adding new payload > > nodes > > > >>> to the running cluster without stopping the running controllers or > > > >>> payload nodes. > > > >>> > > > >>> Steps to run: > > > >>> > > > >>> 1. Add entries for newly added payload nodes in nodes.cfg > > > >>> 2. Run ./immxml-modify-config > > > >>> > > > >>> Hope that helps. > > > >>> > > > >>> Thanks & Regards, > > > >>> Nivrutti > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> On Thu, Dec 5, 2013 at 6:37 AM, Greg Hurlman > > > <[email protected]>wrote: > > > >>> > > > >>>> Hi, > > > >>>> > > > >>>> I have a running cluster with 2 payload nodes created by using > > > >>>> > > > >>>> immxml-clustersize -s 2 -p 2 > > > >>>> > > > >>>> Can I dynamically add a new payload node to this running cluster > > > >>>> without stopping the running controllers or payload nodes? or What > > > >>>> is the best way to do so? Are there any workarounds if not > > > >>>> possible? > > > >>>> > > > >>>> Appreciate for any help. > > > >>>> > > > >>>> Thanks > > > >>>> Greg > > > >>>> > > > >>>> > ------------------------------------------------------------------- > > > >>>> ----------- > > > >>>> Sponsored by Intel(R) XDK > > > >>>> Develop, test and display web and hybrid apps with a single code > > base. > > > >>>> Download it for free now! > > > >>>> > > > >>>> > > > http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/os > > > >>>> tg.clktrk > > _______________________________________________ > > > >>>> Opensaf-users mailing list > > > >>>> [email protected] > > > >>>> https://lists.sourceforge.net/lists/listinfo/opensaf-users > > > >>>> > > > >>> > > > >>> > > > >> > > > > > > > > ------------------------------------------------------------------------------ > > > Sponsored by Intel(R) XDK > > > Develop, test and display web and hybrid apps with a single code base. > > > Download it for free now! > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.cl > > > ktrk > > > _______________________________________________ > > > Opensaf-users mailing list > > > [email protected] > > > https://lists.sourceforge.net/lists/listinfo/opensaf-users > ------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk _______________________________________________ Opensaf-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/opensaf-users
