Okay, I guess I need to revive this again now that I have the SNs deployed and now I am trying to snmove some nodes onto them. The Heirarchical Cluster wiki page is oriented toward those setting up a brand new cluster and not migrating an established cluster to include SNs, so it does not include clear instructions of what commands to run after you have created groups of CNs for SNs to manage. I am assuming that to get nodes to initially "look away" from the MN and put them on an SN for the first time you must execute "snmove" with -d and -D to point to the SN.
My config follows:
I am testing on just two of the nodes in my cluster for now. So first I did
this:
mkdef -t group -o serv1_compute members="node0001,node0002"
Then following the documentation for creating service pools I did this:
chdef -t group serv1_compute servicenode=xcat-serv1,xcat-serv2
# lsdef -t group serv1_compute
Object name: serv1_compute
grouptype=static
members=node0001,node0002
servicenode=xcat-serv1,xcat-serv2
And noderes looks like this now:
#node,servicenode,netboot,tftpserver,tftpdir,nfsserver,monserver,nfsdir,installnic,primarynic,discoverynics,cmdinterface,xcatmaster,current_osimage,next_osimage,nimserver,routenames,nameservers,comments,disable
"user",,"xnba","MN_IP",,"MN_IP",,,"eth0","eth0",,,"MN_IP",,,,,,,
"service",,"xnba","MN_IP",,"MN_IP",,,"mac","mac",,,"MN_IP",,,,,,,
"storage",,"xnba","MN_IP",,"MN_IP",,,"eth1","eth1",,,"MN_IP",,,,,,,
"compute",,"xnba","MN_IP",,"MN_IP",,,"eth0","eth0",,,"MN_IP",,,,,,,
"login",,"xnba","MN_IP",,"MN_IP",,,"eth0","eth0",,,"MN_IP",,,,,,,
"node0059",,"xnba",,,,,,,,,,,,,,,,,
"hinode01",,"xnba",,,,,,,,,,,,,,,,,
"serv1_compute","xcat-serv1,xcat-serv2",,,,,,,,,,,,,,,,,,
"node0001","xcat-serv1,xcat-serv2",,"xcat-serv1",,"xcat-serv1",,,,,,,"xcat-serv1",,,,,,,
"node0002","xcat-serv1,xcat-serv2",,"xcat-serv1",,"xcat-serv1",,,,,,,"xcat-serv1",,,,,,,
I may have a conflict problem though in that the established "compute"
group which node0001 and node0002 are in is pointing to MN_IP (the MN's ip
address) while serv1_compute points to xcat-serv1. I was hoping that since
noderes FURTHER defined the servicenode and xcatmaster for them that it
would override the settings for compute. Will that work or do I have to
remove node0001 & node0002 from compute altogether?
Their nodelist entries look like this:
"node0001","compute,compute-profile,ipmi,dx360m2,rack01,all,serv1_compute","booting","11-24-2013
13:55:00",,,,,"synced","02-05-2014 08:59:57",,
"node0002","compute,compute-profile,ipmi,dx360m2,rack01,all,serv1_compute","booting","11-24-2013
13:55:00",,,,,,,,
--------
Then after all the configuration, I tried an snmove on just node0001:
# snmove serv1_compute -d xcat-serv1 -D xcat-serv1
"Moving nodes to their backup service nodes.
Setting new values in the xCAT database.
node0001: install centos6.4-x86_64-compute
node0002: install centos6.4-x86_64-compute
node0001: install centos6.4-x86_64-compute
node0002: install centos6.4-x86_64-compute
Running postscripts on the nodes.
If you specify the -s flag you must not specify either the -S or -k or -P
flags"
In /var/log/messages I saw: "Allowing nodeset to node0001,node0002 install
for x3650-head01.haib.org <http://x3650-head01.haig.org/> from x3650-head01"
Firstly, why was a nodeset done when I typed snmove? The nodes are already
installed, I don't want to reinstall them.
Secondly, According to the wiki documentation: "If the CNs are up at the
time the *snmove* command is run then snmove will run postscripts on the
CNs to reconfigure them for the new SN."
However I checked files on node0001 like /etc/ntp.conf and their timestamp
had not changed (therefore I deduce the postscript did not run). So I ran
the postscripts manually with "updatenode node0001 syslog,setupntp". I
checked /etc/ntp.conf again and this time the timestamp was updated but
the file's contents were identical to before: it pointed to the MN_IP and
not xcat-serv1 as it should based on the xcatmaster setting in the noderes
table.
What am I doing wrong here?
Thanks,
Josh
On Fri, Jan 10, 2014 at 1:48 PM, Josh Nielsen <jniel...@hudsonalpha.org>wrote:
> Thank you Lissa, that is helpful.
>
> -Josh
>
>
> On Fri, Jan 10, 2014 at 1:25 PM, Lissa Valletta <lis...@us.ibm.com> wrote:
>
>> DNS and DHCP will still work from the Service Node, if setup
>> correctly. In other words, you have configured the service node as the
>> DNS server and/or DHCP server for the nodes and there is no requirement
>> on the Management Node for dns or dhcp. You will not be able to run
>> any xcat commands on the service node, if the Management Node is down.
>> xCAT requires access to the database configured on the MN for the xcat
>> cluster ( mysql, postgresql) to run most xcat commands. Even to recognize
>> that the node is in the xcat cluster.
>>
>> Lissa K. Valletta
>> 8-3/B10
>> Poughkeepsie, NY 12601
>> (tie 293) 433-3102
>>
>>
>>
>> [image: Inactive hide details for Josh Nielsen ---01/10/2014 12:59:12
>> PM---Hi Wang Xiaopeng (I apologize if I got your name wrong befor]Josh
>> Nielsen ---01/10/2014 12:59:12 PM---Hi Wang Xiaopeng (I apologize if I got
>> your name wrong before), You said: "SN cannot work without MN
>>
>> From: Josh Nielsen <jniel...@hudsonalpha.org>
>> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>,
>> Date: 01/10/2014 12:59 PM
>> Subject: Re: [xcat-user] How to create and deploy an xCAT Service Node
>> ------------------------------
>>
>>
>>
>> Hi Wang Xiaopeng (I apologize if I got your name wrong before),
>>
>> You said: "SN cannot work without MN since there's only one database
>> which is running on MN. If MN is down you cannot run lots of commands on
>> SN. In fact xCAT does NOT recommend to run any command on SN."
>>
>> Will DNS and DHCP still work however from an SN when the MN is down? I
>> would need the SN to still resolve the compute nodes' DNS requests on the
>> cluster subnet and issue DHCP leases/renewals as necessary. It looks like
>> SNs are intended to take responsibility on behalf of the MN for serving the
>> compute nodes' necessary services, such that you designate each node's
>> "xcatmaster" as the SN like: chdef -t group compute1 servicenode=sn1
>> xcatmaster=sn1-nic. If that is the case then shouldn't DNS & DHCP still be
>> resolved from the SN by a node even when the MN is down?
>>
>> Thanks,
>> Josh
>>
>>
>> On Wed, Jan 8, 2014 at 6:32 PM, Xiao Peng Wang
>> <*w...@cn.ibm.com*<w...@cn.ibm.com>>
>> wrote:
>>
>> The doc mentioned by Josh will lead you in the correct direction to
>> setup Service Node.
>>
>> Several information for you:
>>
>> Roughly, your understand is correct that you can install and
>> configure a SN automatically when prepare/configure correctly for
>> SNnode
>> definition (correct group like service, correct attribute like
>> postscripts,
>> correct osimage like rhels6.4-x86_64-install-service) and xCAT rpm
>> repository.
>>
>> Generally, the /tftpboot and /install on SN are mounted from xCAT
>> MN, so them will be identical with the dirs on MN in any time.
>>
>> SN cannot work without MN since there's only one database which is
>> running on MN. If MN is down you cannot run lots of commands on SN. In
>> fact
>> xCAT does NOT recommend to run any command on SN.
>>
>>
>> Thanks
>> Best Regards
>> ----------------------------------------------------------------------
>> Wang Xiaopeng (王晓朋)
>> IBM China System Technology Laboratory
>> Tel: 86-10-82453455
>> Email: *w...@cn.ibm.com* <w...@cn.ibm.com>
>> Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road,
>> Haidian District Beijing P.R.China 100193
>>
>> [image: Inactive hide details for Josh Nielsen ---2014/01/09
>> 03:48:01---Hi Russell, For some reason that wiki page never came up in
>> my]Josh
>> Nielsen ---2014/01/09 03:48:01---Hi Russell, For some reason that wiki
>> page
>> never came up in my Google searches even
>>
>> From: Josh Nielsen <*jniel...@hudsonalpha.org*<jniel...@hudsonalpha.org>
>> >
>> To: xCAT Users Mailing list
>> <*xcat-user@lists.sourceforge.net*<xcat-user@lists.sourceforge.net>>,
>>
>> Date: 2014/01/09 03:48
>> Subject: Re: [xcat-user] How to create and deploy an xCAT Service Node
>> ------------------------------
>>
>>
>>
>> Hi Russell,
>>
>> For some reason that wiki page never came up in my Google searches
>> even when I placed quotes around "service node". Thanks! I'll take a look
>> through it.
>>
>> Regards,
>> Josh
>>
>>
>> On Wed, Jan 8, 2014 at 12:25 PM, Russell Jones <
>> *russell-l...@jonesmail.me* <russell-l...@jonesmail.me>> wrote:
>> Hi Josh,
>>
>> This should be helpful for you (it was for me when first learning
>> how to set this up):
>>
>> *http://sourceforge.net/apps/mediawiki/xcat/index.php?title=Setting_Up_a_Linux_Hierarchical_Cluster*<http://sourceforge.net/apps/mediawiki/xcat/index.php?title=Setting_Up_a_Linux_Hierarchical_Cluster>
>>
>>
>>
>> On 1/8/2014 11:52 AM, Josh Nielsen wrote:
>> Hello all,
>>
>> I am trying to move my xCAT installation toward a hierarchical
>> structure by including Service Nodes but I am finding a dearth of
>> information on how to create a service node on the xCAT wiki and
>> even the
>> Sumavi xCAT guide. At first I began following the instructions for a
>> setting up a management node (which involves a new xCAT install, an
>> independent xCAT database, etc.) on a VM intended to the the SN but
>> then
>> realized that that must be wrong.
>>
>> I have noticed the xCAT 'servicenode' database table present on
>> the Management Node (listed with tabdump). I am wondering if I can
>> simply
>> designate a new node (in my case a VM) to be the prospective SN,
>> designate
>> which services (like dhcp, dns, etc.) to enable/disable in the
>> 'servicenode' table, and then do a fresh 'rinstall' to that VM and
>> have it
>> be automatically & fully configured as an SN from postinstall
>> scripts
>> (which I presume are already present and will be automatically
>> applied to
>> that node by some xCAT logic of parsing the 'servicenode' table and
>> associating any relevant postscripts to be run with the node).
>>
>> If so will it create the /install and /tftpboot directories
>> (including ISOs made with copycds) on each SN identical to the
>> contents of
>> the MN? Is it that simple or is there more involved than that?
>>
>> In addition I want to be able to take down the Management Node
>> in the future for maintenance and be able to do just about
>> everything that
>> the MN can do from the SNs (including dhcp leases, rpower commands,
>> updatenode, psh, etc) while the MN is down. Is that possible? Any
>> clarifications would be appreciated.
>>
>> Thank you,
>> Josh Nielsen
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Rapidly troubleshoot problems before they affect your business.
>> Most IT
>> organizations don't have a clear picture of how application
>> performance
>> affects their revenue. With AppDynamics, you get 100%
>> visibility into your
>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>> AppDynamics Pro!
>>
>>
>> *http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk*<http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk>
>>
>>
>> _______________________________________________
>> xCAT-user mailing list
>> *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net>
>> *https://lists.sourceforge.net/lists/listinfo/xcat-user*<https://lists.sourceforge.net/lists/listinfo/xcat-user>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Rapidly troubleshoot problems before they affect your business.
>> Most IT
>> organizations don't have a clear picture of how application
>> performance
>> affects their revenue. With AppDynamics, you get 100% visibility
>> into your
>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>> AppDynamics Pro!
>>
>>
>> *http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk*<http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk>
>> _______________________________________________
>> xCAT-user mailing list
>> *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net>
>> *https://lists.sourceforge.net/lists/listinfo/xcat-user*<https://lists.sourceforge.net/lists/listinfo/xcat-user>
>>
>>
>> ------------------------------------------------------------------------------
>> Rapidly troubleshoot problems before they affect your business. Most
>> IT
>> organizations don't have a clear picture of how application
>> performance
>> affects their revenue. With AppDynamics, you get 100% visibility into
>> your
>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>> AppDynamics Pro!
>>
>>
>> *http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk*<http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk>
>> _______________________________________________
>> xCAT-user mailing list
>> *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net>
>> *https://lists.sourceforge.net/lists/listinfo/xcat-user*<https://lists.sourceforge.net/lists/listinfo/xcat-user>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> Critical Workloads, Development Environments & Everything In Between.
>> Get a Quote or Start a Free Trial Today.
>>
>>
>> *http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk*<http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk>
>> _______________________________________________
>> xCAT-user mailing list
>> *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net>
>> *https://lists.sourceforge.net/lists/listinfo/xcat-user*<https://lists.sourceforge.net/lists/listinfo/xcat-user>
>>
>>
>> ------------------------------------------------------------------------------
>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> Critical Workloads, Development Environments & Everything In Between.
>> Get a Quote or Start a Free Trial Today.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>> _______________________________________________
>> xCAT-user mailing list
>> xCAT-user@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/xcat-user
>>
>>
>>
>> ------------------------------------------------------------------------------
>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> Critical Workloads, Development Environments & Everything In Between.
>> Get a Quote or Start a Free Trial Today.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>> _______________________________________________
>> xCAT-user mailing list
>> xCAT-user@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/xcat-user
>>
>>
>
<<graycol.gif>>
------------------------------------------------------------------------------ Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
_______________________________________________ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
