Xiao,
Okay, so I followed those four steps with some modifications. I did 1 & 4
as instructed with no issues. The service nodes are getting their database
access from the new MN now, and I updated the SN object definitions to
point xcatmaster, tftpserver, and other relevant parameters to the new MN.
I avoided step #3 because I just copied the old /root/.ssh/id_rsa and
corresponding .pub file to the new MN and passwordless logon works fine. I
also tested this from the two service nodes to make sure they could fetch
the host keys: "USEOPENSSLFORXCAT=yes XCATSERVER=<MN_IP>:3001
/xcatpost/getcredentials.awk ssh_rsa_hostkey. Is that sufficient for the
key step?
And lastly for #3 I only selectively updated certain packages on the SNs
like syslog and NTP, because I didn't want to run all of the packages and
in particular the servicenode postscript.
So, I was able to use updatenode with no issues from the new MN to update
the SNs, however when I try to update any cluster client nodes it is having
problems dispatching to the service nodes in the hierarchy:
# updatenode node0010 -P addsiteyum
Error: Failed to dispatch command to any of the following service nodes:
xcat-serv1,xcat-serv2
What is most likely causing that issue?
Thanks,
Josh
On Fri, Jun 3, 2016 at 7:01 AM, Xiao Peng Wang <w...@cn.ibm.com> wrote:
> I think we should talk it as opposite way that how to make the MN to use
> the new SN.
>
> Following steps are necessary to switch a SN:
> 1. rerun 'mysqlsetup -f' to assign the access permission for SN to access
> DB on MN
> 2. run 'updatenode -k <sn>' to set up the ssh key
> 3. run 'updatenode -P' to update the SN
> 4. change the 'servicenode' attribute for compute node accordingly.
>
>
> Thanks
> Best Regards
> ----------------------------------------------------------------------
> Wang Xiaopeng (王晓朋)
> IBM China System Technology Laboratory
> Tel: 86-10-82453455
> Email: w...@cn.ibm.com
> Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road,
> Haidian District Beijing P.R.China 100193
>
>
>
> ----- Original message -----
> From: Josh Nielsen <jniel...@hudsonalpha.org>
> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
> Cc:
> Subject: Re: [xcat-user] How can I migrate to a new xCAT MN in a
> hierarchical environment?
> Date: Thu, Jun 2, 2016 3:49 AM
>
> Can anyone verify if simply updating cfgloc should be all I need to for
> the SNs to start using the new MN? By pointing it to the new MN's MySQL
> instance, which has a site table with the new MN specified as the
> xcatmaster, it should even update the content the the xcatmaster value
> shown in an 'lsdef' of the service nodes automatically, right?
>
> Thanks,
> Josh
>
> On Tue, May 17, 2016 at 3:42 PM, Josh Nielsen <jniel...@hudsonalpha.org>
> wrote:
>
> A correction below for something I wrote previously.
>
> "...and the SNs then shouldn't need newly generated keys (right?)..."
>
> On Tue, May 17, 2016 at 3:36 PM, Josh Nielsen <jniel...@hudsonalpha.org>
> wrote:
>
> I looked at the 'servicenode' postscript and it does _way_ too much for
> what I want to accomplish. I don't think the script was written with
> changes or upgrades in mind. It looks like it freshly copies everything to
> the SNs' $installdir/postscripts and /etc/xcat on the service node and
> generates (new?) keys. The SNs don't need those updates/changes in my case.
> From looking at the following comment in the 'servicenode' postscript and
> the code I'm wondering if all I need to do is manually
> modify /etc/xcat/cfgloc to update the IP for the new MN database location
> and if everything else will be fine. They keys should already be in place
> because I am copying the same keys from the old MN onto the new MN server,
> and the SNs then shouldn't need to keys (right?). Please let me know if you
> see any problems with this.
>
> The comment in the code:
>
> For Linux:
> It calls xcatserver and xcatclient script to get the ssh keys, ssl
> redentials and cfgloc file and transfer from the MN to the SN
> to be able to access the
> database, setup ssh keys on the nodes and have daemon to daemon
> commmunication between the SN and MN and have the SN access the DB.
>
>
> P.S. Also would just giving the new MN the same IP and hostname (even as
> an alias to a different primary hostname) more or less prevent any changes
> from needing to be made on the SNs at all (no postscripts run nor manual
> modifications of files)?
>
> Thanks,
> Josh
>
> On Thu, May 5, 2016 at 11:42 AM, Josh Nielsen <jniel...@hudsonalpha.org>
> wrote:
>
> Hi Christian,
>
> Thanks for the response. So do I actually have to reinstall the SNs and/or
> rerun the service node postscript? If reruning the SN post script just
> makes some minor adjustments but doesn't clear the dhcpd.leases and the
> .conf files for named and dhcp, as I have them configured, then that would
> be fine, but if it blows all that away and starts over that would qualify
> as disruptive for my environment since the cluster depends on slave DNS
> services and dhcp on the SN. I would ideally like minimal changes on the
> SNs except to point them to the new MN.
>
> As far as the postscripts, my question was what common (if not default in
> most installs) postscripts that come with xCAT have code in them that would
> result in the hardcoding of the MN's IP in some configuration file. I
> actually thought of one possible example along those lines, and that is
> whatever configures the client compute nodes to send all their syslog
> messages to the /var/log/messages log on the headnode instead of locally
> will need to be rerun/updated. What will need to be run to change that to
> make the clients log to the new MN server?
>
> Regarding the server identity (even though it will have a new IP address
> and hostname) can we just copy the keys in /etc/ssh/ to the new MN so that
> the SSH fingerprint doesn't change?
>
> Lastly, as regards running updatenode -k I definitely (in this case) do
> not want to replace the root rsa_id private and public keys on the cluster,
> the MN, or the SNs since other critical services like GPFS require the
> current keys to remain in place. Why is rerunning the key deploy necessary
> and is there not a way to make it work with the current keys?
>
> I just need to be very careful with my current setup so that I don't knock
> out critical services while changing the MN, which is why I was wondering
> how disruptive doing this might be. I appreciate the help!
>
> Thanks,
> Josh
>
> On Tue, May 3, 2016 at 10:05 AM, Christian Caruthers <
> ccaruth...@lenovo.com> wrote:
>
> I would begin by looking at the servicenode postscript. It sets up the
> daemon and database communications between SN & MN. Beyond that, the
> default postscripts are listed in the "xcatdefaults" entry of the
> postscripts table. You will probably want to run updatenode -k once you
> have xCAT configured on the new MN. After that, you probably want to rerun
> the remoteshell and syslog postscripts on the cluster members (updatenode
> -P) at the very least.
>
>
>
> Second, you can dump the xCAT DB using dumpxCATdb command. After that,
> grep out the management node (hostname and/or IP) to see where changes need
> to be made for the DB on the new MN.
>
>
>
> If the SNs are handling DHCP, it only needs to be enabled on the MN if you
> plan in reinstaling a SN.
>
>
>
> Anything that resolves DNS through the MN will need an updated resolv.conf.
>
>
>
> Depending on how you're maintaining your /install directory on the SNs,
> that mechanism will need to be updated.
>
>
>
> If your MN is routing for any nodes, that will need to be addressed. You
> might want to check the network configuration on the IMMs. On discovery, if
> you have a gateway defined on your management network (I believe it
> defaults to <xcatmaster>), they might be pointing to the old MN. Shouldn't
> be an issue, but it's something to think about. If you're not routing on
> that network, I would use pasu to set the IMM gateway to 0.0.0.0 and be
> done with it.
>
>
>
> The only other concern I can think of would be the installation repos
> configured on the cluster nodes and SNs. If any point to the MN, they will
> need to be changed.
>
>
>
> Aside from all of that, it really depends on the particulars your cluster.
>
>
>
> Regards,
> *Christian Caruthers*
> Lenovo xESS IT Consultant
>
> Mobile: 757-289-9872
>
>
>
>
>
> *From:* Josh Nielsen [mailto:jniel...@hudsonalpha.org]
> *Sent:* Monday, May 02, 2016 8:32 PM
> *To:* xCAT Users Mailing list
> *Subject:* [xcat-user] How can I migrate to a new xCAT MN in a
> hierarchical environment?
>
>
>
> Hello all,
>
> My team is trying to move the xCAT MN role off of an old server and get it
> over onto new virtual infrastructure, but I am a little unsure about
> whether it is possible to do while leaving everything else in its place as
> we currently have it in our environment. We have an MN with two SNs for our
> xCAT environment, and I would need to make the SNs recognize that the new
> MN (with a new IP and hostname) is now their xcatmaster, and they would
> need to take hierarchical command updates from the new MN, look to the new
> MN for the xCAT database (which is a MySQL database in our environment),
> etc.
>
> So a few questions along those lines.
>
> 1. Which/how many xCAT database fields would I need to update that use the
> MN's IP (other than "master" in the site table), and would I have to
> reinstall or otherwise update anything on the SNs (I imagine restarting the
> daemons is necessary at a minimum) in case they have anything statically
> configured for the current MN's IP?
>
> 2. Do any default postscripts for deployed clients ever place the MN's
> hostname or IP in any config files that would require manual alteration if
> the MN is changed? Our client nodes should, however, have one of the two
> SNs as their designated xcatmaster, instead of the MN, as shown by an
> 'lsdef'.
>
> 3. And as far as DHCP, the MN does not even need DHCP running if the SNs
> are handling DHCP, correct? Would I have to change any of my 'networks'
> table entries and DHCP IP pool config in any case, or should simply dumping
> and importing the current DB settings in to the new MN instance be seamless?
>
> DNS I think (hope) should be an easier matter, since we already have an
> external DNS server configured that the MN pushes entries to with a
> 'makedns -e', so no DNS dependency lies on the present MN itself. I imagine
> I'd have to copy the /etc/hosts from the current MN over to the new though
> for the makedns (and other things) to continue working.
>
> I have attached an image with a simplified sketch of what our xCAT
> environment looks like. Overall I'm just wondering what changes would I
> need to make for this to be possible.
>
> Thanks for your input.
>
> Josh Nielsen
>
>
>
>
> ------------------------------------------------------------------------------
> Find and fix application performance issues faster with Applications
> Manager
> Applications Manager provides deep performance insights into multiple
> tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
>
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
>
>
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports. http://sdm.link/zohomanageengine
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
