For the record, this transition of changing MNs with all the other
hierarchy and SNs in place and not being reinstalled actually worked. I
more or less followed those steps except for #2, which proved to be
unnecessary for me (and would have been problematic to regenerate the keys,
for other reasons I outlined before).
The mysqlsetup (and some manual double-checking of cfgloc) took care of the
database switch over. Initially I did run into problems after the
switch-over when trying to issue commands from the MN that used the
hierarchy, and I got a permission denied error, which led me to realize
that the policy database table needed to be updated with the new MN's
hostname and marked as 'trusted', which one mailing list thread tipped me
off about: https://sourceforge.net/p/xcat/mailman/message/27717510/
Overall though I have provided a proof of concept that it is possible to
change MNs in an established hierarchical environment in my own instance.
If anyone ever needs help in this area feel free to shoot me an email.
Regards,
Josh Nielsen
On Fri, Jun 3, 2016 at 7:01 AM, Xiao Peng Wang <w...@cn.ibm.com> wrote:
> I think we should talk it as opposite way that how to make the MN to use
> the new SN.
>
> Following steps are necessary to switch a SN:
> 1. rerun 'mysqlsetup -f' to assign the access permission for SN to access
> DB on MN
> 2. run 'updatenode -k <sn>' to set up the ssh key
> 3. run 'updatenode -P' to update the SN
> 4. change the 'servicenode' attribute for compute node accordingly.
>
>
> Thanks
> Best Regards
> ----------------------------------------------------------------------
> Wang Xiaopeng (王晓朋)
> IBM China System Technology Laboratory
> Tel: 86-10-82453455
> Email: w...@cn.ibm.com
> Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road,
> Haidian District Beijing P.R.China 100193
>
>
>
> ----- Original message -----
> From: Josh Nielsen <jniel...@hudsonalpha.org>
> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
> Cc:
> Subject: Re: [xcat-user] How can I migrate to a new xCAT MN in a
> hierarchical environment?
> Date: Thu, Jun 2, 2016 3:49 AM
>
> Can anyone verify if simply updating cfgloc should be all I need to for
> the SNs to start using the new MN? By pointing it to the new MN's MySQL
> instance, which has a site table with the new MN specified as the
> xcatmaster, it should even update the content the the xcatmaster value
> shown in an 'lsdef' of the service nodes automatically, right?
>
> Thanks,
> Josh
>
> On Tue, May 17, 2016 at 3:42 PM, Josh Nielsen <jniel...@hudsonalpha.org>
> wrote:
>
> A correction below for something I wrote previously.
>
> "...and the SNs then shouldn't need newly generated keys (right?)..."
>
> On Tue, May 17, 2016 at 3:36 PM, Josh Nielsen <jniel...@hudsonalpha.org>
> wrote:
>
> I looked at the 'servicenode' postscript and it does _way_ too much for
> what I want to accomplish. I don't think the script was written with
> changes or upgrades in mind. It looks like it freshly copies everything to
> the SNs' $installdir/postscripts and /etc/xcat on the service node and
> generates (new?) keys. The SNs don't need those updates/changes in my case.
> From looking at the following comment in the 'servicenode' postscript and
> the code I'm wondering if all I need to do is manually
> modify /etc/xcat/cfgloc to update the IP for the new MN database location
> and if everything else will be fine. They keys should already be in place
> because I am copying the same keys from the old MN onto the new MN server,
> and the SNs then shouldn't need to keys (right?). Please let me know if you
> see any problems with this.
>
> The comment in the code:
>
> For Linux:
> It calls xcatserver and xcatclient script to get the ssh keys, ssl
> redentials and cfgloc file and transfer from the MN to the SN
> to be able to access the
> database, setup ssh keys on the nodes and have daemon to daemon
> commmunication between the SN and MN and have the SN access the DB.
>
>
> P.S. Also would just giving the new MN the same IP and hostname (even as
> an alias to a different primary hostname) more or less prevent any changes
> from needing to be made on the SNs at all (no postscripts run nor manual
> modifications of files)?
>
> Thanks,
> Josh
>
> On Thu, May 5, 2016 at 11:42 AM, Josh Nielsen <jniel...@hudsonalpha.org>
> wrote:
>
> Hi Christian,
>
> Thanks for the response. So do I actually have to reinstall the SNs and/or
> rerun the service node postscript? If reruning the SN post script just
> makes some minor adjustments but doesn't clear the dhcpd.leases and the
> .conf files for named and dhcp, as I have them configured, then that would
> be fine, but if it blows all that away and starts over that would qualify
> as disruptive for my environment since the cluster depends on slave DNS
> services and dhcp on the SN. I would ideally like minimal changes on the
> SNs except to point them to the new MN.
>
> As far as the postscripts, my question was what common (if not default in
> most installs) postscripts that come with xCAT have code in them that would
> result in the hardcoding of the MN's IP in some configuration file. I
> actually thought of one possible example along those lines, and that is
> whatever configures the client compute nodes to send all their syslog
> messages to the /var/log/messages log on the headnode instead of locally
> will need to be rerun/updated. What will need to be run to change that to
> make the clients log to the new MN server?
>
> Regarding the server identity (even though it will have a new IP address
> and hostname) can we just copy the keys in /etc/ssh/ to the new MN so that
> the SSH fingerprint doesn't change?
>
> Lastly, as regards running updatenode -k I definitely (in this case) do
> not want to replace the root rsa_id private and public keys on the cluster,
> the MN, or the SNs since other critical services like GPFS require the
> current keys to remain in place. Why is rerunning the key deploy necessary
> and is there not a way to make it work with the current keys?
>
> I just need to be very careful with my current setup so that I don't knock
> out critical services while changing the MN, which is why I was wondering
> how disruptive doing this might be. I appreciate the help!
>
> Thanks,
> Josh
>
> On Tue, May 3, 2016 at 10:05 AM, Christian Caruthers <
> ccaruth...@lenovo.com> wrote:
>
> I would begin by looking at the servicenode postscript. It sets up the
> daemon and database communications between SN & MN. Beyond that, the
> default postscripts are listed in the "xcatdefaults" entry of the
> postscripts table. You will probably want to run updatenode -k once you
> have xCAT configured on the new MN. After that, you probably want to rerun
> the remoteshell and syslog postscripts on the cluster members (updatenode
> -P) at the very least.
>
>
>
> Second, you can dump the xCAT DB using dumpxCATdb command. After that,
> grep out the management node (hostname and/or IP) to see where changes need
> to be made for the DB on the new MN.
>
>
>
> If the SNs are handling DHCP, it only needs to be enabled on the MN if you
> plan in reinstaling a SN.
>
>
>
> Anything that resolves DNS through the MN will need an updated resolv.conf.
>
>
>
> Depending on how you're maintaining your /install directory on the SNs,
> that mechanism will need to be updated.
>
>
>
> If your MN is routing for any nodes, that will need to be addressed. You
> might want to check the network configuration on the IMMs. On discovery, if
> you have a gateway defined on your management network (I believe it
> defaults to <xcatmaster>), they might be pointing to the old MN. Shouldn't
> be an issue, but it's something to think about. If you're not routing on
> that network, I would use pasu to set the IMM gateway to 0.0.0.0 and be
> done with it.
>
>
>
> The only other concern I can think of would be the installation repos
> configured on the cluster nodes and SNs. If any point to the MN, they will
> need to be changed.
>
>
>
> Aside from all of that, it really depends on the particulars your cluster.
>
>
>
> Regards,
> *Christian Caruthers*
> Lenovo xESS IT Consultant
>
> Mobile: 757-289-9872
>
>
>
>
>
> *From:* Josh Nielsen [mailto:jniel...@hudsonalpha.org]
> *Sent:* Monday, May 02, 2016 8:32 PM
> *To:* xCAT Users Mailing list
> *Subject:* [xcat-user] How can I migrate to a new xCAT MN in a
> hierarchical environment?
>
>
>
> Hello all,
>
> My team is trying to move the xCAT MN role off of an old server and get it
> over onto new virtual infrastructure, but I am a little unsure about
> whether it is possible to do while leaving everything else in its place as
> we currently have it in our environment. We have an MN with two SNs for our
> xCAT environment, and I would need to make the SNs recognize that the new
> MN (with a new IP and hostname) is now their xcatmaster, and they would
> need to take hierarchical command updates from the new MN, look to the new
> MN for the xCAT database (which is a MySQL database in our environment),
> etc.
>
> So a few questions along those lines.
>
> 1. Which/how many xCAT database fields would I need to update that use the
> MN's IP (other than "master" in the site table), and would I have to
> reinstall or otherwise update anything on the SNs (I imagine restarting the
> daemons is necessary at a minimum) in case they have anything statically
> configured for the current MN's IP?
>
> 2. Do any default postscripts for deployed clients ever place the MN's
> hostname or IP in any config files that would require manual alteration if
> the MN is changed? Our client nodes should, however, have one of the two
> SNs as their designated xcatmaster, instead of the MN, as shown by an
> 'lsdef'.
>
> 3. And as far as DHCP, the MN does not even need DHCP running if the SNs
> are handling DHCP, correct? Would I have to change any of my 'networks'
> table entries and DHCP IP pool config in any case, or should simply dumping
> and importing the current DB settings in to the new MN instance be seamless?
>
> DNS I think (hope) should be an easier matter, since we already have an
> external DNS server configured that the MN pushes entries to with a
> 'makedns -e', so no DNS dependency lies on the present MN itself. I imagine
> I'd have to copy the /etc/hosts from the current MN over to the new though
> for the makedns (and other things) to continue working.
>
> I have attached an image with a simplified sketch of what our xCAT
> environment looks like. Overall I'm just wondering what changes would I
> need to make for this to be possible.
>
> Thanks for your input.
>
> Josh Nielsen
>
>
>
> ------------------------------------------------------------
> ------------------
> Find and fix application performance issues faster with Applications
> Manager
> Applications Manager provides deep performance insights into multiple
> tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
>
>
> ------------------------------------------------------------
> ------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
>
>
>
> ------------------------------------------------------------
> ------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
>
>
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
