Re: [xcat-user] Update schema after upgrade prematurely terminated
I actually checked that yesterday when you mentioned it and I had installed all the latest RPMs the first time around, and I had already restarted the daemon several times. So I'm not sure why it was misreporting the version for a while. -Josh On Thu, Jan 16, 2014 at 7:04 AM, Lissa Valletta lis...@us.ibm.com wrote: I expect xcatd restarted cleanly for some reason.Did you run yum update '*xCAT*' finally instead of yum update xcat. That would have done it also. Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 [image: Inactive hide details for Josh Nielsen ---01/14/2014 03:37:53 PM---Suddenly the lsxcatd command is returning 2.8.3 to me now. I]Josh Nielsen ---01/14/2014 03:37:53 PM---Suddenly the lsxcatd command is returning 2.8.3 to me now. I'm not sure why it would take time to up From: Josh Nielsen jniel...@hudsonalpha.org To: xCAT Users Mailing list xcat-user@lists.sourceforge.net, Date: 01/14/2014 03:37 PM Subject: Re: [xcat-user] Update schema after upgrade prematurely terminated -- Suddenly the lsxcatd command is returning 2.8.3 to me now. I'm not sure why it would take time to update, but it looks fine now: [root@x3650-head01 init.d]# lsxcatd -a Version 2.8.3 (built Tue Nov 12 23:16:15 EST 2013) This is a Management Node cfgloc=mysql:dbname=xcatdb;host=10.X.X.X|xx dbengine=mysql dbname=xcatdb dbhost=10.X.X.X dbadmin=xx And here is my rpm query: [root@x3650-head01 init.d]# rpm -qa | grep xCAT xCAT-nbroot-oss-ppc64-2.0-snap200801291320 xCAT-nbkernel-x86-2.6.18_128-8 xCAT-genesis-scripts-x86_64-2.8.3-snap201311122318 perl-xCAT-2.8.3-snap201311122316 xCAT-server-2.8.3-snap201311122316 xCAT-nbroot-core-x86-2.3.1-snap200911062009 xCAT-2.8.3-snap201311122318 xCAT-nbroot-oss-x86_64-2.0-snap200801291344 xCAT-nbroot-core-x86_64-2.3.1-snap200911062009 xCAT-nbkernel-x86_64-2.6.18_128-8 xCAT-genesis-base-x86_64-2.8-snap201308090229 xCAT-nbroot-oss-x86-2.0-snap200804021050 xCAT-nbroot-core-ppc64-2.3.1-snap200911062009 xCAT-nbkernel-ppc64-2.6.18_92-4 xCAT-buildkit-2.8.3-snap201311122318 xCAT-client-2.8.3-snap201311122316 Except for the ppc64 packages (which I don't really need anyway since we use x86_64) everything is up to the current version. The xCAT nbroot-core and nbkernel RPMs aren't up to 2.8.X but it seems that they are still at the most current version available (I have my yum repos pointing to the latest tarballs of the core and dep packages). Thanks for all the help suggestions. I'm just going to try using it now. This was all prep so that I can deploy some service nodes (as mentioned in my other emails to the mailing list). Thanks, Josh On Tue, Jan 14, 2014 at 1:55 PM, Lissa Valletta *lis...@us.ibm.com*lis...@us.ibm.com wrote: Two possibilities the daemon did not restart correctly and that has been common when upgrading from 2.7.6 to higher levels. service xcatd stop ps -ef | grep xcatd kill -9 any hung xcatd processes service xcatd start The other thing is make sure your rpms are all at the 2.8.3 level. rpm -qa | grep xCAT Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 [image: Inactive hide details for Josh Nielsen ---01/14/2014 01:05:08 PM---I seem to be getting myself in all kinds of trouble with xCA]Josh Nielsen ---01/14/2014 01:05:08 PM---I seem to be getting myself in all kinds of trouble with xCAT lately. I was upgrading from xCAT 2.7. From: Josh Nielsen *jniel...@hudsonalpha.org*jniel...@hudsonalpha.org To: xCAT Users Mailing list *xcat-user@lists.sourceforge.net*xcat-user@lists.sourceforge.net, Date: 01/14/2014 01:05 PM Subject: [xcat-user] Update schema after upgrade prematurely terminated -- I seem to be getting myself in all kinds of trouble with xCAT lately. I was upgrading from xCAT 2.7.3 to 2.8.3, I grabbed the latest core and dep tarballs, and made local yum repos, and installed all the newer packages from them. Then to check if the daemon had been upgraded I did an 'lsxcatd -v' and it still said 2.7.3. So I decided to run the xcat daemon manually from the command line with the verbose flag but wasn't expecting to see the schema updates (honestly all I saw were SQL commands scrolling in the screen - I didn't know they were for the schema at the time) and I panicked and hit CRTL+C. This halted the daemon updates but the next time I tried to run it I got an error although the daemon ran in the background. This is from my terminal: root@x3650-head01 yum.repos.d]# /opt/xcat/sbin/xcatd -v updateschema: Running ALTER TABLE mpa ADD `displayname` TEXT updateschema: Running ALTER TABLE mpa ADD `slots` TEXT updateschema: Running ALTER TABLE mpa ADD `urlpath` TEXT updateschema: Running ALTER TABLE servicenode ADD `dhcpinterfaces` TEXT
[xcat-user] Makedns wiping out DNS records
Regardless of the version of xCAT we have used in the past, my coworkers and I have noticed that running makedns with any following options can be dangerous for us, because something happens where it will wipe out all the records in DNS. If we run makedns with no options generally any new nodes that have been added to /etc/hosts will be parsed and added appropriately. Maybe I am misunderstanding what happens when the command is run. Here is a recent example. My entry for a new node xcat-serv1 for some reason didn't have the reverse lookup from the IP in the DNS file for that subnet (db.10.20), and so I decided to try to remove just the DNS entry for that node and then re-add it. Here is what I ran: # makedns -d xcat-serv1 Handling xcat-serv1 in /etc/hosts. Getting reverse zones, this may take several minutes for a large cluster. Completed getting reverse zones. Updating zones. Completed updating zones. Updating DNS records, this may take several minutes for a large cluster. Completed updating DNS records. Then I ran it with the -n parameter to re-add that node (with XCATBYPASS=1): # export XCATBYPASS=1 [root@x3650-head01 centos]# makedns -n xcat-serv1 Unrecognized escape \h passed through at /opt/xcat/lib/perl/xCAT/zvmUtils.pm line 2408. Subroutine handled_commands redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 28. Subroutine delnode redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pmline 33. Subroutine addnode redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pmline 58. Subroutine build_line redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pmline 117. Subroutine addotherinterfaces redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 182. Subroutine delotherinterfaces redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 209. Subroutine add_hosts_content redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 234. Subroutine process_request redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 305. Subroutine writeout redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pmline 511. Subroutine donics redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 545. Subroutine getIPdomain redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pmline 801. Handling xcat-serv1 in /etc/hosts. Getting reverse zones, this may take several minutes for a large cluster. Completed getting reverse zones. Stopping named: . [ OK ] Updating zones. Completed updating zones. Restarting named Restarting named complete Updating DNS records, this may take several minutes for a large cluster. Completed updating DNS records. DNS setup is completed When I did an 'ls' in /var/named many records were missing and the ones still left we virtually empty except one entry for the headnode: BEFORE: # ls /var/named data db.10.29 db.10.40 db.127.0.0db.172.26.41 db.172.29 db.192.168.100 db.haig db.morgan.haib.org.jnl db.10.29.jnl db.10.60 db.172.20 db.172.26.42 db.172.30 db.cache db.morgan slaves db.10.20 db.10.30 db.10.60.jnl db.172.26.40 db.172.26.43 db.172.40 db.haib db.morgan.haib.org AFTER: # ls /var/named data db.10.20.jnl db.10.29.jnl db.10.30.jnl db.10.40.jnl db.10.60.jnl db.172.26.41 db.172.26.43db.morgan.haib.org.jnl db.10.20 db.10.29 db.10.30 db.10.40 db.10.60 db.172.26.40 db.172.26.42 db.morgan.haib.org slaves As you can see several files are missing, Luckily I made a full backup yesterday and was able to restore the old files. From /var/log/messages you can see the initial deletion message for xcat-serv1: Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '11.102.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '7.101.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '10.102.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '27.102.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '33.101.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '1.102.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone Then for the makedens -a xcat-serv you actually see it update EVERY ZONE: Jan 16 10:32:42 x3650-head01 named[8249]: client 10.20.0.1#43788: updating zone '30.10.IN-ADDR.ARPA/IN': adding an RR at '1.50.30.10.IN-ADDR.ARPA' PTR Jan 16 10:32:42 x3650-head01 named[8249]: client 10.20.0.1#43788: updating zone '30.10.IN-ADDR.ARPA/IN':
Re: [xcat-user] Makedns wiping out DNS records
We have an entire doc on name resolution also I would not make it a common practice to use XCATBYPASS mode. Running with the xcatd is much more secure and some things do not work well in XCATBYPASS mode. https://sourceforge.net/apps/mediawiki/xcat/index.php?title=Cluster_Name_Resolution Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 From: Josh Nielsen jniel...@hudsonalpha.org To: xCAT Users Mailing list xcat-user@lists.sourceforge.net, Date: 01/16/2014 12:03 PM Subject:[xcat-user] Makedns wiping out DNS records Regardless of the version of xCAT we have used in the past, my coworkers and I have noticed that running makedns with any following options can be dangerous for us, because something happens where it will wipe out all the records in DNS. If we run makedns with no options generally any new nodes that have been added to /etc/hosts will be parsed and added appropriately. Maybe I am misunderstanding what happens when the command is run. Here is a recent example. My entry for a new node xcat-serv1 for some reason didn't have the reverse lookup from the IP in the DNS file for that subnet (db.10.20), and so I decided to try to remove just the DNS entry for that node and then re-add it. Here is what I ran: # makedns -d xcat-serv1 Handling xcat-serv1 in /etc/hosts. Getting reverse zones, this may take several minutes for a large cluster. Completed getting reverse zones. Updating zones. Completed updating zones. Updating DNS records, this may take several minutes for a large cluster. Completed updating DNS records. Then I ran it with the -n parameter to re-add that node (with XCATBYPASS=1): # export XCATBYPASS=1 [root@x3650-head01 centos]# makedns -n xcat-serv1 Unrecognized escape \h passed through at /opt/xcat/lib/perl/xCAT/zvmUtils.pm line 2408. Subroutine handled_commands redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 28. Subroutine delnode redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 33. Subroutine addnode redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 58. Subroutine build_line redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 117. Subroutine addotherinterfaces redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 182. Subroutine delotherinterfaces redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 209. Subroutine add_hosts_content redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 234. Subroutine process_request redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 305. Subroutine writeout redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 511. Subroutine donics redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 545. Subroutine getIPdomain redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 801. Handling xcat-serv1 in /etc/hosts. Getting reverse zones, this may take several minutes for a large cluster. Completed getting reverse zones. Stopping named: . [ OK ] Updating zones. Completed updating zones. Restarting named Restarting named complete Updating DNS records, this may take several minutes for a large cluster. Completed updating DNS records. DNS setup is completed When I did an 'ls' in /var/named many records were missing and the ones still left we virtually empty except one entry for the headnode: BEFORE: # ls /var/named data db.10.29 db.10.40 db.127.0.0 db.172.26.41 db.172.29 db.192.168.100 db.haig db.morgan.haib.org.jnl db.10.29.jnl db.10.60 db.172.20 db.172.26.42 db.172.30 db.cache db.morgan slaves db.10.20 db.10.30 db.10.60.jnl db.172.26.40 db.172.26.43 db.172.40 db.haib db.morgan.haib.org AFTER: # ls /var/named data db.10.20.jnl db.10.29.jnl db.10.30.jnl db.10.40.jnl db.10.60.jnl db.172.26.41 db.172.26.43 db.morgan.haib.org.jnl db.10.20 db.10.29 db.10.30 db.10.40 db.10.60 db.172.26.40 db.172.26.42 db.morgan.haib.org slaves As you can see several files are missing, Luckily I made a full backup yesterday and was able to restore the old files. From /var/log/messages you can see the initial deletion message for xcat-serv1: Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '11.102.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '7.101.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '10.102.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '27.102.40.10.IN-ADDR.ARPA' PTR Jan 16 10:32:55 x3650-head01 named[8249]: client 10.20.0.1#56080: updating zone '40.10.IN-ADDR.ARPA/IN': adding an RR at '33.101.40.10.IN-ADDR.ARPA' PTR Jan
Re: [xcat-user] Makedns wiping out DNS records
Okay, I will look through that document. This is the first and only time I've used XCATBYPASS with that command (to get more clarity into what was going on in the background). Makedns has been presenting us with the problem(s) I mentioned for well over a year now, with previous versions of xCAT as well. I'm guessing that if this doesn't happen for other people that it must have been some setting when xCAT was originally installed which was misconfigured, or maybe something funny in one of the xCAT db tables like 'site'. I just wanted to see if anyone has had problems with using the -n or -d options deleting more than they are supposed to. -Josh On Thu, Jan 16, 2014 at 11:23 AM, Lissa Valletta lis...@us.ibm.com wrote: We have an entire doc on name resolution also I would not make it a common practice to use XCATBYPASS mode. Running with the xcatd is much more secure and some things do not work well in XCATBYPASS mode. https://sourceforge.net/apps/mediawiki/xcat/index.php?title=Cluster_Name_Resolution Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 [image: Inactive hide details for Josh Nielsen ---01/16/2014 12:03:28 PM---Regardless of the version of xCAT we have used in the past,]Josh Nielsen ---01/16/2014 12:03:28 PM---Regardless of the version of xCAT we have used in the past, my coworkers and I have noticed that run From: Josh Nielsen jniel...@hudsonalpha.org To: xCAT Users Mailing list xcat-user@lists.sourceforge.net, Date: 01/16/2014 12:03 PM Subject: [xcat-user] Makedns wiping out DNS records -- Regardless of the version of xCAT we have used in the past, my coworkers and I have noticed that running makedns with any following options can be dangerous for us, because something happens where it will wipe out all the records in DNS. If we run makedns with no options generally any new nodes that have been added to /etc/hosts will be parsed and added appropriately. Maybe I am misunderstanding what happens when the command is run. Here is a recent example. My entry for a new node xcat-serv1 for some reason didn't have the reverse lookup from the IP in the DNS file for that subnet (db.10.20), and so I decided to try to remove just the DNS entry for that node and then re-add it. Here is what I ran: # makedns -d xcat-serv1 Handling xcat-serv1 in /etc/hosts. Getting reverse zones, this may take several minutes for a large cluster. Completed getting reverse zones. Updating zones. Completed updating zones. Updating DNS records, this may take several minutes for a large cluster. Completed updating DNS records. Then I ran it with the -n parameter to re-add that node (with XCATBYPASS=1): # export XCATBYPASS=1 [root@x3650-head01 centos]# makedns -n xcat-serv1 Unrecognized escape \h passed through at /opt/xcat/lib/perl/xCAT/zvmUtils.pm line 2408. Subroutine handled_commands redefined at /opt/xcat/lib/perl/xCAT_plugin/ *hosts.pm* http://hosts.pm/ line 28. Subroutine delnode redefined at /opt/xcat/lib/perl/xCAT_plugin/*hosts.pm*http://hosts.pm/ line 33. Subroutine addnode redefined at /opt/xcat/lib/perl/xCAT_plugin/*hosts.pm*http://hosts.pm/ line 58. Subroutine build_line redefined at /opt/xcat/lib/perl/xCAT_plugin/ *hosts.pm* http://hosts.pm/ line 117. Subroutine addotherinterfaces redefined at /opt/xcat/lib/perl/xCAT_plugin/ *hosts.pm* http://hosts.pm/ line 182. Subroutine delotherinterfaces redefined at /opt/xcat/lib/perl/xCAT_plugin/ *hosts.pm* http://hosts.pm/ line 209. Subroutine add_hosts_content redefined at /opt/xcat/lib/perl/xCAT_plugin/ *hosts.pm* http://hosts.pm/ line 234. Subroutine process_request redefined at /opt/xcat/lib/perl/xCAT_plugin/ *hosts.pm* http://hosts.pm/ line 305. Subroutine writeout redefined at /opt/xcat/lib/perl/xCAT_plugin/*hosts.pm*http://hosts.pm/ line 511. Subroutine donics redefined at /opt/xcat/lib/perl/xCAT_plugin/*hosts.pm*http://hosts.pm/ line 545. Subroutine getIPdomain redefined at /opt/xcat/lib/perl/xCAT_plugin/ *hosts.pm* http://hosts.pm/ line 801. Handling xcat-serv1 in /etc/hosts. Getting reverse zones, this may take several minutes for a large cluster. Completed getting reverse zones. Stopping named: . [ OK ] Updating zones. Completed updating zones. Restarting named Restarting named complete Updating DNS records, this may take several minutes for a large cluster. Completed updating DNS records. DNS setup is completed When I did an 'ls' in /var/named many records were missing and the ones still left we virtually empty except one entry for the headnode: BEFORE: # ls /var/named data db.10.29 db.10.40 db.127.0.0db.172.26.41 db.172.29 db.192.168.100 db.haig db.morgan.haib.org.jnl db.10.29.jnl db.10.60 db.172.20 db.172.26.42 db.172.30 db.cachedb.morgan slaves db.10.20 db.10.30
Re: [xcat-user] Makedns wiping out DNS records
You will see in the document that makedns is very dependent on what you do to setup you Management node. Your /etc/hosts /etc/resolv.conf, etc Setting that must be added to the database for your cluster such as in the site table, defining your network in the networks table. You can use that document to check your database configuration. Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 From: Josh Nielsen jniel...@hudsonalpha.org To: xCAT Users Mailing list xcat-user@lists.sourceforge.net, Date: 01/16/2014 12:44 PM Subject:Re: [xcat-user] Makedns wiping out DNS records Okay, I will look through that document. This is the first and only time I've used XCATBYPASS with that command (to get more clarity into what was going on in the background). Makedns has been presenting us with the problem(s) I mentioned for well over a year now, with previous versions of xCAT as well. I'm guessing that if this doesn't happen for other people that it must have been some setting when xCAT was originally installed which was misconfigured, or maybe something funny in one of the xCAT db tables like 'site'. I just wanted to see if anyone has had problems with using the -n or -d options deleting more than they are supposed to. -Josh On Thu, Jan 16, 2014 at 11:23 AM, Lissa Valletta lis...@us.ibm.com wrote: We have an entire doc on name resolution also I would not make it a common practice to use XCATBYPASS mode. Running with the xcatd is much more secure and some things do not work well in XCATBYPASS mode. https://sourceforge.net/apps/mediawiki/xcat/index.php?title=Cluster_Name_Resolution Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 Inactive hide details for Josh Nielsen ---01/16/2014 12:03:28 PM---Regardless of the version of xCAT we have used in the past, Josh Nielsen ---01/16/2014 12:03:28 PM---Regardless of the version of xCAT we have used in the past, my coworkers and I have noticed that run From: Josh Nielsen jniel...@hudsonalpha.org To: xCAT Users Mailing list xcat-user@lists.sourceforge.net, Date: 01/16/2014 12:03 PM Subject: [xcat-user] Makedns wiping out DNS records Regardless of the version of xCAT we have used in the past, my coworkers and I have noticed that running makedns with any following options can be dangerous for us, because something happens where it will wipe out all the records in DNS. If we run makedns with no options generally any new nodes that have been added to /etc/hosts will be parsed and added appropriately. Maybe I am misunderstanding what happens when the command is run. Here is a recent example. My entry for a new node xcat-serv1 for some reason didn't have the reverse lookup from the IP in the DNS file for that subnet (db.10.20), and so I decided to try to remove just the DNS entry for that node and then re-add it. Here is what I ran: # makedns -d xcat-serv1 Handling xcat-serv1 in /etc/hosts. Getting reverse zones, this may take several minutes for a large cluster. Completed getting reverse zones. Updating zones. Completed updating zones. Updating DNS records, this may take several minutes for a large cluster. Completed updating DNS records. Then I ran it with the -n parameter to re-add that node (with XCATBYPASS=1): # export XCATBYPASS=1 [root@x3650-head01 centos]# makedns -n xcat-serv1 Unrecognized escape \h passed through at /opt/xcat/lib/perl/xCAT/zvmUtils.pm line 2408. Subroutine handled_commands redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 28. Subroutine delnode redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 33. Subroutine addnode redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 58. Subroutine build_line redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 117. Subroutine addotherinterfaces redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 182. Subroutine delotherinterfaces redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 209. Subroutine add_hosts_content redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 234. Subroutine process_request redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 305. Subroutine writeout redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 511. Subroutine donics redefined at /opt/xcat/lib/perl/xCAT_plugin/hosts.pm line 545. Subroutine getIPdomain redefined at /opt/xcat/lib/perl/xCAT_plugin/ hosts.pm line 801. Handling xcat-serv1 in /etc/hosts. Getting reverse zones, this may take several minutes for a large cluster. Completed getting reverse zones. Stopping named: . [ OK ] Updating zones. Completed updating zones. Restarting named Restarting named complete Updating DNS records, this may take several minutes for a large cluster. Completed updating DNS records. DNS setup is completed
Re: [xcat-user] Makedns wiping out DNS records
Just a tip and something that bit me before - bind updates its data in the journal and won't dump new data to the zone files immediately. You can force it do dump the data so that you can see it by restarting bind. You can also test to make sure DNS is being added by digging for the record. IE, dig @localhost dayhoff.morgan.haib.org. If you get an A record back, your DNS is working fine. On 1/16/2014 12:39 PM, Josh Nielsen wrote: Is there any way to debug what exactly the makedns command does to named/bind, as in what command it issues? Or does it directly edit the named files? I could run the named daemon with "-d 9" for debugging level 9 but I've done that before and the output is daunting and nearly incomprehensible for the non-veteran in DNS. As I said, running "makedns" by it self works fine, but using -d or -n causes problems. I have no way of knowing what xCAT does in the background with those options. I checked and /etc/hosts is correct, and /etc/resolv.conf is as well. The site table has all five possible dns attributes (domain, dnshandler, forwarders, master, and nameservers) set correctly, and the networks are defined for every subnet we use. There is nothing that I can see that would cause it to parse incorrectly, especially since it echoed to /var/log/messages that it was going through /etc/hosts and creating entries (but really it didn't) for each host. I did have to add one entry manually into /etc/hosts though for a SAN storage system. Would seeing an unrecognized node cause problems? And if so why is there nothing in the logs about it? I saw it say that it was adding an entry for that node (even though it didn't - like every other entry): "Jan 16 10:32:53 x3650-head01 named[8249]: client 10.20.0.1#56079: updating zone 'morgan.haib.org/IN': adding an RR at 'dayhoff.morgan.haib.org' A" If I can find out what xCAT is doing to make the changes perhaps I can reproduce it manually in a test environment and see what is causing it to wipe out the entries. -Josh On Thu, Jan 16, 2014 at 12:01 PM, Lissa Valletta lis...@us.ibm.com wrote: You will see in the document that makedns is very dependent on what you do to setup you Management node. Your /etc/hosts /etc/resolv.conf, etc Setting that must be added to the database for your cluster such as in the site table, defining your network in the networks table. You can use that document to check your database configuration. Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 Josh Nielsen ---01/16/2014 12:44:27 PM---Okay, I will look through that document. This is the first and only time I've used XCATBYPASS with t From: Josh Nielsen jniel...@hudsonalpha.org To: xCAT Users Mailing list xcat-user@lists.sourceforge.net, Date: 01/16/2014 12:44 PM Subject: Re: [xcat-user] Makedns wiping out DNS records Okay, I will look through that document. This is the first and only time I've used XCATBYPASS with that command (to get more clarity into what was going on in the background). Makedns has been presenting us with the problem(s) I mentioned for well over a year now, with previous versions of xCAT as well. I'm guessing that if this doesn't happen for other people that it must have been some setting when xCAT was originally installed which was misconfigured, or maybe something funny in one of the xCAT db tables like 'site'. I just wanted to see if anyone has had problems with using the "-n" or "-d" options deleting more than they are supposed to. -Josh On Thu, Jan 16, 2014 at 11:23 AM, Lissa Valletta
Re: [xcat-user] Makedns wiping out DNS records
Are you running makedns or makedns -n? Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 From: Russell Jones russell-l...@jonesmail.me To: xcat-user@lists.sourceforge.net, Date: 01/16/2014 01:53 PM Subject:Re: [xcat-user] Makedns wiping out DNS records Just a tip and something that bit me before - bind updates its data in the journal and won't dump new data to the zone files immediately. You can force it do dump the data so that you can see it by restarting bind. You can also test to make sure DNS is being added by digging for the record. IE, dig @localhost dayhoff.morgan.haib.org. If you get an A record back, your DNS is working fine. On 1/16/2014 12:39 PM, Josh Nielsen wrote: Is there any way to debug what exactly the makedns command does to named/bind, as in what command it issues? Or does it directly edit the named files? I could run the named daemon with -d 9 for debugging level 9 but I've done that before and the output is daunting and nearly incomprehensible for the non-veteran in DNS. As I said, running makedns by it self works fine, but using -d or -n causes problems. I have no way of knowing what xCAT does in the background with those options. I checked and /etc/hosts is correct, and /etc/resolv.conf is as well. The site table has all five possible dns attributes (domain, dnshandler, forwarders, master, and nameservers) set correctly, and the networks are defined for every subnet we use. There is nothing that I can see that would cause it to parse incorrectly, especially since it echoed to /var/log/messages that it was going through /etc/hosts and creating entries (but really it didn't) for each host. I did have to add one entry manually into /etc/hosts though for a SAN storage system. Would seeing an unrecognized node cause problems? And if so why is there nothing in the logs about it? I saw it say that it was adding an entry for that node (even though it didn't - like every other entry): Jan 16 10:32:53 x3650-head01 named[8249]: client 10.20.0.1#56079: updating zone 'morgan.haib.org/IN': adding an RR at ' dayhoff.morgan.haib.org' A If I can find out what xCAT is doing to make the changes perhaps I can reproduce it manually in a test environment and see what is causing it to wipe out the entries. -Josh On Thu, Jan 16, 2014 at 12:01 PM, Lissa Valletta lis...@us.ibm.com wrote: You will see in the document that makedns is very dependent on what you do to setup you Management node. Your /etc/hosts /etc/resolv.conf, etc Setting that must be added to the database for your cluster such as in the site table, defining your network in the networks table. You can use that document to check your database configuration. Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 Inactive hide details for Josh Nielsen ---01/16/2014 12:44:27 PM---Okay, I will look through that document. This is the first aJosh Nielsen ---01/16/2014 12:44:27 PM---Okay, I will look through that document. This is the first and only time I've used XCATBYPASS with t From: Josh Nielsen jniel...@hudsonalpha.org To: xCAT Users Mailing list xcat-user@lists.sourceforge.net, Date: 01/16/2014 12:44 PM Subject: Re: [xcat-user] Makedns wiping out DNS records Okay, I will look through that document. This is the first and only time I've used XCATBYPASS with that command (to get more clarity into what was going on in the background). Makedns has been presenting us with the problem(s) I mentioned for well over a year now, with previous versions of xCAT as well. I'm guessing that if this doesn't happen for other people that it must have been some setting when xCAT was originally installed which was misconfigured, or maybe something funny in one of the xCAT db tables like 'site'. I just wanted to see if anyone has had problems with using the -n or -d options deleting more than they are supposed to. -Josh On Thu, Jan 16, 2014 at 11:23 AM, Lissa Valletta lis...@us.ibm.com wrote: We have an entire doc on name resolution also I would not make it a common practice to use XCATBYPASS mode. Running with the xcatd is much more secure and some things do not work well in XCATBYPASS mode. https://sourceforge.net/apps/mediawiki/xcat/index.php?title=Cluster_Name_Resolution Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 Inactive hide details for Josh Nielsen
Re: [xcat-user] Makedns wiping out DNS records
If you ran makedns -n node1 it is only going to set up DNS for node1. Check the examples in man makedns. The makedns -d node1 should only have removed node1. It you want to set up a new DNS for all nodes in /etc/hosts run makedns -n Examples 1.To set up DNS for all the hosts in /etc/hosts file. makedns 2.To set up DNS for node1. makedns node1 3.To create a new named configuration and db files for all hosts in /etc/hosts. makedns -n 4.To delete the DNS records for node1. makedns -d node1 Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 From: Lissa Valletta/Poughkeepsie/IBM@IBMUS To: xCAT Users Mailing list xcat-user@lists.sourceforge.net, Cc: xcat-user@lists.sourceforge.net Date: 01/16/2014 02:22 PM Subject:Re: [xcat-user] Makedns wiping out DNS records Are you running makedns or makedns -n? Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 Inactive hide details for Russell Jones ---01/16/2014 01:53:16 PM---Just a tip and something that bit me before - bind updates Russell Jones ---01/16/2014 01:53:16 PM---Just a tip and something that bit me before - bind updates its data in the journal and won't dump n From: Russell Jones russell-l...@jonesmail.me To: xcat-user@lists.sourceforge.net, Date: 01/16/2014 01:53 PM Subject: Re: [xcat-user] Makedns wiping out DNS records Just a tip and something that bit me before - bind updates its data in the journal and won't dump new data to the zone files immediately. You can force it do dump the data so that you can see it by restarting bind. You can also test to make sure DNS is being added by digging for the record. IE, dig @localhost dayhoff.morgan.haib.org. If you get an A record back, your DNS is working fine. On 1/16/2014 12:39 PM, Josh Nielsen wrote: Is there any way to debug what exactly the makedns command does to named/bind, as in what command it issues? Or does it directly edit the named files? I could run the named daemon with -d 9 for debugging level 9 but I've done that before and the output is daunting and nearly incomprehensible for the non-veteran in DNS. As I said, running makedns by it self works fine, but using -d or -n causes problems. I have no way of knowing what xCAT does in the background with those options. I checked and /etc/hosts is correct, and /etc/resolv.conf is as well. The site table has all five possible dns attributes (domain, dnshandler, forwarders, master, and nameservers) set correctly, and the networks are defined for every subnet we use. There is nothing that I can see that would cause it to parse incorrectly, especially since it echoed to /var/log/messages that it was going through /etc/hosts and creating entries (but really it didn't) for each host. I did have to add one entry manually into /etc/hosts though for a SAN storage system. Would seeing an unrecognized node cause problems? And if so why is there nothing in the logs about it? I saw it say that it was adding an entry for that node (even though it didn't - like every other entry): Jan 16 10:32:53 x3650-head01 named[8249]: client 10.20.0.1#56079: updating zone 'morgan.haib.org/IN': adding an RR at ' dayhoff.morgan.haib.org' A If I can find out what xCAT is doing to make the changes perhaps I can reproduce it manually in a test environment and see what is causing it to wipe out the entries. -Josh On Thu, Jan 16, 2014 at 12:01 PM, Lissa Valletta lis...@us.ibm.com wrote: You will see in the document that makedns is very dependent on what you do to setup you Management node. Your /etc/hosts /etc/resolv.conf, etc Setting that must be added to the database for your cluster such as in the site table, defining your network in the networks table. You can use that document to check your database configuration. Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 Inactive hide details for Josh Nielsen ---01/16/2014 12:44:27 PM---Okay, I will look through that document. This is the first aJosh Nielsen ---01/16/2014 12:44:27 PM---Okay, I will look through that document. This is the first and only time I've used XCATBYPASS with t From: Josh Nielsen jniel...@hudsonalpha.org To: xCAT Users Mailing list xcat-user@lists.sourceforge.net, Date: 01/16/2014 12:44 PM Subject: Re: [xcat-user] Makedns wiping out DNS records Okay, I will look through that document. This is the first and only time I've used XCATBYPASS with that command (to get more
Re: [xcat-user] Makedns wiping out DNS records
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/01/14 04:00, Josh Nielsen wrote: If it just something with our installation or does makedns do this foe other people as well? FWIW we do not run DNS on the management nodes at all, we have 4 separate xCAT built clusters (3 HPC clusters and 1 for our GPFS/TSM infrastructure and our BG/Q service and front ends) and so to let them all populate DNS with a single, consistent view we have two external DNS servers that they all send updates to with makedns -e. One kink I think we've noticed there is that even with makedns -e xCAT seems to start up bind on the local system which isn't what we want at all. Good luck! Chris - -- Christopher SamuelSenior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.org.au/ http://twitter.com/vlsci -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlLYZ4sACgkQO2KABBYQAh8pwACfSEi4RePX5F2hG5E1AtrAdy77 PtMAni1KEwsOM/az4z0U7o4aZZOEuO/+ =fPWd -END PGP SIGNATURE- -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
