[xcat-user] ESXi VM Discovery Deployment
Hello all, Our organization is in the process of shifting our HPC model from an all physical/bare metal compute cluster to a virtualized compute cluster, making each physical compute node a standalone ESXi host (without vCenter licensing or central management). Because we are not using vCenter the vm-specific xCAT commands are not of much use to us, but I'm not so much concerned about that as with how to redesign/organize the discovery deployment process for VMs on the ESXi hosts. With our current physical compute cluster we had used the ultra handy SNMP switch port discovery method to identify and label nodes with regular expressions, creating compute hosts with simple names like node0001, node0002, etc. Now the ESXi hosts take on those names and use the SNMP switch port discovery method for their naming, IP addresses, etc. But once that is done I need to determine how best to deploy VMs on top of those ESXi hosts and how discovery will work with them. Our intended naming scheme will be to name each VM, per host, after the name of the ESXi host with letters appended to them. So say ESXi host node0001 will have three VMs deployed: we would name them node0001a, node0001b, and node0001c. From what I can tell I cannot use the SNMP method of identifying those VMs. Since I may have to create the VMs by hand anyway (or deploy from a template), perhaps I can use the most tedious method of manually populating the MAC addresses, but I am wondering if anyone has any better ideas for ways to accomplish that. I would welcome any suggestions or pointers for things that I haven't thought of yet. Thanks! Josh Nielsen -- Don't Limit Your Business. Reach for the Cloud. GigeNET's Cloud Solutions provide you with the tools and support that you need to offload your IT needs and focus on growing your business. Configured For All Businesses. Start Your Cloud Today. https://www.gigenetcloud.com/___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] ESXi VM Discovery Deployment
Also, what will the 'switch' xCAT table look like with multiple VMs on the same physical host, since the man page for it says contains what switch port numbers each node is connected to? On Mon, Jul 6, 2015 at 1:31 PM, Josh Nielsen jniel...@hudsonalpha.org wrote: Hello all, Our organization is in the process of shifting our HPC model from an all physical/bare metal compute cluster to a virtualized compute cluster, making each physical compute node a standalone ESXi host (without vCenter licensing or central management). Because we are not using vCenter the vm-specific xCAT commands are not of much use to us, but I'm not so much concerned about that as with how to redesign/organize the discovery deployment process for VMs on the ESXi hosts. With our current physical compute cluster we had used the ultra handy SNMP switch port discovery method to identify and label nodes with regular expressions, creating compute hosts with simple names like node0001, node0002, etc. Now the ESXi hosts take on those names and use the SNMP switch port discovery method for their naming, IP addresses, etc. But once that is done I need to determine how best to deploy VMs on top of those ESXi hosts and how discovery will work with them. Our intended naming scheme will be to name each VM, per host, after the name of the ESXi host with letters appended to them. So say ESXi host node0001 will have three VMs deployed: we would name them node0001a, node0001b, and node0001c. From what I can tell I cannot use the SNMP method of identifying those VMs. Since I may have to create the VMs by hand anyway (or deploy from a template), perhaps I can use the most tedious method of manually populating the MAC addresses, but I am wondering if anyone has any better ideas for ways to accomplish that. I would welcome any suggestions or pointers for things that I haven't thought of yet. Thanks! Josh Nielsen -- Don't Limit Your Business. Reach for the Cloud. GigeNET's Cloud Solutions provide you with the tools and support that you need to offload your IT needs and focus on growing your business. Configured For All Businesses. Start Your Cloud Today. https://www.gigenetcloud.com/___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] ESXi VM Discovery Deployment
From my point view that you should consider the bare-metal and virtual machine management to be two levels. First, use switch-based discovery to discovery bare-metal node and install ESXI as a general operating system; Second, manually or using script to define the virtual machine against certain host. After the vm node definition has done, the hardware control and OS deployment for vm node will be simple since it's very similar with the bare-metal node. You mentioned the discovery of vm. I am curious about this requirement. Is that because the vm was not created by xcat (like mkvm command), so you need to discovery the vm from certain host? xCAT does not have command to discovery/scan host to get vm list. A simple way is to use 'xdsh' to run virsh command against the host. BTW, don't your organization think the performance might be a problem to move from bare-metal to virtual machine? Thanks Best Regards -- Wang Xiaopeng (王晓朋) IBM China System Technology Laboratory Tel: 86-10-82453455 Email: w...@cn.ibm.com Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China 100193 From: Josh Nielsen jniel...@hudsonalpha.org To: xCAT Users Mailing list xcat-user@lists.sourceforge.net Date: 2015/07/07 08:28 Subject:Re: [xcat-user] ESXi VM Discovery Deployment Also, what will the 'switch' xCAT table look like with multiple VMs on the same physical host, since the man page for it says contains what switch port numbers each node is connected to? On Mon, Jul 6, 2015 at 1:31 PM, Josh Nielsen jniel...@hudsonalpha.org wrote: Hello all, Our organization is in the process of shifting our HPC model from an all physical/bare metal compute cluster to a virtualized compute cluster, making each physical compute node a standalone ESXi host (without vCenter licensing or central management). Because we are not using vCenter the vm-specific xCAT commands are not of much use to us, but I'm not so much concerned about that as with how to redesign/organize the discovery deployment process for VMs on the ESXi hosts. With our current physical compute cluster we had used the ultra handy SNMP switch port discovery method to identify and label nodes with regular expressions, creating compute hosts with simple names like node0001, node0002, etc. Now the ESXi hosts take on those names and use the SNMP switch port discovery method for their naming, IP addresses, etc. But once that is done I need to determine how best to deploy VMs on top of those ESXi hosts and how discovery will work with them. Our intended naming scheme will be to name each VM, per host, after the name of the ESXi host with letters appended to them. So say ESXi host node0001 will have three VMs deployed: we would name them node0001a, node0001b, and node0001c. From what I can tell I cannot use the SNMP method of identifying those VMs. Since I may have to create the VMs by hand anyway (or deploy from a template), perhaps I can use the most tedious method of manually populating the MAC addresses, but I am wondering if anyone has any better ideas for ways to accomplish that. I would welcome any suggestions or pointers for things that I haven't thought of yet. Thanks! Josh Nielsen -- Don't Limit Your Business. Reach for the Cloud. GigeNET's Cloud Solutions provide you with the tools and support that you need to offload your IT needs and focus on growing your business. Configured For All Businesses. Start Your Cloud Today. https://www.gigenetcloud.com/ ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user -- Don't Limit Your Business. Reach for the Cloud. GigeNET's Cloud Solutions provide you with the tools and support that you need to offload your IT needs and focus on growing your business. Configured For All Businesses. Start Your Cloud Today. https://www.gigenetcloud.com/___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Multiple IPs during postinstall in kickstart causing DDNS issues
Your explanation make the issue more clear. The code has been checked in,
it will get into 2.10 which will be released at end of July.
BTW, the confusing thing was why an interface will get another IP when do
the dhcp request again? You were using dynamic Ip for installnic?
Thanks
Best Regards
--
Wang Xiaopeng (王晓朋)
IBM China System Technology Laboratory
Tel: 86-10-82453455
Email: w...@cn.ibm.com
Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road,
Haidian District Beijing P.R.China 100193
From: Samveen ListReceipient list-u...@samveen.in
To: xCAT Users Mailing list xcat-user@lists.sourceforge.net
Date: 2015/07/06 12:13
Subject:Re: [xcat-user] Multiple IPs during postinstall in kickstart
causing DDNS issues
- the node -
[root@xcatmn01 postscripts]# lsdef compute1
Object name: compute1
arch=x86_64
currchain=boot
currstate=install rhels6.6-x86_64-compute
groups=compute,ipmi,all
initrd=xcat/osimage/rhels6.6-x86_64-install-compute/initrd.img
installnic=eth0
ip=10.4.4.50
kcmdline=quiet repo=http://!myipfn!:80/install/rhels6.6/x86_64
ks=http://!myipfn!:80/install/autoinst/compute1 ksdevice=eth0
kernel=xcat/osimage/rhels6.6-x86_64-install-compute/vmlinuz
mac=00:25:90:14:FA:30!compute1-s|00:25:90:14:FA:31!compute1|
00:25:90:14:f3:49!compute1-m
netboot=xnba
nichostnamesuffixes.ipmi=-m
nichostnamesuffixes.eth0=-s
nicips.eth1=10.4.4.50
nicips.ipmi=10.1.240.50
nictypes.ipmi=Ethernet
nictypes.eth1=Ethernet
nictypes.eth0=Ethernet
nodetype=osi
os=rhels6.6
postbootscripts=otherpkgs
postscripts=syslog,remoteshell,syncfiles,create_root_authorized_keys
primarynic=eth1
profile=compute
provmethod=rhels6.6-x86_64-install-compute
PLEASE NOTE THAT ETH0 IS NOT STATICALLY ASSIGNED
- the process -
1. Node boots up
2. Node asks for IP via dhcp (eg 10.2.72.50) on mac=00:25:90:14:FA:30
3. Server assigns the lease IP AND creates 2 DDNS entries:
compute1 A 10.2.72.50
50 PTR compute1-s.poc.lzd.co.
4. Node starts netboot/xNBA process
5. Kickstart starts and asks for DHCP again.
6. Server assigns lease IP and
7. kickstart completes and starts the postscripts from the templates (
/opt/xcat/share/xcat/install/scripts/post.rh.common )
- Here the problem starts -
As my primarynic=eth1, the script tries to see if eth1 is up or not. If it
is not, it tries to bring it up.
However Line 41 of /opt/xcat/share/xcat/install/scripts/post.rh.common
is :
8. 41 dhclient eth0
This causes the eth0 interface to get a 2nd IP
9. Server assigns another IP to node (eg. 10.2.72.51) and updates DDNS:
compute1 A 10.2.72.51
51 PTR compute1-s.poc.lzd.co.
So on the server:
[root@xcatmn01 scripts]# host 10.2.72.50
Host 50.72.2.10.in-addr.arpa. not found: 3(NXDOMAIN)
[root@xcatmn01 scripts]# host 10.2.72.51
51.72.2.10.in-addr.arpa domain name pointer compute1.poc.lzd.co.
And on the node:
bash-4.1.2 # ip -4 a s eth0
2: eth0: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP qlen
1000
inet 10.2.72.50/24 brd 10.2.72.255 scope global eth0
inet 10.2.72.51/24 brd 10.2.72.255 scope global eth0
Now to connect to the server, the node address used is '10.2.72.50', but
the ip does not resolve to a name anymore.
So the script /install/postscripts/updateflag.awk keeps on trying to update
the server with it's status, and the server keeps ignoring the update,
leaving it in a stuck state.
I hope this clarifies the problem I face.
Regards
On Mon, Jul 6, 2015 at 9:40 AM, Xiao Peng Wang w...@cn.ibm.com wrote:
In your case, why the first line in my email did not get an IP in your
scenario? Even your IP changed in the dhcp server, the PRINIC should
have the first IP in this situation.
And what's your specific requirement to have two IPs?
IP=$(ip addr show dev $PRINIC | grep inet | grep -v inet6 | awk '{print
$2}' | head -n 1 | awk -F '/' '{print $1}')
if [ -z $IP ]
then
dhclient eth0
#IP=$(ifconfig $PRINIC | grep inet | awk '{print $2}' | awk -F:
'{print $2}')
IP=$(ip addr show dev $PRINIC | grep inet | grep -v inet6 | awk
'{print $2}' | head -n 1 | awk -F '/' '{print $1}')
fi
Thanks
Best Regards
--
Wang Xiaopeng (王晓朋)
IBM China System Technology Laboratory
Tel: 86-10-82453455
Email: w...@cn.ibm.com
Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road,
Haidian District Beijing P.R.China 100193
Inactive hide details for Samveen ListReceipient ---2015/07/01
12:59:45---The IP didn't change. Instead 2 IPs were assigned to Samveen
ListReceipient ---2015/07/01 12:59:45---The IP didn't change. Instead 2
IPs were assigned to
