On Jan 22, 2014, at 8:30 AM, Jonathan Mills <jonmi...@renci.org> wrote:
> Comments inline...
>
> On 1/22/14, 8:08 AM, David D Johnson wrote:
>> I've been lurking on this discussion, and just checked to see what we've
>> got -- nbroot or genesis -- and we have both of them.
>>
>> I had given up on node discovery years ago, we originally used the
>> switch port numbers and forwarding tables to assign node names. Now I
>> use ASU to collect the macs, and populate the mac table with a bit of
>> grep and awk. But last week we powered on a rack of 20 non-ibm nodes,
>> and I was wishing we had something easier since ASU didn't work for them.
>
> Precisely! That's what I've been doing for IBM and Dell gear for a long
> time. In fact, here's what I do:
>
> for i in `seq 1 100`
> do
> MAC=$(rinv node${i} mac | grep 'MAC Address 1' | cut -d " " -f 5)
> chtab node="node${i}" mac.mac=$MAC mac.interface="eth0"
> done;
>
> ....or similar.
>
Nice, "rinv mac" seems to take much less time that "asu show --group PXE",and
it can run in parallel on a node range. Unfortunately neither works for these
(SupermicrO) nodes.
>>
>> So I have two questions --
>> 1) Can I safely delete the xCAT-nbroot-core* RPMS ?
>
> I still don't know! Because if using the chain-loading, I don't see how
> the first stage is installed by the xCAT-genesis-* RPMS.
>
>> 2) What is the current best practice method? What about for non-ibm
>> hardware?
>
> This is also what I'm trying to establish. I had been attempting to use
> sequential node discovery for Cisco UCS-B series equipment, since the
> switch method would be hard to use (since UCS's Fabric Interconnect is
> kinda like a switch but not totally).
>
> I am also familiar with ROCKS Clusters method of sequential node
> discovery (insert-ethers), which literally scrapes MAC addresses out of
> the dhcpd lines which appear in /var/log/messages -- IMHO that would
> have worked far better for xCAT than by the method of PXE booting (or
> chain-loading) the Genesis kernel -- which may or may not have kernel
> modules for your NIC hardware -- to send messages back to xcatmaster:3001.
>
Our clusters from 2006-2009 were all based on Rocks, and that was the one
feature I miss the most.
This way would work for any hardware type.
The Rocks web-GUI database was OK for the small clusters, but they moved to the
"rocks" cli for making changes just about the time we went to xcat-2. We in
fact used xcat-1 for all the hardware management, rcons rpower, etc.
I miss the ability to hack on the python scripts.
>>
>> No, three questions
>> 3) How do you get IBM manufacturing to use a specific different 172.29.X
>> for each rack they build for you?
>> We've had three racks arrive in different months all with 172.29.101
>> addresses for the IMM, and I have to spend 5-10 minutes reprogramming
>> each one. I can't put them on the same network until the conflicts are
>> gone.
>
> For a price, IBM has an integration center with technicians can make
> such things happen. For another price, they even offer a kind of DMZ
> they call "the yellowzone" where you can SSH into their lab and
> pre-configure your gear before it ships. But it isn't worth the effort
> unless you're going to be buying a lot of things with some frequency.
>
We get the racks prebuilt and shipped from Hong Kong, and they do program the
IMM addresses, but they never ask us which rack number to use, it's always A1.
If they're going to do it at all, they should do it right.
>> What's hard for me is that we get new nodes only a couple times a year,
>> and I forget everything in between.
>>
>> Thanks,
>> -- ddj
>>
>> On Jan 22, 2014, at 7:12 AM, Lissa Valletta <lis...@us.ibm.com
>> <mailto:lis...@us.ibm.com>> wrote:
>>
>>> xCAT-nbroot-core* was replaced by xCAT-genesis-* in xCAT 2.8.
>>>
>>> Lissa K. Valletta
>>> 8-3/B10
>>> Poughkeepsie, NY 12601
>>> (tie 293) 433-3102
>>>
>>>
>>>
>>> <graycol.gif>Xiao Peng Wang ---01/22/2014 02:58:13 AM---Why do you say
>>> that you need nbk.x86_64? Is this file listed in the
>>> </tftpboot/xcat/xnba/nets/>?
>>>
>>> From: Xiao Peng Wang <w...@cn.ibm.com <mailto:w...@cn.ibm.com>>
>>> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net
>>> <mailto:xcat-user@lists.sourceforge.net>>,
>>> Cc: xCAT Users Mailing list <xcat-user@lists.sourceforge.net
>>> <mailto:xcat-user@lists.sourceforge.net>>
>>> Date: 01/22/2014 02:58 AM
>>> Subject: Re: [xcat-user] Frustrating time with sequential node discovery
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>>
>>> Why do you say that you need nbk.x86_64? Is this file listed in the
>>> </tftpboot/xcat/xnba/nets/>?
>>>
>>> With the latest xCAT build, it needs
>>> /tftpboot/xcat/genesis.kernel.x86_64 instead of nbk.*
>>>
>>> Thanks
>>> Best Regards
>>> ----------------------------------------------------------------------
>>> Wang Xiaopeng (王晓朋)
>>> IBM China System Technology Laboratory
>>> Tel: 86-10-82453455
>>> Email: w...@cn.ibm.com <mailto:w...@cn.ibm.com>
>>> Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road,
>>> Haidian District Beijing P.R.China 100193
>>>
>>> <graycol.gif>Jonathan Mills ---2014/01/22 14:10:32---It would seem to
>>> me that what I am missing is the whole of the xCAT-nbroot
>>> infrastructure...because
>>>
>>> From: Jonathan Mills <jonmi...@renci.org <mailto:jonmi...@renci.org>>
>>> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net
>>> <mailto:xcat-user@lists.sourceforge.net>>,
>>> Date: 2014/01/22 14:10
>>> Subject: Re: [xcat-user] Frustrating time with sequential node discovery
>>> ------------------------------------------------------------------------
>>>
>>>
>>>
>>> It would seem to me that what I am missing is the whole of the
>>> xCAT-nbroot infrastructure...because it isn't part of xcat-core, nor
>>> xcat-dep. So I didn't grab it. But it just so happens...you need it.
>>>
>>> The file
>>>
>>> /tftpboot/xcat/nbk.x86_64
>>>
>>> is provided by the RPM xCAT-nbkernel-x86_64. Which is missing from my
>>> yum repo mirrior, and from my hosts.
>>>
>>>
>>> Anything else I'm missing? Hopefully if I grab correct copies of
>>> xCAT-nbkernel and xCAT-nbroot (or xCAT-nbroot2?) then node discovery
>>> will actually work.
>>>
>>> On 1/22/14, 12:08 AM, Xiao Peng Wang wrote:
>>>> Both Josh and Russell are correct.
>>>>
>>>> xNBA is a customized pxe and genesis is a xCAT customized diskless linux
>>>> system to run discovery and other tasks like 'bmcsetup'. It does not
>>>> need the /tftpboot/pxelinux.cfg/.* to load the genesis.
>>>>
>>>> For discovery, if a node is not defined in xCAT, the dhcp configuration
>>>> in the /etc/dhcp/dhcpd.conf or /etc/dhcpd.conf is used to reply the dhcp
>>>> request from not-discovered node.
>>>>
>>>> In your dhcpd.conf, it should have the following part for your
>>>> deployment network. If not, run 'makedhcp -n' to recreate your
>>> dhcpd.conf.
>>>> if option user-class-identifier = "xNBA" and option
>>>> client-architecture = 00:00 { #x86, xCAT Network Boot Agent
>>>> always-broadcast on;
>>>> filename =
>>> "_http://10.1.0.207/tftpboot/xcat/xnba/nets/10.1.0.0_16_";;
>>>> } else if option user-class-identifier = "xNBA" and option
>>>> client-architecture = 00:09 { #x86, xCAT Network Boot Agent
>>>> filename =
>>>> "_http://10.1.0.207/tftpboot/xcat/xnba/nets/10.1.0.0_16.uefi_";;
>>>> } else if option client-architecture = 00:00 { #x86
>>>> filename "xcat/xnba.kpxe";
>>>> } else if option vendor-class-identifier = "Etherboot-5.4" { #x86
>>>> filename "xcat/xnba.kpxe";
>>>> } else if option client-architecture = 00:07 { #x86_64 uefi
>>>> filename "xcat/xnba.efi";
>>>> } else if option client-architecture = 00:09 { #x86_64 uefi
>>>> alternative id
>>>> filename "xcat/xnba.efi";
>>>> }
>>>>
>>>> During the boot process of a not-discovered node, dhcpd will tell the
>>>> node to load xcat/xnba.kpxe first and then the configuration file
>>>> http://<xcat mn>/tftpboot/xcat/xnba/nets/10.1.0.0_16. Then the xnba will
>>>> load the genesis.
>>>>
>>>> Take a look of the syslog to see whether the xnba was downloaded
>>>> successfully from tftp server. And look into the httpd log to see
>>>> whether the genesis has been downloaded successfully.
>>>>
>>>>
>>>> Thanks
>>>> Best Regards
>>>> ----------------------------------------------------------------------
>>>> Wang Xiaopeng (王晓朋)
>>>> IBM China System Technology Laboratory
>>>> Tel: 86-10-82453455
>>>> Email: w...@cn.ibm.com <mailto:w...@cn.ibm.com>
>>>> Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road,
>>>> Haidian District Beijing P.R.China 100193
>>>>
>>>> Inactive hide details for Josh Nielsen ---2014/01/22 05:56:00---Ah, I
>>>> see what you are saying now. Well, I hope the thread I stJosh Nielsen
>>>> ---2014/01/22 05:56:00---Ah, I see what you are saying now. Well, I hope
>>>> the thread I stumbled on that Jarrod replied to help
>>>>
>>>> From: Josh Nielsen <jniel...@hudsonalpha.org
>>> <mailto:jniel...@hudsonalpha.org>>
>>>> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net
>>> <mailto:xcat-user@lists.sourceforge.net>>,
>>>> Date: 2014/01/22 05:56
>>>> Subject: Re: [xcat-user] Frustrating time with sequential node discovery
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>> Ah, I see what you are saying now. Well, I hope the thread I stumbled
>>>> on that Jarrod replied to helps figure out why his configuration is
>>>> looking to the outdated (according to what Jarrod said) configuration
>>>> files in /tftpboot/pxelinux.cfg/. Looks like it is either
>>>> /etc/dhcpd.conf or /var/lib/dhcpd/dhcpd.leases related in that case.
>>>>
>>>> On Tue, Jan 21, 2014 at 3:51 PM, Russell Jones
>>>> <russell-l...@jonesmail.me <mailto:russell-l...@jonesmail.me>> wrote:
>>>>> It *should* work with xNBA and Genesis - xNBA is the PXE image that
>>>>> loads Genesis. :-)
>>>>>
>>>>> Genesis is the utility image that handles shell commands,
>>> runimages, etc.
>>>>>
>>>>> Don't confuse NBFS with xNBA - NBFS is deprecated via Genesis.
>>> xNBA is
>>>>> the gpxe image that loads Genesis or your normal OS image
>>> depending on
>>>>> what you sent via nodeset. Genesis would not be able to load without
>>>>> xNBA (or standard PXE), and neither would any netboot images.
>>>>>
>>>>> On 1/21/2014 3:33 PM, Josh Nielsen wrote:
>>>>>> my case it still works with
>>>>>> both xnba and genesis because of the nature of PXE chainloading. It
>>>>>> probably adds deployment time, but it actually works in such a mixed
>>>>>> configuration.
>>>>>>
>>>>>> -Josh
>>>>>
>>>>>
>>>>>
>>>>
>>> ------------------------------------------------------------------------------
>>>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>>>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>>>>> Critical Workloads, Development Environments & Everything In Between.
>>>>> Get a Quote or Start a Free Trial Today.
>>>>>
>>>>
>>> _http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk_
>>>>> _______________________________________________
>>>>> xCAT-user mailing list
>>>>> xCAT-user@lists.sourceforge.net
>>> <mailto:xCAT-user@lists.sourceforge.net>
>>>>> _https://lists.sourceforge.net/lists/listinfo/xcat-user_
>>>>
>>>>
>>> ------------------------------------------------------------------------------
>>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>>>> Critical Workloads, Development Environments & Everything In Between.
>>>> Get a Quote or Start a Free Trial Today.
>>>>
>>> _http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk_
>>>> _______________________________________________
>>>> xCAT-user mailing list
>>>> xCAT-user@lists.sourceforge.net <mailto:xCAT-user@lists.sourceforge.net>
>>>> _https://lists.sourceforge.net/lists/listinfo/xcat-user_
>>>>
>>>>
>>>
>>> --
>>> Jonathan Mills
>>> Systems Administrator
>>> Renaissance Computing Institute
>>> UNC-Chapel Hill
>>>
>>> ------------------------------------------------------------------------------
>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>>> Critical Workloads, Development Environments & Everything In Between.
>>> Get a Quote or Start a Free Trial Today. _
>>> __http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk_
>>> _______________________________________________
>>> xCAT-user mailing list
>>> xCAT-user@lists.sourceforge.net <mailto:xCAT-user@lists.sourceforge.net>_
>>> __https://lists.sourceforge.net/lists/listinfo/xcat-user_
>>> ------------------------------------------------------------------------------
>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>>> Critical Workloads, Development Environments & Everything In Between.
>>> Get a Quote or Start a Free Trial Today.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk_______________________________________________
>>> xCAT-user mailing list
>>> xCAT-user@lists.sourceforge.net <mailto:xCAT-user@lists.sourceforge.net>
>>> https://lists.sourceforge.net/lists/listinfo/xcat-user
>>>
>>> ------------------------------------------------------------------------------
>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>>> Critical Workloads, Development Environments & Everything In Between.
>>> Get a Quote or Start a Free Trial Today.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk_______________________________________________
>>> xCAT-user mailing list
>>> xCAT-user@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/xcat-user
>>
>
> --
> Jonathan Mills
> Systems Administrator
> Renaissance Computing Institute
> UNC-Chapel Hill
>
> ------------------------------------------------------------------------------
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
