Bill,
I have a great deal of *painful* experience with the PDSMi and
systemimager. The problem you are experiencing is that the 82573V and
82573E (one of each on the PDSMi) require a very recent version of the
e1000 driver that will not be present in systemimager's pxe kernel. You
will need to regenerate a boot kernel that contains e1000 source around
7.0.41 or later (what I tested personally).
Unfortunately getting over the pxe e1000 driver issue will only move
you forward to the next, and more significant problem. You will find
that when running heavy mpi i/o the PDSMi will drop a large number of
packets and this will occur on all machines participating in the mpi
job. Running applications like Pallas (Intel IMB) or netpipe over as few
as 4 nodes you can drop as much as 12% of your receive packets.
This issue was escalated up to senior engineering at Supermicro and
Intel and was not resolved. It remains a mystery. We found two paths to
resolution. Disabling the onboard 82573 nics in the PDSMi and installing
a PCIe 82572EI (Pro1000/T) card in each machine (requires a PCIe riser)
or scrap the PDSMi nodes altogether and replacing with Asus RS120
systems that are the same as the PDSMi except they use a Broadcom PCIe
nic that does not suffer from the same ills.
To pursue the path of adding the PCIe nic to the PDSMi nodes you
will need: (per node)
Supermicro PCIe 1U riser (Supermicro P/N# CSE-RR1U-ELi)
Intel Pro1000/T PCIe gigabit card (Intel P/N# EXPI9300PT)
To pursue replacing the PDSMi with Asus I have tested the Asus RS120-E3
If you are doing HPC work I would strongly suggest the Asus approach
because it just works out of the box. The Intel card is a workable
solution but still requires a customized pxe kernel with updated e1000
driver and lots of bios setting changes and motherboard jumper settings
to make the added PCIe nic card eth0 and the primary pxe device. If you
are only doing file/print, mail or web services and using systemimager
to deploy then the PDSMi would likely suffice without modification.
--Jeff
Bill McKie wrote:
> Hi
>
> We've been happily & gratefully using systemimager (v3.2.0-4) for some time
> now
> to manage system installs & updates for several linux clusters.
>
> We're now adding new cluster boxes that are built using the SuperMicro PDSMi
> motherboard, which includes 2 Intel 82573 gigabit ethernet interfaces, an
> IDE/ATA hard disk, and Pentium-4 CPU.
>
> We prepared for autoinstalling with systemimager on these boxes in our usual
> way:
>
> o Custom configured a FC4 golden client on one of these PDSMi boxes.
>
> o Installed our usual set of client side sis RPMs on the golden client:
>
> rpm -Uvh \
> perl-AppConfig-1.52-4.noarch.rpm \
> systemconfigurator-2.0.9-1.noarch.rpm \
> systemimager-common-3.2.0-4.noarch.rpm \
> systemimager-client-3.2.0-4.noarch.rpm
>
> o Our systemimager image server (running FC4) has the following RPMs
> installed:
>
> perl-XML-Simple-1.08-1.noarch.rpm
> perl-AppConfig-1.52-4.noarch.rpm
> systemimager-i386boot-standard-3.2.0-4.noarch.rpm
> systemimager-common-3.2.0-4.noarch.rpm
> systemimager-server-3.2.0-4.noarch.rpm
> systemconfigurator-2.0.9-1.noarch.rpm
>
> o Successfully tested golden client network communications through its
> 82573 interfaces with the image server, including dhcp IP assignment.
>
> o Successfully ran the usual prepareclient on the golden client
>
> o Successfully pulled the golden client's image to the image server
> using getimage.
>
> o Ensured that PXE network booting was set up on the image server
> under /tftpboot. [The imager server is also the dhcp server.]
>
> o Set up dhcp server entries for fixed-address IP assignment and PXE
> network autoinstalling for a fresh PDSMi box that does not yet have
> an OS installed.
>
> o Set up network booting through one of the 82573 interfaces in the
> BIOS of the fresh PDSMi box.
>
> o Ensured that the systemimager rsyncd server is running on the
> image server.
>
> o Booted the fresh PDSMi box.
>
> o Watched the dhcp server successfully assign the expected IP to the
> fresh PDSMi box. [I.e. PXE booting within the 82573 ethernet
> interface successfully communicates with the dhcp server.]
>
> o Watched pxelinux successfully boot across the network. [I.e.
> the dhcp server is successfully handing off network booting
> to tftpd.]
>
> o Watched the systemimager kernel download and then take control of
> the client PDSMi box, successfully detecting the IDE/ATA disk. But
> it then stalled when it tried to configure the ethernet interface.
> Manually trying a few ifconfig commands from the breakout shell
> indicated that no eth0 or eth1 devices were known to the systemimager
> kernel.
>
> o Replaced the standard systemimager kernel under /tftpboot with
> Peter Mueller's kernel, repeated the client autoinstall boot,
> and again the ethernet interface was not detected.
>
> So it appears that I need a systemimager kernel that supports the
> intel 82573 ethernet interface and is compatible with systemimager
> v3.2.0-4. Does such a kernel exist, and if yes, where can I obtain
> it?
>
> [I've been reading about UYOK under newer systemimager version(s),
> but it is not yet clear to me whether or not this is available now
> in a stable version of systemimager. And since everything except the
> kernel is working fine with our current systemimager 3.2.0-4 setup,
> it would be desirable to not change anything except the autoinstall
> kernel if that is possible.]
>
> Following is the output from the lspci command for the PDSMi box,
> showing the presence of the intel 82573 ethernet interfaces:
>
> 00:00.0 Host bridge: Intel Corporation E7230 Memory Controller Hub (rev 81)
> 00:01.0 PCI bridge: Intel Corporation E7230 PCI Express Root Port (rev 81)
> 00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1
> (rev 01)
> 00:1c.4 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI
> Express Port 5 (rev 01)
> 00:1c.5 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI
> Express Port 6 (rev 01)
> 00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI #1
> (rev 01)
> 00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI #2
> (rev 01)
> 00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI #3
> (rev 01)
> 00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI #4
> (rev 01)
> 00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI
> Controller (rev 01)
> 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1)
> 00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface
> Bridge (rev 01)
> 00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller
> (rev 01)
> 00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev
> 01)
> 02:00.0 PCI bridge: Intel Corporation 6702PXH PCI Express-to-PCI Bridge A
> (rev 09)
> 02:00.1 PIC: Intel Corporation 6700/6702PXH I/OxAPIC Interrupt Controller A
> (rev 09)
> 04:00.0 Ethernet controller: Intel Corporation 82573E Gigabit Ethernet
> Controller (Copper) (rev 03)
> 05:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet
> Controller
> 0a:00.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
>
>
> Thanks,
> Bill McKie
> NASA Ames Research Center
> [EMAIL PROTECTED]
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Sisuite-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/sisuite-users
>
--
Best Regards,
Jeff Johnson
Vice President
Engineering/Technology
Western Scientific, Inc
[EMAIL PROTECTED]
http://www.wsm.com
9445 Farnham Street - San Diego, CA 92123
Tel 800.443.6699 +001.858.565.6699
Fax +001.858.565.6938
"Abra Capocus" - Bugs Bunny
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Sisuite-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sisuite-users
