>>>>> "Daniel" == Daniel Feenberg <[email protected]> writes:
Daniel> We want to PXE boot a dozen compute and file servers with over 10GBE Daniel> ethernet. All of them boot fine with the motherboard NICs. We have Daniel> a Brocade Ironport switch and a dozen direct attach cables. We also Daniel> have samples of 3 brands of 10GBE NICs to test. Why are you booting file servers via PXE? Shouldn't the be your core servers which stay up all the time and provide services? Daniel> 1) The HP card boots correctly when PXE is enabled on the NIC. Daniel> 2) Chelsio N320E gives a "PXE-E61 - Media Failure" error and Daniel> the NIC link light never comes on. It could be a bad card, how many have you tested? Daniel> 3) Brocade 1010 - says "Adaptor 1/0 link initialization failed. Daniel> Disabling BIOS" or "No target devices or link down or init Daniel> failed" depending on the NIC BIOS setting. Again, the NIC link Daniel> light does not come on. Daniel> We discount a bad cable (it works with the HP, and we have tried Daniel> several) or a motherboard incompatibility (if we boot RHEL from Daniel> a local drive and enable eth2 we can use the Chelsio or Brocade Daniel> cards). Is there some configuration issue we are missing? Chelsio Daniel> support did not offer a solution, we haven't contacted Brocade yet. I'm not sure you can discount motherboard issues, since the PXE stuff is directly tied into the BIOS and how the BIOS initialized the card and how the on-card BIOS handles PXE support. Daniel> The motherboard is a Gigabyte GA-P55M-UB2. All the cards have the Daniel> latest firmware. Is the motherboard at the latest level of firmware? Daniel> Since the failure occurs before any packets are sent it can't Daniel> be a dhcpd or tftp problem. Is the problem that some cards Daniel> offer less than full support for direct attach? Or is direct Daniel> attach not fully standardized? Should be try fiber optic Daniel> cables? The documentation for all the cards suggests that Daniel> their primary purpose is ethernet SANs. Perhaps the vendors Daniel> don't care about other uses? Daniel> We only need a single port per server, while the HP offers Daniel> 2. Because of heat, power and cost reasons, we would prefer a Daniel> single port card. How much have you spent already in terms of your time debugging this? I'd just got with the HP cards and get on with life. Get the systems up and running and then keep playing with other cards in the lab. I don't think you're going to notice heat and power by just having a second port on the system which isn't used. It will be in the noise I suspect. Daniel> Any wisdom greatly appreciated. This is our first experience Daniel> with 10GBE. We could boot over 1Gb and then switch to 10GBE Daniel> for file service, but we wish to reduce the amount of cabling. Are you also running a management network on these systems for IPMI or remote console stuff? Could you boot off that? Daniel> Two odd details - even booting from the local drive the Daniel> Chelsio card fails on intensive use if IRQPOLL is enabled. An Daniel> additional advantage of the HP card is that brief network Daniel> interuptions do not affect it, while the Chelsio card will Daniel> hang the computer if the switch reboots or a cable is moved Daniel> from one port to another. Sounds like you need to toss that Chelsio card out the window, it's not reliable and you'll just end up with all kinds of black magic voodoo. Now of course it could just be a motherboard interaction issue. Have you tried another brand of motherboard? John _______________________________________________ bblisa mailing list [email protected] http://www.bblisa.org/mailman/listinfo/bblisa
