On Tue, Feb 14, 2012 at 09:34:50PM +1100, Jonathan Gray wrote: > On Tue, Feb 14, 2012 at 10:52:49AM +0100, David Imhoff wrote: > > >Synopsis: Intel PRO/1000 PF em() network card not working with MSI > > >on Dell R610 > > >Category: Kernel/i386 > > >Environment: > > System : OpenBSD 5.1 > > Details : OpenBSD 5.1 (GENERIC.MP) #5: Mon Feb 13 12:12:18 CET > > 2012 > > > > [email protected]:/usr/src/sys/arch/i386/compile/GENERIC.MP > > > > Architecture: OpenBSD.i386 > > Machine : i386 > > >Description: > > My Intel PRO/1000 PF network card in a Dell PowerEdge R610 server, > > stopped > > functioning after upgrading to OpenBSD 5.0. The card does not > > transmit any > > data. When trying to send a packet the following error appears in the > > kernel logging: > > "em0: watchdog timeout -- resetting" > > > > I found that disabling MSI in the em* driver, by always setting > > 'sc->osdep.em_pa.pa_flags &= ~PCI_FLAGS_MSI_ENABLED;' in em_attach(), > > made the card function again. Of course this is not a structural fix. > > > > The problem does not seem card or driver related. Since the same model > > and subtype card, in a Dell PowerEdge 1950, does function properly > > with > > MSI enabled. I also tried the same setup with multiple revisions of > > the > > R610, with the same result. > > This appears to be a silicon level flaw: > > From "Intel 82571EB/82572EI Ethernet Controller Specification Update": > > 63. Byte Enables 2 and 3 Are Not Set on MSI Writes Problem: MSI > (format code definition Message Signal Interrupts) writes on the 82571EB > will not have the upper two Byte Enables (BE) set. > > Implication: The PCI specification requires Byte Enables 2 and 3 to be > set even though that data will always be zero. Because the > 82571/82572 does not set these Byte Enables, MSI writes fail to > generate interrupts on systems with chipsets that have been designed > to require these Bytes Enables to be set. This errata only applies > when MSI is supported and enabled by the system and OS. > > Workaround: None, As long as MSI is being used, Byte Enables 2 and 3 > will not be set. > > Status: No Fix. > > -- > > So it seems to be safe we should not try to enable MSI on 82571/82572 > at all. > > Index: if_em.c > =================================================================== > RCS file: /cvs/src/sys/dev/pci/if_em.c,v > retrieving revision 1.261 > diff -u -p -r1.261 if_em.c > --- if_em.c 5 Oct 2011 02:52:09 -0000 1.261 > +++ if_em.c 14 Feb 2012 10:27:03 -0000 > @@ -329,8 +329,11 @@ em_attach(struct device *parent, struct > /* Determine hardware revision */ > em_identify_hardware(sc); > > - /* Only use MSI on the newer PCIe parts */ > - if (sc->hw.mac_type < em_82571) > + /* > + * Only use MSI on the newer PCIe parts, with the exception > + * of 82571/82572 due to "Byte Enables 2 and 3 Are Not Set" errata > + */ > + if (sc->hw.mac_type <= em_82572) > sc->osdep.em_pa.pa_flags &= ~PCI_FLAGS_MSI_ENABLED; > > /* Parameters (to be read from user) */ >
OK claudio@ -- :wq Claudio
