On Mon, Apr 03, 2006 at 09:36:00PM -0700, Grant Grundler wrote: > On Mon, Apr 03, 2006 at 03:14:56PM -0700, Greg Lindahl wrote: > > Red Hat has started turning off CONFIG_PCI_MSI in their kernels (FC5 > > and the latest FC4 update). I remember a while back there was a > > discussion about how MSI made the Mellanox HCA run faster, can someone > > please add some concrete details about this to the bug? Thanks. > I also don't see why MSI support should be disabled. > MSI should work fine on most of the platforms out there. > For the cases where MSI does not work correctly and there > is no workaround in place yet, Matthew Wilcox proposed a > "platform" MSI disable patch. User can then disable > MSI globally at boot time.
One thing that struck me as strange with the MSI code was that it just unconditionally turns on without any checks that the system actually supports it, so that it breaks in some cases doesn't seem to be too surprising. MSI requires end device support and something in a bridge to transform the MSI message into an APIC message, but the kernel currently only looks for end device support. Presumably only some subset of Intel PCI/PCI-X bridges support MSI translation to an APIC message on the FSB, and we know on HT systems that the MSI PCI capability block is required (and must be *enabled*) in the PCI/PCI-X/PCIe bridge for there to be support. IMHO what is needed is that if a device X wants MSI on then the path from X to the root bridge must hit a MSI capable bridge[1]. White list the known good intel chips, detect HT capability blocks, turn on MSI for *every* device (ie 100% of PCIe devices), not just high performance ones that specifically ask for it - then black list any broken hardware ;> In the long term as PCIe becomes dominant this is best since linux becomes firmly in control of all interrupt routing and doesn't rely on working APCI for IOAPIC tables.. It is hard to blame RedHat for turning it off by default when I know some BIOS vendors are not enabling MSI translation in the HT chipsets. I have some MBs that are broken like that and MSI only works with a kernel patch that forces the MSI translation to be turned on. Ideally the kernel would detect that and just refuse to do MSI. [1: In this same way the current hack to disable MSI if an 8131 is detected is bogus, there are systems with 8131's and other nvidia bridges that support MSI on PCIe slots (via a nvidia bridge) but not on PCI-X (since the 8131 doesn't support it)] Regards, Jason _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
