On Mon, Jan 31, 2005 at 12:54:47PM -0800, Roland Dreier wrote:
> I seem to remember someone quoting the PCI spec saying that this was
> not allowed, but I can't find it right now. Grant, do you happen to
> know if legacy INTx interrupts are allowed to pass writes to memory
> once they leave PCI Express and make it into the chipset?
I don't know offhand and (sorry) can't look it up right now.
Poke me again later this week (thurs) if you need a solid answer
and no one else has provided one.
I do expect ordering issues with the path between the device and
the bridge chip that is handling legacy INTx support. This is basically
the same DMA ordering issues we see today with Line based interrupts.
It would make sense for the rest of the path to NOT have strict ordering
given this hole.
Ordering matrix is documented within the PCIe base 1.0a specification. The rules are fairly straight forward depending upon the capabilities being accessed. Too much to type in here but to give you an idea:
- INTx are treated as writes from a PCIe transaction perspective.
- In general, a TC defines an ordering domain. The TC to VC mapping is used to control how a transaction (aka TLP) is serviced on a per link basis. Consistency in how a TC is mapped can be important if the application is operating across multiple TC. However, servers do not require multiple TC or VC (waste of resources) nor really does any application so one can design under the assumption that a single TC is sufficient. All transactions should therefore have the same TC or you are headed for a bit of complexity and pain.
- Within a link, all TLP are processed on a given VC in the order they are presented. They are always strongly ordered across a given link. If an intermediate device such as a PCIe switch is present, it must obey the PCI ordering rules and avoid re-ordering where prohibited (e.g. a Read getting ahead of a Write which is a common flush the path technique).
- The PCIe specifications only define how a TLP is process to / from a RC to another endpoint or between endpoints. They do not define how a RC is implemented or how it interacts with the memory subsystem.
- Weak ordering is allowed in PCIe just like for PCI / PCI-X (our objective was to enable compatible operations while minimizing the pain to migrate to a new interconnect technology).
- In general, all interrupts (line or MSI/MSI-X) should be strongly ordered relative to other write operations to avoid silent data corruption from occurring. As such, an interrupt should not pass a memory write when being processed by the chipset.
If you have any other questions, I'm happy to try and answer or one can read the friendly specification for more specifics.
Mike
_______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
