Roland,
I checked with our chief hardware engineer and here's what he had to say :-
Yes, config writes are non-posted, but, no, it should not
block everything.
From the 1.0 version of the PCI-X spec, sec 8.4.4 Transaction
Ordering and Passing Rules for Bridges, p 153
"Split Requests are permitted to be blocked by or to pass
other Split Requests. (These Split Transactions in PCI-X have
the same requirements as Delayed Transactions in conventional
PCI.)"
(Appendix E of the version 2.3 PCI spec says the same thing
about Delayed Transactions)
And on p151 of the PCI-X spec:
"If an initiator requires two Split Transactions to complete
in order, the initiator must not issue the second request
until the first Split Transaction completes."
So in the failing example, the bridge chip is issuing the 2nd Split
Transaction (the read) before the 1st (the config write) completes.
From my hardware bias, it seems like the driver should be in
charge of determining whether or not there's an ordering dependence
and take the appropriate action. Otherwise, the hardware has no way
of knowing whether or not there's an order requirement and will
have to serialize everything. That's a negative impact on
performance and kind of renders the spec-allowed option of passing
or not passing meaningless.
Roland Dreier wrote:
> > The posted/non-posted write stuff in the spec really only means that a
> > split completion is generated for that transaction on the bus. There
> > is no bus-level requirement that the bus halt while an outstanding
> > split is pending. In fact, the PCI-X ordering rules in this case
> > actually would allow your config read and memory read to be re-ordered
> > by a the bridge (table 8-3). ``Split requests are permitted to be
> > blocked by or pass other split requests.''
> >
> > Most implementations block the CPU on a non-posted write which
> > provides the necessary serialization, but Altix clearly didn't..
>
> Thanks, I think I get it now. It does seem like this behavior skirts
> right inside the boundary of what the PCI spec allows. What was
> confusing me was the section:
>
> Non-posted transactions reach their ultimate destination before
> completing at the originating device. The master cannot proceed
> with any other work until the transaction has completed at the
> ultimate destination (if a dependency exists).
>
> I just saw "the master cannot proceed" but the first few times I read
> this, I didn't see the "(if a dependency exists)." Since no
> dependency exists between the config write and the subsequent memory
> read, the master _can_ proceed in this case. So I do agree that this
> patch looks correct and needed, although the Altix behavior is
> somewhat unusual.
So, do you need anything alse from me like an attached copy of the patch ?
Thanks
John
--
John Partridge
Silicon Graphics Inc
Tel: 651-683-3428
Vnet: 233-3428
E-Mail: [EMAIL PROTECTED]
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
