This message is from the T13 list server.
Pat,
Your latest covers the same ground we have already covered before. In DATA
IN operations in ATAPI (or ATA) you always physically transfer (and thus
physically take up in the buffer memory) an even number of bytes (CF devices
being a notable, but irrelevant, exception). This is simply because the ATA
physical/electrical bus is 16 bits wide, and you always use all of the bits.
This is true for PIO AND DMA (SW, MW or Ultra).
So a byte in the host buffer memory can never be "stepped on" in DMA due to
returning an even number of bytes when an odd allocation length was
specified by the host. If that were true, you would be seeing the exact
same problem in every host today using PIO transfers. Since you are not,
you are not going to see the problem in DMA.
I suppose hosts either pad out that last byte in their memory (so they get
an even number of bytes, but the last byte steps on nothing) or some agent
in the data stream on the host side suitably truncates the returned data to
the odd number of bytes originally specified by the host. The details are
not very important, since the fact remains that the host deals fine with
even byte returns for odd byte allocation lengths today using PIO.
Basically, can you provide data that shows an actual problem here?
On DATA OUT operations, the transferring of less bytes than specified in the
command would result in a hang since the device is still waiting for the
remaining bytes (does that make sense?). So the only possible issue is
sending more bytes than specified originally by the host.
In SCSI this is never an issue, since the device paces the transfers, and so
will simply not ask for more bytes than specified. The one exception to
this is the case of an odd transfer length (as per your example), where a
wide SCSI bus has to request an even number of bytes. But in this case the
device knows to discard the last byte since it is over the amount originally
specified by the host.
Imagine now that we expanded SCSI to 32 bits (as indeed has been does in
some applications). Now you potentially have to discard up to the last
three bytes transferred. Once again, no problem - the device has all of the
information it needs to accomplish this task.
Now look at PIO. Like wide SCSI, all physical transfers are in a multiple
of an even number of bytes. So the device has to be able to discard the
last byte if the transfer length was an odd number of bytes. If more bytes
come in, then the device is explicitly allowed to discard them and complete
the command successfully. I'd be a harder task master here, since the host
never has a good reason to send out more bytes that it originally specified
in the command, but the ATA standard allows hosts to do this.
In sum, there is no problem I can see with DATA IN or DATA OUT. Using wider
buses and the like do complicate things over the old 8 bit SCSI, but the
standards and the industry appear to handle them well. Specifically, I
still know of no examples not covered by these provisions (including the
examples you include below - see embedded notes).
Jim
-----Original Message-----
From: Pat LaVarre [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, January 02, 2002 5:36 PM
To: [EMAIL PROTECTED]
Subject: [t13] AtapiDma byte counts how imprecise?
This message is from the T13 list server.
> > ... I'm more confident in algebra than in English ...
> > If [D = H] ... "has to be" ... what makes it so?
> "Mcgrath, Jim" <[EMAIL PROTECTED]> 01/02/02 05:49PM
> all of your examples have been DATA IN, not DATA OUT.
Thanks for explaining here how I failed to hear what you heard:
tellingly, this is not MY memory of this confused conversation. I wonder
how little of what I rehearsed in thought I did then post? Below at the
next occurrence of the seven char string "UDmaOut" I (re?)post a Data OUT
example.
> Harlan Andrews <[EMAIL PROTECTED]> 01/02/02 03:43PM
> What are you talking about?
Thank you likewise for you continued interest, and your help in
making my concerns here less muddy. I'll try now to restate my question in
short. Then I will follow with the two most clearly helpful AtapiDmaIn and
AtapiDmaOut examples people here have helped me remember ...
> What are you talking about?
I'm trying to make it harder for an app to discover when (and thus
to choke if) a host & my new device negotiate an AtapiDma transfer where old
hosts and old devices always arranged for an AtapiPio transfers to occur.
I've concluded I need help from the host.
By "the host" I agree with Hale: I mean the OS for a direct-attached
Atapi device, the bridge for an Atapi device connected via a Usb/Atapi
bridge, and so on.
I'd very much like to be told I'm wrong. Otherwise I'd like to see
us all extend the Atapi standard to let Atapi devices invite Atapi aware
hosts to help. Then we could fix this once in the OS/ bridge/ whatever,
rather than repeatedly in every app.
> SwDmaIn/ UDmaIn
Specifically, I'm having trouble designing an AtapiDma device to
cooperate with the host to reproduce precisely the legacy AtapiPio byte
counts.
One byte count that matters is the count of bytes written or read
that the OS reports to the app.
Another byte count that matters is the count of bytes actually
written or read.
> Perhaps you could give an example ...
Suppose we try:
iocb x 12 0 0 0 FF 0 /i x1000
In English, this means the app sent the Cdb x 12 0 0 0 FF 0 (Inquiry
For Up To xFF Bytes) and the app asked the OS to move up to 4KiB of data
bytes In.
Now suppose the device under test makes x91 bytes of Inquiry data
available. With an AtapiPio device we see the host copies in x91 bytes.
With an AtapiDma device we see the host copies in x92 bytes.
JPM - this is incorrect. At the ATA interface level you always see 92
bytes, whether it is PIO or DMA. It is up to something else on the host
side that knows to ignore the last byte. Ultimately this is due to the fact
that the command, with the 91 byte provision, came from the host anyway.
You are focusing on the bridge situation, but you have to remember that
ATAPI works fine in a native ATA host - something in the host software is
stripping out that last byte (or causing it to be ignored). Now true, the
USB host software is not the same as the ATAPI software - but that just
means the USB software has to emulate the ATAPI software and ignore that
last byte. I assume that is a function of the USB Mass Storage Class
driver, but I'm not an expert here. Since USB products are shipping fine
today, I assume there is not a problem with the USB software stack (but I
could be wrong).
Whoops.
We're left hoping that that x92'nd byte of memory was pinned,
allocated, and never will be fetched to branch on. And we hope the app
doesn't care about precision in the count of bytes moved that the OS
reports.
> UDmaOut
In simulation, me, I also see that bridging to Atapi UDma OUT moves
as many as 2 * X + 1 extra bytes out, where X is the burst "word" length
indeterminacy built into Ide Udma protocols that rises with burst rate.
> Perhaps you could give an example ...
Suppose we try a WriteBuffer of x1FD bytes from a x1000 byte buffer:
iocb x 3B:02 00:00:00:00 0 01:FD 0 /o x1000
With an AtapiPio device, I see precisely x1FD bytes moved out.
With an AtapiUDma33 device I see x1FE, x200, or x202 bytes move,
randomly by how the buffering/timing worked out in the bridge. With hosts
that pause for a turnaround delay at x200 byte boundaries, I most commonly
see x200 bytes move. Never do I see x1FD bytes move.
JPM - once again, overruns of this sort in ATAPI/ATA are explicitly allowed
for in the ATA standard, and the device is told to ignore the extra bytes
(and the deice knows they are extra since it knows the transfer length
provided by the host). So there is no problem.
Whoops.
Again, we're left hoping those x202 bytes of memory were all pinned
and allocated, no matter that the legacy only tells us x1FD bytes of memory
were ok. And again we hope the app doesn't care about precision in the
count of bytes moved that the OS reports.
JPM - Why are you worried about host memory anyway on a DATA OUT? You are
reading from host memory - it makes no difference if the bytes were pinned
or not, since they were not altered (and are discarded by the device
anyway). As to the OS report, once again this is being done today routinely
using the ATAPI software stack, so clearly the OS is perfectly fine with it.
While there could be a problem with the USB stack, the fact that products
are shipping today makes that unlikely (once again, I could be wrong on the
USB software side of things).
Me, in my own app code, I align the i/o buffer in virtual space and
I include one extra physical page on the end, which covers the long history
of pain I've had with byte counts being imprecise.
For me, how much imprecision to disregard in byte counts is a
configuration parameter, often zero for block transfers and something less
persnickety for plug 'n play transactions.
> UDmaIn
In simulation, the imprecision in byte counts is fully symmetric:
the device can ask to move X * 2 + 1 more bytes than the host expects.
JPM - your simulation is incorrect. Once again, the DATA In and DATA OUT
are not symmetric. The ATA standard allows for the host to overrun the
device in a DATA OUT:
A host may send extra data words on the last Ultra DMA burst of a data-out
command. If a device
determines that all data has been transferred for a command, the device
shall terminate the burst. A device
may have already received more data words than were required for the
command. These extra words are
used by both the host and the device to calculate the CRC, but, on an Ultra
DMA data-out burst, the extra
words shall be discarded by the device.
There is no similar provision for DATA OUT.
Jim argued forcefully that such a device is broken, I agree.
If you asked, I'd argue that a host should cope: it should report
that the device is broken. But I don't mind leaving that "should" out of
the standard at least until we can agree on what goes wrong when neither the
host or the device is broken.
> MwDmaIn/ MwDmaOut
I'm told the MwDma standard as written neglects to require the
device to turnaround DMARQ quickly enough to avoid the host thinking it
wants to move another two bytes than it does.
But I also hear in practice this isn't much of an issue. People
found this was enough of a problem that they taught their devices to
turnaround DMARQ quickly. Accordingly, I don't mind leaving MwDma out of
the discussion until we can agree exactly how imprecise UDma is.
> ...
I am any clearer than mud, yet? Happy new year?
Thanks in advance, hope this helps. Pat LaVarre
Subscribe/Unsubscribe instructions can be found at www.t13.org.
Subscribe/Unsubscribe instructions can be found at www.t13.org.
