Gerard Roudier wrote...
>
>
> On Thu, 6 May 1999, Kenneth D. Merry wrote:
>
> > Well, the major bug in LXY4 isn't really related to tagged queueing
> > specifically. It's that the drive locks up under load. But it's easiest
> > to reach that sort of load by enabling tagged queueing.
>
> Hmmm ...
> You should be right, but very partially, in my opinion, since you omit
> something important, I think.
>
> Some firmwares, and notably the Atlas' ones, seem to complete immediately
> SCSI transactions when TGQ is used and they are lacking of some resources.
> Such a situation can be avoided by not enabling write caching and using
> a reasonnable number of tags (32 for the ATlas II works).
>
> Such a behaviour makes the old Atlas I L912 return QUEUE FULL with no more
> that ZERO command disconnected for example, when it is heavily stressed
> with write caching enabled.
> I never got that with the ATlas II LXY4, but I got some QUEUE FULL with no
> more than 2 disconnect commands. When write caching is disabled, the
> device does not experience those weirdnesses.
Hmm, that's interesting. I just did a quick iozone run on this drive:
da0 at ahc0 bus 0 target 0 lun 0
da0: <QUANTUM XP34550W LYK8> Fixed Direct Access SCSI-2 device
da0: 40.000MB/s transfers (20.000MHz, offset 8, 16bit), Tagged Queueing Enabled
da0: 4341MB (8890760 512 byte sectors: 255H 63S/T 553C)
And you're right, with write caching disabled, I didn't get any QUEUE FULL
responses. (The maximum number of transactions for that drive is set to
32, so presumably the drive can handle at least that.)
> By the way, under Linux the write caching does not help a lot for
> performances since everything is cached by the kernel. This is probably
> very different with O/Ses that do synchronous IOs, as seen by the kernel,
> for meta-data. Note that the disk write caching is just doing the
> asynchronous IO that the kernel has been too fearfull to do by itself. ;-)
Well, for FreeBSD we've got "softupdates", which is essentially asychronous
filesystem metadata updates. If softupdates are enabled, enabling write
caching on a drive won't really make any difference, and can make
performance a little more uneven.
If softupdates aren't enabled, though, enabling write caching on a drive
generally yields higher performance on random I/O especially.
> > Well, the main bug in LYK8 is the same bug that most every high-end Quantum
> > drive has nowadays. It continually returns Queue Full under high load, even
> > when there are very few transactions queued to the drive.
>
> This is not a bug, in my opinion and conforms to SCSI specifications.
It may conform to the SCSI specs, but it is annoying.
> The only thing that is questionnable is to return QUEUE FULL with ZERO
> command disconnected. Such a situation is very painfull to handle from
> the kernel/driver. It is kind of 'full and empty condition' that cannot be
> handled without wasting either time or SCSI BUS bandwitch.
> Anyway, it is still compliant with SCSI specifications.
You're right, it does make things difficult for an operating system to deal
with when a drive says that it's full and there are no commands oustanding.
:)
IBM and Seagate disks don't exhibit the same behavior, I wish Quantum would
follow suit.
> > We "fixed" it in FreeBSD by setting a lower bound on the number of
> > transaction slots (24) for Atlas II and III drives. Without that, we
> > would automatically reduce the number of slots (or queue depth) to the
> > system-wide minimum for devices with tagged queueing enabled (2
> > transactions).
>
> The ncr53c8xx + sym53c8xx driver of Linux (that started from the ncr
> driver, as you probably know) are handling this situation.
>
> In order to handle intelligently and dynamically the device (logical unit)
> CCB queue depth, we must have knowledge of the _actual_ number of
> disconnected CCB when the QUEUE FULL occurs. When a QUEUE FULL occurs, we
> must immediately reduce (temporalily) the device queue depth to something
> not greater than the number of the disconnected CCBs at the time the QUEUE
> FULL has been returned, or wait for a completion before _actually_
> queuing to the a new command to the device.
>
> On the other hand, the _order_ of commands must be preserved.
>
> I donnot know about the FreeBSD heuristic, but if it is not based on what
> I just mentionned, it should be not that smart.
> May-be you would want to tell us how it works.
It works more or less like you described above. When the FreeBSD SCSI
layer gets a queue full, it reduces the number of possible oustanding
transactions for a device to the number of transactions currently
oustanding. e.g., there are 64 transactions oustanding to a drive, and
when we try to queue a 65th transaction, we get a queue full. The limit
for that device is then set to 64. I don't think we yet have logic in
place to increase the number of transactions after it has decreased, but
that is in the works.
Transaction ordering is maintained whenever an I/O request is rejected with
queue full.
Most of the tagged queueing logic in FreeBSD is handled in the CAM
transport layer (or "mid-layer"). There are also limits imposed on a
per-controller basis, depending on the capabilities of the controller and
driver. For instance, I have a Seagate Barracuda on an NCR 810 controller:
ncr0: <ncr 53c810a fast10 scsi> at pci0.18.0
ncr0: interrupting at irq 10
da0 at ncr0 bus 0 target 0 lun 0
da0: <SEAGATE ST34371N 0484> Fixed Direct Access SCSI-2 device
da0: 10.000MB/s transfers (10.000MHz, offset 8), Tagged Queueing Enabled
da0: 4148MB (8496884 512 byte sectors: 255H 63S/T 528C)
Generally, Seagate Barracudas can handle 63 transactions at a time.
However, for this drive, the limit is 32 because that is the hard limit for
the FreeBSD NCR driver at the moment (most other drivers don't have
driver-based limitations on the number of transactions):
# ./camcontrol inquiry -n da -u 0 -v
pass0: <SEAGATE ST34371N 0484> Fixed Direct Access SCSI-2 device
pass0: Serial Number JDB176660L0J88
pass0: 10.000MB/s transfers (10.000MHz, offset 8), Tagged Queueing Enabled
# ./camcontrol tags -n da -u 0 -v
(pass0:ncr0:0:0:0): dev_openings 32
(pass0:ncr0:0:0:0): dev_active 0
(pass0:ncr0:0:0:0): devq_openings 32
(pass0:ncr0:0:0:0): devq_queued 0
(pass0:ncr0:0:0:0): held 0
(pass0:ncr0:0:0:0): mintags 2
(pass0:ncr0:0:0:0): maxtags 255
'mintags' and 'maxtags' are the transport layer hard minimum and maximum
number of transactions for the device. The 'dev_openings' parameter is the
number of transaction slots available on the drive. 'dev_active' is the
number of transactions oustanding on the drive. 'devq_openings' is the
number of transaction slots available in the kernel queue, and
'devq_queued' is the number of items currently in the kernel queue.
To deal with the Atlas II drives (when write caching is enabled), we set
mintags to 24 and maxtags to 32 via a kernel device quirk entry. So no
matter how many times the drive returns queue full, we don't reduce the
number of slots below 24. We just retry the command after that...
Ken
--
Kenneth Merry
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to [EMAIL PROTECTED]
