Re: SCSI testing/USB devices are amazing
On 04/28/13 18:19, ronnie sahlberg wrote: Hi List, Interested in SCSI tests? I have a reasonable SCSI (mainly SBC) testsuite at https://github.com/sahlberg/libiscsi while I am mainly interested in testing of iSCSI targets, most of my tests so far are for the SCSI protocol so they work quite well on devices connected to any kind of transport, as long as you re-export them via iSCSI. (For example using a simple passthrough device with TGT.) At some stage I will redo some of the framework so that it can optionally talk directly to a /dev/sg* device, but reexporting via TGT works well enough for now. Hello Ronnie, Thanks for making this software publicly available. I have a question about one particular test though. In the output of the test suite I found the following: quoteTest READ16 with non-zero RDPROTECT. Device does not support/use protection information. All commands should fail./quote However, in SBC-3 I found the following: quoteIf type 0 protection is enabled and the RDPROTECT field, the WRPROTECT field, the VRPROTECT field, or the ORPROTECT field is set to a non-zero value, then medium access commands are invalid and may be terminated by the device server with CHECK CONDITION status with the sense key set to ILLEGAL REQUEST and the additional sense code set to INVALID FIELD IN CDB./quote Apparently your test suite considers not failing READ16 with non-zero RDPROTECT field as an error. However, if I interpret the above paragraph from SBC-3 correctly that behavior seems compliant to me ? Bart. -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SCSI testing/USB devices are amazing
Hi Bart,
Thanks for trying the test tool.
You have a more detailed description furhter down at the definition of
READ10. Look there instead.
Looking at sbc3r35b.pdf 5.13 for READ16 it refers to READ10 for the
bits in the CDB.
Looking at 5.11 READ10 : Table 59 RDPROTECT field we have all non-zero
codes footnoted with b) that says :
If the logical unit does not support protection information, then
the device server should terminate the
command with CHECK CONDITION status with the sense key set to
ILLEGAL REQUEST and the
additional sense code set to INVALID FIELD IN CDB.
which covers the case where the device does not support PI.
In that table we also have that when Logical unit formatted with
protection information is set to No for all non-zero RDPROTECT values
footnoted by a) which says
If a logical unit supports protection information (see 4.22) butand
has not been formatted with
protection information, then the device server shall terminate a
command specifying a verify medium
operation to the logical unit with CHECK CONDITION status with the
sense key set to ILLEGAL
REQUEST and the additional sense code set to INVALID FIELD IN CDB.
which covers the cases where the devices does support protection
information but the device is not formatted with it.
So I think these two together mean that the only time a device may not
fail a request with non-zero RDPROTECT field is for the case where
1, the device does support PI
2, the medium is formatted with type!=0 protection
The READ16 RDPROTECT checks are conditional on :
if (!inq-protect || (rc16 != NULL !rc16-prot_en)) {
I.e. if the device said it does not support protection or if the
device iether did not support READCAPACITY16 or if RC16 said that the
medium is not formatted with protection information.
(The check also means that if a device does not support the
READCAPACITY16 opcode then I assume that
it can not use PI either, or that the protection type == 0. I think
that is a reasonable extra condition outside of what SBC3 technically
requires)
regards
ronnie sahlberg
On Wed, May 22, 2013 at 7:07 AM, Bart Van Assche
bart.vanass...@gmail.com wrote:
On 04/28/13 18:19, ronnie sahlberg wrote:
Hi List,
Interested in SCSI tests?
I have a reasonable SCSI (mainly SBC) testsuite at
https://github.com/sahlberg/libiscsi
while I am mainly interested in testing of iSCSI targets, most of my
tests so far are for the SCSI protocol
so they work quite well on devices connected to any kind of transport,
as long as you re-export them via iSCSI.
(For example using a simple passthrough device with TGT.)
At some stage I will redo some of the framework so that it can
optionally talk directly to a /dev/sg* device, but reexporting via TGT
works well enough for now.
Hello Ronnie,
Thanks for making this software publicly available. I have a question about
one particular test though. In the output of the test suite I found the
following: quoteTest READ16 with non-zero RDPROTECT. Device does not
support/use protection information. All commands should fail./quote
However, in SBC-3 I found the following: quoteIf type 0 protection is
enabled and the RDPROTECT field, the WRPROTECT field, the VRPROTECT field,
or the ORPROTECT field is set to a non-zero value, then medium access
commands are invalid and may be terminated by the device server with CHECK
CONDITION status with the sense key set to ILLEGAL REQUEST
and the additional sense code set to INVALID FIELD IN CDB./quote
Apparently your test suite considers not failing READ16 with non-zero
RDPROTECT field as an error. However, if I interpret the above paragraph
from SBC-3 correctly that behavior seems compliant to me ?
Bart.
--
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SCSI testing/USB devices are amazing
On 13-05-17 07:53 PM, ronnie sahlberg wrote:
Martin,
Thansk for your suggestions.
On Tue, May 14, 2013 at 3:43 PM, Martin K. Petersen
martin.peter...@oracle.com wrote:
Ronnie == ronnie sahlberg ronniesahlb...@gmail.com writes:
...
Ronnie I have added tests for the block limits VPD as
Ronnie SCSI.Inquiry.InquiryBlockLimits. It checks that the pagelength
Ronnie is valid. 3C if SBC3 is claimed and 0C if prior to SBC3.
Well, there are devices out there that claim SPC3/SBC2 compliance but do
support some of the newer features from SPC4/SBC3.
In this case I'd rely on the supported VPD page list. And if the BL VPD
is present and the device reports SPC3/SBC2 I'd print a warning.
I have updated the tests so that IF the device claims SBC-3 then the
SBC-3 page of pagelength 0x3c must be present.
If the device is not SBC-3, then I allow either the SBC-2 version of
the page that is 0x0c in length OR the SBC-3 version.
If the device returns a SBC-3 style page without claiming SBC-3
support I print a warning but allow the test to pass.
This is now part of the test --test SCSI.Inquiry.InquiryBlockLimits
Ronnie The other fields I had a hard time to come up with good sanity
Ronnie tests for. Any suggestions ? Do you have examples of things
Ronnie that vendors get wrong here ?
Maximum Write Same Length vs. support for WS10 and WS16.
Another interesting Write Same test: I have several devices that support
WS16 but which only support a 2-byte block count in WS16. I.e. you get
the larger LBA but not a bigger block count with WS16.
I have added tests to verify that WS16 supports 16 bit block count.
I also added tests that both WS10 and WS16 support 8 bit block counts.
These are part of:
--test SCSI.WriteSame10.WriteSame10Unmap
--test SCSI.WriteSame16.WriteSame16Unmap
Could you check if these tests catch the devices that you have that
had problem in this area?
There's also the Logical Block Provisioning VPD page. You could verify
that UNMAP is supported when LBPU=1. Repeat for LBPWS and LBPWS10.
I added tests that IF UNMAP works, then both LBPU and LPBME must be clear.
Analog for WS10/WS16.
If either of them does not work, then I check that the corresponding
LBPU/LBPWS10/LBPWS flags are clear.
This is tested in
--test SCSI.Unmap.UnmapVPD
--test SCSI.WriteSame10.WriteSame10UnmapVPD
--test SCSI.WriteSame16.WriteSame16UnmapVPD
You could verify that the device actually returns zeroes when LBPRZ=1.
Tested in
--test SCSI.Unmap.Unmap
--test SCSI.WriteSame10.WriteSame10Unmap
--test SCSI.WriteSame16.WriteSame16Unmap
Ronnie I will add tests for when protection information is enabled in
Ronnie the future, I will need to find time to add it to tgt first.
I have a fairly extensive set of PI tests in my test suite. But that
gets pretty involved. We can deal with those later.
I would love to implement those tests in my testsuite, but that is
probably not near future since I would have to add support for PI to
both wireshark and tgtd first.
Other tests you would like to see in the test tool ?
Hi,
If you are working in the WRITE SAME (WS) area, you could make
this addition: T10/Robert Elliott just added a NDOB bit
('no data-out buffer') to WS(16) and WS(32) in sbc3r35d.pdf .
Using it vastly simplifies my sg_write_same code but I don't
understand why NDOB=1,UNMAP=0 is disallowed.
Doug Gilbert
--
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SCSI testing/USB devices are amazing
Martin, Thansk for your suggestions. On Tue, May 14, 2013 at 3:43 PM, Martin K. Petersen martin.peter...@oracle.com wrote: Ronnie == ronnie sahlberg ronniesahlb...@gmail.com writes: ... Ronnie I have added tests for the block limits VPD as Ronnie SCSI.Inquiry.InquiryBlockLimits. It checks that the pagelength Ronnie is valid. 3C if SBC3 is claimed and 0C if prior to SBC3. Well, there are devices out there that claim SPC3/SBC2 compliance but do support some of the newer features from SPC4/SBC3. In this case I'd rely on the supported VPD page list. And if the BL VPD is present and the device reports SPC3/SBC2 I'd print a warning. I have updated the tests so that IF the device claims SBC-3 then the SBC-3 page of pagelength 0x3c must be present. If the device is not SBC-3, then I allow either the SBC-2 version of the page that is 0x0c in length OR the SBC-3 version. If the device returns a SBC-3 style page without claiming SBC-3 support I print a warning but allow the test to pass. This is now part of the test --test SCSI.Inquiry.InquiryBlockLimits Ronnie The other fields I had a hard time to come up with good sanity Ronnie tests for. Any suggestions ? Do you have examples of things Ronnie that vendors get wrong here ? Maximum Write Same Length vs. support for WS10 and WS16. Another interesting Write Same test: I have several devices that support WS16 but which only support a 2-byte block count in WS16. I.e. you get the larger LBA but not a bigger block count with WS16. I have added tests to verify that WS16 supports 16 bit block count. I also added tests that both WS10 and WS16 support 8 bit block counts. These are part of: --test SCSI.WriteSame10.WriteSame10Unmap --test SCSI.WriteSame16.WriteSame16Unmap Could you check if these tests catch the devices that you have that had problem in this area? There's also the Logical Block Provisioning VPD page. You could verify that UNMAP is supported when LBPU=1. Repeat for LBPWS and LBPWS10. I added tests that IF UNMAP works, then both LBPU and LPBME must be clear. Analog for WS10/WS16. If either of them does not work, then I check that the corresponding LBPU/LBPWS10/LBPWS flags are clear. This is tested in --test SCSI.Unmap.UnmapVPD --test SCSI.WriteSame10.WriteSame10UnmapVPD --test SCSI.WriteSame16.WriteSame16UnmapVPD You could verify that the device actually returns zeroes when LBPRZ=1. Tested in --test SCSI.Unmap.Unmap --test SCSI.WriteSame10.WriteSame10Unmap --test SCSI.WriteSame16.WriteSame16Unmap Ronnie I will add tests for when protection information is enabled in Ronnie the future, I will need to find time to add it to tgt first. I have a fairly extensive set of PI tests in my test suite. But that gets pretty involved. We can deal with those later. I would love to implement those tests in my testsuite, but that is probably not near future since I would have to add support for PI to both wireshark and tgtd first. Other tests you would like to see in the test tool ? regards ronnie sahlberg -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SCSI testing/USB devices are amazing
Ronnie == ronnie sahlberg ronniesahlb...@gmail.com writes:
Ronnie,
Ronnie I have added tests for the block limits VPD as
Ronnie SCSI.Inquiry.InquiryBlockLimits. It checks that the pagelength
Ronnie is valid. 3C if SBC3 is claimed and 0C if prior to SBC3.
Well, there are devices out there that claim SPC3/SBC2 compliance but do
support some of the newer features from SPC4/SBC3.
In this case I'd rely on the supported VPD page list. And if the BL VPD
is present and the device reports SPC3/SBC2 I'd print a warning.
Ronnie It then validates that the UNMAP counts are sane. Sane being
Ronnie that if LBPU==0 then these must be 0, and if LBPU==1 then these
Ronnie must be 1, must be than 2**LBPPBE and either 0x or
Ronnie 1M. (1M is arbitrary for crazy large number of blocks)
That's a good start, anyway.
Ronnie The other fields I had a hard time to come up with good sanity
Ronnie tests for. Any suggestions ? Do you have examples of things
Ronnie that vendors get wrong here ?
Maximum Write Same Length vs. support for WS10 and WS16.
Another interesting Write Same test: I have several devices that support
WS16 but which only support a 2-byte block count in WS16. I.e. you get
the larger LBA but not a bigger block count with WS16.
There's also the Logical Block Provisioning VPD page. You could verify
that UNMAP is supported when LBPU=1. Repeat for LBPWS and LBPWS10.
You could verify that the device actually returns zeroes when LBPRZ=1.
Ronnie I will add tests for when protection information is enabled in
Ronnie the future, I will need to find time to add it to tgt first.
I have a fairly extensive set of PI tests in my test suite. But that
gets pretty involved. We can deal with those later.
Ronnie Very nice document. Section 1.3 could update
Ronnie though. READCAPACITY16 is only mandatory in SBC-2 IF the device
Ronnie supports protection information, but optional if it does not.
Ronnie In SBC-3 it is always mandatory though. Thin provisioning and
Ronnie different logical/physical block sizes were only added to this
Ronnie command in SBC-3 not SBC-2.
I describe what we actually do, not what's spec compliant :)
This is our current heuristic for READ CAPACITY(16):
static int sd_try_rc16_first(struct scsi_device *sdp)
{
if (sdp-host-max_cmd_len 16)
return 0;
if (sdp-scsi_level SCSI_SPC_2)
return 1;
if (scsi_device_protection(sdp))
return 1;
return 0;
}
Ronnie I have thus updated my READCAPACITY16 tests so that IF SBC-3 is
Ronnie claimed, or if IQN-PROTECT is set then the device must support
Ronnie this opcode or the test will fail. Otherwise, if it is not
Ronnie SBC-3 and if PROTECT is clear, then the test will be skipped but
Ronnie not fail if the opcode is missing.
*nod*
--
Martin K. Petersen Oracle Linux Engineering
--
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SCSI testing/USB devices are amazing
Hi Martin, Thanks for your input, and for the link to your nice writeup. On Thu, May 9, 2013 at 7:43 PM, Martin K. Petersen martin.peter...@oracle.com wrote: Ronnie == ronnie sahlberg ronniesahlb...@gmail.com writes: Ronnie * please have a look at the tests and the test results I linked Ronnie to above. I currently have quite a few tests but am happy to Ronnie add more. Aside from issuing commands with various bits set, I'd like to see some more sanity checking. Mainly to see whether it responds correctly to the features is claims to support. I will start to add such. I think your document will provide ideas on what to start writing tests for. I.e. do the values reported in the Block Limits VPD look sane given what we know about the device in general (SCSI level, capacity, PI supported, discard supported, etc.)? I have added tests for the block limits VPD as SCSI.Inquiry.InquiryBlockLimits. It checks that the pagelength is valid. 3C if SBC3 is claimed and 0C if prior to SBC3. It then validates that the UNMAP counts are sane. Sane being that if LBPU==0 then these must be 0, and if LBPU==1 then these must be 1, must be than 2**LBPPBE and either 0x or 1M. (1M is arbitrary for crazy large number of blocks) The other fields I had a hard time to come up with good sanity tests for. Any suggestions ? Do you have examples of things that vendors get wrong here ? Right now you always expect RDPROTECT/WRPROTECT 0 to fail, but they should succeed if the device is formatted with PI. I have added a check so that the tests are skipped if the device does not support protection information or if it supports protection information but but it is not enabled. I will add tests for when protection information is enabled in the future, I will need to find time to add it to tgt first. FWIW, my slightly outdated document is here: https://oss.oracle.com/~mkp/docs/linux-advanced-storage.pdf Very nice document. Section 1.3 could update though. READCAPACITY16 is only mandatory in SBC-2 IF the device supports protection information, but optional if it does not. In SBC-3 it is always mandatory though. Thin provisioning and different logical/physical block sizes were only added to this command in SBC-3 not SBC-2. I have thus updated my READCAPACITY16 tests so that IF SBC-3 is claimed, or if IQN-PROTECT is set then the device must support this opcode or the test will fail. Otherwise, if it is not SBC-3 and if PROTECT is clear, then the test will be skipped but not fail if the opcode is missing. If you want to, have time and if you have specific opcodes/features that vendors get wrong, I am happy to add tests if you point me in the right direction for what you want me to add tests for. regards ronnie sahlberg -- Martin K. Petersen Oracle Linux Engineering -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SCSI testing/USB devices are amazing
Ronnie == ronnie sahlberg ronniesahlb...@gmail.com writes: Ronnie * please have a look at the tests and the test results I linked Ronnie to above. I currently have quite a few tests but am happy to Ronnie add more. Aside from issuing commands with various bits set, I'd like to see some more sanity checking. Mainly to see whether it responds correctly to the features is claims to support. I.e. do the values reported in the Block Limits VPD look sane given what we know about the device in general (SCSI level, capacity, PI supported, discard supported, etc.)? Right now you always expect RDPROTECT/WRPROTECT 0 to fail, but they should succeed if the device is formatted with PI. FWIW, my slightly outdated document is here: https://oss.oracle.com/~mkp/docs/linux-advanced-storage.pdf -- Martin K. Petersen Oracle Linux Engineering -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
SCSI testing/USB devices are amazing
Hi List, Interested in SCSI tests? I have a reasonable SCSI (mainly SBC) testsuite at https://github.com/sahlberg/libiscsi while I am mainly interested in testing of iSCSI targets, most of my tests so far are for the SCSI protocol so they work quite well on devices connected to any kind of transport, as long as you re-export them via iSCSI. (For example using a simple passthrough device with TGT.) At some stage I will redo some of the framework so that it can optionally talk directly to a /dev/sg* device, but reexporting via TGT works well enough for now. The test suite is fairly comprehensive for SBC so far, you can find a list of all the current tests at https://sites.google.com/site/libiscsitarballs/libiscsitarballs/ Based on hearing about the problems with USB devices, I added a special family with the tests I think would be relevant to USB SBC devices. Oh boy, I thought soft iSCSI targets were bad... USB seems like a completely new level of badness. In testing some random sticks I had I found no usb stick actually claiming SBC support in the standard inq page, but one stick claimed support for SSC ! Another stick never really reported LBA OUT OF RANGE and it just kept wrapping the LBA once you wrote beyond the end of the stick. I.e. write at LBA modulo size-of-stick. Another one didnt supported VERIFY10 with bytchk set (whats the point supporting VERIFY10 at all then?), and if you sent too many VERIFY10 to it it would lock up completely. Wow. And I thought iSCSI targets were bad. I added changes to the tests so that IF a device claims SBC-3 support then both READCAPACITY16 and READ16 are mandatory so the tests will fail if the commands are missing. If SBC-3 is not claimed, I allow these commands to fail and just skip that particular test. To get an overview of what it tests so far for SCSI-USB-SBC I have pasted the output of ./bin/iscsi-test-cu iscsi://127.0.0.1/iqn.ronnie.test/3 --dataloss -V --usb --test SCSI-USB-SBC also at https://sites.google.com/site/libiscsitarballs/libiscsitarballs/ I think this could be very useful for testing USB devices and maybe if you can get vendors to test their devices before release ? I am very happy to add more tests for specific pain point you have with devices but since I have little experience with USB devices I would need your help to identify what I should add tests for. So to sum it up: * please have a look at the tests and the test results I linked to above. I currently have quite a few tests but am happy to add more. * do you think this is something you guys could find useful ? * then if so, do you want to work with me to improve it so it tests and flags the problem areas you find with usb (and other) devices ? regards ronnie sahlberg -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
