Re: SCSI test tool for linux (and others)
On Mon, Sep 22, 2014 at 9:34 AM, Bart Van Assche bvanass...@acm.org wrote: On 21/09/2014 9:58, ronnie sahlberg wrote: For now I populated it with a bunch of tests I copied fro the SCSI family. Some, not all, tests. Missing are a whole bunch of tests that require that you have multiple I_T nexuses, such as all the persistent reservation tests, which would be difficult to implement on /dev/sg* devices. (all users of a specific /dev/sg* device ends up on the same nexus, right, the nexus for kernel-device ?) Hello Ronnie, Have you considered to allow multiple /dev/sg* arguments to be specified and to let the tests that require multiple I_T nexuses use one /dev/sg* argument per nexus ? Do you think such a feature would be helpful for testing FC and RDMA over SCSI initiator drivers in a setup with multiple storage paths between initiator and target systems ? Good idea. I have added that as an Issue to the githup repo so I do not forget. -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
SCSI test tool for linux (and others)
List, I have recently converted my iscsi based test suite to allow running it against /dev/sg* devices in addition to iscsi targets. The purpose here is to create a tool that can be used by vendors to test that their device will be well supported by the Linux SCSI stack. To have a tool that you can send to vendors and ask that please run these tests, if you pass you have higher probability to interoperate well with us. In order to do this I added a new Family of tests to my tool, which I called LINUX. The tests in this tool are divided up in Families/Suites/Testarea and you/we/I can add whatever tests we need or want to the LINUX family. For example, while T10 does not require that you MUST put a proper SPC/SBC standard in the inquiry standards page/version descriptions I can't really add such tests to the SCSI family since that would imply compliance to the T10 specs. But adding such a test to the LINUX family is perfectly fine. ./test-tool/iscsi-test-cu --list | grep LINUX will list all the tests in the Linux family. For now I populated it with a bunch of tests I copied fro the SCSI family. Some, not all, tests. Missing are a whole bunch of tests that require that you have multiple I_T nexuses, such as all the persistent reservation tests, which would be difficult to implement on /dev/sg* devices. (all users of a specific /dev/sg* device ends up on the same nexus, right, the nexus for kernel-device ?) To run all the Linux tests you can do : sudo ./test-tool/iscsi-test-cu /dev/sg2 --test LINUX.*.* --usb -V To just run the Read10 suite, you can do sudo ./test-tool/iscsi-test-cu /dev/sg2 --test LINUX.Read10.* --usb -V etc etc. The --usb flag is used to tell the test suite that we use a USB device and thus we need to clamp any I/O to = 120kbyte. (Which is a restriction of the USB transport in Linux, and which also is a restriction that is DIFFICULT to discover automagically due to lack of a proper ioctl() to discover this. I will send you a patch to add such an ioctl at some stage, but for now you will have to live with : use --usb if the device is usb attached.) -V is to get extra verbose output. This makes the tool print every scsi cdb it sends and what it expects. This makes it easier to see what the tool does but the output gets pretty busy so drop -V if you want. When things fail, it is reasonably easy to see why. This is an example of a device that does not respect the RDPROTECT field in the READ CDBs. ... Suite: Read10 Test: ReadProtect ... Test READ10 with non-zero RDPROTECT Device does not support/use protection information. All commands should fail. ... Send READ10 (Expecting CHECK_CONDITION) LBA:0 blocks:1 rdprotect:7 dpo:0 fua:0 fua_nv:0 group:0 [FAILED] READ10 successful but should have failed with ILLEGAL_REQUEST(0x05)/INVALID_FIELD_IN_CDB(0x2400) FAILED ... --Run Summary: Type Total Ran Passed Failed suites1 1 n/a 0 tests 1 1 0 1 asserts 7 7 0 7 Tests completed with return value: 0 Lots of low end devices have a lot more interesting failures, but I am lazy and don't feel like looking for an example right now. However, if you plan to start using the VERIFY10/12/16 commands at some stage, lots of low end devices always return SUCCESS even when you know there is a miscompare there :-) So beware :-) What now then? == LINUX.*.* currently runs some 150 different suites/tests and hundreds of thousands of individual checks/asserts. For most USB devices most of these tests will be automatically skipped since they don't support much of the more interesting opcodes, like OrWrite, CompareAndWrite or WriteSame10/16. So all those tests are automagically [SKIPPED]. It may be that you don't want/need many of these tests and then I can drop them from the LINUX tests. But they will do no harm either other than distract from more important issues. Right now, I would like feedback and users to use this tool. I am also very interested in starting a dialogue with you guys, if you think this is worthwhile, to find test cases for things that cause you pain and you would like to see in a Linux compliance test suite. Anything you want to test I can try to add. At some stage later we could then end up with a tool you guys can send to the vendors and ask them run this, pass the tests, and linux scsi guys gives two thumbs up. To install: The main repository is here : https://github.com/sahlberg/libiscsi I had to export some extra symbols from the library to do the /dev/sg* support so you need to build and 'sudo make install' the library. The /dev/sg* support itself is currently in a separate branch for this project, so : git checkout sgio make and that should build the tool. regards ronnie sahlberg -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord
Re: SCSI test tool for linux (and others)
Let me follow up: The test suite depends on CUnit, so you need to install the proper package so configure can find it. libcunit1-dev is what you need on debian based systems. On Sun, Sep 21, 2014 at 8:58 AM, ronnie sahlberg ronniesahlb...@gmail.com wrote: List, I have recently converted my iscsi based test suite to allow running it against /dev/sg* devices in addition to iscsi targets. The purpose here is to create a tool that can be used by vendors to test that their device will be well supported by the Linux SCSI stack. To have a tool that you can send to vendors and ask that please run these tests, if you pass you have higher probability to interoperate well with us. In order to do this I added a new Family of tests to my tool, which I called LINUX. The tests in this tool are divided up in Families/Suites/Testarea and you/we/I can add whatever tests we need or want to the LINUX family. For example, while T10 does not require that you MUST put a proper SPC/SBC standard in the inquiry standards page/version descriptions I can't really add such tests to the SCSI family since that would imply compliance to the T10 specs. But adding such a test to the LINUX family is perfectly fine. ./test-tool/iscsi-test-cu --list | grep LINUX will list all the tests in the Linux family. For now I populated it with a bunch of tests I copied fro the SCSI family. Some, not all, tests. Missing are a whole bunch of tests that require that you have multiple I_T nexuses, such as all the persistent reservation tests, which would be difficult to implement on /dev/sg* devices. (all users of a specific /dev/sg* device ends up on the same nexus, right, the nexus for kernel-device ?) To run all the Linux tests you can do : sudo ./test-tool/iscsi-test-cu /dev/sg2 --test LINUX.*.* --usb -V To just run the Read10 suite, you can do sudo ./test-tool/iscsi-test-cu /dev/sg2 --test LINUX.Read10.* --usb -V etc etc. The --usb flag is used to tell the test suite that we use a USB device and thus we need to clamp any I/O to = 120kbyte. (Which is a restriction of the USB transport in Linux, and which also is a restriction that is DIFFICULT to discover automagically due to lack of a proper ioctl() to discover this. I will send you a patch to add such an ioctl at some stage, but for now you will have to live with : use --usb if the device is usb attached.) -V is to get extra verbose output. This makes the tool print every scsi cdb it sends and what it expects. This makes it easier to see what the tool does but the output gets pretty busy so drop -V if you want. When things fail, it is reasonably easy to see why. This is an example of a device that does not respect the RDPROTECT field in the READ CDBs. ... Suite: Read10 Test: ReadProtect ... Test READ10 with non-zero RDPROTECT Device does not support/use protection information. All commands should fail. ... Send READ10 (Expecting CHECK_CONDITION) LBA:0 blocks:1 rdprotect:7 dpo:0 fua:0 fua_nv:0 group:0 [FAILED] READ10 successful but should have failed with ILLEGAL_REQUEST(0x05)/INVALID_FIELD_IN_CDB(0x2400) FAILED ... --Run Summary: Type Total Ran Passed Failed suites1 1 n/a 0 tests 1 1 0 1 asserts 7 7 0 7 Tests completed with return value: 0 Lots of low end devices have a lot more interesting failures, but I am lazy and don't feel like looking for an example right now. However, if you plan to start using the VERIFY10/12/16 commands at some stage, lots of low end devices always return SUCCESS even when you know there is a miscompare there :-) So beware :-) What now then? == LINUX.*.* currently runs some 150 different suites/tests and hundreds of thousands of individual checks/asserts. For most USB devices most of these tests will be automatically skipped since they don't support much of the more interesting opcodes, like OrWrite, CompareAndWrite or WriteSame10/16. So all those tests are automagically [SKIPPED]. It may be that you don't want/need many of these tests and then I can drop them from the LINUX tests. But they will do no harm either other than distract from more important issues. Right now, I would like feedback and users to use this tool. I am also very interested in starting a dialogue with you guys, if you think this is worthwhile, to find test cases for things that cause you pain and you would like to see in a Linux compliance test suite. Anything you want to test I can try to add. At some stage later we could then end up with a tool you guys can send to the vendors and ask them run this, pass the tests, and linux scsi guys gives two thumbs up. To install: The main repository is here : https://github.com/sahlberg/libiscsi I had to export some extra symbols from the library to do
Re: SCSI testing/USB devices are amazing
Hi Bart,
Thanks for trying the test tool.
You have a more detailed description furhter down at the definition of
READ10. Look there instead.
Looking at sbc3r35b.pdf 5.13 for READ16 it refers to READ10 for the
bits in the CDB.
Looking at 5.11 READ10 : Table 59 RDPROTECT field we have all non-zero
codes footnoted with b) that says :
If the logical unit does not support protection information, then
the device server should terminate the
command with CHECK CONDITION status with the sense key set to
ILLEGAL REQUEST and the
additional sense code set to INVALID FIELD IN CDB.
which covers the case where the device does not support PI.
In that table we also have that when Logical unit formatted with
protection information is set to No for all non-zero RDPROTECT values
footnoted by a) which says
If a logical unit supports protection information (see 4.22) butand
has not been formatted with
protection information, then the device server shall terminate a
command specifying a verify medium
operation to the logical unit with CHECK CONDITION status with the
sense key set to ILLEGAL
REQUEST and the additional sense code set to INVALID FIELD IN CDB.
which covers the cases where the devices does support protection
information but the device is not formatted with it.
So I think these two together mean that the only time a device may not
fail a request with non-zero RDPROTECT field is for the case where
1, the device does support PI
2, the medium is formatted with type!=0 protection
The READ16 RDPROTECT checks are conditional on :
if (!inq-protect || (rc16 != NULL !rc16-prot_en)) {
I.e. if the device said it does not support protection or if the
device iether did not support READCAPACITY16 or if RC16 said that the
medium is not formatted with protection information.
(The check also means that if a device does not support the
READCAPACITY16 opcode then I assume that
it can not use PI either, or that the protection type == 0. I think
that is a reasonable extra condition outside of what SBC3 technically
requires)
regards
ronnie sahlberg
On Wed, May 22, 2013 at 7:07 AM, Bart Van Assche
bart.vanass...@gmail.com wrote:
On 04/28/13 18:19, ronnie sahlberg wrote:
Hi List,
Interested in SCSI tests?
I have a reasonable SCSI (mainly SBC) testsuite at
https://github.com/sahlberg/libiscsi
while I am mainly interested in testing of iSCSI targets, most of my
tests so far are for the SCSI protocol
so they work quite well on devices connected to any kind of transport,
as long as you re-export them via iSCSI.
(For example using a simple passthrough device with TGT.)
At some stage I will redo some of the framework so that it can
optionally talk directly to a /dev/sg* device, but reexporting via TGT
works well enough for now.
Hello Ronnie,
Thanks for making this software publicly available. I have a question about
one particular test though. In the output of the test suite I found the
following: quoteTest READ16 with non-zero RDPROTECT. Device does not
support/use protection information. All commands should fail./quote
However, in SBC-3 I found the following: quoteIf type 0 protection is
enabled and the RDPROTECT field, the WRPROTECT field, the VRPROTECT field,
or the ORPROTECT field is set to a non-zero value, then medium access
commands are invalid and may be terminated by the device server with CHECK
CONDITION status with the sense key set to ILLEGAL REQUEST
and the additional sense code set to INVALID FIELD IN CDB./quote
Apparently your test suite considers not failing READ16 with non-zero
RDPROTECT field as an error. However, if I interpret the above paragraph
from SBC-3 correctly that behavior seems compliant to me ?
Bart.
--
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SCSI testing/USB devices are amazing
Martin, Thansk for your suggestions. On Tue, May 14, 2013 at 3:43 PM, Martin K. Petersen martin.peter...@oracle.com wrote: Ronnie == ronnie sahlberg ronniesahlb...@gmail.com writes: ... Ronnie I have added tests for the block limits VPD as Ronnie SCSI.Inquiry.InquiryBlockLimits. It checks that the pagelength Ronnie is valid. 3C if SBC3 is claimed and 0C if prior to SBC3. Well, there are devices out there that claim SPC3/SBC2 compliance but do support some of the newer features from SPC4/SBC3. In this case I'd rely on the supported VPD page list. And if the BL VPD is present and the device reports SPC3/SBC2 I'd print a warning. I have updated the tests so that IF the device claims SBC-3 then the SBC-3 page of pagelength 0x3c must be present. If the device is not SBC-3, then I allow either the SBC-2 version of the page that is 0x0c in length OR the SBC-3 version. If the device returns a SBC-3 style page without claiming SBC-3 support I print a warning but allow the test to pass. This is now part of the test --test SCSI.Inquiry.InquiryBlockLimits Ronnie The other fields I had a hard time to come up with good sanity Ronnie tests for. Any suggestions ? Do you have examples of things Ronnie that vendors get wrong here ? Maximum Write Same Length vs. support for WS10 and WS16. Another interesting Write Same test: I have several devices that support WS16 but which only support a 2-byte block count in WS16. I.e. you get the larger LBA but not a bigger block count with WS16. I have added tests to verify that WS16 supports 16 bit block count. I also added tests that both WS10 and WS16 support 8 bit block counts. These are part of: --test SCSI.WriteSame10.WriteSame10Unmap --test SCSI.WriteSame16.WriteSame16Unmap Could you check if these tests catch the devices that you have that had problem in this area? There's also the Logical Block Provisioning VPD page. You could verify that UNMAP is supported when LBPU=1. Repeat for LBPWS and LBPWS10. I added tests that IF UNMAP works, then both LBPU and LPBME must be clear. Analog for WS10/WS16. If either of them does not work, then I check that the corresponding LBPU/LBPWS10/LBPWS flags are clear. This is tested in --test SCSI.Unmap.UnmapVPD --test SCSI.WriteSame10.WriteSame10UnmapVPD --test SCSI.WriteSame16.WriteSame16UnmapVPD You could verify that the device actually returns zeroes when LBPRZ=1. Tested in --test SCSI.Unmap.Unmap --test SCSI.WriteSame10.WriteSame10Unmap --test SCSI.WriteSame16.WriteSame16Unmap Ronnie I will add tests for when protection information is enabled in Ronnie the future, I will need to find time to add it to tgt first. I have a fairly extensive set of PI tests in my test suite. But that gets pretty involved. We can deal with those later. I would love to implement those tests in my testsuite, but that is probably not near future since I would have to add support for PI to both wireshark and tgtd first. Other tests you would like to see in the test tool ? regards ronnie sahlberg -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SCSI testing/USB devices are amazing
Hi Martin, Thanks for your input, and for the link to your nice writeup. On Thu, May 9, 2013 at 7:43 PM, Martin K. Petersen martin.peter...@oracle.com wrote: Ronnie == ronnie sahlberg ronniesahlb...@gmail.com writes: Ronnie * please have a look at the tests and the test results I linked Ronnie to above. I currently have quite a few tests but am happy to Ronnie add more. Aside from issuing commands with various bits set, I'd like to see some more sanity checking. Mainly to see whether it responds correctly to the features is claims to support. I will start to add such. I think your document will provide ideas on what to start writing tests for. I.e. do the values reported in the Block Limits VPD look sane given what we know about the device in general (SCSI level, capacity, PI supported, discard supported, etc.)? I have added tests for the block limits VPD as SCSI.Inquiry.InquiryBlockLimits. It checks that the pagelength is valid. 3C if SBC3 is claimed and 0C if prior to SBC3. It then validates that the UNMAP counts are sane. Sane being that if LBPU==0 then these must be 0, and if LBPU==1 then these must be 1, must be than 2**LBPPBE and either 0x or 1M. (1M is arbitrary for crazy large number of blocks) The other fields I had a hard time to come up with good sanity tests for. Any suggestions ? Do you have examples of things that vendors get wrong here ? Right now you always expect RDPROTECT/WRPROTECT 0 to fail, but they should succeed if the device is formatted with PI. I have added a check so that the tests are skipped if the device does not support protection information or if it supports protection information but but it is not enabled. I will add tests for when protection information is enabled in the future, I will need to find time to add it to tgt first. FWIW, my slightly outdated document is here: https://oss.oracle.com/~mkp/docs/linux-advanced-storage.pdf Very nice document. Section 1.3 could update though. READCAPACITY16 is only mandatory in SBC-2 IF the device supports protection information, but optional if it does not. In SBC-3 it is always mandatory though. Thin provisioning and different logical/physical block sizes were only added to this command in SBC-3 not SBC-2. I have thus updated my READCAPACITY16 tests so that IF SBC-3 is claimed, or if IQN-PROTECT is set then the device must support this opcode or the test will fail. Otherwise, if it is not SBC-3 and if PROTECT is clear, then the test will be skipped but not fail if the opcode is missing. If you want to, have time and if you have specific opcodes/features that vendors get wrong, I am happy to add tests if you point me in the right direction for what you want me to add tests for. regards ronnie sahlberg -- Martin K. Petersen Oracle Linux Engineering -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
SCSI testing/USB devices are amazing
Hi List, Interested in SCSI tests? I have a reasonable SCSI (mainly SBC) testsuite at https://github.com/sahlberg/libiscsi while I am mainly interested in testing of iSCSI targets, most of my tests so far are for the SCSI protocol so they work quite well on devices connected to any kind of transport, as long as you re-export them via iSCSI. (For example using a simple passthrough device with TGT.) At some stage I will redo some of the framework so that it can optionally talk directly to a /dev/sg* device, but reexporting via TGT works well enough for now. The test suite is fairly comprehensive for SBC so far, you can find a list of all the current tests at https://sites.google.com/site/libiscsitarballs/libiscsitarballs/ Based on hearing about the problems with USB devices, I added a special family with the tests I think would be relevant to USB SBC devices. Oh boy, I thought soft iSCSI targets were bad... USB seems like a completely new level of badness. In testing some random sticks I had I found no usb stick actually claiming SBC support in the standard inq page, but one stick claimed support for SSC ! Another stick never really reported LBA OUT OF RANGE and it just kept wrapping the LBA once you wrote beyond the end of the stick. I.e. write at LBA modulo size-of-stick. Another one didnt supported VERIFY10 with bytchk set (whats the point supporting VERIFY10 at all then?), and if you sent too many VERIFY10 to it it would lock up completely. Wow. And I thought iSCSI targets were bad. I added changes to the tests so that IF a device claims SBC-3 support then both READCAPACITY16 and READ16 are mandatory so the tests will fail if the commands are missing. If SBC-3 is not claimed, I allow these commands to fail and just skip that particular test. To get an overview of what it tests so far for SCSI-USB-SBC I have pasted the output of ./bin/iscsi-test-cu iscsi://127.0.0.1/iqn.ronnie.test/3 --dataloss -V --usb --test SCSI-USB-SBC also at https://sites.google.com/site/libiscsitarballs/libiscsitarballs/ I think this could be very useful for testing USB devices and maybe if you can get vendors to test their devices before release ? I am very happy to add more tests for specific pain point you have with devices but since I have little experience with USB devices I would need your help to identify what I should add tests for. So to sum it up: * please have a look at the tests and the test results I linked to above. I currently have quite a few tests but am happy to add more. * do you think this is something you guys could find useful ? * then if so, do you want to work with me to improve it so it tests and flags the problem areas you find with usb (and other) devices ? regards ronnie sahlberg -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[LSF/MM TOPIC] Making sure soft SCSI Targets are Valid
Hi All I would like to attend LSF/MM and talk with people about scsi and iscsi testing. I am the author/maintainer for libiscsi which is userspace initiator used primarily by KVM/QEMU. This package also includes a pretty big test suite for mainly SCSI but also a lot of interesting iSCSI tests. As someone hacking on a testsuite, I would like to meet with initiator and target implementors and talk about the existing test suites, and how we can work together from here. What can i bring to FSF/MM? I like SCSI and I like writing tests. I have a big existing testsuite that I think at least some of you can benefit from. I would like to talk to people on how to make the test suite even better. Can we work out a platform where we can build even better and more comprehensive test suites ? etc. regards ronnie sahlberg On Tue, Jan 29, 2013 at 11:13 AM, Lee Duncan ldun...@suse.com wrote: Hi: I'm not sure if there is much interest in this, but I've recently realized that there is no good free software to validate iSCSI targets, not to mention FCOE targets, IB soft targets, etc. There's just no way to know if any change you make is legal short of learning to speak SCSI geek spec (or waiting to see what fails when you make a subtle change). So I have been working with the (user-space) libiscsi creator and maintainer, Ronnie Sahlberg, to enhance his test suite. But this only addresses iSCSI targets. Some of his tests have already shown problems like kernel panics when an incorrect bit is injected, showing the need for such testing. It occurs to me it would be most valuable if we had more generic SCSI tests, not even limited to soft targets, available to developers and manufacturers. How best to support such tests with our SCSI layer, and what tests are needed now and in the future may be a good topic for discussion. -- Lee Duncan SUSE Labs -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[LSF/MM TOPIC] Making sure soft SCSI Targets are Valid
Hi, This is my attend request to to co-present a paper with Lee Duncan. regards ronnie sahlberg On Tue, Jan 29, 2013 at 11:13 AM, Lee Duncan ldun...@suse.com wrote: Hi: I'm not sure if there is much interest in this, but I've recently realized that there is no good free software to validate iSCSI targets, not to mention FCOE targets, IB soft targets, etc. There's just no way to know if any change you make is legal short of learning to speak SCSI geek spec (or waiting to see what fails when you make a subtle change). So I have been working with the (user-space) libiscsi creator and maintainer, Ronnie Sahlberg, to enhance his test suite. But this only addresses iSCSI targets. Some of his tests have already shown problems like kernel panics when an incorrect bit is injected, showing the need for such testing. It occurs to me it would be most valuable if we had more generic SCSI tests, not even limited to soft targets, available to developers and manufacturers. How best to support such tests with our SCSI layer, and what tests are needed now and in the future may be a good topic for discussion. -- Lee Duncan SUSE Labs -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
