Re: [openssl-dev] Where is the sample-comprehensive CAVS test vectors' set with all 259 test vectors
On 04/14/2016 07:20 AM, cyriac wrote: > Thanx! That link works now. Infact, we had some samples from there already. > We understand now that the test vectors do change over time and there is > nothing like a "final" set. > > And yes, we are working with an accredited test lab already. The intention > behind getting hold of a complete set in advance was to have a trial run of > the tests in advance till we wait for official test vectors from the lab. IMHO the algorithm testing process is tedious enough as it is; since in general you cannot get a "complete set" in advance because the format changes so frequently, you're just asking for unnecessary grief and frustration. You'll encounter enough of that in the normal course of events without seeking it out :-) Your lab should have told you that... > And as I understand, officially, these have to be verified with the CAVS > tool which can be done only by the lab. Correct. > However, the perl script fipsalgtest.pl is capable to verify the .rsp files > against the .fax files (provided along with the vectors) and to provide a > test summary report. (With the exception of some key gen vectors which could > be verified only by CAVS tool) > We have done this for a set of vectors and it passes too. > > *Only one clarification sought for.. If fipsalgtest.pl tells me that my > vectors are verified without errors, should I still be skeptical until the > lab confirms it ?* Yes, for several reasons: 1) That check only compares the results from a presumed known good platform against the target response files. 2) The test vector set you're using is probably obsolete, and so is no good for your intended outcome even if "correct". 3) Even of "current", with "correct" response files relative to the request files, the request files may be wrong (as in not what is required by the CAVP). Those files are generated from the CAVS tool via a labor intensive manual process, and the CAVS tool is updated frequently and sometimes has bugs. Errors in one or both (manual process or tool) are not at all uncommon; I'd say the error rate is in the 10% range. So you can find that the test vectors you processed without apparent error, and even that the test lab confirmed, can still turn out to be unsuitable. Usually you don't have to reprocess them all, though I usually do given that it's easier to use fipsalgtest.pl on a full test vector set than to manually manipulate individual request files. Note I like to hang on to the test device until the CMVP formally approves the related validation action, as on occasion we've have to re-do testing that was first done months ago. -Steve M. -- Steve Marquess OpenSSL Validation Services, Inc. 1829 Mount Ephraim Road Adamstown, MD 21710 USA +1 877 673 6775 s/b +1 301 874 2571 direct marqu...@openssl.com gpg/pgp key: http://openssl.com/docs/0x6D1892F5.asc -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Where is the sample-comprehensive CAVS test vectors' set with all 259 test vectors
Thanx! That link works now. Infact, we had some samples from there already. We understand now that the test vectors do change over time and there is nothing like a "final" set. And yes, we are working with an accredited test lab already. The intention behind getting hold of a complete set in advance was to have a trial run of the tests in advance till we wait for official test vectors from the lab. And as I understand, officially, these have to be verified with the CAVS tool which can be done only by the lab. However, the perl script fipsalgtest.pl is capable to verify the .rsp files against the .fax files (provided along with the vectors) and to provide a test summary report. (With the exception of some key gen vectors which could be verified only by CAVS tool) We have done this for a set of vectors and it passes too. *Only one clarification sought for.. If fipsalgtest.pl tells me that my vectors are verified without errors, should I still be skeptical until the lab confirms it ?* -Cyriac -- View this message in context: http://openssl.6102.n7.nabble.com/Where-is-the-sample-comprehensive-CAVS-test-vectors-set-with-all-259-test-vectors-tp65538p65541.html Sent from the OpenSSL - Dev mailing list archive at Nabble.com. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Where is the sample-comprehensive CAVS test vectors' set with all 259 test vectors
On 04/14/2016 06:09 AM, cyriac wrote: > Hi, > > *In FIPS Userguide 2.0*, Appendix B, about CAVS testing, I could find: > Note this step requires a large directory tree of input test data files > produced by the > testing lab using a NIST provided tool (CAVS); several sets of input and > response values can be > found http://openssl.com/testing/validation-2.0/testvectors/. The file > *http://openssl.com/testing/validation-2.0/testvectors/tv.tar.gz > contains a complete set of 259 test vector files with correct responses that > can be used for a single > comprehensive test. *Note the number and format of these test vector files > changes over time, so this > set may not correspond exactly to what the CAVS tool currently produces. > > Unfortunately, this sample comprehensive test vector tar-ball (tv.tar.gz) is > not present in this location. > I have been searching all out, but I could not get hold of this set with all > 259 vectors from anywhere. > Could I know how to get hold of this complete test vector set. (Any web link > available?). Kindly help… The tv.tar.gz symlink was missing; I've restored it. Unfortunately that doesn't do you much good. You can find a huge collection of historical test vectors at: http://openssl.com/testing/validation-2.0/testvectors/ and tv.tar.gz is now pointing to one of them. But, the format and contents of these test vector data sets change over time, frequently. Having one of them doesn't do you much good for a number of reasons: 1) Even if you appear to have processed them without error, you can't properly verify them without an accredited test lab, and if you were working with an accredited test lab they would supply you with a current set of test vectors. 2) There is no reason to fool with these test vectors unless you're trying for your own validation using the OpenSSL FIPS module code, in which case you'll have to engage an accredited test lab. 3) Even if you have a current set (unlikely), any official algorithm validation action requires a unique new set of test vectors (which ... wait for it ... you can only get from an accredited test lab). 4) If you're working with a non-current set of test vectors (which is usually all of them as the format changes frequently), you'll waste time barking up the wrong tree. They can change substantially in a short period of time; note for instance the file count is no longer 259. Notice I mention "accredited test lab" a lot. You're wasting your time if you've not engaged one. Our open source test suite software makes the mechanics of validation a lot easier, but you still have to use a test lab. Yes, you have to pay the lab, but welcome to the wonderful world of FIPS 140-2. -Steve M. -- Steve Marquess OpenSSL Validation Services, Inc. 1829 Mount Ephraim Road Adamstown, MD 21710 USA +1 877 673 6775 s/b +1 301 874 2571 direct marqu...@openssl.com gpg/pgp key: http://openssl.com/docs/0x6D1892F5.asc -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
[openssl-dev] Where is the sample-comprehensive CAVS test vectors' set with all 259 test vectors
Hi, *In FIPS Userguide 2.0*, Appendix B, about CAVS testing, I could find: Note this step requires a large directory tree of input test data files produced by the testing lab using a NIST provided tool (CAVS); several sets of input and response values can be found http://openssl.com/testing/validation-2.0/testvectors/. The file *http://openssl.com/testing/validation-2.0/testvectors/tv.tar.gz contains a complete set of 259 test vector files with correct responses that can be used for a single comprehensive test. *Note the number and format of these test vector files changes over time, so this set may not correspond exactly to what the CAVS tool currently produces. Unfortunately, this sample comprehensive test vector tar-ball (tv.tar.gz) is not present in this location. I have been searching all out, but I could not get hold of this set with all 259 vectors from anywhere. Could I know how to get hold of this complete test vector set. (Any web link available?). Kindly help… Thanks, Cyriac -- View this message in context: http://openssl.6102.n7.nabble.com/Where-is-the-sample-comprehensive-CAVS-test-vectors-set-with-all-259-test-vectors-tp65538.html Sent from the OpenSSL - Dev mailing list archive at Nabble.com. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
