Re: tstclnt exit code: do we really need to distinguish between failures during PR_Send and PR_Recv?

[Robert Relyea]

On 10/31/2011 01:45 AM, Brian Smith wrote:
 I am modifying tstclnt to test my patch for bug 542832. With my patch, 
 tstclnt always succeeds or fails like it did before, but in some of the 
 failure cases it exits with exit code 1 when the testcase expects it to exit 
 with exit code 254, and sometimes it exits with exit code 254 when the 
 testcase expects it to exit with exit code 1.

 Basically, the current rule is that tstclnt exits with an exit code of 254 if 
 PR_Send failed, and 1 for any other error. This doesn't seem like a useful 
 rule. Instead, a modification of the proposal by Glen Beasley in bug 402058 
 comment 9 [1] seems better: If the handshake failed because of a certificate 
 validation failure, return 254; if it failed due to any other error, return 
 1. In the future, we could distinguish other types of errors (e.g. SSL 
 version mismatch could be 253, TLS renegotiation support mismatch could be 
 252, etc.). This seems like it would be much more useful than distinguishing 
 whether we failed during PR_Send or PR_Recv.

 Thoughts?

 Cheers,
 Brian
I would look at the checks and see what kind of failures it is expecting.

I think the main thing is to distinguish a normal connection failure
with a client auth authorization failure.

That being said, I suspect that we currently don't distinguish between
these two, and wind up with a false success on expected client auth
failure cases when the server is down. In practice that's not too much
of a problem because several of the other tests in the suite will fail
in that case.

Upshot: creating a rationalized error system from tstclnt sounds fine,
as long that the tests are adjusted accordingly. (longer term, we may
want to set the error code based on the PORT_GetError() value).

bob
 [1] https://bugzilla.mozilla.org/show_bug.cgi?id=402058#c9

 Additional background on error codes returned by tstclnt:

 [2] https://bugzilla.mozilla.org/show_bug.cgi?id=402058#c7
 [3] https://bugzilla.mozilla.org/show_bug.cgi?id=402058#c5
 [4] https://bugzilla.mozilla.org/show_bug.cgi?id=86528#c9
 [5] https://bugzilla.mozilla.org/show_bug.cgi?id=68869


-- 
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto

Re: tstclnt exit code: do we really need to distinguish between failures during PR_Send and PR_Recv?

[Brian Smith]

Robert Relyea wrote:
 That being said, I suspect that we currently don't distinguish between
 these two, and wind up with a false success on expected client auth
 failure cases when the server is down. In practice that's not too much
 of a problem because several of the other tests in the suite will fail
 in that case.
 
 Upshot: creating a rationalized error system from tstclnt sounds fine,
 as long that the tests are adjusted accordingly. (longer term, we may
 want to set the error code based on the PORT_GetError() value).

Bob, I think your suggestion makes a lot of sense. We can only return 255 
distinct error codes from tstclnt but there are many more possible error codes 
from PORT_GetError(). How about this updated proposal:

* Add an additional command-line flag -E error-id (e.g. -E 
SSL_ERROR_CERT_DOMAIN) to tstclnt.
* If the flag is given and PR_Read() or PR_Write() fails with the given error, 
exit with code 2, which would mean failed with the expected error.
* Otherwise, if error flag was given, and an error was encountered, exit with 
code 3, which would mean failed with the wrong error.
* Otherwise, if the error flag was given, and no error was encountered, exit 
with code 4, which would mean succeeded but expected to fail.
* Otherwise, return 1 on any error or 0 if there were no errors.

We could then make an analogous change to selfserv (hopefully in another bug).

Thoughts?

Cheers,
Brian
-- 
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto