Re: Help Needed: SSL Connect starting from a weird state

[Jeff Saremi]

My initial analysis of this was very misleading. I have to apologize for
that.
The problem was that during the first part of the handshake
(clienthello), the call failed without anything being written out.
Tracing ssl23_client_hello() in s23_clnt.c showed that the following
statement returned false and as a result -1 was returned as the error.
if (RAND_pseudo_bytes(...) =0)
  return -1;

And for any instances of error for which an internal OpenSSL ERR is not
set, SSL_ERROR_SYSCALL is used, which is further misleading.

I did a cursory search of anywhere that a call to RAND_pseudo_bytes can
fail and there are tens of such instances for which OpenSSL ERR is not
set. In fact, there's only one instance of a call to RANDerr which is
inside md_rand.c. I guess this would be something for OpenSSL guys to
ponder.

Another strange thing is no matter how many times we ran the
application, the call always failed on the same spot; the same call to
RAND_pseudo_byes each time, not before or after. This was regardless of
how many successful calls were made prior to.

Jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Help Needed: SSL Connect starting from a weird state

[Jeff Saremi]

We've been running our SSL code for a while now with no issues. But
recently one of our developers started encountering this problem.
We did the best we could to troubleshoot to no avail. I know the 
problem is not OpenSSL and it's something we're doing incorrectly,
probably at the start up.

The problem:
SSL completed without having done a single send or receive during the
handshake.

What we get in the print out, after issuing SSL_connect() is this:

Printout:
18:13:56.925 [4228] connect
18:13:56.927 [4228] SSL nonblock rc:-1 shutdown:0 state:23WCHA
(from:UNKWN )
18:13:56.928 [4228] ssl_err:5 SSL_ERROR_SYSCALL

The rough version of the code printing the above is this:
printf(connect\n);
const char *fromState = SSL_state_string(mSsl);
rc = SSL_connect(mSsl);
printf(SSL nonblock rc:%d shutdown:%d state:%s (from:%s)\n,
rc,
SSL_get_shutdown(mSsl),
SSL_state_string(mSsl),
fromState);
int ssl_error = SSL_get_error(mSsl, rc);
switch(ssl_error)
{
case SSL_ERROR_SYSCALL:
  printf(%d SSL_ERROR_SYSCALL\n, SSL_ERROR_SYSCALL);
...


What I would expect to see would be something along the lines of the
following:

SSL nonblock rc:1 shutdown:0 state:SSLOK (from:UNKWN )

or
SSL nonblock rc:-1 shutdown:0 state:SSLOK (from:SSLOK )


For additional debugging I have enabled callbacks using the following
too:
SSL_set_msg_callback

And I see a lot of that happening but not in this case.
In this particular case, after switching the destination IP and port all
we get is what I showed you. Not even one single byte is exchanged
anywhere.

Looking inside ssl_stat.c I see the following:
case SSL23_ST_CW_CLNT_HELLO_A:  str=23WCHA; break;

Looking inside s23_clnt.c I see these lines near the beginning of
ssl23_client_hello():

buf=(unsigned char *)s-init_buf-data;
if (s-state == SSL23_ST_CW_CLNT_HELLO_A)

How can my code start in this state?

Any hints would be appreciated.
thanks
jeff