Dr. Henson, (and the rest of the list)- I'm still having problems getting my server to renegotiate correctly. I'll describe the basic scenario again and how I'm handling it, then ask some questions. Again, I'm running 0.9.8r 1. My server is sending a large amount of data to the peer, so SSL_write() is being called over and over, getting blocked several time, then continuing when the socket is writeable again. 2. While the server is sending data, the client sends a ClientHello(with renegotation info). I verify this with traces in the client. 3. After that, a call to SSL_write() returns -1, and calling SSL_get_error() returns SSL_ERROR_WANT_WRITE.* As I understand it, at this point, I need to make sure the socket is writeable and then call SSL_write() again. 4. The server poll()s on the socket several times until the file descriptor is writeable. I should note that when the server polls, it doesn't even poll for readability on the socket, POLLIN is removed from events in the pollinfo struct. The server doesn't care if it's readable, it needs to call SSL_write() for the socket. (I mention this because when I did poll for reads, I encountered the error scenario in my previous email. The socket would be readable, and the server would call SSL_read(), the ClientHello would be read in, but the server gets a "bad write retry" error. The way my server is coded, its' easier to not even ask for readability on the socket. 5. The socket tells me it is writeable, so the server calls SSL_write(). I have message callbacks implemented in my server so I can examine SSL packets coming in. I would expect the SSL_write() to read in the ClientHello at this point, since the peer sent it. But why wouldn't the SSL_write have returned WANT_READ at this point, when the ClientHello is coming in? Just curious, but I am doing what the protocol tells me to do by calling SSL_write() 6. The call to SSL_write essentially just keeps sending the data that in the buffer I was sending from. I also know that internally in OpenSSL, there is data to write in it's buffer. In the function do_ssl3_write(), I log what's in s->s3->wbuf.left if it is not zero, which it's not at this point. So after SSL_write() keeps returning either the number of bytes it has written, or that it's blocked and I need to poll again, I keep calling it, all of the data is written out, and the ClientHello is never read by the protocol. Once the data is all written, my server closes the connection. So my most general question is, is there something obvious I'm doing wrong? Do I need to be polling for reads in step 4? I don't think I should care if the socket is readable, since I need to wait until the socket is writable before calling SSL_write() again. At least that's what OpenSSL is telling me. Why does SSL_write() never return WANT_READ when the ClientHello will be read in internally by the protocol? Seems backwards to me. Do I need to do anything in particular in step 5 when calling SSL_write()? If I understand correctly, I'm supposed to call the function with the same params as the last call, which is what I'm pretty sure I'm doing. So why does the protocol never see the ClientHello? Any help is appreciated, because I think I'm doing what OpenSSL is telling me to do. > Date: Tue, 7 Feb 2012 19:47:59 +0100 > From: st...@openssl.org > To: jetso...@hotmail.com > Subject: Re: Renegotiation > > On Tue, Feb 07, 2012, Jason Schultz wrote: > > > > > Dr. Henson- > > > > I've sent this question to the OpenSSL mailing list, but have not received > > any responses to this point. I hope I am sending it to the correct list. > > I know you're one of the main OpenSSL developers, so I'm sending this > > directly to you to hopefully help figure out what might be happening. I > > know it's somewhat of an involved question, but I'm struggling to figure it > > out. At this point, it seems like OpenSSL is not handling the situation > > correctly. I would be willing to do some more diagnostic work to drill > > down into what the problem is, per your suggestions, or anyone elses. The > > following is the original message to the list. Thanks for any help you can > > provide. > > > > I have implemented a server using OpenSSL 0.9.8r. If I use s_client to > > open a > > connection to a listenening SSL port on the server, and use the R commend > > to > > initiate a rehandshake, the rehandshake completes successfully(as > > expected). I > > have verified this using both SSL 3.0 and TLS 1.0. > > > > > > Another client(non-OpenSSL) opens a connection to my server with a > > successful > > initial handshake as well. The client is basically an FTP client, and it > > then > > initiates a data transfer with a get, so my server is sending data to the > > client. During this transfer, the client sends a Client Hello to > > renegotiate. > > My server, using poll() to look for activated sockets, ends up calling > > SSL_read() and reads in the Client Hello. However, OpenSSL gets an error, > > ?SSL3_WRITE_PENDING:bad write retry?. I?ve dug around a little in the > > OpenSSL > > code and added some debug code to try to follow what is happening. > > > > >From what I can tell, the following is taking place: > > > > - Client and server establish a connection and my server is > > sending > > data with repeated calls to SSL_write() > > > > - While server is writing data, client initiates a renegotiation, > > and > > sends a Client Hello > > > > - SSL_write() on the server returns -1 with SSL_ERROR_WANT_READ, > > presumably after the Client Hello comes in > > > > - Server calls SSL_read() and reads in the Client Hello > > (I have info_cb functionality implemented that dump out handshake messages, > > the > > message looks OK) > > > > - OpenSSL calls ssl3_get_client_hello(); I added a syslog() when > > the > > state changes to SSL3_ST_SR_CLNT_HELLO_B > > > > - ssl3_get_client_hello() returns successfully; I added a syslog > > when > > the state gets changed to SSL3_ST_SW_SRVR_HELLO_A to verify this. > > > > - ssl3_send_server_hello() is called > > > > - The ssl3_send_server_hello() will lead to calls to the following > > functions, in order: ssl3_write_bytes(), do_ssl3_write(), and > > ssl3_write_pending(). > > > > - Meanwhile, the previous calls to SSL_write have data left to > > send > > and a call to ssl3_write_pending() is being made from do_ssl3_write(). I?m > > not > > sure of the code path from the server calling SSL_write() to > > do_ssl3_write()/ssl3_write_pending(), but I see do_ssl3_write() calling > > ssl3_write_pending() several times during the file transfer. > > > > > > Does this scenario sound like it should be working? I?m not sure what my > > server could be doing wrong? A couple notes, > > SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER is not set on my server. I?m not sure > > that > > it should or needs to be, I did notice in the code that that mode is one of > > the > > conditions that is checked for when throwing the SSL_R_BAD_WRITE_RETRY > > error. > > >From what I?ve read, the SSL3_WRITE_PENDING error is caused by calling > > SSL_write() after getting SSL_ERROR_WANT_READ/WRITE with a different buffer > > than before. However, my server is not calling SSL_write() after it gets > > the > > SSL_ERROR_WANT_READ error when the Client Hello comes in. So it seems to > > me > > the error is being thrown because OpenSSL is internally calling > > ssl3_write_pending() incorrectly. > > > > > > I have some traces of the scenario but I did not include them, they may > > make > > things more confusing. They exists in two different places, logging in my > > server before/after SSL_read and SSL_write > > calls(SSL_get_error()/ERR_error_string() calls) as well as info_cb and > > msg_cb > > message that display the SSL records coming in and going out. A separate > > trace > > exists of some syslog() calls I added to the OpenSSL code. I know there is > > a > > way to build OpenSSL in debug mode, but I haven?t taken the time to do this > > yet. Please advise if I should look at doing that, or include the logging > > I do > > have. > > > > If an SSL_write or SSL_read calls returns an SSL_ERROR_WANT_* condition you > should retry that call until it succeeds. That means if you are calling > SSL_write and you get and SSL_ERROR_WANT_READ you should retry the SSL_write > call that produced the condition. > > Steve. > -- > Dr Stephen N. Henson. OpenSSL project core developer. > Commercial tech support now available see: http://www.openssl.org
