Re: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
Hi The c-client library/API does its own socket I/O for non-SSL sessions, but in SSL the socket I/O is delegated to OpenSSL. When c-client does its own socket I/O, it sets a timeout (normally 15 seconds) on a select() call prior to doing any read() or write() calls. Thus, c-client never does a read() or write() that would block. If the select() timeout hits, there is a callback to the application to enquire whether the application wants to continue waiting. If the application chooses to continue waiting, the select() is restarted with an updated timeout. Otherwise, the socket is closed and the application receives a corresponding I/O error. The net effect is that a non-SSL I/O can wait forever as long as the application consents. c-client does not unilaterally disconnect. My problem is that this doesn't happen with SSL sessions because the socket I/O has been delegated to OpenSSL. There is no obvious way to instruct OpenSSL to timeout its socket I/O, much less do the mechanism described above. So, the questions are: (1) Is there a way to set a timeout for OpenSSL's socket I/O (given that it has been delegated to OpenSSL)? If so, how? (2) If the answer is yes, is there a way to do the query type timeout described above? If so, how? (3) If the answer to either (1) or (2) is no, then how would we go about altering the OpenSSL consumer (which, in this case, is c-client) so that OpenSSL uses the consumer's socket I/O code instead of OpenSSL's socket I/O code? I'm hoping that you will tell me that there's some callback function pointer that can be passed. -Parimal -- -- Warm Regards, Parimal Das
Re: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
Sorry, I forgot to mention that c-client library/API is part of IMAP-2009 library (http://www.panda.com/imap/) I am using IMAP library, which in turn using OpenSSL. I am re-posting my last post here. The IMAP(2009) c-client library/API does its own socket I/O for non-SSL sessions, but in SSL the socket I/O is delegated to OpenSSL. When c-client does its own socket I/O, it sets a timeout (normally 15 seconds) on a select() call prior to doing any read() or write() calls. Thus, c-client never does a read() or write() that would block. If the select() timeout hits, there is a callback to the application to enquire whether the application wants to continue waiting. If the application chooses to continue waiting, the select() is restarted with an updated timeout. Otherwise, the socket is closed and the application receives a corresponding I/O error. The net effect is that a non-SSL I/O can wait forever as long as the application consents. c-client does not unilaterally disconnect. My problem is that this doesn't happen with SSL sessions because the socket I/O has been delegated to OpenSSL. There is no obvious way to instruct OpenSSL to timeout its socket I/O, much less do the mechanism described above. So, the questions are: (1) Is there a way to set a timeout for OpenSSL's socket I/O (given that it has been delegated to OpenSSL)? If so, how? (2) If the answer is yes, is there a way to do the query type timeout described above? If so, how? (3) If the answer to either (1) or (2) is no, then how would we go about altering the OpenSSL consumer (which, in this case, is c-client) so that OpenSSL uses the consumer's socket I/O code instead of OpenSSL's socket I/O code? I'm hoping that you will tell me that there's some callback function pointer that can be passed. I again apologies for reposting. Parimal Das
RE: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
Parimal Das wrote: The IMAP(2009) c-client library/API does its own socket I/O for non-SSL sessions, but in SSL the socket I/O is delegated to OpenSSL. When c-client does its own socket I/O, it sets a timeout (normally 15 seconds) on a select() call prior to doing any read() or write() calls. Thus, c-client never does a read() or write() that would block. Actually, they still can block. Suppose 'select' gets a 'write' hit because X bytes can be written without blocking, but then you call 'write' to send X+1 bytes. Boom, you block. So unless the library actually uses non-blocking socket operations, it can always block due to various types of ambush. If the select() timeout hits, there is a callback to the application to enquire whether the application wants to continue waiting. If the application chooses to continue waiting, the select() is restarted with an updated timeout. Otherwise, the socket is closed and the application receives a corresponding I/O error. The net effect is that a non-SSL I/O can wait forever as long as the application consents. c-client does not unilaterally disconnect. You can code the same thing with OpenSSL. Use non-blocking sockets and non-blocking BIOs. Really, the only difference is that with TCP, you can 'select' before or after you try to 'read' or 'write'. (You can call 'read' and then only call 'select' if the read would block, or you can call 'select' first and only call 'read' if 'select' tells you to; same for write.) With SSL, you must try to SSL_read or SSL_write *before* you 'select'. Otherwise, you may wind up waiting for something that has already happened. (And how would you know whether to 'select' for read or write anyway? You may need to write to the socket to read decrypted data, and you may need to read from the socket to send encrypted data depending on what's going on with the SSL engine.) My problem is that this doesn't happen with SSL sessions because the socket I/O has been delegated to OpenSSL. There is no obvious way to instruct OpenSSL to timeout its socket I/O, much less do the mechanism described above. Set the sockets and BIOs non-blocking. If SSL_read/SSL_write tells you they will block, then 'select' in the appropriate direction. If 'select' times out, then do your callback thing. So, the questions are: (1) Is there a way to set a timeout for OpenSSL's socket I/O (given that it has been delegated to OpenSSL)? If so, how? No, but you don't need to. (3) If the answer to either (1) or (2) is no, then how would we go about altering the OpenSSL consumer (which, in this case, is c-client) so that OpenSSL uses the consumer's socket I/O code instead of OpenSSL's socket I/O code? I'm hoping that you will tell me that there's some callback function pointer that can be passed. You can do that, but it's a lot more work. Just call 'select' when OpenSSL tells you to and callback the application if the 'select' times out. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
Parimal Das wrote: Its the second case Darry, Here the 'sleep' is Operating System Sleep mode induced by closing the lid of laptop. After opening the laptop, when the system wakes up, My application is always hanging at the same place. Bug is in your code. It is doing what you asked it do -- waiting up to forever for data from the other side. The other side will never send anything because it has long forgotten about the connection. Your application will never send anything because it is blocked in a read function. TCP and UDP will do the same thing if you call 'read' or 'recv' and block for data that will never arrive. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
Hello,
Here is my test code. I am downloading a file with https connection.
This is compiled as $g++ -lssl -lcrypto sslShow.cpp. on OS X 10.5.8
Using default OS X libs (libcrypto 0.9.7 and libssl 0.9.7)
When it has downloaded some 2MB data, I closed my laptop lid (OSX induced
sleep)
After 5 minutes when i open my laptop, the process hangs at the same place
as before.
I have reproduced the same with latest 0.9.8k version also.
Please suggest.
1. What i should include in this code to correct this hang?
2. How to set read/write timeouts?
Thanks a lot guys.
(the Test Code Call Trace is as follows )
CALL TRACE===
Call graph:
811 Thread_2507
811 start
811 main
811 BIO_read
811 ssl_read
811 ssl3_read_internal
811 ssl3_read_bytes
811 ssl3_read_n
811 BIO_read
811 read$UNIX2003
811 read$UNIX2003
MY TEST CODE =
#include openssl/ssl.h
#include openssl/err.h
#include openssl/bio.h
#include iostream
#define MAX_PACKET_SIZE 1
int main() {
BIO * bio;
SSL * ssl;
SSL_CTX * ctx;
/* Initializing OpenSSL */
SSL_load_error_strings();
ERR_load_BIO_strings();
OpenSSL_add_all_algorithms();
SSL_library_init(); //mandatory and missing from some examples
ctx = SSL_CTX_new(SSLv23_client_method());
if (ctx == NULL) {
std::cout Ctx is null std::endl;
ERR_print_errors_fp(stderr);
}
//using a store from examples
if(! SSL_CTX_load_verify_locations(ctx,
/Users/pd/workspace/openssl/TrustStore.pem, NULL))
{/* Handle failed load here */
std::cout Faild load verify locations std::endl;
}
bio = BIO_new_ssl_connect(ctx);
BIO_get_ssl(bio, ssl);
SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);
//replace with your own test server
BIO_set_conn_hostname(bio, www.myDomain.com:https);
if(BIO_do_connect(bio) = 0) {
std::coutFailed connection std::endl;
} else {
std::coutConnected std::endl;
}
if(SSL_get_verify_result(ssl) != X509_V_OK)
{
/* Handle the failed verification */
std::cout Failed get verify result std::endl;
fprintf(stderr, Certificate verification error: %i\n,
SSL_get_verify_result(ssl));
//do not exit here (but some more verification would not hurt)
because if you are using a self-signed certificate you will receive 18
//18 X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT which is not an
error}
char *write_buf = GET /downloads/goodApp.exe / HTTP/1.0\n\n;
if(BIO_write(bio, write_buf, strlen(write_buf)) =
0){
if(! BIO_should_retry(bio)){
/* Handle failed write here */
}
/* Do something to handle the retry */
std::cout Failed write std::endl;
}
char buf[MAX_PACKET_SIZE];
int p;
char r[1024];
FILE *fp;
fp = fopen(something.abc, a+);
for(;;){
p = BIO_read(bio, r, 1023);
if(p = 0) break;
r[p] = 0;
fprintf(fp, %s, r);
}
fclose(fp);
std::cout Done reading std::endl;
/* To free it from memory, use this line */
ERR_print_errors_fp(stderr);
BIO_free_all(bio);
}
return 0;
}
On Thu, Oct 29, 2009 at 4:57 PM, David Schwartz dav...@webmaster.comwrote:
Parimal Das wrote:
Its the second case Darry,
Here the 'sleep' is Operating System Sleep mode induced by closing the
lid
of laptop.
After opening the laptop, when the system wakes up,
My application is always hanging at the same place.
Bug is in your code. It is doing what you asked it do -- waiting up to
forever for data from the other side. The other side will never send
anything because it has long forgotten about the connection. Your
application will never send anything because it is blocked in a read
function. TCP and UDP will do the same thing if you call 'read' or 'recv'
and block for data that will never arrive.
DS
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
--
--
Warm Regards,
Parimal Das
RE: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
Parimal Das wrote: Please suggest. 1. What i should include in this code to correct this hang? It depends on what your code should do in this case. Do you want to wait a limited amount of time for the other side to reply? Or do you want to wait possibly forever? Your current code specifically elects to wait possibly forever, but then you complain when it waits possibly forever. If the user should interrupt it, then this is sensible. If not, then it's not. 2. How to set read/write timeouts? Well, 'alarm' is the easiest way in toy code like this. You can reset the 'alarm' every time you receive a certain amount of data. It depends how you want to handle cases like where the other side 'dribbles' data at you. Decide how long is reasonable to wait, and code that. But this is another reason that blocking socket operations are difficult to use. In what realistic situation do you want to wait forever? So you make a blocking operation and then, surprise, have to work around the annoying fact that it blocks. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
google: TCP OPTION KEEPALIVE http://tldp.org/HOWTO/html_single/TCP-Keepalive-HOWTO/ You would be best with an application level timeout, which would send an application enquiry (heartbeat) from the laptop to the server. Waking up from sleeping, the application would send the next heartbeat. (any data has the same effect, the problem was that the laptop was silent). That would get an immediate response, from the remote OS, that the TCP connection was dismantled ages ago, whilst the laptop was comatose. That would appear to the SSL BIO as select_says_fd_is_readable and when it reads from the socket-fd, an error eof. Then SSL knows and lets you know too. If you dont want to change the application protocol, you can have the OS do exactly that for that socket. No DATA.data will appear on the application side of the socket-fd, just hidden TCP control messages, on the network side. That will also keep any NAT router happy. In an ideal world, the wake-from-sleep should do that for each TCP connection. Maybe it leaves it for each app to figure it out on its own. Graham 2009/10/29 Parimal Das parimal@webyog.com When it has downloaded some 2MB data, I closed my laptop lid (OSX induced sleep) After 5 minutes when i open my laptop, the process hangs at the same place as before.
Re: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
Parimal Das wrote: Its the second case Darry, Here the 'sleep' is Operating System Sleep mode induced by closing the lid of laptop. After opening the laptop, when the system wakes up, My application is always hanging at the same place. It is possible there is something specific OSX does in relation to restarting the application. Knowing the call function trace is a start but you need to rebuild the application with full debugging and insert some logging. What you want to be able to do to this mailing list is describe the sequence of event that take place. So you need to describe the entire situation: * From the context of the kernel system calls the application is making, including return codes and data. On linux strace is a command line tool that can do this. * Dig into the code and describe the sleep/wake mechanism used. You then need to work out from the code base of the application which party is taking responsibility for what matters relating to socket IO. I.e. the application or the OpenSSL library. Its possible to delegate most of the work to the OpenSSL library but its also possible to override and provide custom handling inside the application. Can you write a simple test case, this is the smallest possible program that can reliably demonstrate the problem. If you are not yourself a developer (software engineer) then it is unlikely you will be able to make further progress on this matter. It is not even clear at this time the problem is with the OpenSSL library so the onus is on your to provide test cases to prove that. In doing that then it maybe possible for non-OSX users to assist with the problem and provide patches for you to test. Darryl __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
after wakeup from sleep ? What do you mean ? Do you mean you used an API like poll() or select() to put the thread to sleep and then it wakes up and the socket readability was indicated ? If so please post an outline of the code your sleep/wake mechanism you are using. Do you mean your operating system put the host computer (possibly a laptop) to sleep, as in low power mode ? Not sure I can help you on this one, maybe someone else can. Darryl __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: ssl_read() hangs after wakeup from sleep in OSX 10.5.8
Its the second case Darry, Here the 'sleep' is Operating System Sleep mode induced by closing the lid of laptop. After opening the laptop, when the system wakes up, My application is always hanging at the same place.* * 562 ssl_getbuffer 562 ssl_getdata 562 ssl3_read_internal 562 ssl3_read_bytes 562 ssl3_read_n 562 BIO_read 562 read$UNIX2003 Thanks Again -Parimal Das On Wed, Oct 28, 2009 at 12:32 AM, Darryl Miles darryl-mailingli...@netbauds.net wrote: after wakeup from sleep ? What do you mean ? Do you mean you used an API like poll() or select() to put the thread to sleep and then it wakes up and the socket readability was indicated ? If so please post an outline of the code your sleep/wake mechanism you are using. Do you mean your operating system put the host computer (possibly a laptop) to sleep, as in low power mode ? Not sure I can help you on this one, maybe someone else can. Darryl __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
