RE: Does Openssl cache App data -- 2nd Try..
-Original Message-From: [EMAIL PROTECTED][mailto:[EMAIL PROTECTED]]On Behalf Of David SchwartzSent: Friday, July 07, 2006 10:57 AMTo: openssl-users@openssl.orgSubject: RE: Does Openssl cache App data -- 2nd Try.. a. Does this msg_callback get executed after peek? or after read? if former, I could simply discard the buffer as peek does not dequeue the pkt, so the kernel can read it again. If its a "read", then I need to pass on that buffer to the kernel module which does the actual decryption. I can't quite follow you. The whole thing you are trying to avoid iscaching of application data, but this is SSL_peek's whole point. The maindifference between SSL_read and SSL_peek is that SSL_peek caches theapplication data (so you can peek at it again or read it later) whereasSSL_read discards it. I was thinking SSL_peek does a buffer copy internally and gives that to the application, and the actual bytes are still available in the TCP receive queue..the idea is although I tried to peek, the data is still available for a read by kernel, and so I neednt make use of the cache, inspite of one being available. As you said, kernel has to do a lot of processing wherein it should read the ssl record header, and if not(application data) hand the fd control back to userspace. We are going to somehow poll from userspace as well as kernel for the same connection, and do a fd transfer from userspace to kernel. This seems like an overly-complex solution. The kernel should always ownthe SSL connection. It should analyze received data to determine if it isprotocol or application. If application data, it should decrypt it andreturn it as application data. If protocol data, it should pass it touser-space for SSL protocol processing. This seems like a clean and simpleapproach. This is exactly what we want to do, but is not an FDrequired in the userspace to POLL and read the SSL Control Packets? Once the KERNEL sees the 1st Control Pkt on an FD, it will handover control back to the userspace module to continue processing that FD, until an application data packet is received, wherein control will be switched back to the KERNEL. so what I have now learnt from the responses is that I can expect that openssl will end up caching application data, as as the control pkts gets processed, userspace could endup reading app data..so if I do an SSL_peek before every SSL_read can I prevent processing of application data? I don't understand what SSL_peek and SSL_read are meant to be in thecontext you are using them. These are user-space OpenSSL functions and youare supposed to be doing SSL in kernel. Yes, I wanted to call them from userspace openssl module only inorder to process control packets, as SSL_read is the one which receives control packets as well and internally triggers renegotiations and change cipher specs..so if I can peek and check if its an application data packet, I can simply discard the buffer given by peek and give back control to KERNEL, which will READ the TCP receive queue and get back the same DATA, hoping SSL_peek is same as TCP_PEEK!! This is the clarification I require. From KERNEL openssl will never be invoked, we have our own kernel library which can only encrypt/decrypt. SSL_accept is done from userspace. You can think of this as an hardware accelerator working from kernel, except that its still software routines.. PS: If i am not making sense in more than 1 way(s) beg apoligies, am a newbie.. I guess I can't seem to follow your main architecture. Again, I recommendthe following:1) The kernel should always manage the SSL connection, it should probablypresent the SSL connection to the SSL user-space code and to the applicationusing the connection as two separate objects. Userspace accepts the incoming SSL connection using openssl SSL_accept, until the kernel has no work. Once accept is completed, userspace will transfer that FD to a kernel thread, which will continue polling on that FD using sock_poll(). Userspace will continue polling on the accepted FD. Note here we now have an FD pair, as we have transferred the userspace ssl accepted FD to a kernel thread using send_fd (AF_UNIX socket).2) For received data, the kernel should analyze it and determine if it'sapplication or protocol. Exactly, thats what will happen after sock_poll() returns.3) Received application data should be decrypted in the kernel and returnedas normal data to the application using the SSL connection. Correct4) Received protocol data should be passed to the user-space SSL protocolengine application. Precisely, so now the sock_poll() will not be polling on this FD, the userspace will start its poll. Call SSL_read() until openssl says that app data is now on the wire so I cant process anymore. But I dont want SSL_read to read the app data meaning I prefer to get some notification that thepkt is app data pkt using *(msg_handler) registered to
RE: Does Openssl cache App data -- 2nd Try..
So in a crux, what can I do to ensure that openssl does not read app data pkts at all.. Nothing. You have no guarantee how many bytes the kernel's read() will hand back up to you. You will have to severely hack on your kernel and networking implementation to make this happen. You might find it easier to have the kernel do 'all' SSL, and then do callbacks/upcalls to user space for the non-kernel stuff. /r$ -- SOA Appliances Application Integration Middleware __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Does Openssl cache App data -- 2nd Try..
Thanks david..let me add more clarity to my requirement... Is there someway for the application to know before calling ssl_read, that some app data buffer can be got for sure? Only by putting that data in a buffer, which seems to be what you don't want. Or can SSL_peek prevent me from hitting the case you have mentioned? SSL_peek does precisely what you claim you don't want, which is to put the data in a cache. The reason is we are in a scenerio wherein all ssl control pkts are gonna be processed from userspace using openssl and all app data pkts are gonne be processed from kernel using our own ssl implementation. So when userspace openssl does manage to read app data, I need to ensure that whatever it has cached (add data bytes), has to be cleared and copied into our ssl library in kernel, so that it can wait for the remaining bytes and do the decryption. So in a crux, what can I do to ensure that openssl does not read app data pkts at all.. You have to separate the data in the kernel. I would not recommend trying to hack OpenSSL's high-level functions for this purpose but use the low-level ones instead. Ignore SSL_read/SSL_peak/SSL_write and the like. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Does Openssl cache App data -- 2nd Try..
Thanks once again for the responses...I would like to add few more points here..there seems to be a callback function (msg_callback) within the ssl_st structure, which when registered by the application for a specific content type, SSL_read, seems to call that if a pkt of that content type is seen on the wire. I was thinking of making use of this, now my questions are a. Does this msg_callback get executed after peek? or after read? if former, I could simply discard the buffer as peek does not dequeue the pkt, so the kernel can read it again. If its a read, then I need to pass on that buffer to the kernel module which does the actual decryption. b. I dont see this msg_callback getting called for content type Application data, I see the code only for the other ssl record types..now am I missing something here. As you said, kernel has to do a lot of processing wherein it should read the ssl record header, and if not(application data) hand the fd control back to userspace. We are going to somehow poll from userspace as well as kernel for the same connection, and do a fd transfer from userspace to kernel. so what I have now learnt from the responses is that I can expect that openssl will end up caching application data, as as the control pkts gets processed, userspace could endup reading app data..so if I do an SSL_peek before every SSL_read can I prevent processing of application data? Also there seems to be no low/high level api to access this read buffer cache of ssl.. or is there? This read buffer anyhow is a pointer to the application read buffer rite? or is there some bufcopy happening internally within openssl? Coz if its just a pointer I am thinking if I could just pass a minimal buffer size wherein it can read only handshake record size pkts, then indirectly I could end up controlling the read. PS: If i am not making sense in more than 1 way(s) beg apoligies, am a newbie.. Thanks --Gayathri -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of David Schwartz Sent: Friday, July 07, 2006 1:47 AM To: openssl-users@openssl.org Subject: RE: Does Openssl cache App data -- 2nd Try.. Thanks david..let me add more clarity to my requirement... Is there someway for the application to know before calling ssl_read, that some app data buffer can be got for sure? Only by putting that data in a buffer, which seems to be what you don't want. Or can SSL_peek prevent me from hitting the case you have mentioned? SSL_peek does precisely what you claim you don't want, which is to put the data in a cache. The reason is we are in a scenerio wherein all ssl control pkts are gonna be processed from userspace using openssl and all app data pkts are gonne be processed from kernel using our own ssl implementation. So when userspace openssl does manage to read app data, I need to ensure that whatever it has cached (add data bytes), has to be cleared and copied into our ssl library in kernel, so that it can wait for the remaining bytes and do the decryption. So in a crux, what can I do to ensure that openssl does not read app data pkts at all.. You have to separate the data in the kernel. I would not recommend trying to hack OpenSSL's high-level functions for this purpose but use the low-level ones instead. Ignore SSL_read/SSL_peak/SSL_write and the like. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Does Openssl cache App data -- 2nd Try..
a. Does this msg_callback get executed after peek? or after read? if former, I could simply discard the buffer as peek does not dequeue the pkt, so the kernel can read it again. If its a read, then I need to pass on that buffer to the kernel module which does the actual decryption. I can't quite follow you. The whole thing you are trying to avoid is caching of application data, but this is SSL_peek's whole point. The main difference between SSL_read and SSL_peek is that SSL_peek caches the application data (so you can peek at it again or read it later) whereas SSL_read discards it. As you said, kernel has to do a lot of processing wherein it should read the ssl record header, and if not(application data) hand the fd control back to userspace. We are going to somehow poll from userspace as well as kernel for the same connection, and do a fd transfer from userspace to kernel. This seems like an overly-complex solution. The kernel should always own the SSL connection. It should analyze received data to determine if it is protocol or application. If application data, it should decrypt it and return it as application data. If protocol data, it should pass it to user-space for SSL protocol processing. This seems like a clean and simple approach. so what I have now learnt from the responses is that I can expect that openssl will end up caching application data, as as the control pkts gets processed, userspace could endup reading app data..so if I do an SSL_peek before every SSL_read can I prevent processing of application data? I don't understand what SSL_peek and SSL_read are meant to be in the context you are using them. These are user-space OpenSSL functions and you are supposed to be doing SSL in kernel. PS: If i am not making sense in more than 1 way(s) beg apoligies, am a newbie.. I guess I can't seem to follow your main architecture. Again, I recommend the following: 1) The kernel should always manage the SSL connection, it should probably present the SSL connection to the SSL user-space code and to the application using the connection as two separate objects. 2) For received data, the kernel should analyze it and determine if it's application or protocol. 3) Received application data should be decrypted in the kernel and returned as normal data to the application using the SSL connection. 4) Received protocol data should be passed to the user-space SSL protocol engine application. 5) Sent application data should be encrypted by the kernel and sent. 6) SSL protocol data should be accepted by the kernel from the user-space SSL protocol engine and sent. For connection establishment, for example, you have an application that asks the kernel to setup an SSL connection. The kernel makes the underlying TCP connection and informs the SSL user-space application about the new connection. Then the user-space SSL application uses the kernel to manage the establishment of the connection and then tells the kernel the connection is ready. Then the kernel unblocks the socket in the application that requested the connection. For inbound connections, the kernel must know that some sockets always get SSL connections. When they get a new connection, they inform both the user-space SSL application and the application that opened the socket in the first place. The user-space SSL application manages the negotiation and passes the symettric key to the kernel, which then lets the application that opened the socket send and receive. Again, you should totally ignore high-level functions like SSL_read and SSL_peek, they don't do anything even remotely resembling what you want. The way you seem to want to do it seems much, much harder to me. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
