RE: Is full-duplex socket use possible with OpenSSL?

2009-11-02 Thread Mark
Hi Darrel, Thanks for the very useful and clear explanation. Mark wrote: There is one added complication in that the protocol is a datagram protocol at a higher level (although it uses TCP). I am concerned that the whole protocol could block if there is not enough data to encrypt a

RE: Is full-duplex socket use possible with OpenSSL?

2009-11-02 Thread Mark
Hi Ger, There is one added complication in that the protocol is a datagram protocol at a higher level (although it uses TCP).  I am concerned that the whole protocol could block if there is not enough data to encrypt a whole outgoing message but the peer cannot continue until it

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-30 Thread David Schwartz
Mark wrote: I may be making a wrong assumption but if the cypher used is a block cypher does it not wait until a full block of data is ready before it can encrypt and send the data? If a message does not consist of enough data to fill a block, could there be unencrypted data left in a

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-29 Thread David Schwartz
Mark Williams wrote: I can think of one. In the near future I will need to add SSL support to a legacy application which uses two threads to read/write from/to a socket. If SSL supported this it would make my life much easier. As the situation stands I am not sure how to tackle this

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-29 Thread Mark
Mark Williams wrote: I can think of one. In the near future I will need to add SSL support to a legacy application which uses two threads to read/write from/to a socket. If SSL supported this it would make my life much easier. As the situation stands I am not sure how to

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-29 Thread David Schwartz
Mark Williams wrote: 2) Let the two threads read and write to your own two independent queues and service the application side of the SSL connection with your own code to and from the read and write queues. Won't I still need to combine the reading and writing to the SSL object into a

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-29 Thread Mark
Mark Williams wrote: 2) Let the two threads read and write to your own two independent queues and service the application side of the SSL connection with your own code to and from the read and write queues. Won't I still need to combine the reading and writing to the SSL

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-29 Thread David Schwartz
Mark Williams wrote: There is one added complication in that the protocol is a datagram protocol at a higher level (although it uses TCP). I am concerned that the whole protocol could block if there is not enough data to encrypt a whole outgoing message but the peer cannot continue until

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-29 Thread Mark
Hi David, There is one added complication in that the protocol is a datagram protocol at a higher level (although it uses TCP). I am concerned that the whole protocol could block if there is not enough data to encrypt a whole outgoing message but the peer cannot continue until

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-29 Thread Darryl Miles
Mark wrote: There is one added complication in that the protocol is a datagram protocol at a higher level (although it uses TCP). I am concerned that the whole protocol could block if there is not enough data to encrypt a whole outgoing message but the peer cannot continue until it gets the

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-29 Thread Ger Hobbelt
There is one added complication in that the protocol is a datagram protocol at a higher level (although it uses TCP).  I am concerned that the whole protocol could block if there is not enough data to encrypt a whole outgoing message but the peer cannot continue until it gets the message.

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-29 Thread Darryl Miles
Ger Hobbelt wrote: It is presumed that every SSL_write() requires a flush (at TCP level this mechanism is called a Push). This basically means the data needs to flush to the reading API at the far end on exactly the byte boundary (or more) data than you sent. This mean you have a guarantee to

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-28 Thread Mark
On Fri, Oct 23, 2009 at 03:47:51PM +0100, Darryl Miles wrote: I advocate that some users would find it useful to be able to invoke SSL_read() and SSL_write() from exactly two threads on the same 'SSL *' simultaneously. There is merit in this and as things stands OpenSSL does

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-26 Thread Darryl Miles
David Schwartz wrote: Darryl Miles wrote: I do not believe the SSL_write() call is allowed to access the underlying BIO/kernel-socket to read in more data. I think SSL_write() is allowed to process any data already read into buffer (from kernel to OpenSSL library internal buffer) in an

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-26 Thread Konstantin Ivanov
Hi all, I am developing a server application which is based on Windows IO Completion ports which basically means that the reads and write to the socket are asynchronous. This also means that I cannot use the SSL_read and SSL_write functions which are tied to the socket fd if I am correct. So

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-26 Thread Kyle Hamilton
My understanding is that if SSL_ERROR_WANT_WRITE happened with SSL_read(), the next SSL_read() would actually call write() to make the forward progress. -Kyle H On Sun, Oct 25, 2009 at 11:03 PM, Darryl Miles darryl-mailingli...@netbauds.net wrote: David Schwartz wrote: Darryl Miles wrote: I

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-26 Thread Darryl Miles
Kyle Hamilton wrote: My understanding is that if SSL_ERROR_WANT_WRITE happened with SSL_read(), the next SSL_read() would actually call write() to make the forward progress. Yes that is possible, as the data for the write is already inside OpenSSL library. Infact all the write to BIO/kernel

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-26 Thread Darryl Miles
Konstantin Ivanov wrote: I am developing a server application which is based on Windows IO Completion ports which basically means that the reads and write to the socket are asynchronous. This also means that I cannot use the SSL_read and SSL_write functions which are tied to the socket fd if

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-26 Thread David Schwartz
Konstantin Ivanov wrote: I am developing a server application which is based on Windows IO Completion ports which basically means that the reads and write to the socket are asynchronous. This also means that I cannot use the SSL_read and SSL_write functions which are tied to the socket fd if

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-24 Thread Darryl Miles
Victor Duchovni wrote: SSL is a state-machine, not a pipe. Reading data may require writes, and writing data may require reads (e.g. when re-negotiating). If you want to write and read as data arrives in either direction, don't block, and enter the state machine to move data in either direction

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-24 Thread Darryl Miles
The issue is down to the OpenSSL API thread-safety rules (which are dictated to by the internal design of OpenSSL). I covered those thread-safety rules in a previous posting. Yes the common application design pattern for full-duplex SSL streams is to only ever have one thread doing the

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-24 Thread David Schwartz
Darryl Miles wrote: I do not believe the SSL_write() call is allowed to access the underlying BIO/kernel-socket to read in more data. I think SSL_write() is allowed to process any data already read into buffer (from kernel to OpenSSL library internal buffer) in an attempt to unstall the

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread David Schwartz
Darryl Miles wrote: But this flag (while documented to the contrary) does nothing inside libssl. So yes the documentation says you should set it, prove to me that OpenSSL behaves in a different way because you set it. One of the biggest downsides of open source software is that encourages

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread Darryl Miles
David Schwartz wrote: Darryl Miles wrote: But this flag (while documented to the contrary) does nothing inside libssl. So yes the documentation says you should set it, prove to me that OpenSSL behaves in a different way because you set it. One of the biggest downsides of open source

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread Darryl Miles
David Schwartz wrote: Darryl Miles wrote: This is how everything else works, it's odd to say it's somehow a limitation of OpenSSL that it works the same way everything else works. Try to read to a string in one thread while you write to it from another. The general rule of thread

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread Victor Duchovni
On Fri, Oct 23, 2009 at 03:47:51PM +0100, Darryl Miles wrote: I advocate that some users would find it useful to be able to invoke SSL_read() and SSL_write() from exactly two threads on the same 'SSL *' simultaneously. There is merit in this and as things stands OpenSSL does not allow it

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread Jason Pettiss
Now the next question you might want to ask, is it allowed for exactly two threads to operate specifically the SSL_read() and SSL_write() on the _SAME_ 'SSL *' instance at the same time ? My understanding would be that the answer is NO. This is a limitation in the OpenSSL

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread Victor Duchovni
On Fri, Oct 23, 2009 at 08:50:38AM -0700, Jason Pettiss wrote: However, it's clearly alright to read a socket from one thread while writing a socket from another: indeed, this is the purpose of a socket. That OpenSSL doesn't allow this usage seems like a limitation of the library. (Although

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread Jason Pettiss
I advocate that some users would find it useful to be able to invoke SSL_read() and SSL_write() from exactly two threads on the same 'SSL *' simultaneously.  There is merit in this and as things stands OpenSSL does not allow it due to a design choice (aka design limitation). You

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread Victor Duchovni
On Fri, Oct 23, 2009 at 09:15:35AM -0700, Jason Pettiss wrote: We could take turns sending discrete pieces of each file but that's silly and slow. Assuming we can load these gigantic files into memory to make the example simpler, we could both do this to write: It is possible to use

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread Jason Pettiss
It is possible to use non-blocking SSL_read() SSL_write() calls that are interleaved, but not without a mutex or a separate thread that owns all SSL I/O that consumes requests to read/write. It is simpler to use two SSL connections. SSL is a state-machine, not a pipe. Awesome the former

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread Victor Duchovni
On Fri, Oct 23, 2009 at 09:34:22AM -0700, Jason Pettiss wrote: It is possible to use non-blocking SSL_read() SSL_write() calls that are interleaved, but not without a mutex or a separate thread that owns all SSL I/O that consumes requests to read/write. It is simpler to use two SSL

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-23 Thread David Schwartz
Darryl Miles wrote: Kernel objects are the exception, only because we cannot allow a program (broken or valid) to screw up kernel objects. So the kernel has no choice but to overserialize. FYI modern kernel's do not need to serialize (let alone overserialize, whatever that means,

Is full-duplex socket use possible with OpenSSL?

2009-10-22 Thread Jason Pettiss
I have a server which reads/writes a socket independently; that is to say, at the same time (not a request-response model). I note in the FAQ it says I must not allow multiple threads to use an SSL connection, so clearly if my sockets are blocking I cannot support full-duplex traffic (because

RE: Is full-duplex socket use possible with OpenSSL?

2009-10-22 Thread David Schwartz
Jason Pettiss wrote: I have a server which reads/writes a socket independently; that is to say, at the same time (not a request-response model). I note in the FAQ it says I must not allow multiple threads to use an SSL connection, so clearly if my sockets are blocking I cannot support

Re: Is full-duplex socket use possible with OpenSSL?

2009-10-22 Thread Darryl Miles
David Schwartz wrote: Jason Pettiss wrote: I have a server which reads/writes a socket independently; that is to say, at the same time (not a request-response model). I note in the FAQ it says I must not allow multiple threads to use an SSL connection, so clearly if my sockets are blocking I