Re: DTLS server implementation experiences and documentation
On Mon, 26 Jan 2009 18:19:02 +0100, Daniel Mentz danie...@sent.com said: DM I have some comments regarding your wiki article. But first of all DM thanks for taking the time writing down all this information: Your welcome, and I'm sorry for taking so long to reply to your note... I was both sick for a week and out of town for a week and am just getting caught up again. (btw, I suspect if you followed the conversation that Robin's patches [1] to fix OpenSSL's DTLS handling are potentially much better than my hackary.) DM I do not use a memory BIO for sending data. I create a datagram BIO DM instead and let OpenSSL write to this datagram BIO directly. Yep, you're right that this would work as well. Outgoing wise, it's not an issue like it is incoming-wise. DM When it comes to receiving data from the UDP socket I create a new DM memory BIO for *each* packet I received via recvfrom() and pass that DM memory BIO to OpenSSL. Hmm... ok, though I'm not sure why you can't reuse an existing BIO unless you're battling memory issues (which I suspect is the case). Either way, thanks for your notes on your implementation. DM I've got a question regarding your solution for sending data: How do DM ensure that the message boundaries are preserved? ... DM What happens if the SSL_* function wants to send more than one UDP DM datagram at once? I *guess* this could happen if someone wants to send DM a very large certificate (chain). To me it seems like that you're DM assuming that OpenSSL sends only a single packet during one invocation DM of SSL_*, aren't you? If OpenSSL happens to send two packets you're DM going to concatenate the payload and send out one large datagram DM instead of two smaller ones. You're right... It technically should never happen according to the DTLS specs, *assuming* that the implementation isn't trying to send a second packet before waiting for the response to the first. The DTLS RFC requires that all DTLS messages must fit in a single UDP datagram and I doubt the internal implementation is sending multiple messages at once. But you're right, if it does try to I'm stuck as is and will fire both off in a single packet. On the bright side though, I suspect that if the other side is simply reading a bunch of data from the UDP socket and is openssl based it too will not care because the message boundaries on the incoming side are also ignored. This only works if both sides are equally broken and ignoring the UDP framing :-/ DM As regards DTLS Cookie handling I suggest to ignore the information DM hiding/data abstraction principle for now and to access OpenSSL's DM internal variables in order to find out in which state the OpenSSL DM state machine is right now. I guess that there's some way to find out DM whether OpenSSL sent out a HelloVerifyRequest or a ServerHello. If it DM just sent a HelloVerifyRequest we could just destroy the SSL object DM and wait for the client to send back the Cookie. Yep. I've thought of that as well. sigh... but the right solution would be to fix the OpenSSL implementation. Footnotes: [1] http://www.net-snmp.org/wiki/index.php/DTLS_Implementation_Notes#Writing_a_new_UDP_BIO -- In the bathtub of history the truth is harder to hold than the soap, and much more difficult to find. -- Terry Pratchett __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
On Thu, 22 Jan 2009 06:10:36 +0100, Robin Seggelmann seggelm...@fh-muenster.de said: RS To avoid getting into trouble with already fixed bugs you should apply RS the patches I sent to the dev list. I'll set up a website with a patch RS collection and some instructions soon. You've certainly posted a lot! I've found a few and documented them at [1] but I'm sure the list isn't complete. A web page, when you get the chance, describing your work would be very helpful! Footnotes: [1] http://www.net-snmp.org/wiki/index.php/DTLS_Implementation_Notes#Writing_a_new_UDP_BIO -- In the bathtub of history the truth is harder to hold than the soap, and much more difficult to find. -- Terry Pratchett __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
On Jan 26, 2009, at 5:24 PM, Daniel Mentz wrote:
I'm surprised that you can use accept() on UDP sockets. I checked
the man pages of a Debian GNU/Linux system. They say that you can
use accept() only with connection-based socket types (SOCK_STREAM,
SOCK_SEQPACKET). Is this something specific to FreeBSD? Could you
please provide me with more information. Maybe you could make your
source code available.
Hi Daniel,
sorry for causing confusion, of course you can't use accept() with
UDP. You have to create a new socket, use bind() to assign it to a
specific connection and create another socket. Everything else is like
handling TCP connections. Unfortunately it was too long ago that I
have written the code so I hadn't had all of it in mind anymore.
Probably you can use connected UDP sockets with Linux, but I'm not
sure. I'm working with FreeBSD and Mac OS X which support them.
Sample UDP server code:
while (1) {
memset((void *) client_addr, 0, sizeof(client_addr));
accfd = (int*) malloc (sizeof(int));
*accfd = socket(AF_INET, SOCK_DGRAM, 0);
setsockopt (*accfd, SOL_SOCKET, SO_REUSEADDR, optval, sizeof(optval));
bind(*accfd, (const struct sockaddr *) server_addr,
sizeof(server_addr));
bio = BIO_new_dgram(*accfd, BIO_NOCLOSE);
ssl = SSL_new(ctx);
SSL_set_bio(ssl, bio, bio);
SSL_accept(ssl);
info = (struct pass_info*) malloc (sizeof(struct pass_info));
info-fd = *accfd;
info-client_addr = client_addr;
info-ssl = ssl;
rc = pthread_create( tid, NULL, connection_handle, info);
if (rc != 0) {
perror(pthread_create);
exit(1);
}
}
Regards,
Robin
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
Robin Seggelmann wrote: As a workaround you can use connected UDP sockets. Just use accept() and connect() as you would with TCP connections and create new BIO and SSL objects for every connection. I have tested that and it works pretty well so far. Hi Robin, I'm surprised that you can use accept() on UDP sockets. I checked the man pages of a Debian GNU/Linux system. They say that you can use accept() only with connection-based socket types (SOCK_STREAM, SOCK_SEQPACKET). Is this something specific to FreeBSD? Could you please provide me with more information. Maybe you could make your source code available. Thanks Daniel __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
Wes Hardaker wrote: http://www.net-snmp.org/wiki/index.php/DTLS_Implementation_Notes Hi Wes, I have some comments regarding your wiki article. But first of all thanks for taking the time writing down all this information: I'm trying to implement IPFIX on top of DTLS so I also made some experiences with DTLS and OpenSSL. I do not use a memory BIO for sending data. I create a datagram BIO instead and let OpenSSL write to this datagram BIO directly. When it comes to receiving data from the UDP socket I create a new memory BIO for *each* packet I received via recvfrom() and pass that memory BIO to OpenSSL. Here's some code: len = read(socket,recvbuf,sizeof(recvbuf)); /* Free existing BIO */ /* TODO: Check whether EOF reached. */ BIO_free(ssl-rbio); /* Create new BIO */ ssl-rbio = BIO_new_mem_buf(recvbuf,len); BIO_set_mem_eof_return(ssl-rbio,-1); I've got a question regarding your solution for sending data: How do ensure that the message boundaries are preserved? What you are doing is basically: 1. Call some SSL_* function like SSL_write, SSL_connect or SSL_accept. 2. Perform a BIO_read on the for_writing BIO 3. Use sendto to send data just read in step 2. What happens if the SSL_* function wants to send more than one UDP datagram at once? I *guess* this could happen if someone wants to send a very large certificate (chain). To me it seems like that you're assuming that OpenSSL sends only a single packet during one invocation of SSL_*, aren't you? If OpenSSL happens to send two packets you're going to concatenate the payload and send out one large datagram instead of two smaller ones. As regards DTLS Cookie handling I suggest to ignore the information hiding/data abstraction principle for now and to access OpenSSL's internal variables in order to find out in which state the OpenSSL state machine is right now. I guess that there's some way to find out whether OpenSSL sent out a HelloVerifyRequest or a ServerHello. If it just sent a HelloVerifyRequest we could just destroy the SSL object and wait for the client to send back the Cookie. Daniel __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
On Sat, 2009-01-24 at 00:13 +0100, Georges Le grand wrote: I wonder if you could give out a reference on how to establish a VPN using DTLS or to tell how to do so. We are just using Cisco's AnyConnect VPN, which runs over an HTTPS 'CONNECT' and will use DTLS for subsequent data transfer if it can. The client code is at git://git.infradead.org/users/dwmw2/openconnect.git (viewable in gitweb by changing git:// to http:// in that URL). That code works on Linux and MacOS, and if anyone wants to provide a patch to make it work on other BSD systems that would be much appreciated. Since Cisco use an old version of OpenSSL on the server side, you'll need to patch OpenSSL to make it compatible with its own pre-RFC version of DTLS -- see http://rt.openssl.org/Ticket/Display.html?id=1751 for the patch. The VPN will work over HTTPS if you don't patch OpenSSL, but VPN over TCP is a very suboptimal solution. I haven't done server-side code yet; the point of this was to interoperate with the existing servers, and I have no immediate need to _replace_ them. It really wouldn't be hard though -- it's all fairly trivial stuff. You might also be interested in http://campagnol.sourceforge.net/ -- dwmw2 __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
Dear David, Thanks for the reply. So it is alike SSL VPN with data encapsulated into HTTP Packets, but I don't get how does HTTP run over UDP. Kind regards GLG 2009/1/24 David Woodhouse dw...@infradead.org On Sat, 2009-01-24 at 00:13 +0100, Georges Le grand wrote: I wonder if you could give out a reference on how to establish a VPN using DTLS or to tell how to do so. We are just using Cisco's AnyConnect VPN, which runs over an HTTPS 'CONNECT' and will use DTLS for subsequent data transfer if it can. The client code is at git://git.infradead.org/users/dwmw2/openconnect.git (viewable in gitweb by changing git:// to http:// in that URL). That code works on Linux and MacOS, and if anyone wants to provide a patch to make it work on other BSD systems that would be much appreciated. Since Cisco use an old version of OpenSSL on the server side, you'll need to patch OpenSSL to make it compatible with its own pre-RFC version of DTLS -- see http://rt.openssl.org/Ticket/Display.html?id=1751 for the patch. The VPN will work over HTTPS if you don't patch OpenSSL, but VPN over TCP is a very suboptimal solution. I haven't done server-side code yet; the point of this was to interoperate with the existing servers, and I have no immediate need to _replace_ them. It really wouldn't be hard though -- it's all fairly trivial stuff. You might also be interested in http://campagnol.sourceforge.net/ -- dwmw2
Re: DTLS server implementation experiences and documentation
On Sat, 2009-01-24 at 23:03 +0100, Georges Le grand wrote: So it is alike SSL VPN with data encapsulated into HTTP Packets, but I don't get how does HTTP run over UDP. Probably best explained by the code... it just uses HTTP for the initial setup -- a CONNECT request with an HTTP cookie for authentication, and you get IP address etc. in the headers of the response. Then you're connected with an SSL connection, you can forget HTTP, and run IP packets over that connection. In the headers of the initial exchange you _also_ set up parameters for a DTLS connection, over which you can pass packets. -- dwmw2 __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
On Thu, 22 Jan 2009 06:10:36 +0100, Robin Seggelmann seggelm...@fh-muenster.de said: RS As a workaround you can use connected UDP sockets. Just use accept() RS and connect() as you would with TCP connections and create new BIO and RS SSL objects for every connection. I have tested that and it works RS pretty well so far. And that prevents OpenSSL from reading too much data from the socket? If so, that's certainly a good thing. -- In the bathtub of history the truth is harder to hold than the soap, and much more difficult to find. -- Terry Pratchett __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
Hello David, I wonder if you could give out a reference on how to establish a VPN using DTLS or to tell how to do so. Kind regards, GLG On Thu, Jan 22, 2009 at 7:47 AM, David Woodhouse dw...@infradead.org wrote: On Thu, 2009-01-22 at 06:10 +0100, Robin Seggelmann wrote: To avoid getting into trouble with already fixed bugs you should apply the patches I sent to the dev list. I'll set up a website with a patch collection and some instructions soon. Is there anyone who actually cares about DTLS and getting patches applied? I've had patches to make OpenSSL capable of talking to production servers out there in the wild, which use the OpenSSL-specific pre-RFC version of DTLS and I've been able to write a complete VPN client along with NetworkManager support, and get it into Linux distributions, in the time it's taken to get the patch into OpenSSL... and I'm still waiting... It's getting to the point where I wonder if it would be quicker and easier just to reimplement DTLS in GNUTLS and use that.
Re: DTLS server implementation experiences and documentation
On Thu, 22 Jan 2009 06:10:36 +0100, Robin Seggelmann seggelm...@fh-muenster.de said: RS As a workaround you can use connected UDP sockets. Just use accept() RS and connect() as you would with TCP connections and create new BIO and RS SSL objects for every connection. I have tested that and it works RS pretty well so far. And that keeps OpenSSL from reading too much data from a single socket? If so, that'd certainly be a good thing. -- In the bathtub of history the truth is harder to hold than the soap, and much more difficult to find. -- Terry Pratchett __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
On Jan 22, 2009, at 5:11 PM, Wes Hardaker wrote: RS As a workaround you can use connected UDP sockets. Just use accept() RS and connect() as you would with TCP connections and create new BIO and RS SSL objects for every connection. I have tested that and it works RS pretty well so far. And that keeps OpenSSL from reading too much data from a single socket? If so, that'd certainly be a good thing. If you meant keeps reading data from more than one connection, then yes. With connected UDP sockets the kernel handles the differentiation between connections and distributes incoming data to the corresponding sockets. Regards, Robin __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
DTLS server implementation experiences and documentation
I recently tried playing with the DTLS code within OpenSSL, but it contains multiple problems. I know from reading the archives that other people have had issues trying to figure out the details of what is needed to use the DTLS code. So, since I actually got things working (albeit in an interesting way) I thought I'd actually write up the results so that others might benefit. The biggest issue comes from needing to deal with multiple clients trying to talk through the same UDP port, which isn't handled by at least the 0.9.8i code at least. I'd love feedback on some of my conclusions. I'm not in any way an OpenSSL exert and dove further into the code this time than I have in the past. But I'm not swimming in the deep end yet. The write up is written on the Net-SNMP wiki, since that's what I was working on it for. It's written fairly generically so you don't need to understand SNMP (and people who know SNMP don't need to know a huge amount about OpenSSL). IE, There is a fair amount of introductory text. http://www.net-snmp.org/wiki/index.php/DTLS_Implementation_Notes -- In the bathtub of history the truth is harder to hold than the soap, and much more difficult to find. -- Terry Pratchett __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
Hi Wes, The biggest issue comes from needing to deal with multiple clients trying to talk through the same UDP port, which isn't handled by at least the 0.9.8i code at least. Using multiple connections with one socket is still not implemented as the OpenSSL architecture and API does not support this. To be able to choose which connection within a socket you want to send to or receive from, major changes would be necessary, starting from SSL_read, SSL_write, etc. and ending with the relation between BIO and SSL objects. As a workaround you can use connected UDP sockets. Just use accept() and connect() as you would with TCP connections and create new BIO and SSL objects for every connection. I have tested that and it works pretty well so far. I'd love feedback on some of my conclusions. I'm not in any way an OpenSSL exert and dove further into the code this time than I have in the past. But I'm not swimming in the deep end yet. The write up is written on the Net-SNMP wiki, since that's what I was working on it for. It's written fairly generically so you don't need to understand SNMP (and people who know SNMP don't need to know a huge amount about OpenSSL). IE, There is a fair amount of introductory text. http://www.net-snmp.org/wiki/index.php/DTLS_Implementation_Notes To avoid getting into trouble with already fixed bugs you should apply the patches I sent to the dev list. I'll set up a website with a patch collection and some instructions soon. Regards, Robin __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
On Thu, 2009-01-22 at 06:10 +0100, Robin Seggelmann wrote: To avoid getting into trouble with already fixed bugs you should apply the patches I sent to the dev list. I'll set up a website with a patch collection and some instructions soon. Is there anyone who actually cares about DTLS and getting patches applied? I've had patches to make OpenSSL capable of talking to production servers out there in the wild, which use the OpenSSL-specific pre-RFC version of DTLS and I've been able to write a complete VPN client along with NetworkManager support, and get it into Linux distributions, in the time it's taken to get the patch into OpenSSL... and I'm still waiting... It's getting to the point where I wonder if it would be quicker and easier just to reimplement DTLS in GNUTLS and use that. -- dwmw2 __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: DTLS server implementation experiences and documentation
On Jan 22, 2009, at 7:47 AM, David Woodhouse wrote: Is there anyone who actually cares about DTLS and getting patches applied? Very good question. I hope someone will apply at least the bugfixes soon. I have several patches adding new features in the pipe, but they understandably rely on the bugfixes. However, I'll probably publish a list of patches on a website with an explanation which of them have to be applied in which order for a feature. Not very satisfying, but that seems to be the only solution by now. I've had patches to make OpenSSL capable of talking to production servers out there in the wild, which use the OpenSSL-specific pre-RFC version of DTLS and I've been able to write a complete VPN client along with NetworkManager support, and get it into Linux distributions, in the time it's taken to get the patch into OpenSSL... and I'm still waiting... It's getting to the point where I wonder if it would be quicker and easier just to reimplement DTLS in GNUTLS and use that. Ya, I also thought about that already, but was afraid of starting from scratch yet. Regards, Robin __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
