Re: [openssl-dev] [EXTERNAL] Re: PKCS12 safecontents bag type deviation from spec
I think the change is justified. — Regards, Uri > On Jan 16, 2018, at 14:31, Sands, Daniel wrote: > > On Tue, 2018-01-16 at 14:50 +, Salz, Rich via openssl-dev wrote: >> OpenSSL defines it as a SET OF and the spec says it’s a SEQUENCE >> OF. Ouch! Will that cause interop problems if we change it? (I >> don’t remember the DER encoding rules) >> >> >> > > Well, a SEQUENCE uses tag 16 while a SET uses tag 17, according to a > quick reference I found. So that could be an interoperability concern. > But maybe this is the first actual use of nested safecontents, since > this difference flew under the radar for so long :) > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [openssl/openssl] Dtls listen refactor (#5024)
On 16/01/18 19:44, Michael Richardson wrote: > > Matt Caswell wrote: > >> a) when the existing FD is connect(2) any future traffic to the bound > >> port will get rejected with no port. So the application really has to > >> open a new socket first. The application can do this two ways: it can > >> open a new socket on which to receive new connections, or it can open > >> a new socket on which to communicate with the new client. The second > >> method is better for reason (b) below. Either way, it socket to > >> communicate with the client needs to be bind(2) to the address that > >> the client used to communicate with the server, and DTLSv1_listen() > >> didn't collect or return that information. > > > The second way is what is intended. > > Unfortunately, there remains a race condition because we have to call bind() > before connect() on the new socket. Under load, if a packet is received > between the bind() and the connect(), it might go onto the wrong socket > queue. So some packets that could have been processed will get dropped and > have to be retransmitted by the client. This seems like a non-issue to me. At this point in the handshake the client will have sent its ClientHello and won't progress until it gets the server's flight back (ServerHello etc), i.e. in the vast majority of cases it won't be sending anything. It is possible that the client may retransmit the ClientHello if the server hasn't responded within a timely manner. Retransmit times are usually quite a while (relatively speaking) after the original transmission, so the chances of this happening immediately after we've processed the original ClientHello and before we've called connect() seem quite small - although possible. If a retransmitted ClientHello arrives *after* the connect() it will be dropped anyway by OpenSSL so we really don't care about these messages going missing. Another possibility is that the client transmits an alert of some form. This also seems quite unlikely (i.e. send a ClientHello and then immediately send an alert before the server has had a chance to respond) - but again, its possible. It would be bad luck indeed for this unlikely scenario to happen and then for the alert to not make it onto the right fd due to the race: but if it did its not a big deal. This connection is doomed in any case (an alert was sent) and we have to be prepared to deal with packets getting dropped anyway (this is DTLS!). It is very common for clients to just abort without sending an alert anyway - so this will just appear like that. > > It could be solved if there was a way to punt packets received on the wrong > socket to the correct BIO on the input side. I looked into this, but decided > it was too difficult... > > That would also let one operate a multitude of DTLS connections using single > socket which might be a boon to some users. Mis-designed it would scale > badly on multi-threaded machines and involve lots of ugly locks. > I don't want to consider the impacts if one had to pass packets between > processes... > I don't advocate a solution like this (I'll live with the dropped packets), > but I think it's worth making people aware that they might see mis-directed > packets get dropped. > > > Maybe I misunderstand your point - > > but the client address *is* returned? Admittedly its wrapped in a > > BIO_ADDR, but its easy to get the underlying "raw" address using > > BIO_ADDR_rawaddress(): > > > Why isn't recvfrom() suitable (which is what the code currently uses to > > get the address)? > > The address of the remote client is returned ("getpeername()") by > DTLSv1_listen(). > That's all that recvfrom() gives you. > > recvfrom() was a reasonable API for SunOS 3.x machines with a single 10Mb/s > interface with a single IPv4 address. I loved all that at the time... > But it doesn't work that well when we might have a dozen different kind of > IPv6 addresses on each virtual interface. > > The address that I'm talking about needing is the one the remote client used > to reach us. That's the destination IP of the incoming packet > ("getsockname()" in TCP speak). Ahhhits the *server's* address you are after. This requirement seems more reasonable. I think the API is designed to expect you to only bind to a single IP. I'd be interested in Richard Levitte's opinion on this. It seems like a fairly simple solution could solve this. Currently we have BIO_dgram_get_peer() which returns the peer's address for the last message read from a BIO. You could imagine a new call being introduced to get our own address. You could then call that immediately after a successful DTLSv1_listen() call. Obviously we'd have to change the dgram BIO to use recvmsg for this to work. Matt -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [openssl/openssl] Dtls listen refactor (#5024)
Matt Caswell wrote: >> a) when the existing FD is connect(2) any future traffic to the bound >> port will get rejected with no port. So the application really has to >> open a new socket first. The application can do this two ways: it can >> open a new socket on which to receive new connections, or it can open >> a new socket on which to communicate with the new client. The second >> method is better for reason (b) below. Either way, it socket to >> communicate with the client needs to be bind(2) to the address that >> the client used to communicate with the server, and DTLSv1_listen() >> didn't collect or return that information. > The second way is what is intended. Unfortunately, there remains a race condition because we have to call bind() before connect() on the new socket. Under load, if a packet is received between the bind() and the connect(), it might go onto the wrong socket queue. So some packets that could have been processed will get dropped and have to be retransmitted by the client. It could be solved if there was a way to punt packets received on the wrong socket to the correct BIO on the input side. I looked into this, but decided it was too difficult... That would also let one operate a multitude of DTLS connections using single socket which might be a boon to some users. Mis-designed it would scale badly on multi-threaded machines and involve lots of ugly locks. I don't want to consider the impacts if one had to pass packets between processes... I don't advocate a solution like this (I'll live with the dropped packets), but I think it's worth making people aware that they might see mis-directed packets get dropped. > Maybe I misunderstand your point - > but the client address *is* returned? Admittedly its wrapped in a > BIO_ADDR, but its easy to get the underlying "raw" address using > BIO_ADDR_rawaddress(): > Why isn't recvfrom() suitable (which is what the code currently uses to > get the address)? The address of the remote client is returned ("getpeername()") by DTLSv1_listen(). That's all that recvfrom() gives you. recvfrom() was a reasonable API for SunOS 3.x machines with a single 10Mb/s interface with a single IPv4 address. I loved all that at the time... But it doesn't work that well when we might have a dozen different kind of IPv6 addresses on each virtual interface. The address that I'm talking about needing is the one the remote client used to reach us. That's the destination IP of the incoming packet ("getsockname()" in TCP speak). With TCP this is never an issue because the kernel creates the new socket and copies the right stuff in for us when it creates the socket. With UDP, the source address for outgoing packets needs to match or the client may get a response from an address that it didn't connect to. Worse, there might be firewalls or policy routing that would cause the packet to disappear or get routed differently. In my application, I definitely dealing with connections over IPv6 Link-Local addresses with a multitude of virtual links. In your code example: bind(client_fd, &server_addr, sizeof(struct sockaddr_in6)); server_addr has to be set from the destination address of the incoming packet, it's not a global that the admin set, or the SIP negotiated. In the bad old days, this meant opening a socket for every interface on the machine, and re-reading the list of interfaces based upon some heuristic. (routing socket, SIGHUP, ...) Even getting a list of interfaces (and their addresses) is itself a OS-dependant activity! And, if you use the old BSD interface on Linux, you'll miss a bunch of interfaces, because the Linux kernel people thought that it would confuse BSD APIs if interfaces were returned that the BSD interface didn't create. So you can't even win there. The IPv6 API gives us recvmsg() and ipi6_pktinfo, which makes it all sane. But, we never got a standard interface for IPv4: Linux uses something that looks identical to IPv6, and BSD has something with slightly different names. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works| network architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ signature.asc Description: PGP signature -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [EXTERNAL] Re: PKCS12 safecontents bag type deviation from spec
On Tue, 2018-01-16 at 14:50 +, Salz, Rich via openssl-dev wrote: > OpenSSL defines it as a SET OF and the spec says it’s a SEQUENCE > OF. Ouch! Will that cause interop problems if we change it? (I > don’t remember the DER encoding rules) > > > Well, a SEQUENCE uses tag 16 while a SET uses tag 17, according to a quick reference I found. So that could be an interoperability concern. But maybe this is the first actual use of nested safecontents, since this difference flew under the radar for so long :) -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [openssl/openssl] Dtls listen refactor (#5024)
On 16/01/18 15:32, Michael Richardson wrote: > > a) when the existing FD is connect(2) any future traffic to the bound port >will get rejected with no port. So the application really has to open a >new socket first. >The application can do this two ways: it can open a new socket on which to > receive >new connections, or it can open a new socket on which to communicate with >the new client.The second method is better for reason (b) below. >Either way, it socket to communicate with the client needs to be bind(2) >to the address that the client used to communicate with the server, and >DTLSv1_listen() didn't collect or return that information. The second way is what is intended. Maybe I misunderstand your point - but the client address *is* returned? Admittedly its wrapped in a BIO_ADDR, but its easy to get the underlying "raw" address using BIO_ADDR_rawaddress(): https://www.openssl.org/docs/man1.1.0/crypto/BIO_ADDR_rawaddress.html i.e. call BIO_ADDR_rawaddress() on the BIO_ADDR returned by DTLSv1_listen() and then do something like this: /* Handle client connection */ int client_fd = socket(AF_INET6, SOCK_DGRAM, 0); bind(client_fd, &server_addr, sizeof(struct sockaddr_in6)); connect(client_fd, &client_addr, sizeof(struct sockaddr_in6)); /* Set new fd and set BIO to connected */ BIO *cbio = SSL_get_rbio(ssl); BIO_set_fd(cbio, client_fd, BIO_NOCLOSE); BIO_ctrl(cbio, BIO_CTRL_DGRAM_SET_CONNECTED, 0, &client_addr); /* Finish handshake */ SSL_accept(ssl); > > b) the existing FD might have additional packets from other clients. This >argues for opening a new socket for the new client, and leaving the queue >on the existing FD. Which is what I recommend. > I absolutely need to have recvmsg()/sendmsg() in the bss_dgram.c in order to > get the destination address used. This IPv6 code is portable, since the RFC > API says how to do it. Why isn't recvfrom() suitable (which is what the code currently uses to get the address)? Matt -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [openssl/openssl] Dtls listen refactor (#5024)
please see https://github.com/openssl/openssl/pull/5024 mattcaswell asks on github: mattcaswell> I am unclear about the underlying premise of this PR: mcr> This patch refactors the DTLSv1_listen() to create an mcr> alternative API that is called DTLSv1_accept(). mcr> DTLSv1_accept() is useable by programs that need to treat mcr> DTLS interfaces in a way similar to TCP: new connections mcr> must be accepted, and new sockets created. mattcaswell> Your going to give more justification than this. Why is it mattcaswell> that DTLSv1_listen() is not appropriate for your use case? As I understand using DTLSv1_listen(), one does the following: 1) open a UDP socket, bind(2) it appropriately. {in an RTP context, one might already know the remote port numbers, and one could connect(2) it already. In the CoAP case, that certainly is not the case} Put the socket into an SSL, do appropriate blocking or non-blocking event handling in application. 2) call DTLSv1_listen() when there is traffic. DTLSv1_listen() will process (via peek) the first packet in the socket. If it's a Client Hello without a cookie, then a Hello Verify is sent back (%). If it ate the packet, then it loops until it find something it can't handle or runs out of packets. 3) DTLSv1_listen(), when it finds a Client Hello with a cookie, marks the provided SSL as having transitioned to a state where things can start, and it returns 1, along with the BIO_ADDR of the newly Verify Hello'ed client. 4) the application is now expected to connect() the FD to the BIO_ADDR, and call SSL_accept(), and then to proceed with SSL_read()/SSL_write(), etc. Perhaps I've gotten something wrong with this process. This flow is entirely appropriate for a RTP user, but for a CoAP server there are a number of problems: a) when the existing FD is connect(2) any future traffic to the bound port will get rejected with no port. So the application really has to open a new socket first. The application can do this two ways: it can open a new socket on which to receive new connections, or it can open a new socket on which to communicate with the new client.The second method is better for reason (b) below. Either way, it socket to communicate with the client needs to be bind(2) to the address that the client used to communicate with the server, and DTLSv1_listen() didn't collect or return that information. b) the existing FD might have additional packets from other clients. This argues for opening a new socket for the new client, and leaving the queue on the existing FD. c) the DTLSv1_listen() uses the SSL* (and associated CTX) that is provided to it for callbacks, and cookie verification. It modifies the state of that SSL* to continue the transaction. I think that the roles should be split up. also, from point (2) above: (%) - the send that DTLSv1_listen() depends upon the socket having been bind(2) with a non-INADDR_ANY/IN6ADDR_ANY_INIT IP address, or that the system in question has only a single IP address. This is because the write that is done relies upon the kernel to pick the right source address, which appears to be easy for IPv4 with a single interface, but trivially can fail for IPv6 even with a single interface due to temporary, stable-private, and link-local addresses. DTLSv1_accept() takes two SSL*. The first is used for cookie verification, while the second is filled in with a new FD that has been bind/connect to the client and state advanced to be ready for SSL_accept(). It also returns the same BIO_ADDR for the client, but that could be removed as it can trivially be retrieved from the new SSL*. mattcaswell> In any case the PR as it currently stands is a very long way mattcaswell> off being acceptable: I totally agree, but I had to post something to start the conversation. mattcaswell> * As you point out the use of the POSIX socket APIs is mattcaswell> unacceptable and is at the wrong level of abstraction. I mattcaswell> might perhaps expect to see this sort of thing in the BIO mattcaswell> layer. a) I could move the socket creation code into BIO layer, a new BIO_ctrl method could be created to "duplicate" the BIO. This would probably eliminate having to expose BIO_ADDR_sockaddr{,_size} from libcrypto->libssl. b) creation of a new socket could be a new callback. c) DTLSv1_accept() could return at: "now set up a socket based upon the original rbio's peer/addr" as all of the subsequent operations could be done by the application given BIO_dgram_get_peer(rbio, client) and BIO_dgram_get_addr(rbio, ouraddr) d) a combination of (a) and (c), where the duplication code is provided by the BIO layer, but the application could do something different if it needed to. My preference is for (d), because I think that it's common code and the
Re: [openssl-dev] PKCS12 safecontents bag type deviation from spec
OpenSSL defines it as a SET OF and the spec says it’s a SEQUENCE OF. Ouch! Will that cause interop problems if we change it? (I don’t remember the DER encoding rules) -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev