[openssl.org #2573] [PATCH] DTLS buffering and decryption bug

[Robin Seggelmann via RT]

DTLS buffers records of the next epoch while listening, although it's not 
supposed to change its state. Additionally, when the decryption of buffered 
records fails, the connection is dropped with a BadRecord alert, instead of 
just silently discarding the message. With this patch nothing is buffered while 
listening and messages that cannot be decrypted will be ignored.

Thanks to Yogesh Chopra for providing hints!

Best regards
Robin



--- ssl/d1_pkt.c        4 Jan 2011 19:33:22 -0000       1.27.2.26
+++ ssl/d1_pkt.c        28 Jul 2011 10:48:48 -0000
@@ -409,13 +409,13 @@
        enc_err = s->method->ssl3_enc->enc(s,0);
        if (enc_err <= 0)
                {
-               if (enc_err == 0)
-                       /* SSLerr() and ssl3_send_alert() have been called */
-                       goto err;
-
-               /* otherwise enc_err == -1 */
-               al=SSL_AD_BAD_RECORD_MAC;
-               goto f_err;
+               /* decryption failed, silently discard message */
+               if (enc_err < 0)
+                       {
+                       rr->length = 0;
+                       s->packet_length = 0;
+                       }
+               goto err;
                }
 
 #ifdef TLS_DEBUG
@@ -658,10 +658,12 @@
 
        /* If this record is from the next epoch (either HM or ALERT),
         * and a handshake is currently in progress, buffer it since it
-        * cannot be processed at this time. */
+        * cannot be processed at this time. However, do not buffer
+        * anything while listening.
+        */
        if (is_next_epoch)
                {
-               if (SSL_in_init(s) || s->in_handshake)
+               if ((SSL_in_init(s) || s->in_handshake) && !s->d1->listen)
                        {
                        dtls1_buffer_record(s, &(s->d1->unprocessed_rcds), 
rr->seq_num);
                        }

dtls-decryption-buffer-bug-0.9.8.patch
Description: Binary data

dtls-decryption-buffer-bug-1.0.0.patch
Description: Binary data

dtls-decryption-buffer-bug-1.0.1.patch
Description: Binary data