This problem has been observed on multiple platforms (Windows, Linux)
using openssl-0.9.8f.
When sending application data of size larger than current MTU, the
function dtls1_write_app_data_bytes attempts to fragment the data into
MTU-sized chunks. This is of questionable utility, but results in very
bad behavior due to incorrect calculation of remaining data to send.
The return value of SSL_write() is also wrong if
SSL_MODE_ENABLE_PARTIAL_WRITE has been selected.
The root of the problem is that dtls1_write_bytes returns the total
number of bytes in the buffer that have been sent so far (including
previous fragments), but the return value is used as the number of bytes
newly sent. Following the values n, tot, and i through the loop:
(without ENABLE_PARTIAL_WRITE)
init:
n = 4000 bytes to send; mtu = 1472
after 1 send:
i = 1472; tot = 1472; n = 4000-1472 = 2528
after 2 sends:
i = 2944; tot = 1472 + 2944 = 4416; n = 2528 - 2944 = -416 (actually,
4294966880 because n is unsigned)
This will take some time to terminate for the loop condition "while (n)
{ ... }". The attached patch is one possible way to address the problem.
--
- Ariel Salomon / Senior Software Engineer
Real-Time Innovations (RTI) / www.rti.com
408 200-4739 / [EMAIL PROTECTED]
RTI - The Real-Time Middleware Experts
Index: ssl/d1_pkt.c
===================================================================
RCS file: /home/local/cvsroot/usermodules/ariel/openssl/ssl/d1_pkt.c,v
retrieving revision 1.4.2.9
diff -u -r1.4.2.9 d1_pkt.c
--- ssl/d1_pkt.c 3 Oct 2007 10:18:06 -0000 1.4.2.9
+++ ssl/d1_pkt.c 13 Nov 2007 08:42:17 -0000
@@ -1156,6 +1156,7 @@
i = dtls1_write_bytes(s, type, buf_, len);
if (i <= 0) return i;
+ i -= tot; /* dtls1_write_bytes returns _total_ length; want
just delta */
if ((i == (int)n) ||
(type == SSL3_RT_APPLICATION_DATA &&
(s->mode & SSL_MODE_ENABLE_PARTIAL_WRITE)))
