DES_CBC_CKSUM in SSL and Kerberos.
Title: DES_CBC_CKSUM in SSL and Kerberos. Hi, I have a customer with a Kerberos V4 application who is trying to decide if they can substitute their existing Kerberos V4 DES encryption capability with SSL's DES encryption support. When calling DES_CBC_CKSUM() from the Kerberos library, the checksum output buffer and the function's return value read : OUTBUF= 1234ABCD... // char *, 8 Byte checksum RetVal = ABCD // unsigned long, Low order(right half) 4 bytes of 8byte checksum When calling SSL's DES_CBC_CKSUM(), the values read: OUTBUF= 1234ABCD... // char *, 8 Byte checksum RetVal = DCBA // unsigned long, Low order(right half) 4 bytes of 8byte checksum SSL's RetVal produces a checksum error in the KerberosV4 application on return because it is in the oposite byte order than what Kerberos expects. The Kerberos and SSL macros used to initialize the return value from the OUTBUF follow respectively: Kerberos: #define GET_HALF_BLOCK(lr, ip) \ (lr) = ((unsigned int)(*(ip)++)) 24; \ (lr) |= ((unsigned int)(*(ip)++)) 16; \ (lr) |= ((unsigned int)(*(ip)++)) 8; \ (lr) |= (unsigned int)(*(ip)++) SSL: #define c2l(c,l) (l =((DES_LONG)(*((c)++))) , \ l|=((DES_LONG)(*((c)++))) 8L, \ l|=((DES_LONG)(*((c)++)))16L, \ l|=((DES_LONG)(*((c)++)))24L) We do not understand why SSL is swapping the bytes when initializing the output longword return value with the c2l macro in the DES_CBC_CKSUM() function. I have a test program that shows the differences. As you can see from the output below, the output is reversed between ssl and krb. $ run test_cbc_cksum set key 0 ssl 0x738af841, 0x80 0x16 0xd6 0x6b 0x41 0xf8 0x8a 0x73 krb 0x41f88a7 I then added a new macro, ksg_c2l, and had it do the same order of shifting as the get_half_block. They output is now in the correct order. $ run test_cbc_cksum2 set key 0 ssl 0x738af841, 0x80 0x16 0xd6 0x6b 0x41 0xf8 0x8a 0x73 krb 0x41f88a73 c2l 0x738af841 ksg_c2l 0x41f88a73 I have included the program with this mail message, and here are the environmental details we were operating in: OpenSSL 0.9.6b with Security patches. OpenVMS Alpha V7.2 TCP/IP V5.0A DEC C V6.2 Kerberos V4 and 5 Thanks, Kevin Kevin Greaney SSL for OpenVMS Team Hewlett Packard Company OpenVMS Engineering Group 110 Spitbrook Road Nashua, NH 03062 (603) 884-5099 test_cbc_cksum.c test_cbc_cksum.c Description: test_cbc_cksum.c
Re: DES_CBC_CKSUM in SSL and Kerberos.
The answer is: MIT DES and OpenSSL DES use different internal representations of the data. You cannot replace the MIT DES with OpenSSL DES unless you also recompile MIT Kerberos 4 to use the OpenSSL DES as well. Several people have done it in the past but it is not recommended. OpenSSL has gone to great lengths to clean the namespace so that each implementation can be used within the same application. Hi, I have a customer with a Kerberos V4 application who is trying to decide if they can substitute their existing Kerberos V4 DES encryption capability with SSL's DES encryption support. When calling DES_CBC_CKSUM() from the Kerberos library, the checksum output buffer and the function's return value read : OUTBUF=3D 1234ABCD... // char *, 8 Byte checksum RetVal =3D ABCD // unsigned long, Low order(right half) 4 bytes of 8byte checksum When calling SSL's DES_CBC_CKSUM(), the values read: OUTBUF=3D 1234ABCD... // char *, 8 Byte checksum RetVal =3D DCBA // unsigned long, Low order(right half) 4 bytes of 8byte checksum SSL's RetVal produces a checksum error in the KerberosV4 application on return because it is in the oposite byte order than what Kerberos expects. The Kerberos and SSL macros used to initialize the return value from the OUTBUF follow respectively: Kerberos: #define GET_HALF_BLOCK(lr, ip) \ (lr) =3D ((unsigned int)(*(ip)++)) 24; \ (lr) |=3D ((unsigned int)(*(ip)++)) 16; \ (lr) |=3D ((unsigned int)(*(ip)++)) 8; \ (lr) |=3D (unsigned int)(*(ip)++) =20 SSL: #define c2l(c,l) (l =3D((DES_LONG)(*((c)++))), \ l|=3D((DES_LONG)(*((c)++))) 8L, \ l|=3D((DES_LONG)(*((c)++)))16L, \ l|=3D((DES_LONG)(*((c)++)))24L) =20 We do not understand why SSL is swapping the bytes when initializing the output longword return value with the c2l macro in the DES_CBC_CKSUM() function. I have a test program that shows the differences. As you can see from the output below, the output is reversed between ssl and krb. $ run test_cbc_cksum set key 0 ssl 0x738af841, 0x80 0x16 0xd6 0x6b 0x41 0xf8 0x8a 0x73 krb 0x41f88a7 I then added a new macro, ksg_c2l, and had it do the same order of shifting as the get_half_block. They output is now in the correct order. $ run test_cbc_cksum2 set key 0 ssl 0x738af841, 0x80 0x16 0xd6 0x6b 0x41 0xf8 0x8a 0x73 krb 0x41f88a73 c2l 0x738af841 ksg_c2l 0x41f88a73 I have included the program with this mail message, and here are the environmental details=20 we were operating in: OpenSSL 0.9.6b with Security patches. OpenVMS Alpha V7.2 TCP/IP V5.0A DEC C V6.2 Kerberos V4 and 5 Thanks, Kevin =09 Kevin Greaney SSL for OpenVMS Team Hewlett Packard Company OpenVMS Engineering Group 110 Spitbrook Road =20 Nashua, NH 03062 (603) 884-5099 test_cbc_cksum.c=20 --_=_NextPart_002_01C27085.410E8F0F Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable !DOCTYPE HTML PUBLIC -//W3C//DTD HTML 3.2//EN HTML HEAD META HTTP-EQUIV=3DContent-Type CONTENT=3Dtext/html; = charset=3Dus-ascii META NAME=3DGenerator CONTENT=3DMS Exchange Server version = 6.0.6249.1 TITLEDES_CBC_CKSUM in SSL and Kerberos./TITLE /HEAD BODY !-- Converted from text/rtf format -- BR PFONT SIZE=3D2 FACE=3DArialnbsp;Hi,/FONT UL PFONT SIZE=3D2 FACE=3DArialnbsp;nbsp;nbsp; I have a customer = with a Kerberos V4 application who is trying to decide if they can = substitute their existing Kerberos V4 DES encryption capability with = SSL's DES encryption support.nbsp; When calling DES_CBC_CKSUM() from = the Kerberos library, the checksum output buffer and the function's = return value read :/FONT/P PFONT SIZE=3D2 FACE=3DArialnbsp;nbsp;nbsp; OUTBUF=3D = 1234ABCD...nbsp;nbsp; // char *, 8 Byte checksum/FONT BRFONT SIZE=3D2 FACE=3DArialnbsp;nbsp;nbsp; RetValnbsp;nbsp; = =3D = ABCDnbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nb= sp; // unsigned long, Low order(right half) 4 bytes of 8byte = checksum/FONT /P PFONT SIZE=3D2 FACE=3DArialnbsp;When calling SSL's = DES_CBC_CKSUM(), the values read:/FONT /P PFONT SIZE=3D2 FACE=3DArialnbsp;nbsp;nbsp; OUTBUF=3D = 1234ABCD...nbsp;nbsp; // char *, 8 Byte checksum/FONT BRFONT SIZE=3D2 FACE=3DArialnbsp;nbsp;nbsp; RetValnbsp;nbsp; = =3D = DCBAnbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nbsp;nb= sp; // unsigned long, Low order(right half) 4 bytes of 8byte = checksum/FONT /P PFONT SIZE=3D2 FACE=3DArialSSL's RetVal produces a checksum error = in the
RE: DES_CBC_CKSUM in SSL and Kerberos.
Thanks very much for your reply, Jeffery. Our initial note may have led you to believe that a wholesale replacement of Kerberos encryption with SSL encryption was being considered beyond the customer's application. That was my wording error. The customer simply wants to use SSL for their application's data encryption tasks (which they manage), and so far, this is the only issue that they report (and their workaround is to simply swap the return value from SSL's DES_CBC_CKSUM call). Our follow-up question is whether DES_CBC_CKSUM() is used/required by SSL itself for its encryption processing, or if it was only provided for KRB4 application support? If it is only for KRB4/5 application support, then it looks like we need to at least re-define the c2l macro in the cbc_cksum.c module to return the correct byte order of the checksum's right-half in the longword return value. If people consider this a bugfix issue for this function, we like this suggested fix. If the DES_CBC_CKSUM() function is used by SSL for its own encryption processing, then it is presumed correct and must be left alone. Our course here is to define an equivalent function that returns the longword in the expected byte order for Kerberos callers. Lastly, note that the OUTBUF stream for both SSL and Kerberos are identical. It is only the 4 byte return value from the function that differs, so we do not seem to have any DES encryption incompatibility between SSL and Kerberos (even accepting that internal data representations are different between the two technologies). Thanks for any feedback on this! Al... -Original Message- From: Jeffrey Altman [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 10, 2002 1:56 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; Greaney, Kevin; Jamison, Alan Subject: Re: DES_CBC_CKSUM in SSL and Kerberos. The answer is: MIT DES and OpenSSL DES use different internal representations of the data. You cannot replace the MIT DES with OpenSSL DES unless you also recompile MIT Kerberos 4 to use the OpenSSL DES as well. Several people have done it in the past but it is not recommended. OpenSSL has gone to great lengths to clean the namespace so that each implementation can be used within the same application. Hi, I have a customer with a Kerberos V4 application who is trying to decide if they can substitute their existing Kerberos V4 DES encryption capability with SSL's DES encryption support. When calling DES_CBC_CKSUM() from the Kerberos library, the checksum output buffer and the function's return value read : OUTBUF=3D 1234ABCD... // char *, 8 Byte checksum RetVal =3D ABCD // unsigned long, Low order(right half) 4 bytes of 8byte checksum When calling SSL's DES_CBC_CKSUM(), the values read: OUTBUF=3D 1234ABCD... // char *, 8 Byte checksum RetVal =3D DCBA // unsigned long, Low order(right half) 4 bytes of 8byte checksum SSL's RetVal produces a checksum error in the KerberosV4 application on return because it is in the oposite byte order than what Kerberos expects. The Kerberos and SSL macros used to initialize the return value from the OUTBUF follow respectively: Kerberos: #define GET_HALF_BLOCK(lr, ip) \ (lr) =3D ((unsigned int)(*(ip)++)) 24; \ (lr) |=3D ((unsigned int)(*(ip)++)) 16; \ (lr) |=3D ((unsigned int)(*(ip)++)) 8; \ (lr) |=3D (unsigned int)(*(ip)++) =20 SSL: #define c2l(c,l) (l =3D((DES_LONG)(*((c)++))), \ l|=3D((DES_LONG)(*((c)++))) 8L, \ l|=3D((DES_LONG)(*((c)++)))16L, \ l|=3D((DES_LONG)(*((c)++)))24L) =20 We do not understand why SSL is swapping the bytes when initializing the output longword return value with the c2l macro in the DES_CBC_CKSUM() function. I have a test program that shows the differences. As you can see from the output below, the output is reversed between ssl and krb. $ run test_cbc_cksum set key 0 ssl 0x738af841, 0x80 0x16 0xd6 0x6b 0x41 0xf8 0x8a 0x73 krb 0x41f88a7 I then added a new macro, ksg_c2l, and had it do the same order of shifting as the get_half_block. They output is now in the correct order. $ run test_cbc_cksum2 set key 0 ssl 0x738af841, 0x80 0x16 0xd6 0x6b 0x41 0xf8 0x8a 0x73 krb 0x41f88a73 c2l 0x738af841 ksg_c2l 0x41f88a73 I have included the program with this mail message, and here are the environmental details=20 we were operating in: OpenSSL 0.9.6b with Security patches. OpenVMS Alpha V7.2 TCP/IP V5.0A DEC C V6.2 Kerberos V4 and 5 Thanks, Kevin =09 Kevin Greaney SSL for OpenVMS Team
