Hi,

I have encountered weird behavior with openssl that I can't explain, and I'm 
wondering if it's a bug.

I have compiled the openssl-0.9.6j library on Linux and Solaris,using the 
default Configure options. ("Configure linux-elf" for Linux and "Configure 
solaris-sparcv7-cc" for Solaris).

I'm finding that the output from RC4 is different for Linux and Solaris once 
the key strength > 144.  However, Linux and Win32 produce the same RC4 
results up to 2048 bits.

I have including a short program that can reproduce the following output:

When I set RC4_KEYSIZE to 152 and run the program on Linux, I get the 
following output:

Initial:        74 65 72 72 79
Encrypt:        be 72 fe 4f 46
Decrypt:        74 65 72 72 79

When I run it on Solaris, I get the following output:

Initial:        74 65 72 72 79
Encrypt:        a4 1e 73 3a de
Decrypt:        74 65 72 72 79

This means that when I encrypt something > 152 on Solaris, I cannot use the 
ciphertext on Linux.

I was, however, able to make it work.  On Linux, I hand-edited the Configure 
file to add the RC4_CHAR and RC4_CHUNK options to linux-elf, and used the 
following command for Configure:

Configure no-asm linux-elf

The resulting libcrypto.a after I compiled it gave the same results as 
Solaris up to 2048 bit key strength.

Is this a bug?  I'm wondering why the default Configure options work only 
until 144 bits... what is magical about the 144 bit/152 bit boundary?

Thanks for any help,

Terry

Tested on Linux Red Hat Advanced Server 2.1 (gcc) and Solaris 7 (Workshop 
5.0)
To compile:
cc/gcc -o main.c -o rc4test -I <openssldir>/include <openssldir>/libcrypto.a


#include <stdio.h>

#include "openssl/evp.h"
#include "openssl/rc4.h"

#define RC4_KEYSIZE         152

int main (void)
{
   unsigned int setKeyLen = 0;
   EVP_CIPHER *cipher = 0;
   EVP_CIPHER_CTX ctx;
   unsigned char iv[8];
   unsigned char buffer[2048];
   unsigned char *encryptOutput = buffer;
   int rc;
   unsigned char key[RC4_KEYSIZE];
   int i;
   int encryptOutputLen;
   unsigned int outLen;

   unsigned char *plainText = (unsigned char *) "terry";
   printf("Initial:\t");
   for (i=0; i < strlen(plainText); i++)
      printf("%02x ", plainText[i]);
   printf("\n");


   cipher = EVP_rc4();
   setKeyLen = RC4_KEYSIZE/8;

   memset(&iv, 0, sizeof(iv));
   memset(key, 1, sizeof(key));

   /* initialize encryption */
   rc = EVP_EncryptInit(&ctx, cipher, key, iv);
   EVP_CIPHER_CTX_set_key_length(&ctx, setKeyLen);
   rc = EVP_EncryptInit(&ctx, 0, key, 0);

   encryptOutputLen = 0;
   rc = EVP_EncryptUpdate(&ctx, encryptOutput, &encryptOutputLen, plainText, 
strlen(plainText));
   encryptOutput += encryptOutputLen; encryptOutputLen = 0;

   rc = EVP_EncryptFinal(&ctx, encryptOutput, &encryptOutputLen);
   encryptOutput += encryptOutputLen; encryptOutputLen = 0;

   /* Get the length of the output */
   encryptOutputLen = encryptOutput - buffer;

   printf("Encrypt:\t");
   for (i=0; i < encryptOutputLen; i++)
      printf("%02x ", buffer[i]);
   printf("\n");

   /*Decrypt */
   {

      EVP_CIPHER_CTX ctx2;
      unsigned char *cipherText = buffer;
      unsigned int cipherTextLen = encryptOutputLen;
      unsigned char buffer2[2048];
      unsigned char *decryptTxt = buffer2;
      int decryptLen = 0;

      memset(buffer2, 0, sizeof(buffer2));

      EVP_DecryptInit(&ctx2, cipher, key, iv);
      EVP_CIPHER_CTX_set_key_length(&ctx2, setKeyLen);
      EVP_DecryptInit(&ctx2, 0, key, 0);

      rc = EVP_DecryptUpdate(&ctx2, decryptTxt, &decryptLen, cipherText, 
cipherTextLen);
      decryptTxt += decryptLen; decryptLen = 0;

      rc = EVP_DecryptFinal(&ctx2, decryptTxt, &decryptLen);
      decryptTxt += decryptLen; decryptLen = 0;

      outLen = decryptTxt - buffer2;

      printf("Decrypt:\t");
      for (i=0; i < outLen; i++)
         printf("%02x ", buffer2[i]);
      printf("\n");
   }

   return 0;
}

_________________________________________________________________
Add photos to your messages with MSN 8. Get 2 months FREE*. 
http://join.msn.com/?page=features/featuredemail

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [EMAIL PROTECTED]
Automated List Manager                           [EMAIL PROTECTED]
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [EMAIL PROTECTED]
Automated List Manager                           [EMAIL PROTECTED]

Reply via email to