Encrypt and decrypt large files with auto-padding?

[wiired]

I need to encrypt and decrypt large files (up to a few GB) so I need
to do the encryption/decryption one chunk at a time (128KB) to keep
memory utilization low.

Processing in chunks _and_ using padding appears to be more complex
than I thought. Here is the (slightly simplified) technique I'm using.
LSource and LDest are the input and output files read from and written
to directly.

Is there a more simple solution that I'm missing?

Also, is there much benefit in using a random iv? Doesn't that mean
that it would need to be stored with the file and therefore little
more secure than using a fixed value?

    const int CBufferSize = Blowfish::BLOCKSIZE * 16384; // 128KB

Encryption:

    byte* plainbuffer = NULL;
    byte* cypherbuffer = NULL;
    try {
      plainbuffer = new byte[CBufferSize];
      // Allow for padding in encrypted buffer - up to an extra block
size is added.
      cypherbuffer = new byte[CBufferSize + Blowfish::BLOCKSIZE];

      // Initialise
      byte iv[Blowfish::BLOCKSIZE] = { ... }; // 8 bytes
      Blowfish::Encryption blowfishEncryption(AKey, AKeyLen);
      CBC_Mode_ExternalCipher::Encryption cbcEncryption
(blowfishEncryption, iv);

      int LBytesLeft;
      int LBytesProcessed;
      while ((LBytesLeft = LSource->Size - LSource->Position) > 0) {
        LBytesProcessed = LSource->Read(plainbuffer, CBufferSize);

        // If at end use a StreamTransformationFilter to handle
padding
        if (LBytesLeft <= CBufferSize) {
          StreamTransformationFilter cbcEncryptor(cbcEncryption);
          cbcEncryptor.Put(plainbuffer, LBytesProcessed);
          cbcEncryptor.MessageEnd();
          LBytesProcessed = cbcEncryptor.MaxRetrievable();
          LBytesProcessed = cbcEncryptor.Get(cypherbuffer,
LBytesProcessed);
        } else {
          cbcEncryption.ProcessData(cypherbuffer, plainbuffer,
LBytesProcessed);
        }

        LDest->Write(cypherbuffer, LBytesProcessed);
      }
    }
    __finally {
      delete[] cypherbuffer;
      delete[] plainbuffer;
    }

Decryption:

    byte* plainbuffer = NULL;
    byte* cypherbuffer = NULL;
    try {
      plainbuffer = new byte[CBufferSize];
      // Allow read of padding in encrypted buffer - up to an extra
block size is added.
      cypherbuffer = new byte[CBufferSize + Blowfish::BLOCKSIZE];

      // Initialise
      byte iv[Blowfish::BLOCKSIZE] = { ... }; // 8 bytes
      Blowfish::Decryption blowfishDecryption(AKey, AKeyLen);
      CBC_Mode_ExternalCipher::Decryption cbcDecryption
(blowfishDecryption, iv);

      int LBytesLeft;
      int LBytesProcessed;
      while ((LBytesLeft = LSource->Size - LSource->Position) > 0) {
        // If at end use a StreamTransformationFilter to handle
padding. Not sure
        // if StreamTransformationFilter handles reading just the
padding block at the end
        // so group it with the last data blocks.
        if (LBytesLeft <= CBufferSize + Blowfish::BLOCKSIZE) {
          LBytesProcessed = LSource->Read(cypherbuffer, CBufferSize +
Blowfish::BLOCKSIZE);
          StreamTransformationFilter cbcDecryptor(cbcDecryption);
          cbcDecryptor.Put(cypherbuffer, LBytesProcessed);
          cbcDecryptor.MessageEnd();
          LBytesProcessed = cbcDecryptor.MaxRetrievable();
          LBytesProcessed = cbcDecryptor.Get(plainbuffer,
LBytesProcessed);
        } else {
          LBytesProcessed = LSource->Read(cypherbuffer, CBufferSize);
          cbcDecryption.ProcessData(plainbuffer, cypherbuffer,
LBytesProcessed);
        }

        LDest->Write(plainbuffer, LBytesProcessed);
      }
    }
    __finally {
      delete[] cypherbuffer;
      delete[] plainbuffer;
    }

Now this seems to work ok but the conditional code to detect the end
of the input buffer and then use a StreamTransformationFilter seems
like a hack. It would be nice to be able to reuse a single instance of
StreamTransformationFilter and reuse it to process every chunk (Put
followed by a Get) but with padding only at the end of the entire data
(not at the end of each chunk), and to be able to reset its internal
buffers for each chunk so that it doesn't build up the entire data in
its buffers.

Regards,

Jarrod Hollingworth
Complete Time Tracking
http://www.complete-time-tracking.com/

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the "Crypto++ Users" 
Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscr...@googlegroups.com.
More information about Crypto++ and this group is available at 
http://www.cryptopp.com.
-~----------~----~----~----~------~----~------~--~---