On 12/10/2012 9:38 PM, Markus Wernig wrote:
Hi everyone!
I have a fairly basic question about how to use blowfish-cbc in an
application. Here's the scanario:
An application will receive arbitrary amount of data (potentially
multi-gigabyte) via a tcp/ssl socket, multiple files from multiple
senders over long periods of time (i.e. not in the same tcp session).
Before processing it, the whole stream needs to be saved to disk, in the
end assembling it into a structured file (xml). The data being
sensitive, the files on disk need to be encrypted from the start. The
application has access to a persistant secret key (length to be
defined?). All resulting files should be recoverable with the same key.
Processing of the file is done asynchronously, ie. not immediately after
receiving.
So the idea was to use blowfish-cbc with the secret key to sequentially
encrypt the data in arrival and append the crypttext to the saved file.
After receiving all data, the file can be in turn decrypted later on
with the secret key and processed by the application.
Now the basic questions that I was unable to answer:
- With a given key being reused for all encrypted files, the IV from my
understanding is central to the strength of the encryption. So a unique
random IV needs to be used for each file. Does this mean that for every
file I have to record the IV in order to decrypt it later? Or is my
understanding wrong?
Yes, you need to know the IV to decrypt correctly, and it should not be
predictable by anyone without the key, but it does not need to be secret.
Some experts believe that setting the IV to
Encrypt(key=key,IV=0,data=filenumber) is safe, others disagree, but the
mathematical arguments on both sides of the debate seems to make a lot
of dubious assumptions, so I am undecided as to what I should believe.
When appending to an existing CBC encrypted file, the IV to use for
continuing as one long encryption stream can be recovered as follows:
1. Grab the last 2 blocks of the already encrypted file, and make an in
memory copy of this.
2. Decrypt the copy as if IV=000000. Then throw away the decryption of
the first block, leaving you with the decryption of the last block.
3. Assuming you are using the standard SSL-style padding for files not a
multiple of the block size, this last decrypted block will give you
the 0 to (blocksize-1) bytes at the end of the already encrypted part of
the file. Push those in front of the data you are going to append
to the file.
4. Set your IV to the second-last block of the encrypted file (from step
1 above) and start encrypting, overwriting the last block of the
encrypted file with the first new encrypted block.
- Is there a reasonably secure way to use the same secret key for all
encryption, and if so, how should it be used by the application?
The safe way to encrypt multiple files with the same symmetric key is to
use different IVs for each file.
There is also a security limit as to how many bytes (total) you should
directly encrypt with a single key. To get around that, for each lump
of X Gigabytes (where X is less than that security limit), use the
secure RNG to generate a new random key, encrypt that random key with
your persistent master key, and store the encrypted key in front of the
lump that will then be encrypted with the new key. In this case the
master key is often referred to as a KEK or "Key Encryption Key" and
should not be used to encrypt anything other than keys.
- Is the above feasible at all? Is there a more logical way to do this?
With the notes I gave above, this is a completely normal way to do it.
You should also add some provision for recovering if the receiving
computer crashes halfway through the process, leaving it with improperly
terminated partial files. One way would be to keep a separate on-disk
record of the status of each encrypted file as either:
0=="Encryption in progress, no attempt has been made to write the final
padded block" or N="Encryption (almost) stopped cleanly, the final
padded block will cause the file length to become N*blocksize bytes".
Then if the recorded state is 0 or more than the file size, the get old
IV procedure changes to simply skipping steps 2 and 3 and restarting
download slightly earlier than what the other end previously sent you
(because some data did not make it to disk before the crash).
Thanks in advance for shedding some light for a lost soul ;-)
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com
Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users@openssl.org
Automated List Manager majord...@openssl.org
