Hi Erik, > Sorry for the delayed answer, I've been swamped with work and haven't had the > time to deal with this for a while. Looks like I will have quite some time > this week though, and I hope to commit by the end of the week. My suggestion > is to commit on a separate branch and we can then build it out further from > there.
no problem - we're all doing this for fun ;) > Anyhow, here's an outline of what we've done so far; Sound fantastic what I read. Again: a nice performance comparison would be great for promotion. > I'd also like to point out that we've only changed the parts that are > critical for us. This would mainly be AES encryption/decryption and > generating SHA-1 hashes. Ok - so might be a long term goal to apply your changes also to the rest. > Since encryption/decryption can be a time consuming task we decided to do it > from web workers. Are you aware of this branch? https://github.com/openpgpjs/openpgpjs/tree/webworker > Another problem we ran into is that window.crypto.getRandomValues() is not > defined for Firefox (and not defined in a web worker context for any other > browser either). Currently we will instead use SJCL (the crypto lib from > Stanford) to get random numbers for Firefox. Very well. See https://github.com/openpgpjs/openpgpjs/issues/22 and https://github.com/openpgpjs/openpgpjs/tree/master/Dependencies (added the dependency already a year ago). > Currently all of this work with Firefox, Chrome and IE10. It is possible to > add support for Safari as well and I hope to be able to do this this week, > although that's nothing I can promise. Other browsers are not tested (except > for lower versions of IE that will not work). > Anyhow, > Sorry for the much delayed answer, let me know if there's any questions.. Your contribution is very welcome. If you or anyone on this list is interested, we can apply for the current call for OpenITP proposals[1]. It's about $5k-$30k USD, the deadline is 31.03.2013, it's simple to apply and I think there is a high chance to get accepted with an OpenPGP.js related project. Best regards, Alex [1] http://openitp.org/?q=openitp_first_round_of_2013_project_funding_now_open_for_proposals Best regards, Alex [1] http://openitp.org/?q=openitp_first_round_of_2013_project_funding_now_open_for_proposals On 12.02.2013, at 16:42, Erik Larsson <[email protected]> wrote: > Hi Alex, > > Sorry for the delayed answer, I've been swamped with work and haven't had the > time to deal with this for a while. Looks like I will have quite some time > this week though, and I hope to commit by the end of the week. My suggestion > is to commit on a separate branch and we can then build it out further from > there. > > Anyhow, here's an outline of what we've done so far; > > In order to be able to deal with larger files we now work extensively with > blobs and typed arrays rather than strings. Blob's are a good way to handle > large chunks of data since they are stored on the file system and not just > kept in memory. Rather than dealing with blob objects right away we created > two new buffer objects that basically reads in a small part of the blob to > avoid keeping it all in memory. It reads in a small chunk and returns a typed > array. By doing this we can perform normal array operations on the data which > is more convenient than dealing with a blob. Currently we have on buffer for > uint8array (used for encryption/decryption) and one for int32Arrays (for SHA > generation). > > I'd also like to point out that we've only changed the parts that are > critical for us. This would mainly be AES encryption/decryption and > generating SHA-1 hashes. > > Below is a more detailed description of the code changes we made. > > Encryption: > We've created a new function in openpgp.packet.literaldata named > write_packet_large. It takes a buffer object as input and returns a new > buffer containing the data + header. > > We've also created a new function in openpgp_packet_encrypteddata again > called write_packet_large. This function is not very different from the > normal one besides that it of course deals with the new buffer object rather > than strings. > > From there openpgp_crypto_symmetricEncrypt is called. We always use AES when > doing symmetric encryption so we haven't modified any of the other ones. > > Last but not least we've added a new openpgp_cfb_encrypt_large function which > takes in a buffer object. We've modified the original function so it will now > read in chunks from the blob and encrypt them. When done it returns a new > buffer containing the cipher text. Besides that it is also possible to pass > in a callback object to which progress is reported back. > > One note here is that we always use the resync option when encrypting which > means we have so far just converted that. I could possible convert the non > resync case before committing as well. > > Decryption > The changes made to the decryption flow is similar to the ones we did for > encryption. Our typical flow looks like this: > We added a new read_packet_large to openpgp_packet. In here we made one major > change. Previously there was a loop to determine the length for a packet of > indeterminate length (https://tools.ietf.org/html/rfc4880#section-4.2.1, > https://tools.ietf.org/html/rfc4880#section-4.2.2). We found that this loop > was delaying the decryption process quite a bit for larger files so it was > removed. I don't believe there is really no need to know the length in > advance, correct me if I'm wrong? Instead we're calculating the length while > decrypting. > > Just as previously, the functions returns a package. The difference is that > the encrypted data is stored as a buffer rather than a string. Again, only > the methods we are using have been converted. The only packages using large > amounts of data that we use is the type 9 packages. The decryption is then > done very similar to how the encryption was accomplished. > > SHA-1 Generation > Similarly we've modified the SHA generation to accept a buffer as input. > Again, We've appended _large to the modified functions to ensure that regular > strings works as before. > > At last > Since encryption/decryption can be a time consuming task we decided to do it > from web workers. There is currently a JQuery reference when encoding a html > string which won't work when used in a Web Worker context. We never used it > so we just ended up commenting it out although that is of course not a good > solution. Another problem we ran into is that window.crypto.getRandomValues() > is not defined for Firefox (and not defined in a web worker context for any > other browser either). Currently we will instead use SJCL (the crypto lib > from Stanford) to get random numbers for Firefox. This is however handled > outside of OpenPGP since we need to generate the numbers from the main thread > and pass them to the worker. > > Currently all of this work with Firefox, Chrome and IE10. It is possible to > add support for Safari as well and I hope to be able to do this this week, > although that's nothing I can promise. Other browsers are not tested (except > for lower versions of IE that will not work). > > Anyhow, > Sorry for the much delayed answer, let me know if there's any questions.. -- http://openpgpjs.org
