Agreed! It was great working with you! -Yan
On 08/19/2014 03:32 PM, Jacob S Hoffman-Andrews wrote: > Thanks very much for your contributions, Red! It's been great to have > you on the team this summer, and the work you've done is really valuable. > > On 08/18/2014 08:11 PM, Red wrote: >> Hello everyone! >> >> Today is the official "pencils down" date for Google Summer of Code, so >> I am going to outline the results of my work this Summer as a way of >> concluding my project. >> >> The first thing I had done at the start of my project was learn about >> Mozilla's XPCOM components and how they are used to interface with Gecko >> and other Firefox internals from extension code. I also learned about >> special files included with any extension such as install.rdf which are >> used to describe an extension's version information and >> signature-verifying public key, among other things. This lead me to >> understanding how extensions are updated and discover the kinds of >> limitations I would find in trying to update HTTPS Everywhere's rulesets >> library without changing the rest of the extension. >> >> The next thing I did was draft the specification for the format of the >> JSON manifest used to describe ruleset updates and explain how the >> process of updating the rulesets library would work. This work has >> since been built on and improved, and the existing specification >> document can be found in my fork of the extension, and is linked below[0]. >> This document describes everything one needs to know about the ruleset >> updater. It details: >> >> * How an object signing certificate can be generated using certutil from >> the NSS toolset. >> * How to create an update.json manifest file using the >> ruleset_update_manifest.py utility script I created.[1] >> * How to sign the update.json file using pk1sign and an object signing >> certificate. >> * How update information is fetched and how download failures are handled. >> * How the contents of update.json and the rulesets library database file >> are verified to be authentic. >> * What the significance of each field in update.json is. >> * Approximately how the ruleset updating mechanism functions in pseudocode. >> >> After the details of how updating would work were outlined in >> updateJSONSpec.md, I began implementing the ruleset updater module code >> for the extension. I also started working on a way to write tests for >> HTTPS Everywhere that I would initially use to test my code. I >> discovered that Firefox extensions could import XPCOM components >> exported by other extensions added to a user's browser, and so began >> work on a separate extension[2] that could be installed and used solely >> for testing HTTPS Everywhere. This work was soon improved upon by my >> mentor, Yan, who had the idea of creating a skeleton of an extension >> inside of HTTPS Everywhere's own codebase[3]. This extension used the >> more modern Addon SDK, which includes a unit testing suite, enabling >> HTTPS Everywhere developers to start writing proper unit tests that >> could be included in the HTTPS-Everywhere repository and run from the >> command line. >> >> While the unit testing environment I now had available was very >> effective, I had run into some surprising difficulty with Mozilla's >> nsIDataSignatureVerified XPCOM component, used to verify the signatures >> we were generating. By this point, Yan and I had agreed to try to >> compute the signature over the SHA256 hash of update.json's contents, so >> that this data could be more easily transcribed from a regular work >> computer to the airgapped machine EFF uses to house an offline signing >> key. It was at this point that we found that the key and signatures >> OpenSSL were generating were, somehow, incompatible with >> nsIDataSignatureVerifier, and Yan managed to get the verification unit >> test to pass by using certutil and pk1sign. The same problem of >> verifying signatures was encountered again when I got around to testing >> the actual ruleset updater code again, and had to generate new data, >> despite using certutil and pk1sign. Quite unfortunately, this problem >> drained almost four whole weeks of my time, as I had been looking into >> solutions involving porting a Firefox extension signing tool called >> Uhura (written in Perl), which meant also trying to understand what the >> tool was doing with ASN.1 specifications, among other things. Finally, >> I found that the nsIDataSignatureVerifier tool successfully verified the >> signature I had created for testing when it was not the hash of >> update.json's contents that were signed and supplied to the verifier, >> but the contents of the file themselves. The component is poorly >> documented and, after reaching out to developers in Mozilla's ecosystem, >> I found that it isn't very well understood in general. As such, even >> now I do not really understand why the verifier did not behave as >> expected in this scenario. Regardless, while it is somewhat less >> convenient here to sign update.json, the signature verification process >> works. >> >> Finally, I was able to move on to solve the problems of downloading and >> verifying the authenticity of the rulesets database file, and of >> reinitializing the extension's rulesets with those of the new database >> file. These problems were technically nuanced, but have been solved. >> The ruleset updater has been integrated into my fork of the extension's >> repository on github. My tests of its performance have passed, and I am >> pleased to say that it appears to be working. I have since submitted a >> pull request[4] to have my work reviewed and merged, and have even made >> changes requested by Yan, who has reviewed my work already. >> >> I'd like to conclude by giving my thanks to everyone in both of these >> mailing lists. I have received some invaluable support and advice that >> has had a profoundly positive impact on my experience. I learned a lot >> from everyone that lent a hand, and I feel the project has benefited >> greatly from each of the contributions I received to my efforts. My >> mentor, Yan, has also been a pleasure to work with. Our weekly meetings >> added very significant structure to my workflow, helped to keep me >> organized, and were also a great opportunity for me to pique her brain >> for ideas and possible solutions to some of the tougher problems I've >> encountered. I sincerely regard this as having been a positive >> experience, and I hope that everyone who has helped me until now is >> satisfied with my efforts and the actual results of my work. >> >> Thanks, everyone! >> >> [0] - >> https://github.com/redwire/https-everywhere/blob/master/doc/updateJSONSpec.md >> [1] - >> https://github.com/redwire/https-everywhere/blob/master/utils/ruleset_update_manifest.py >> [2] - https://github.com/redwire/https-everywhere-testing >> [3] - >> https://github.com/EFForg/https-everywhere/tree/master/https-everywhere-tests >> [4] - https://github.com/EFForg/https-everywhere/pull/437 >> >> >> >> _______________________________________________ >> HTTPS-Everywhere mailing list >> [email protected] >> https://lists.eff.org/mailman/listinfo/https-everywhere > _______________________________________________ HTTPS-Everywhere mailing list [email protected] https://lists.eff.org/mailman/listinfo/https-everywhere
