On Fri, Jan 17, 2014 at 05:51:13PM -0800, Jacob Hoffman-Andrews wrote: > > > On the other hand, the SQLite solution should be let us scale not just > > to ~10K rulesets but if we're lucky to more like ~100K. > > > This is one of the things I like about it. Also it reduces memory use, > which is > especially valuable on mobile.
That's probably true, although for optimal performance we may hope that the whole .sqlite file is in the OS'es page cache; it looks like it takes up about 20MB. Perhaps on any OS where we think we aren't RAM constrained we should just cat the file to /dev/null (or whatever the local equivalent operation is) in the background. Another downside is that the .sqlite file format roughly doubles the size of our .xpi file from ~1MB to ~2MB (in the current master branch). I guess we can live with that. We should warn EFF's ops team though :). On balance that cost seems outweighed by the startup time benefits, so I think we should merge the pull request. After that I'm wondering whether we should port in Nick Semenkovich's fancier LRU caching layer (which is the chrome version of the extension only right now) in place of the one you used; that has a few benefits: (1) prevents us from needing to keep all of the targets table in RAM; (2) optimises away all the expensive string manipulations in potentiallyApplicableRulesets; (3) it's safe against having the negative cache bloat indefinitely if the user never restarts their browser and meets Lots and Lots of domains. -- Peter Eckersley [email protected] Technology Projects Director Tel +1 415 436 9333 x131 Electronic Frontier Foundation Fax +1 415 436 9993 _______________________________________________ HTTPS-Everywhere mailing list [email protected] https://lists.eff.org/mailman/listinfo/https-everywhere
