The proposal to merge lp:~michael.nelson/launchpad/982938-dont-load-all-tokens-for-all-ppas-into-memory into lp:launchpad has been updated.
Description changed to: Summary ======= This branch aims to reduce the load/memory consumption on germanium while generating htpasswd files. Proposed fix ============ Use a generator to return a storm result set of tokens for each PPA, rather than loading all tokens for all PPAs into a dict. It will result in a larger query count. jml is working on a script that will generate data to be able to verify whether this branch does indeed help. Pre-implementation notes ======================== Referring to https://bugs.launchpad.net/launchpad/+bug/982938/comments/6 12:27 < noodles> jml: if you think the first of those points (in my comment) is very likely to be causing performance issues, I could start coding that straight away... 12:27 < noodles> (given the numbers achuni mentioned above, it would seem likely, but only the measurements will tell). 12:27 < jml> as you say, only measurements will tell :) 12:28 < jml> noodles: I reckon code it up straight away. :) Implementation details ====================== Sheesh - this is the quick cover letter? :P LOC Rationale ============= Yay - 12 lines of credit. Tests ===== ./bin/test -vvct TestPPAHtaccessTokenGeneration ./bin/test -vvct TestHtpasswdGeneration Demo and Q/A ============ If this script runs on staging, we could QA by comparing output before and after running it. Lint ==== Checking for conflicts and issues in changed files. Linting changed files: lib/lp/archivepublisher/scripts/generate_ppa_htaccess.py For more details, see: https://code.launchpad.net/~michael.nelson/launchpad/982938-dont-load-all-tokens-for-all-ppas-into-memory/+merge/107980 -- https://code.launchpad.net/~michael.nelson/launchpad/982938-dont-load-all-tokens-for-all-ppas-into-memory/+merge/107980 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. _______________________________________________ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
