John A Meinel has proposed merging
lp:~jameinel/launchpad/loggerhead-clear-cache into lp:launchpad.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~jameinel/launchpad/loggerhead-clear-cache/+merge/119793
This is a small tweak for the Launchpad Loggerhead app that drive
bazaar.launchpad.net.
Specifically, every request to loggerhead gets a new bzrlib.branch.Branch
object created. Which then loads data for all the indexes, etc. A lot of this
gets cached on the assumption that the object will get re-used. However, these
objects are *not* shared between Loggerhead requests. It is a known issue that
they also don't go away by simple ref-count (they require the garbage collector
to run to remove circular references).
So this is just helping things along. Once we've finished an HTTP request,
directly request that caches at least get cleared out.
In doing a test of just spidering of the launchpad codebase using
bazaar.launchpad.dev on my local machine. This dropped the peak memory of
'start-loggerhead.py' down from 500+MB down to around 130-200MB.
A different option would be to look into changing how branches are opened, and
possibly caching them. It would theoretically help performance a lot, we
wouldn't be re-reading all the same content over and over. However at this
point, I'd rather drop a cache than add a new one.
--
https://code.launchpad.net/~jameinel/launchpad/loggerhead-clear-cache/+merge/119793
Your team Launchpad code reviewers is requested to review the proposed merge of
lp:~jameinel/launchpad/loggerhead-clear-cache into lp:launchpad.
=== modified file 'lib/launchpad_loggerhead/app.py'
--- lib/launchpad_loggerhead/app.py 2012-06-29 08:40:05 +0000
+++ lib/launchpad_loggerhead/app.py 2012-08-15 21:05:07 +0000
@@ -286,6 +286,12 @@
served_url=None, private=private)
return view.app(environ, start_response)
finally:
+ bzr_branch.repository.revisions.clear_cache()
+ bzr_branch.repository.signatures.clear_cache()
+ bzr_branch.repository.inventories.clear_cache()
+ if bzr_branch.repository.chk_bytes is not None:
+ bzr_branch.repository.chk_bytes.clear_cache()
+ bzr_branch.repository.texts.clear_cache()
bzr_branch.unlock()
finally:
lp_server.stop_server()
_______________________________________________
Mailing list: https://launchpad.net/~launchpad-reviewers
Post to : [email protected]
Unsubscribe : https://launchpad.net/~launchpad-reviewers
More help : https://help.launchpad.net/ListHelp
