Martin Pool has proposed merging lp:~mbp/launchpad/690021-rlimit into
lp:launchpad.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
Related bugs:
#690021 scan_branches terminated for excessive memory abuse
https://bugs.launchpad.net/bugs/690021
This change limits the address space of scan_branches.py to 2GB. If it tries
to allocate more, it will crash. See
<https://bugs.launchpad.net/launchpad-code/+bug/690021>
After some discussion with spm and mwh on irc this seemed like a good idea. It
is consistent with what we do in other scripts.
2Gb is a compromise between I hope being small enough to stop the machine
bogging down, but not so small that scans that would otherwise succeed will now
fail.
It would be nice if the limit was not hard coded, but this seems like the most
pragmatic thing for now. I added a note on
https://dev.launchpad.net/LEP/FeatureFlags that we could potentially do it
there.
Obviously this will not actually fix whatever the bad scaling behaviour is in
bug 690021; we should probably split that to a separate bug.
I would welcome opinions on how to test this. Do these jobs run on staging or
qastaging? Perhaps we should just let it get to production and then watch for
it to fail cleanly.
--
https://code.launchpad.net/~mbp/launchpad/690021-rlimit/+merge/43733
Your team Launchpad code reviewers is requested to review the proposed merge of
lp:~mbp/launchpad/690021-rlimit into lp:launchpad.
=== modified file 'cronscripts/scan_branches.py'
--- cronscripts/scan_branches.py 2010-04-27 19:48:39 +0000
+++ cronscripts/scan_branches.py 2010-12-15 05:38:55 +0000
@@ -9,6 +9,8 @@
import _pythonpath
+import resource
+
from lp.services.job.runner import JobCronScript
from lp.code.interfaces.branchjob import IBranchScanJobSource
@@ -21,5 +23,8 @@
if __name__ == '__main__':
+
+ resource.setrlimit(resource.RLIMIT_AS, (2L << 30, 2L << 30))
+
script = RunScanBranches()
script.lock_and_run()
_______________________________________________
Mailing list: https://launchpad.net/~launchpad-reviewers
Post to : [email protected]
Unsubscribe : https://launchpad.net/~launchpad-reviewers
More help : https://help.launchpad.net/ListHelp
