I may be out on paternity leave starting any time now, so I'm going to start writing these messages at the end of every day so whatever I'm working on will be easy to pick up.
Right now I have lp:~benji/launchpad/bug-894177-2 landing. It just got merged into devel as revision 14476. Once qastaging has been updated to that version it can be QAed using the instructions at https://code.launchpad.net/~benji/launchpad/bug-894177-2/+merge/84676 I'm going to be out Monday so if anyone has time to shepherd the 894177 branch through QA and a no-downtime deployment, I'd appreciate it. I've also been handing over bug 808930 to Graham. Here are the notes I promised him on analysing the behavior of the branch scan job. First, the built-in profiling doesn't work (--profile=FILE); it will just profile the spawning of a subprocess that actually does all the work. Printing anything to stdout won't work either, since the subprocess captures all output and sends it... somewhere. I had good luck initially with strace to figure out roughly where time was being spent, especially helpful are -f and -tt (see man). Here's one of the command-lines I used: strace -tt -f -T -o strace-b cronscripts/scan_branches.py The stracing showed me that a very large number of queries were being executed. That lead me to try the LP_DEBUG_SQL and LP_DEBUG_SQL_EXTRA environment variables (https://dev.launchpad.net/Debugging) but neither worked, presumably because they weren't passed through to the subprocess. That lead me to hack Storm to record similar info. Here's the diff: http://paste.ubuntu.com/765479/ The vast majority of the DB traffic is generated by lib/lp/codehosting/scanner/bzrsync.py so I instrumented that to give me some rough timing info: http://paste.ubuntu.com/765475/ I haven't had the time to analyze that much yet. Looking at the output of the traceback dump indicated that line 286 of lib/lp/code/model/revision.py RevisionSet.new was the primary culprit, generating most of the SELECTS and INSERTS. That's all I have for the moment. -- Benji York -- Mailing list: https://launchpad.net/~yellow Post to : [email protected] Unsubscribe : https://launchpad.net/~yellow More help : https://help.launchpad.net/ListHelp
