On Tue, Aug 3, 2010 at 11:02 AM, Robert Collins <[email protected]> wrote: > On Tue, Aug 3, 2010 at 12:49 PM, Robert Collins > <[email protected]> wrote: >> Just a reminder - I'm spending today, all day, just poking at >> performance; I'm 100% interruptible for performance related questions >> / tasks / todos & gotchas. >> >> And if anyone wants to join in, please do so! >> >> Right now, I'm about to finish up a small backlog of emails, then look >> into why https://api.staging.launchpad.net/beta/~ubuntu-dev/participants >> takes 8 seconds. > > So, I'm heading off to sleep, and I haven't got a mergable branch yet, > though I do have some small test cleanups. > > Largely I've been on a voyage of discovery of a few bits of the > system, which is ok :). >
Thanks for sending these notices, btw. It's nice to see what's going on. ... > I grabbed a profile off of staging. This required some bootstrapping > of that facility. Remember the excellent kcachegrind stuff Maris wrote > about? Well we can get that from staging, with LOSA cooperation. Its a > little manual at the moment, so its worth seeing if you can reproduce > things locally first. > > To get a profile from staging, follow these basic steps: > - grab a LOSA > - ask them to set profile_requests: True in the staging config (on > staging) and bounce the appserver to make it take effect. I have a > branch that makes this trivial which I will get merged right after the > rollout; for now you need to apply the following: > === modified file 'staging-lazr.conf' > --- staging-lazr.conf 2010-07-19 06:53:44 +0000 > +++ staging-lazr.conf 2010-08-03 01:02:11 +0000 > @@ -246,3 +246,9 @@ > [calculate_bug_heat] > oops_prefix: CBHT > error_dir: /srv/staging.launchpad.net/staging-logs/calculate_bug_heat > + > +[profiling] > +profile_requests: True > +profile_dir: /srv/staging.launchpad.net/staging-logs/profiling > > With that applied, every request will generate a prof file > (datestamp-pageid-serial.prof) in > /srv/staging.launchpad.net/staging-logs/profiling. So, hit your page > up, and then > - ask your friendly LOSA to revert the config and bounce the appserver again > - ask them to rsync the logs/oops etc to devpad > > You can then rsync the profile down locally - rsync -r > devpad.canonical.com:/srv/launchpad.net-logs/staging/asuka/profiling/ > SOMEPATH -. To find the profile you want, look for the datestamp > matching your web request and the page id. This can be a bit fiddly > :(. There should be a oops comment in the page if you hit ctrl-U, but > API pages (which I was looking at) don't render that. > > Anyhow, having found it (thanks Steve!) note that the name - e.g. > 2010-08-03_02:04:43.893-MailingListApplication:MailingListAPIView-OOPS-1676S219-Dummy-4.prof > - has an OOPS code in it. You can look this up on > lp-oops.canonical.com, and correlate the SQL activity and the python > backtrace together. > This sounds like a guide that should be in a doc somewhere, perhaps a wikipage. Could you please do so? I would probably have looked for it in the debugging section of the dev wiki. ... > Now, given Tim has run into this in code, it may be that there is a > general problem pattern we should look into, but we should also fix > this up. In the code API, what they do is a tuple query in storm to > get back all the needed information and then start yielding > pre-populated objects which already have the needed values looked up. > This is a bit manual and error prone but a sensible approach for now. > Where can we find this code? > I hope this slightly rambling story is useful - its probably the case > that its all all documented already and I just missed it. > I think this points to a more general problem with documentation navigation. I think the apidoc work done recently by Salgado & Gary is an excellent first step along these lines. > If not though, please do tell me where you would naively have looked > for the interesting bits of the story, and I'll put them there > (wherever there is) in a more permanent fashion. > As above. jml _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
