Hi Michael, I'm happy to hear you think this would make a good inclusion to 13.04. I have been using the ExecutionTime helper code a lot in tracking down bottle necks it is extremely useful. I have been tracking the bottlenecks I've found in this bug: https://bugs.launchpad.net/ubuntu/+source/software-center/+bug/845579
Last night I tracked down some more bottlenecks in calls to Xapian. Specifically calls to get_mset() these calls take around 4 seconds in total on my machine. Most of these calls are made from _blocking_perform_search() in softwarecenter/db/enquire.py Aside from upstream optimizations to Xapian there doesn't appear to be any easy way around this. The maxitems parameter can be set to 0 to speedup get_mset() this results in only statistical information being returned which seems to be all that is use from these calls however the documentation says this also reduces the accuracy of the mset.get_matches_estimated() call which it seems the software center relies on being accurate so I dont know if change could be made. So with the pickle load time and the Xapian calls that I have tracked so far that adds up to around 20-25% of the startup time. Tim ----- Original Message ----- From: Michael Vogt <[email protected]> To: Timothy Arceri <[email protected]> Cc: "[email protected]" <[email protected]> Sent: Wednesday, 10 October 2012 11:44 PM Subject: Re: [Software-store-developers] Speeding up software center load times On Sun, Oct 07, 2012 at 04:51:11PM -0700, Timothy Arceri wrote: > Hi Guys, Hi Timothy, > Due to time constraints I tend to limit my open source > contributions to resolve personal annoyances. Lately I've turned my attention > to the startup times on the Ubuntu Software Center which although is slightly > better than when it was first release still takes far to long to load on my > Netbook. > > > Anyway I'm tackling this one performance issue at a time and the first one > that has come to my attention is that the software center is slow to load a > Python pickle file that has been generated by apt-xapian-index. This file > simply contains a serialized python dictionary and therefore its possible to > use a more optimized serialization module than pickle. > > My suggestion is to replace pickle with msgpack which reduces load time (save > times are also increased using msgpack ) on my machine from 1.4 seconds to > 0.1 seconds also it reduces the filesize from 3.1MB to 1.7MB. Woah, that sounds like a pretty impressive improvement. Thanks a bunch for looking into this. > Obviously the drawback from my proposal is that it introduces a new > dependency: msgpack-python > Unfortunately we are too late for the 12.10 release as the new dependency needs to get a main approval first and get on the CD etc. But as soon as 13.04 opens we can switch > I have submitted a patch upstream to apt-xapian-index to make this change: > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=689717 Looks like Enrico did not have the time yet to look at this, I am happy to merge this myself, we may need a compatibilty pickle file for some time and also convert the old data over. But that is probably straightforward :) > Anyway I would appreciate some feedback on the likely hood of adopting this > improvement before I move on to tracking down further bottlenecks in software > center. I would certainly love to help with finding more bottlenecks. There is a bit of helper code availalbe: with ExecutionTime("description"): codeblock for example and "software-center --debug-filter=performance" should give some useful hints on areas that are slow. Cheers, Michael _______________________________________________ Mailing list: https://launchpad.net/~software-store-developers Post to : [email protected] Unsubscribe : https://launchpad.net/~software-store-developers More help : https://help.launchpad.net/ListHelp
