AWESOME... Tainted how, you did not add your copyrights :P On Mon, Aug 30, 2010 at 3:27 PM, Mikkel Kamstrup Erlandsen < [email protected]> wrote:
> Mikkel Kamstrup Erlandsen has proposed merging > lp:~zeitgeist/zeitgeist/small-find-events-optimization into lp:zeitgeist. > > Requested reviews: > Zeitgeist Framework Team (zeitgeist) > > > I was seeing query response times of around 200ms when calling ZG from > Unity. As this seems like a suspiciously long query time for my small > database I went looking a bit into this. > > First I noticed that the debug/profiling statements we had for this was not > only imprecise but also directly hiding the truth. So I fixed that. The > problem being that we never really logged the time it took to execute the > query where we find, group, and sort the events, only the time it takes us > to build the event objects after the query ... > > Next thing I noticed was that we spend surprisingly long time in > _find_events() when looking up full events compared to just looking up the > ids. So I made that function always just find the ids and then do another > SQL call to look up the full event structures. Surprisingly adding this > extra SQL call shaves off roughly 50ms on the full query time. > > (Note: This was done in my Canonical time - so if you accept this branch > Zeitgeist will forever be "tainted" bwahaha! ;-P) > > -- > > https://code.launchpad.net/~zeitgeist/zeitgeist/small-find-events-optimization/+merge/34065<https://code.launchpad.net/%7Ezeitgeist/zeitgeist/small-find-events-optimization/+merge/34065> > You are subscribed to branch lp:zeitgeist. > > === modified file '_zeitgeist/engine/main.py' > --- _zeitgeist/engine/main.py 2010-08-02 10:13:12 +0000 > +++ _zeitgeist/engine/main.py 2010-08-30 13:27:45 +0000 > @@ -337,7 +337,7 @@ > if return_mode == 0: > sql = "SELECT DISTINCT id FROM event_view" > elif return_mode == 1: > - sql = "SELECT * FROM event_view" > + sql = "SELECT id FROM event_view" > elif return_mode == 2: > sql = "SELECT subj_uri, timestamp FROM event_view" > else: > @@ -374,14 +374,19 @@ > > result = self._cursor.execute(sql, > where.arguments).fetchall() > > - if return_mode == 1: > - return self.get_events(rows=result, sender=sender) > - if return_mode == 2: > - return map(lambda row: (row[0], row[1]), result) > - else: # return_mode == 0 > + if return_mode == 0: > + log.debug("Found %d event IDs in %fs" % > (len(result), time.time()- t)) > result = [row[0] for row in result] > - log.debug("Fetched %d event IDs in %fs" % > (len(result), time.time()- t)) > - return result > + elif return_mode == 1: > + log.debug("Found %d events in %fs" % (len(result), > time.time()- t)) > + result = self.get_events(ids=[row[0] for row in > result], sender=sender) > + elif return_mode == 2: > + log.debug("Found %d (uri,timestamp) tuples in %fs" > % (len(result), time.time()- t)) > + result = map(lambda row: (row[0], row[1]), result) > + else: > + raise Exception("%d" % return_mode) > + > + return result > > def find_eventids(self, *args): > return self._find_events(0, *args) > > > -- This is me doing some advertisement for my blog http://seilo.geekyogre.com https://code.launchpad.net/~zeitgeist/zeitgeist/small-find-events-optimization/+merge/34065 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~zeitgeist/zeitgeist/small-find-events-optimization into lp:zeitgeist. _______________________________________________ Mailing list: https://launchpad.net/~zeitgeist Post to : [email protected] Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
