[Zeitgeist] Translation template import - zeitgeist in Zeitgeist Framework 0.5
Hello Zeitgeist Framework Team, On 2010-11-16 20:30z (7 minutes ago), you uploaded a translation template for zeitgeist in Zeitgeist Framework 0.5 in Launchpad. The template has now been imported successfully. Thank you, The Launchpad team ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673452] Re: Using the subj_origin column of event_view is slower than it should be
** Changed in: zeitgeist Status: In Progress = Fix Committed -- Using the subj_origin column of event_view is slower than it should be https://bugs.launchpad.net/bugs/673452 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: Let me summarize what we have wrt subj_origin 1) we have a subj_origin column in the event table, which is an integer pointing to an entry in the uri table 2) we have an index on the subj_origin column of the event table 3) we have subj_origin in event view, which is not the id, but the resolved uri 4) we have subj_origin_id in event_view which is in fact the same as event.subj_uri, but we get this value by using a sub-query on the uri table. Points 3) and 4) are the reason why the index on event.subj_origin is never used when querying the event_view, example: DEBUG:zeitgeist.sql:Got query: QUERY: SELECT id FROM event_view WHERE subj_origin_id=1 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] By simply renaming event_view.subj_origin to event_view.subj_origin_uri and event_view.subj_origin_id to event_view.subj_origin in combination with removing the sub-query we can make queries for the origin id on event_view *much* faster. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673394] Re: Queries for subj_uri and subj_origin are using no index
** Changed in: zeitgeist Milestone: None = 0.7.0 -- Queries for subj_uri and subj_origin are using no index https://bugs.launchpad.net/bugs/673394 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: We have a bunch of queries which are using no index and therefor are much slower than others, this is one example: Running: engine.find_eventids(TimeRange.always(), [], StorageState.Any, 6, ResultType.MostRecentSubjects) DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY subj_uri ORDER BY timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.571514s --- MostRecentSubjects: get len(ids)=6 using .find_eventids() in 0.571749s All quries with subj_uri and subj_origin are affected. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673394] Re: Queries for subj_uri and subj_origin are using no index
** Changed in: zeitgeist Status: In Progress = Fix Committed -- Queries for subj_uri and subj_origin are using no index https://bugs.launchpad.net/bugs/673394 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: We have a bunch of queries which are using no index and therefor are much slower than others, this is one example: Running: engine.find_eventids(TimeRange.always(), [], StorageState.Any, 6, ResultType.MostRecentSubjects) DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY subj_uri ORDER BY timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.571514s --- MostRecentSubjects: get len(ids)=6 using .find_eventids() in 0.571749s All quries with subj_uri and subj_origin are affected. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 639737] Re: Improve insertion times
OK I think the only way we can actually get the insertion time improved ATM is by using WAL for journaling. However it requires sqlite 3.7 and once its set to WAL sqlite 3.6 wont be able to access it. This breaks compatibility with older zeitgeist DBs: However I think this should be doable if we set zeitgeist 0.7 to depend on sqlite 3.7 and upgrade the db scheme to 4 What do u think? -- Improve insertion times https://bugs.launchpad.net/bugs/639737 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: In Progress Bug description: We insert pretty slowly with an average of 0.15 seconds for one event on my core i5 2.5 GHz beast. RainCT had some optimization possibilities: 1) PRAGMA synchronous=OFF 2) PRAGMA journal_mode=OFF The Chat: kamstrup I think we are - but I can't recall... in case of failed transactions - but I don't even know if we use transactions these days... seif RainCT try synchronous=OFF seif RainCT but it can corrupt your database if your phone dies while ZG is inserting seif RainCT and journal_mode=MEMORY seif RainCT or OFF since we don't use rollback anyway seif so maybe journal_mode = OFF is a good start? kamstrup okay, he's probably right... kamstrup 'grep -Ri rollback _zeitgeist/' is your friend :-) kamstrup apparently we are not using rollback... More info can be found here: http://www.sqlite.org/pragma.html In order to get a better picture of what's going on, can you please try to get some more information, like: 1) How many events are in your database? 2) What's the insertion time for one event into an empty db? 3) Out of this 0.15 secs, how many time is spend in our python code, and what's the time of the actual sql action? 4) How much faster is adding 10 events at once compared to adding them one at a time? 5) You think 0.15 secs is slow for inserting one event, what time do you expect, and why? ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Bug 602211] Re: Monitoring Create/Move/Copy Files events
On 15 November 2010 21:08, Siegfried Gevatter rai...@ubuntu.com wrote: 2010/11/15 Michal Hruby michal@gmail.com: I still didn't understand why would the current_uri be part of event, isn't it by definition a property of the subject? That way you don't need to update dozens of events, just one subject entry... A subject is part of an event, there is no global subject entry since the subject represents a snapshot of an object at a particular time instant (with some properties -eg. mimetype- which it has at that instant). I understand your confusion Michal. The deal is that while subjects are conceptually disjoint from the event they are still stored together with the event in the event table (as an optimization). Also as Siegfried says - the event subject is a snapshot (like a normal log statement) so it makes even more sense to store it together with the event as we do. -- Monitoring Create/Move/Copy Files events https://bugs.launchpad.net/bugs/602211 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Unity: Triaged Status in Unity Files Place: Triaged Status in Zeitgeist Framework: Confirmed Status in Zeitgeist Datahub: Confirmed Bug description: An issue we are facing at the moment is that ppl lose track of there files in a timeline if the file was moved around or renamed. I would propose using taskview or patch nautilus to actually grab those events and either: 1) Modify the uris in the uris table 2) Create a new table with| new_id | old_uri_id | event | to map uris to their actual ids and the event that allowed the change, this would allow us to track a history of renaming or moving a file. It will look a bit like the following: 9 | 9 | 48124 # CREATE EVENT 12 | 9 | 48126 # MOVE EVENT In other words the last row means uri 12 was moved from uri 9 with event 48126 UPDATE: 3) Create a changable_uri table that is a map of the uri table. it gets updated upon moved and rename. We then add new resulttype that allow you to ask for either pureSubject or adaptedSubject. depending on which one is chosen we then use the according table in the join of the find_events_query :) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Bug 602211] Re: Monitoring Create/Move/Copy Files events
Agreed I think I am overthinking it a bit. I am pretty ok with this solution but we shouldn't jump dive into it unless we are pretty comfortable all of us. What will break is also an issue for me.. On Tue, Nov 16, 2010 at 11:06 PM, Mikkel Kamstrup Erlandsen mikkel.kamst...@gmail.com wrote: On 15 November 2010 21:07, Seif Lotfy 602...@bugs.launchpad.net wrote: SNIP So lets say I have a file that has been moved from /home/seif/foo to /home/seif/bar. This means I query for all events where subject_uri = /home/seif/foo so I get all results until the point it was moved, right? And if I ask for /home/seif/bar I get all events with subject_uri = /home/seif/foo as well as all subject_uri = /home/seif/bar ... ? We still do strict template matching. If you query for event with subject_uri=bar then you wont get any events with subject_uri=foo, disregarding the subject_current_uri. Otoh if you query for events with subject_current_uri=bar you will get all events for subject_uri=foo too because these events will have been set to subject_current_uri=bar. Note that we are *not* being clever about matching on both the subject_uri and subject_current_uri fields. It's still strict matching on the subject_current_uri field that gives you these results. -- Monitoring Create/Move/Copy Files events https://bugs.launchpad.net/bugs/602211 You received this bug notification because you are a direct subscriber of the bug. Status in Unity: Triaged Status in Unity Files Place: Triaged Status in Zeitgeist Framework: Confirmed Status in Zeitgeist Datahub: Confirmed Bug description: An issue we are facing at the moment is that ppl lose track of there files in a timeline if the file was moved around or renamed. I would propose using taskview or patch nautilus to actually grab those events and either: 1) Modify the uris in the uris table 2) Create a new table with| new_id | old_uri_id | event | to map uris to their actual ids and the event that allowed the change, this would allow us to track a history of renaming or moving a file. It will look a bit like the following: 9 | 9 | 48124 # CREATE EVENT 12 | 9 | 48126 # MOVE EVENT In other words the last row means uri 12 was moved from uri 9 with event 48126 UPDATE: 3) Create a changable_uri table that is a map of the uri table. it gets updated upon moved and rename. We then add new resulttype that allow you to ask for either pureSubject or adaptedSubject. depending on which one is chosen we then use the according table in the join of the find_events_query :) To unsubscribe from this bug, go to: https://bugs.launchpad.net/unity/+bug/602211/+subscribe -- This is me doing some advertisement for my blog http://seilo.geekyogre.com -- Monitoring Create/Move/Copy Files events https://bugs.launchpad.net/bugs/602211 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Unity: Triaged Status in Unity Files Place: Triaged Status in Zeitgeist Framework: Confirmed Status in Zeitgeist Datahub: Confirmed Bug description: An issue we are facing at the moment is that ppl lose track of there files in a timeline if the file was moved around or renamed. I would propose using taskview or patch nautilus to actually grab those events and either: 1) Modify the uris in the uris table 2) Create a new table with| new_id | old_uri_id | event | to map uris to their actual ids and the event that allowed the change, this would allow us to track a history of renaming or moving a file. It will look a bit like the following: 9 | 9 | 48124 # CREATE EVENT 12 | 9 | 48126 # MOVE EVENT In other words the last row means uri 12 was moved from uri 9 with event 48126 UPDATE: 3) Create a changable_uri table that is a map of the uri table. it gets updated upon moved and rename. We then add new resulttype that allow you to ask for either pureSubject or adaptedSubject. depending on which one is chosen we then use the according table in the join of the find_events_query :) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
