looks good
minor detail - Live handler needs to return FALSE!
jamie
On Thu, 2008-05-01 at 17:49 +0200, Philip Van Hoof wrote:
> Newer version
>
> On Thu, 2008-05-01 at 15:15 +0200, Philip Van Hoof wrote:
> > To Jamie, Martyn, Frade, Ottela, etc ...
> >
> > Here's some actual code. Please review this and let me know if you see
> > dings different.
> >
> > On Tue, 2008-04-29 at 17:56 +0200, Philip Van Hoof wrote:
> > > Pre note:
> > >
> > > This is about the Xesam support being done (since this week) in the
> > > indexer-split.
> > >
> > > About:
> > >
> > > Xesam requires notifying live searches about changes that affect them.
> > > We plan to implement this with a "events" table that journals all
> > > creates, deletes and updates that the indexer causes.
> > >
> > > Periodically we will handle and then flush the items in that events
> > > table.
> > >
> > > I made a cracktasty diagram that contains the from-a-high-distance
> > > abstract proposal that we have in mind for this.
> > >
> > >
> > > This is pseudo code that illustrates the periodic handler:
> > >
> > > bool periodic_handler (...)
> > >
> > > {
> > >
> > > lock indexer
> > > update eventstable set beinghandled=1 where 1=1 (all items)
> > > unlock indexer
> > >
> > > foreach query in all livequeries
> > > added, modified, removed = query.execute-on (eventstable)
> > > query.emit_added (added)
> > > query.emit_removed (removed)
> > > query.emit_modified (modified)
> > > done
> > >
> > > lock indexer
> > > delete from eventstable where beinghandled = 1
> > > unlock indexer
> > >
> > > return (!stopping)
> > >
> > > }
> > >
> > >
> > > Here's a piece of IRC log between me and jamiecc about the proposal:
> > >
> > > pvanhoof ping jamiemcc
> > > pvanhoof same thing
> > > pvanhoof I'll make a pdf
> > > jamiemcc oh ok
> > > pvanhoof Sending
> > > pvanhoof ok
> > > pvanhoof so
> > > pvanhoof it's about the hitsadded, hitsremoved and hitsmodified signals
> > > for xesam
> > > pvanhoof What we have in mind is using a "events" table that is a journal
> > > for all creates, deletes and updates
> > > pvanhoof Periodically we will flush that table, each create (insert),
> > > update and each delete we add a record in that table
> > > pvanhoof We'll make sure the table is queryable in a similar fashion as
> > > how the Xesam query will execute
> > > pvanhoof In the periodical handler we'll for each live search check
> > > whether it got affected by the items in the events table
> > > pvanhoof In pseudo, the handler:
> > > jamiemcc sounds feasible
> > > pvanhoof gboolean periodic_handler (void data) {
> > > pvanhoof lock indexer
> > > pvanhoof update eventstable set beinghandled=1 where 1=1 (all items)
> > > pvanhoof unlock indexer
> > > pvanhoof foreach query in all live queries
> > > pvanhoof added, modified, removed = query.execute-on (eventstable)
> > > pvanhoof query.emit_added (added)
> > > pvanhoof query.emit_removed (removed)
> > > pvanhoof query.emit_modified (modified)
> > > pvanhoof done
> > > pvanhoof lock indexer
> > > pvanhoof delete from eventstable where beinghandled = 1
> > > pvanhoof unlock indexer
> > > pvanhoof }
> > > pvanhoof I've send you a diagram that you can look at as if it's a
> > > state-activity one, a ERD and a class diagram :) now how cool is that?? :)
> > > pvanhoof it's just three columns, although the ERD is quite simplistic of
> > > course
> > > jamiemcc yeah just go tit
> > > * fritschy ([EMAIL PROTECTED]) has left #tracker
> > > pvanhoof so, the current idea is to adapt those stored procedures into
> > > transactions that will also add this record to the "events" table
> > > * fritschy ([EMAIL PROTECTED]) has joined #tracker
> > > pvanhoof Which might not be sufficient, and we kinda lack the in-depth
> > > know-how of all the db handling of tracker
> > > pvanhoof So that's a first issue we want to discuss with you
> > > pvanhoof The other is stopping the indexing, restarting it (locking it,
> > > in the pseudo code): what you think about that
> > > jamiemcc ok I will need to think about it - I iwll probably reply later
> > > tonight and we can discuss tomorrow
> > > pvanhoof I adapted my initial proposal to have two short critical
> > > sections rather than letting the entire periodic handler be one critical
> > > section
> > > pvanhoof that way the lock is smaller
> > > jamiemcc the indexer will be seaparte process so will need to be locked
> > > via dbus signals
> > > pvanhoof by just adding a column to the events table
> > > pvanhoof yes but I guess we want any such locking to be short
> > > jamiemcc well yes
> > > pvanhoof then once the items that are to be handled are identified, we
> > > for each live-search check whether the live-search is affected
> > > pvanhoof and we perform the necessary hitsadded, hitsremoved and
> > > hitsmodified signals if needed
> > > pvanhoof if all is done, we simply purge the handled items from the
> > > events table
> > > jamiemcc the query results will be store din temp tables
> > > pvanhoof which is the second location where we want the indexer to be
> > > locked-out
> > > jamiemcc remember a query may be a cursor so wont include entire result
> > > set
> > > pvanhoof No okay, but that's something the check needs to worry about
> > > pvanhoof so ottela is working on a query for the live-search
> > > jamiemcc ok cool
> > > pvanhoof and if we only want to update if the client has the affected
> > > item visible, due to cursor-usage
> > > pvanhoof then i guess we'll somehow need to get that info into trackerd
> > > jamiemcc any reason we dont store whats change din memory rather than
> > > sqlite table?
> > > pvanhoof oh, that's abstract right now
> > > jamiemcc o
> > > jamiemcc ok
> > > pvanhoof "tracker's event table" can also be a hashtable for me ..
> > > jamiemcc yeah fine
> > > pvanhoof implementation detail
> > > pvanhoof since it doesn't need to be persistent ...
> > > pvanhoof difference is that either we use a memory table and still a
> > > transaction for the three stored procedures
> > > pvanhoof or we adapt code
> > > jamiemcc prefer hashtable as amount of data will be small
> > > jamiemcc can even be a list
> > > pvanhoof ok, your comments/ideas on this would of course be very useful
> > > btw
> > > jamiemcc yeah I will think about it more tonight and get back to you
> > > pvanhoof sounds great
> > > pvanhoof I'll make a mail about this to the mailing list? or I await your
> > > ideas tomorrow?
> > > pvanhoof I'll just wait for now
> > > jamiemcc you cna mail if you like
> > > jamiemcc I will reply to it
> > >
> > >
> > > _______________________________________________
> > > tracker-list mailing list
> > > [email protected]
> > > http://mail.gnome.org/mailman/listinfo/tracker-list
> > _______________________________________________
> > tracker-list mailing list
> > [email protected]
> > http://mail.gnome.org/mailman/listinfo/tracker-list
> _______________________________________________
> tracker-list mailing list
> [email protected]
> http://mail.gnome.org/mailman/listinfo/tracker-list
_______________________________________________
tracker-list mailing list
[email protected]
http://mail.gnome.org/mailman/listinfo/tracker-list
