On Fri, 2008-08-08 at 12:49 +0100, Martyn Russell wrote: > Laurent Aguerreche wrote: > > Hello, > > Hi :) > > > Le jeudi 07 août 2008 à 16:38 +0100, Martyn Russell a écrit : > >> Jamie, > >> > >> I think we are ready for merge if you are happy with the current state > >> of the indexer-split branch. > >> > >> If you find anything you think needs fixing before the merge. Let us know. > > > > I'm experiencing two issues with the indexer-split branch : > > - if I terminate trackerd with "kill" (to send a SIGTERM signal) sent to > > it, I see tracker-indexer still running. I can't kill tracker-indexer > > with "kill", I have to use "kill -9" > > - tracker kills the performances of my hard-drive while it starts: GUIs > > of my applications get frozen! > > > > Do you intend to fix these issues before the merge (notably the second > > one...)? > > We have been fixing a few things up today that I noticed last night. > Issues include: > > 1.) Index being too big that indexing is slowed right down:
this is why trunk used index merging so it would flush to separate indexes and then merge them into one when finished. By experimentation 16MB is a good size for an index hence trunk flushed in 16mb lumps doing lots of small writes to a large index causes real IO problems with ext/3 (ext/4 fixes this) - each word being indexed is a separate seek and write on the index so you could easily have 100,000+ small updates being applied. Also when words have hits added the hashtable has to relocate them to free space which also causes fragmentation - the optimize qdbm call recovers this by copying the index into a new one once indexing is finished as you can see index merging eliminates this perfomance overhead or the need to call optimize as the final merged index will have no fragmentation jamie _______________________________________________ tracker-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/tracker-list
