On Mon, 2007-09-24 at 01:45 +0200, Laurent Aguerreche wrote: > Le dimanche 23 septembre 2007 à 14:12 +0100, jamie a écrit : > > Hi All, > > > > the next version of tracker is sitting in svn (there are still a few bug > > fixes to apply but should be stable for the most part) > > > > this version implements index merging which dramatically improves speed > > and scalability of indexing (as well as eliminating fragmentation and > > disk IO issues0. 250 MB of linux kernel source was indexed in less than > > 8 minutes so it should index large amounts in very short time) > > > > because of that there are some major changes in the way things ar e > > handled so it really needs a lot of thorough testing before release > > > > we use by default a 16MB word cache as each flush produces a separate > > mini-index which are later all merged together to form the main index. > > Search results will only be available after the first flush and no new > > search results will be added from subsequent flushes until merging is > > complete > > > > so please can you test the following: > > > > 1) index as normal (no need to use --reindex - it should reindex itself > > auotmatically) > > My previous experience with trackerd going around 180Mo tends to say > that there could be a problem there... More testing needed. > > > 2) keep an eye on memory usage - there may be a few leaks but it should > > plateau around 30mb > > It is more 50Mo or something... Did you introduce Mono, Java or Python > somewhere in trackerd? ;-)
im fixing this - email indexing causes the most memory usage! > > > 3) after index test makking changes to files and moving stuff around > > I moved ~440Mo of code sources/html from one directory to another and > trackerd stayed stuck with CPU 100% several seconds (I had to kill > it...). thats bound to be slow moving around that many files! if it stayed stuck for a few secs I would say its ok > > > 4) do source checkouts so that lots of files are overwritten in one go > > and pay attention to speed/slowdown of system > > It seems to be better than before but I still have emacs freezing > sometimes when I want to save a file. the freezes should be less than a few secs? also can you tell if freezes occur when indexing files or emails? (i suspect emails) or are confined to merging (where heaviest disk IO occurs) > > > 5) create lots of new files and make sure its fast and new results show > > up > > New files are not shown very fast... thats ok so long as they are shown and indexing of these new files is fast > > > > > pls also run with -f in gdb ss thta you can get me backtraces for > > non-crash errors (as well as crashed ones!) > > > > > > if all goes well I will release tomorrow... > > I think we should not! well we will see tomorrow night! _______________________________________________ tracker-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/tracker-list
