On Tue, 2008-02-19 at 17:40 +0000, Jamie Lokier wrote: > Jamie McCracken wrote: > > trackerd is niced +19 so cpu will nose dive to 1% if another app uses > > cpu. Indexing needs lots of cpu which no indexer can escape from. > > Switching to another will not change this > > > (unless you consider slowing indexing to such a crawl that it takes > > days to index which you can do already in tracker-prefs by setting > > the throttle slider) > > Does this work now? It had no noticable effect when I tried it, and > certainly didn't stop the I/O overloading problems. At all.
As I said that cuts cpu usage and not I/O ones. Its mainly intended to stop laptops getting too hot and annoying fans coming on due to constantly high cpu usage. > > > Only slow downs Im aware of is during index merging and these are 100% > > due to I/O issues > > Don't forget scanning files after reboot, just to check they haven't > changed. That takes a few minutes of heavy disk seeking, and is very > annoying after booting a laptop to do something quickly. the scan is done only on directories in order to place watches on them. until the kernel supplies us with recursive folder watching all indexers have to do this. We also check to see if the directory mtime has changed but that adds no overhead to above. I have a cunning plan to solve this but it must wait for another day... > > > during index merging I/O can become overloaded which can starve other > > processes of disk due to bugs in kernel (lots of small writes kills > > performance on ext3 but hoepfully ext4 will have pre allocation to > > alleviate this) > > You shouldn't be doing lots of small writes with fsyncs - that's > application silliness. we dont we batch 5000 small writes then fsync. if perform fast merges is enable din tracker-prefs then we dont do any fsync > > I've explained how to avoid them at the db level but as far as I could > tell you weren't interested. To be fair, index merging is a smart (if > suboptimal) alternative - but you _really_ shouldn't have small writes > with fsync when merging indexes. That's algorithms from the 70s, long > ago solved. (See Knuth). sure but that logic must be applied to sqlite/qdbm or other indexer backend. Theres nothing we can do in tracker source. If you or someone else writes us a smart indexer backend we would be happy to use it. (indexer backend independence will be in the next major release of tracker 0.7) > > Imho, ext4 preallocation and delayed allocation will not make small > single-file writes with fsyncing much faster, if at all. But I would > like to be pleasantly surprised. > wee you can see today - mount ~/.cache/tracker on XFS (which has said pre allocation) and turn fast merges on in tracker-prefs. Index merges are about 100x faster and cause no perceptible I/O overload AFAICT. pre allocation is a highly desirable feature for us.Also note that ext3 is the slowest to write to so some of above overhead might be due to other reasons specific to ext3. > > In any event, next version of tracker (due this week) now pauses > > automatically during merges whenever the user moves the mouse or > > presses a key/button so there should not be any perceptible slow downs. > > Oh boy. Talk about krufty hacks! 'Tis a good idea. What about > watching long TV/animations in Totem and other video players. Is that > affected? It would be a good idea to index during that, provided I/O > + CPU isn't affected. > next version of Tracker will pause when: 1) user activity via mouse/keyboard is detected during merge (must be running tracker-applet as trackerd does not depend on X) 2) Same files change frequently - these are temporarily blacklisted and indexed after an hour or on next restart of tracker 3) Other apps write to disk in any watched directory (current version of tracker already has this) In the future we could get the applet to monitor /proc/diskstats to manage disk IO load better and pause tracker where necessary jamie -- Tracker should not be enabled by default until it doesn't clobber everything https://bugs.launchpad.net/bugs/132741 You received this bug notification because you are a member of Ubuntu Bugs, which is the bug contact for Ubuntu. -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
