https://bugs.kde.org/show_bug.cgi?id=411291
Bug ID: 411291
Summary: filenamesearch is very slow
Product: kio-extras
Version: unspecified
Platform: Other
OS: Linux
Status: REPORTED
Severity: normal
Priority: NOR
Component: default
Assignee: [email protected]
Reporter: [email protected]
Target Milestone: ---
Hi,
I was trying to find a file and usually do: "find /home/mark -iname
"*part_of_the_name*" which returns results within a few seconds. Mind you, my
home folder is (when testing this) 381GB large and containing 1095284 files
(find /home/mark -type f | wc -l). That count also return within about a
second.
That's just to give an impression of number of files that have to be searched
through to find that file i was looking for.
It takes about 2 seconds for the find command to find the files and be done.
It takes about 25 seconds for Dolphin to do the exact same thing (i'm not
searching through content, only filenames). I verified that dolphin is in fact
using this KIO slave :) (as it could be baloo too)
Now i was looking through the code of filenamesearch to figure out why it would
be this slow. Obviously, there is some overhead when using KIO::ListDir or
KCoreDirLister as it goes over the socket connection and does serialization. So
i fully expect it to be (a bit) slower. As those paths have been massively
optimized in the past (partly by me too) i kinda have a feeling what can be
expected as slowdown. 2 or 3x slower can be expected, ~13x like i'm observing
here is not the expected behavior and smells like a severe deficiency in this
code path.
My first gut feeling was to use KIO::listRecursive instead of manually
implementing recursive folder iteration as is done now in filenamesearch. I was
guessing that to be (a lot) faster as the KIO::listRecursive code simply has
seen tons of optimizations over the past, well, decades. That turned out right,
the time is then reduced to about ~8 seconds as opposed to the ~25 it was
before. However, this introduces a much more complicated issue. You do not want
the caching logic of KIO::listRecursive when searching for files. Sadly, there
is no way to tell KIO::listRecursive to not use caching hence going the
KCoreDirLister isn't such a bad idea (that's how it's done now).
But there is more needed to improve searching.
Right now if you search for an entry and it finds one within 300ms (mind you,
KIO does 300ms batching of entries) but then don't find another entry for - say
- 24 seconds, then you will not get any listEntry update for that time. Thus no
GUI entry for an item that might be found all along. You will get the entry as
soon as there is a new match (triggering the 300ms interval) or till finished()
is called. This can be improved, but i'm not yet sure what would be wise.
Leaving it open here as a braindump.
Cheers,
Mark
--
You are receiving this mail because:
You are watching all bug changes.
