https://bugs.kde.org/show_bug.cgi?id=500665
Stefan Brüns <[email protected]> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |[email protected] | |e --- Comment #71 from Stefan Brüns <[email protected]> --- The memory limit as currently applied is totally broken. It does not help at all, but causes significant issues. It accounts for the memory of the main baloo-file daemon, and the extractor process. The baloo-file daemon uses most of its memory for the inotify watches, notably the associated pathes. The inotify events only provides the watch descriptor id, but not the full path (only names of files relative to the directory watch). If you have several 100k of watched directories (the exhausted limit suggests this is the case), and each watched path takes ~100 bytes, you already have some 10s of MByte, maybe more. An idle daemon with a low count of watched directories requires ~10 MByte of RESident memory. Everything on top are required data structures associated with additional watched directories. The extractor process needs memory for loading the file (if the whole file is loaded into memory, or just small fragments, depends on the extractor plugin), and for processing it. It may need significantly more memory than the file size suggests, as files may be compressed, and data may be stored in compact representation, while processing requires a completely different representation. If an extractor plugin requires some memory for a short amount of time, it is best to give it the memory, let it finish, and release the memory again. The amount of memory required temporarily is dependent on the data and almost impossible to predict. When you limit the slice to just 512MByte, you are shooting into your own foot. The process will be actively stalled, and instead of releasing the memory it will hold onto it for an extended period of time. Recent versions of baloo (since KF 6.23) split transactions which use a lot of memory, so in this case each indexed file goes into a separate transaction. But there is a lower limit for each file, and this limit can be significantly larger than 500 MByte. -- You are receiving this mail because: You are watching all bug changes.
