[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #23 from tagwer...@innerjoin.org --- You're the first one that's happened to :-) So, I blindly followed the "How to create useful crash reports" and was able to run coredumpctl and get a list of crash dumps, with their "process numbers', the most recent at the bottom. Then coredumpctl gdb and after lots of stuff had scrolled off the screen bt This was in "Neon Testing". For "Neon" I first had to install systemd-coredump reboot and then trigger a crash to get the same thing. ... Every day a learning day. -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #22 from Peter Wibberley --- (In reply to tagwerk19 from comment #21) > I think we do need someone who knows to come in here... > > A couple of observations might help though, I see lines like: > > ioctl(14, FIONREAD, [32]) = 0 > read(14, "\3\0\0\0\0\2\0\0\0\0\0\0\20\0\0\0testfile2.txt\0\0\0", 32) = 32 > > appear when I write to "testfile2.txt". Could you be logging to a file and > that file being indexed? > > I've also looked at: > > > https://community.kde.org/Guidelines_and_HOWTOs/Debugging/ > How_to_create_useful_crash_reports > > It's being actively updated. Maybe it can give a pointer or two. Doh! The file, "210224Baloo_file", which is referenced 42,835 times is the file to which I was redirecting the strace output! I will exclude this from indexing and see if I get more interesting (and if not more interesting, at least less) output. Many thanks. -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #21 from tagwer...@innerjoin.org --- I think we do need someone who knows to come in here... A couple of observations might help though, I see lines like: ioctl(14, FIONREAD, [32]) = 0 read(14, "\3\0\0\0\0\2\0\0\0\0\0\0\20\0\0\0testfile2.txt\0\0\0", 32) = 32 appear when I write to "testfile2.txt". Could you be logging to a file and that file being indexed? I've also looked at: https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports It's being actively updated. Maybe it can give a pointer or two. -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204
--- Comment #20 from Peter Wibberley ---
(In reply to tagwerk19 from comment #19)
> I would look though at what baloo_file had started to index when it started
> to log errors, maybe it's possible to see something strange...
>
> You can exclude folders from indexing, either through system settings /
> search or by editing the .config/baloofilerc file. Add a line:
>
> exclude folders[$e]=$HOME/dontindexme
>
> and you can skip index that folder (and its subfolders). Maybe you can pin
> down a folder that is giving trouble.
>
> Alas I cannot help with the SIGSEGV, need someone with experience of core
> dumps...
Tagewerk19,
Not a lot of obvious errors. However, the last 130,000 lines of the 210,000
lines from strace (except for the final line, "+++ killed by SIGSEGV +++")
consist of 43,000 repeats of the three lines,
ioctl(14, FIONREAD, [48]) = 0
read(14, "\1\0\0\0\2\0\0\0\0\0\0\0 \0\0\000210224Baloo_file"..., 48) = 48
poll([{fd=3, events=POLLIN}, {fd=9, events=POLLIN}, {fd=10, events=POLLPRI},
{fd=11, events=POLLIN}, {fd=14, events=POLLIN}], 5, 10) = 1 ([{fd=14,
revents=POLLIN}])
That seems a little strange.
--
You are receiving this mail because:
You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #19 from tagwer...@innerjoin.org --- I would look though at what baloo_file had started to index when it started to log errors, maybe it's possible to see something strange... You can exclude folders from indexing, either through system settings / search or by editing the .config/baloofilerc file. Add a line: exclude folders[$e]=$HOME/dontindexme and you can skip index that folder (and its subfolders). Maybe you can pin down a folder that is giving trouble. Alas I cannot help with the SIGSEGV, need someone with experience of core dumps... -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #18 from Peter Wibberley --- (In reply to tagwerk19 from comment #17) Tagwerk19, Thank you for the suggestion. No need to kill the process, as it's dying anyway. You're not wrong about shedloads: over 20MB and 200,000 lines collected in just a few seconds. There's 11973 instances or 'inotify_add_watch', with an 'inotify_init1(IN_CLOEXEC) and an 'inotify_init()' very early on. There appears to be several phases with different clusters of error messages. And it ends with "+++ killed by SIGSEGV +++" and "Segmentation fault", which would seem to explain why it's not running for long. I don't want to burden you with megabytes of probably superfluous detail, but do you have any suggestions about how I should proceed? Thank you again, and regards P > I think that baloo_file watches directories (mostly) and not files. > > I get this info from a trick of killing the baloo_file process and starting > from the command line under strace. So... > > ps ax | grep baloo_file > kill ... > strace baloo_file > > This gives shedloads of information; details of every system call. You can > however see the "inotify_add_watch' calls for each individual directory and > their return values - which can either include a steadily increasing count > or the 'changes will be ignored' message. You should be able to see a bit > more about what's happening > > I haven't worked how to see what info is being passed to the > baloo_file_extractor process and what it is doing. I also haven't discovered > where baloo logs its activities, that would be good to know! -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #17 from tagwer...@innerjoin.org --- I think that baloo_file watches directories (mostly) and not files. I get this info from a trick of killing the baloo_file process and starting from the command line under strace. So... ps ax | grep baloo_file kill ... strace baloo_file This gives shedloads of information; details of every system call. You can however see the "inotify_add_watch' calls for each individual directory and their return values - which can either include a steadily increasing count or the 'changes will be ignored' message. You should be able to see a bit more about what's happening I haven't worked how to see what info is being passed to the baloo_file_extractor process and what it is doing. I also haven't discovered where baloo logs its activities, that would be good to know! -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #16 from Peter Wibberley --- (In reply to Harald Sitter from comment #15) > I can't really say anything about that I know nothing about baloo. The > inotify cap needs fixing one way or another, but if increasing the cap > doesn't help you I guess your problem has nothing to do with it. Understood. Thank you. -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #15 from Harald Sitter --- I can't really say anything about that I know nothing about baloo. The inotify cap needs fixing one way or another, but if increasing the cap doesn't help you I guess your problem has nothing to do with it. -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #14 from Peter Wibberley --- (In reply to Harald Sitter from comment #13) The thing I don't understand is that, up until 17 December, I would have had max_user_watches set at 8192 with the number of files not being significantly less than the current number, and it was working OK. Now, even a 16-fold increase in the value doesn't seem to make any difference. Hence I was wondering whether the problem could have been related to updates. >From my perspective, my problem is getting more diagnostic information. Thanks and regards > (In reply to Jonathan Riddell from comment #10) > > I wonder if this should be built into baloo rather than being distro > > specific > > It will sort itself with kernel 5.11. Well, to a degree. > > If my maths is correct linux 5.11 sets the watch limit to > 32G ~> 260k > 16G ~> 130k > 8G ~> 65k > 4G ~> 30k > > So this can conceivably still be easily exhausted. e.g. the original > reporter would need to have >16G or manually set the watch limit to track > their 200k files. > > As I've said, the UX is fairly weak since we don't inform the user when they > run out of inotify capacity. KDirWatch has the similar problem really > (albeit more with instances than watches). -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 --- Comment #13 from Harald Sitter --- (In reply to Jonathan Riddell from comment #10) > I wonder if this should be built into baloo rather than being distro specific It will sort itself with kernel 5.11. Well, to a degree. If my maths is correct linux 5.11 sets the watch limit to 32G ~> 260k 16G ~> 130k 8G ~> 65k 4G ~> 30k So this can conceivably still be easily exhausted. e.g. the original reporter would need to have >16G or manually set the watch limit to track their 200k files. As I've said, the UX is fairly weak since we don't inform the user when they run out of inotify capacity. KDirWatch has the similar problem really (albeit more with instances than watches). -- You are receiving this mail because: You are watching all bug changes.
[neon] [Bug 433204] Baloo File Indexer will not keep running. Changing fs.inotify.max_user_watches not having any effect!
https://bugs.kde.org/show_bug.cgi?id=433204 Peter Wibberley changed: What|Removed |Added Summary|fs.inotify.max_user_watches |Baloo File Indexer will not |is set to 8192, which is|keep running. Changing |too low for Baloo to work |fs.inotify.max_user_watches |properly|not having any effect! -- You are receiving this mail because: You are watching all bug changes.
