Bug#927168: fails to detect certain ruby processes linked against libssh2
Hi, On Tue, 16 Apr 2019, Antoine Beaupré wrote: I wonder which Debian release is in use. The system information section looks like testing but needrestart 2.11 is from stretch. Sorry, I didn't file the bug report from the affected machine. :/ I hope that's alright, I can re-extract the rest of the data as required if that's needed. No worries! (You may also give needrestart from BPO a try since it contains many fixes missing in stable.) The recent libssh2 upgrade wasn't correctly flagged by needrestart: some proceses were marked as need a restart, but others, specifically those running under the ruby interpreter, were not. Here's what our homegrown system has detected for those: root@gitlab-01:/etc/nagios/nrpe.d# /usr/lib/nagios/plugins/dsa-check-libs --verbose 2>&1 | grep -a -v /log/ Running /usr/bin/lsof -F0 -n [snip] Needrestart finds nothing of the sort: Using lsof alone does not tell if the library is mapped executable (read access on deleted files is ignored by needrestart intentional). Can you please check if those files are mapped executable in /proc/$PID/maps? Unfortunately, the box has since then been rebooted. You could simulate an update using `apt-get install --reinstall libssh2-1`. This looks OK for me. The PID 883 uses a old libssh2 but belongs to a user session of uid 1504. The ruby instances seems not to have libssh2 mapped executable - so they are not reported. Interesting. In which circumstance could a process have a library loaded but not mappex executable? That seems like a paradox. I don't know.A There are also writable mapped libraries: $ cat /proc/$$/maps|grep -v 'xp ' 070-00703000 r--p 0010 103:01 1046603 /bin/bash 00703000-0070c000 rw-p 00103000 103:01 1046603 /bin/bash 0070c000-00716000 rw-p 00:00 0 01e74000-02051000 rw-p 00:00 0 [heap] 7fbbd7dfa000-7fbbd7ff9000 ---p 3000 103:01 791389 /usr/lib/x86_64-linux-gnu/samba/libwinbind-client.so.0 7fbbd7ff9000-7fbbd7ffa000 r--p 2000 103:01 791389 /usr/lib/x86_64-linux-gnu/samba/libwinbind-client.so.0 7fbbd7ffa000-7fbbd7ffb000 rw-p 3000 103:01 791389 /usr/lib/x86_64-linux-gnu/samba/libwinbind-client.so.0 [..] It also seem to fail to find the source code for those files... The "homegrown" tool is actually the one used by DSA to check for upgrades through nagios: Needrestart tries to get the source file from the cmdline which may fail and break the interpreter heuristic which looks for outdated source files. The library detection is done before and not affected by the missing source files. Understood. Anyways, dsa-check-libs doesn't notice anything specifically about ruby source files here anyways, as far as I understand it. I don't think any other restart-detection-tool (checkrestart, whatmaps, ...) does look at script files run by interpreters like ruby. It is just a heuristic in needrestart which may fail since there is no easy way to get the source files and all dependencies. So this might just be false positives on our side. Is that fundamentally your conclusion as well? In this case I guess we can close this until I have more concrete evidence... I would think so :-) Regards, Thomas -- :: WWW:https://fiasko-nw.net/~thomas/ :: ::: GnuPG: 0x49D0C2C3 mailto:tho...@fiasko-nw.net ::: :: flickr: https://www.flickr.com/photos/laugufe/ ::
Bug#927168: fails to detect certain ruby processes linked against libssh2
On 2019-04-15 23:26:02, tho...@fiasko-nw.net wrote: > Hi Antoine, > > > On Mon, 15 Apr 2019, Antoine Beaupre wrote: > >> Package: needrestart >> Version: 2.11-3+deb9u1 > > I wonder which Debian release is in use. The system information section > looks like testing but needrestart 2.11 is from stretch. Sorry, I didn't file the bug report from the affected machine. :/ I hope that's alright, I can re-extract the rest of the data as required if that's needed. >> The recent libssh2 upgrade wasn't correctly flagged by needrestart: >> some proceses were marked as need a restart, but others, specifically >> those running under the ruby interpreter, were not. Here's what our >> homegrown system has detected for those: >> >> root@gitlab-01:/etc/nagios/nrpe.d# /usr/lib/nagios/plugins/dsa-check-libs >> --verbose 2>&1 | grep -a -v /log/ >> Running /usr/bin/lsof -F0 -n > [snip] >> Needrestart finds nothing of the sort: > > Using lsof alone does not tell if the library is mapped executable > (read access on deleted files is ignored by needrestart intentional). Can > you please check if those files are mapped executable in /proc/$PID/maps? Unfortunately, the box has since then been rebooted. >> root@gitlab-01:/etc/nagios/nrpe.d# needrestart -v >> [main] eval /etc/needrestart/needrestart.conf >> [main] needrestart v3.3 >> [main] running in root mode >> [Core] Using UI 'NeedRestart::UI::stdio'... >> [main] systemd detected >> [Core] #843 is a NeedRestart::Interp::Python >> [Python] #843: source=/usr/bin/fail2ban-server >> [Core] #882 is a NeedRestart::Interp::Ruby >> [Ruby] #882: >> source=/srv/dip.torproject.org/home/gitlab/vendor/bundle/ruby/2.3.0/bin/mail_room >> [main] #883 uses deleted /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 >> [main] #883 is a child of #745 >> [Core] #31644 is a NeedRestart::Interp::Ruby >> [Ruby] #31644: source file '' not found, skipping >> [Ruby] #31644: reduced ARGV: >> [Core] #31669 is a NeedRestart::Interp::Ruby >> [Ruby] #31669: source file '' not found, skipping >> [Ruby] #31669: reduced ARGV: >> [Core] #31671 is a NeedRestart::Interp::Ruby >> [Ruby] #31671: source file '' not found, skipping >> [Ruby] #31671: reduced ARGV: >> [Core] #31675 is a NeedRestart::Interp::Ruby >> [Ruby] #31675: source file '' not found, skipping >> [Ruby] #31675: reduced ARGV: >> [Core] #31677 is a NeedRestart::Interp::Ruby >> [Ruby] #31677: source file '' not found, skipping >> [Ruby] #31677: reduced ARGV: >> [main] #745 exe => /lib/systemd/systemd >> [main] #745 part of user manager service: uid=1504 > > This looks OK for me. The PID 883 uses a old libssh2 but belongs to a user > session of uid 1504. The ruby instances seems not to have libssh2 mapped > executable - so they are not reported. Interesting. In which circumstance could a process have a library loaded but not mappex executable? That seems like a paradox. >> It also seem to fail to find the source code for those files... The >> "homegrown" tool is actually the one used by DSA to check for upgrades >> through nagios: > > Needrestart tries to get the source file from the cmdline which may fail > and break the interpreter heuristic which looks for outdated source files. > The library detection is done before and not affected by the missing > source files. Understood. Anyways, dsa-check-libs doesn't notice anything specifically about ruby source files here anyways, as far as I understand it. So this might just be false positives on our side. Is that fundamentally your conclusion as well? In this case I guess we can close this until I have more concrete evidence... Thanks for the prompt reply! A. -- La politique est l'art d'empêcher les gens de se mêler de ce qui les regarde - Paul Valéry
Bug#927168: fails to detect certain ruby processes linked against libssh2
Hi Antoine, On Mon, 15 Apr 2019, Antoine Beaupre wrote: Package: needrestart Version: 2.11-3+deb9u1 I wonder which Debian release is in use. The system information section looks like testing but needrestart 2.11 is from stretch. The recent libssh2 upgrade wasn't correctly flagged by needrestart: some proceses were marked as need a restart, but others, specifically those running under the ruby interpreter, were not. Here's what our homegrown system has detected for those: root@gitlab-01:/etc/nagios/nrpe.d# /usr/lib/nagios/plugins/dsa-check-libs --verbose 2>&1 | grep -a -v /log/ Running /usr/bin/lsof -F0 -n [snip] Needrestart finds nothing of the sort: Using lsof alone does not tell if the library is mapped executable (read access on deleted files is ignored by needrestart intentional). Can you please check if those files are mapped executable in /proc/$PID/maps? root@gitlab-01:/etc/nagios/nrpe.d# needrestart -v [main] eval /etc/needrestart/needrestart.conf [main] needrestart v3.3 [main] running in root mode [Core] Using UI 'NeedRestart::UI::stdio'... [main] systemd detected [Core] #843 is a NeedRestart::Interp::Python [Python] #843: source=/usr/bin/fail2ban-server [Core] #882 is a NeedRestart::Interp::Ruby [Ruby] #882: source=/srv/dip.torproject.org/home/gitlab/vendor/bundle/ruby/2.3.0/bin/mail_room [main] #883 uses deleted /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 [main] #883 is a child of #745 [Core] #31644 is a NeedRestart::Interp::Ruby [Ruby] #31644: source file '' not found, skipping [Ruby] #31644: reduced ARGV: [Core] #31669 is a NeedRestart::Interp::Ruby [Ruby] #31669: source file '' not found, skipping [Ruby] #31669: reduced ARGV: [Core] #31671 is a NeedRestart::Interp::Ruby [Ruby] #31671: source file '' not found, skipping [Ruby] #31671: reduced ARGV: [Core] #31675 is a NeedRestart::Interp::Ruby [Ruby] #31675: source file '' not found, skipping [Ruby] #31675: reduced ARGV: [Core] #31677 is a NeedRestart::Interp::Ruby [Ruby] #31677: source file '' not found, skipping [Ruby] #31677: reduced ARGV: [main] #745 exe => /lib/systemd/systemd [main] #745 part of user manager service: uid=1504 This looks OK for me. The PID 883 uses a old libssh2 but belongs to a user session of uid 1504. The ruby instances seems not to have libssh2 mapped executable - so they are not reported. It also seem to fail to find the source code for those files... The "homegrown" tool is actually the one used by DSA to check for upgrades through nagios: Needrestart tries to get the source file from the cmdline which may fail and break the interpreter heuristic which looks for outdated source files. The library detection is done before and not affected by the missing source files. HTH, Thomas https://salsa.debian.org/dsa-team/mirror/dsa-nagios/blob/master/dsa-nagios-checks/checks/dsa-check-libs It uses lsof to look at opened files... -- Package-specific info: needrestart output: checkrestart output: -- System Information: Debian Release: buster/sid APT prefers testing APT policy: (500, 'testing'), (1, 'experimental'), (1, 'unstable') Architecture: amd64 (x86_64) Kernel: Linux 4.19.0-4-amd64 (SMP w/4 CPU cores) Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=fr_CA.UTF-8, LC_CTYPE=fr_CA.UTF-8 (charmap=UTF-8), LANGUAGE=fr_CA.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages needrestart depends on: ii binutils 2.31.1-15 ii dpkg 1.19.6 ii gettext-base 0.19.8.1-9 ii libintl-perl 1.26-2 ii libmodule-find-perl0.13-1 ii libmodule-scandeps-perl1.27-1 ii libproc-processtable-perl 0.56-1 ii libsort-naturally-perl 1.03-2 ii libterm-readkey-perl 2.38-1 ii perl 5.28.1-6 ii xz-utils 5.2.4-1 Versions of packages needrestart recommends: ii libpam-systemd 241-3 Versions of packages needrestart suggests: ii iucode-tool2.3.1-1 ii libnotify-bin 0.7.7-4 -- debconf-show failed -- :: WWW:https://fiasko-nw.net/~thomas/ :: ::: GnuPG: 0x49D0C2C3 mailto:tho...@fiasko-nw.net ::: :: flickr: https://www.flickr.com/photos/laugufe/ ::
Bug#927168: fails to detect certain ruby processes linked against libssh2
Package: needrestart Version: 2.11-3+deb9u1 Severity: normal The recent libssh2 upgrade wasn't correctly flagged by needrestart: some proceses were marked as need a restart, but others, specifically those running under the ruby interpreter, were not. Here's what our homegrown system has detected for those: root@gitlab-01:/etc/nagios/nrpe.d# /usr/lib/nagios/plugins/dsa-check-libs --verbose 2>&1 | grep -a -v /log/ Running /usr/bin/lsof -F0 -n adding ruby2.3(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding ruby-timer-thr(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding default-executo(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding resolver-execut(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding grpc_global_tim(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding util.rb:23(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 adding grpc_global_tim(883) because of [/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1]: fDELa l tREGD0x801i135145n/usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 The following processes have libs linked that were upgraded: git: default-executo (883), grpc_global_tim (883), resolver-execut (883), ruby-timer-thr (883), ruby2.3 (883), util.rb:23 (883) Needrestart finds nothing of the sort: root@gitlab-01:/etc/nagios/nrpe.d# needrestart -v [main] eval /etc/needrestart/needrestart.conf [main] needrestart v3.3 [main] running in root mode [Core] Using UI 'NeedRestart::UI::stdio'... [main] systemd detected [Core] #843 is a NeedRestart::Interp::Python [Python] #843: source=/usr/bin/fail2ban-server [Core] #882 is a NeedRestart::Interp::Ruby [Ruby] #882: source=/srv/dip.torproject.org/home/gitlab/vendor/bundle/ruby/2.3.0/bin/mail_room [main] #883 uses deleted /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1 [main] #883 is a child of #745 [Core] #31644 is a NeedRestart::Interp::Ruby [Ruby] #31644: source file '' not found, skipping [Ruby] #31644: reduced ARGV: [Core] #31669 is a NeedRestart::Interp::Ruby [Ruby] #31669: source file '' not found, skipping [Ruby] #31669: reduced ARGV: [Core] #31671 is a NeedRestart::Interp::Ruby [Ruby] #31671: source file '' not found, skipping [Ruby] #31671: reduced ARGV: [Core] #31675 is a NeedRestart::Interp::Ruby [Ruby] #31675: source file '' not found, skipping [Ruby] #31675: reduced ARGV: [Core] #31677 is a NeedRestart::Interp::Ruby [Ruby] #31677: source file '' not found, skipping [Ruby] #31677: reduced ARGV: [main] #745 exe => /lib/systemd/systemd [main] #745 part of user manager service: uid=1504 Failed to load NeedRestart::uCode::Intel: [uCode/Intel] iucode-tool not available! [ucode] no supported processor microcode detection [Kernel] Linux: kernel release 4.9.0-8-amd64, kernel version #1 SMP Debian 4.9.144-3.1 (2019-02-19) [Kernel/Linux] /boot/vmlinuz-4.9.0-8-amd64 => 4.9.0-8-amd64
