[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
** Tags removed: server-todo -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in PAM: New Status in landscape-client package in Ubuntu: Fix Released Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: Fix Released Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: Fix Released Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/pam/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
** Changed in: pam Status: Unknown => New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in PAM: New Status in landscape-client package in Ubuntu: Fix Released Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: Fix Released Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: Fix Released Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/pam/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
FYI: Filed upstream at https://github.com/linux-pam/linux-pam/issues/452 ** Bug watch added: github.com/linux-pam/linux-pam/issues #452 https://github.com/linux-pam/linux-pam/issues/452 ** Also affects: pam via https://github.com/linux-pam/linux-pam/issues/452 Importance: Unknown Status: Unknown -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in PAM: Unknown Status in landscape-client package in Ubuntu: Fix Released Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: Fix Released Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: Fix Released Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/pam/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
Overall the majority of this is now fixed and mitigated with the combination of: ubuntu-release-upgrader | 1:22.04.8 | jammy | source landscape-client | 19.12-0ubuntu13| jammy | source, amd64, arm64, armhf, ppc64el, riscv64, s390x update-notifier | 3.192.54 | jammy | source, amd64, arm64, armhf, ppc64el, riscv64, s390x There is still the IMHO valid feature request to pam_motd to not run at all in non-interactive sessions which I'll need to file upstream. But already in a system with these updates: ubuntu@login-jammy:~$ dpkg -l ubuntu-release-upgrader-core landscape-common update-notifier-common Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++--===--=== ii landscape-common 19.12-0ubuntu13 amd64Landscape administration system client - Common files ii ubuntu-release-upgrader-core 1:22.04.8 all manage release upgrades ii update-notifier-common 3.192.54all Files shared between update-notifier and other packages I now get reasonable results. Down from 70-80 seconds to ~20-25 => almost down to 1/4 of the time. At the same time the system is ~16% less busy, so other things running won't stall it that much either and vice versa. What is left looks as in the test sessions. This now mostly comes down to the fact that logging in for every command will in general have overhead to spawn the session. For another gain pam_motd can be disabled as shown above, but that does not reduce it to zero overhead - so as explained any mutli-command submitting solution should still - even with the fix - try to use one login for all of them. # Overhead Command # ... # 32.50% swapper 26.67% sshd 3.53% dbus-daemon 3.37% systemd 2.36% run-parts 2.02% systemd-logind 1.87% find 1.85% gdbus 1.48% cat 1.47% update-motd-fsc 1.22% 50-motd-news 1.17% awk 1.15% systemd-journal 1.11% grep 1.10% bash 1.05% uname 0.98% 00-header 0.93% 91-release-upgr 0.92% 97-overlayroot 0.81% 90-updates-avai 0.80% date 0.72% cut 0.68% 50-landscape-sy 0.62% env 0.59% ksoftirqd/0 0.58% 95-hwe-eol 0.53% stat 0.51% id -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: Fix Released Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: Fix Released Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: Fix Released Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
This bug was fixed in the package update-notifier - 3.192.54 --- update-notifier (3.192.54) jammy; urgency=medium * d/95-hwe-eol: do not update eol data more than once per day (LP: #1893716) -- Christian Ehrhardt Wed, 30 Mar 2022 12:20:47 +0200 ** Changed in: update-notifier (Ubuntu) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: Fix Released Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: Fix Released Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: Fix Released Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
This bug was fixed in the package landscape-client - 19.12-0ubuntu13 --- landscape-client (19.12-0ubuntu13) jammy; urgency=medium * d/landscape-sysinfo.wrapper, d/landscape-common.postrm: avoid too frequent expensive operations (LP: #1893716) - use a cache file and refresh it only once per minute - use a single printf to format output - if the former info was useful and we'd replace it with "sorry, load to high" skip that update - remove the cache file on purge -- Christian Ehrhardt Wed, 30 Mar 2022 12:32:38 +0200 ** Changed in: landscape-client (Ubuntu) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: Fix Released Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: Fix Released Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: In Progress Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
This bug was fixed in the package ubuntu-release-upgrader - 1:22.04.8 --- ubuntu-release-upgrader (1:22.04.8) jammy; urgency=medium [ Brian Murray ] * Update the firefox deb2snap entry so that it includes Ubuntu Studio. * Modify the ubuntu-release-upgrader apport package hook to also gather recent messages in journalctl and a list of crashes in /var/crash as one of those may have negatively affected the upgrade process. [ Christian Ehrhardt ] * d/91-release-upgrade: try to use the less expensive /etc/lsb-release to check if we are in a devel release (LP: #1893716) -- Brian Murray Thu, 31 Mar 2022 14:23:13 -0700 ** Changed in: ubuntu-release-upgrader (Ubuntu) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: In Progress Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: Fix Released Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: In Progress Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
** Merge proposal linked: https://code.launchpad.net/~paelzer/update-notifier/+git/update-notifier/+merge/418122 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: In Progress Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: In Progress Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: In Progress Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/update-notifier/+git/update-notifier/+merge/417912 ** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/ubuntu-release-upgrader/+git/ubuntu-release-upgrader/+merge/417913 ** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/landscape-client/+git/landscape-client/+merge/417914 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: In Progress Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: In Progress Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: In Progress Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
I re-installed the former package content, rebooted the system and gave it some more memory to get rid of any concerns in that regard (from the perf data). The diff of the actual content before/after was all reasonable (new times, different package counts, but otherwise the same) Consumption wise we have: # Before fixes real1m11.731s us sy id wa st 71 20 9 0 0 59.16% landscape-sysin 14.23% swapper 6.16% sshd 5.78% lsb_release 3.35% apt-config 0.86% dbus-daemon 0.80% systemd 0.59% gdbus 0.56% dpkg 0.48% systemd-logind # After fixes real0m21.257s us sy id wa st 32 34 34 0 0 42.00% swapper 22.45% sshd 2.94% dbus-daemon 2.79% systemd 2.08% gdbus 2.07% grep 1.88% run-parts 1.58% systemd-logind 1.46% find 1.23% bash 1.22% update-motd-fsc 1.18% systemd-journal 1.13% cat That is good. Ready to open MRs for this. At a very high summary: - delay cut down by 2/3 of the initial duration - consumption reduced by 71% (duration) and by 26% (consumption while running). This is multiplicative so for the example we have an saving of ~81.27% of cpu cycles -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: In Progress Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: In Progress Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: In Progress Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
With the three combined I get down to: real0m20.900s us sy id wa st 32 33 35 0 0 43.88% swapper 21.48% sshd 3.07% dbus-daemon 2.78% systemd 2.10% gdbus 1.96% grep 1.80% run-parts 1.68% systemd-logind 1.38% find 1.24% systemd-journal 1.23% bash 1.12% update-motd-fsc 1.10% cat 0.95% awk 0.85% 50-motd-news 0.76% uname 0.70% 00-header 0.69% 91-release-upgr 0.67% 97-overlayroot 0.67% 50-landscape-sy 0.62% date 0.61% 90-updates-avai 0.56% 95-hwe-eol 0.54% cut None of the remaining big contributions to consumption is from the MOTD efforts (all <2%). That is reasonable, nice time gain as well as reduced cpu consumption. ** Changed in: landscape-client (Ubuntu) Status: New => In Progress ** Changed in: ubuntu-release-upgrader (Ubuntu) Status: New => In Progress ** Changed in: update-notifier (Ubuntu) Status: New => In Progress ** Changed in: update-notifier (Ubuntu) Assignee: (unassigned) => Christian Ehrhardt (paelzer) ** Changed in: ubuntu-release-upgrader (Ubuntu) Assignee: (unassigned) => Christian Ehrhardt (paelzer) ** Changed in: landscape-client (Ubuntu) Assignee: (unassigned) => Christian Ehrhardt (paelzer) ** Changed in: landscape-client (Ubuntu) Importance: Undecided => Critical ** Changed in: ubuntu-release-upgrader (Ubuntu) Importance: Undecided => High ** Changed in: update-notifier (Ubuntu) Importance: Undecided => High ** Changed in: update-motd (Ubuntu) Importance: High => Medium ** Changed in: pam (Ubuntu) Importance: High => Medium ** Changed in: update-motd (Ubuntu) Status: Confirmed => Invalid -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: In Progress Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: In Progress Status in update-motd package in Ubuntu: Invalid Status in update-notifier package in Ubuntu: In Progress Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
#3 50-landscape-sysinfo
The landscape part already has a statement about "when it is from" as it
is not re-executed on high load. This is handy as it will also ensure
there is no confusion "from when" this info is if we skip for too
frequent invocations.
Since it has an alternate less useful output I've added checks to
replace this more often and not replace a good output with a bad one.
--- orig/50-landscape-sysinfo 2022-03-30 07:53:04.320551811 +
+++ /etc/update-motd.d/50-landscape-sysinfo 2022-03-30 10:04:00.536053398
+
@@ -1,17 +1,39 @@
#!/bin/sh
-# pam_motd does not carry the environment
-[ -f /etc/default/locale ] && . /etc/default/locale
-export LANG
-cores=$(grep -c ^processor /proc/cpuinfo 2>/dev/null)
-[ "$cores" -eq "0" ] && cores=1
-threshold="${cores:-1}.0"
-if [ $(echo "`cut -f1 -d ' ' /proc/loadavg` < $threshold" | bc) -eq 1 ]; then
-echo
-echo -n " System information as of "
-/bin/date
-echo
-/usr/bin/landscape-sysinfo
-else
-echo
-echo " System information disabled due to load higher than $threshold"
+
+# do try refresh this more than once per minute
+# Due to cpu consumption and login delays (LP: #1893716)
+stamp="/var/lib/landscape/landscape-sysinfo.cache"
+NEED_UPDATE="FALSE"
+find $stamp -newermt 'now-1 minutes' 2> /dev/null | grep -q -m 1 '.' ||
NEED_UPDATE="TRUE"
+# If the last report in cache wasn't useful (load was too high) still wait at
least 5 seconds
+if grep -q "System information disabled" $stamp 2> /dev/null; then
+find $stamp -newermt 'now-5 seconds' 2> /dev/null | grep -q -m 1 '.' ||
NEED_UPDATE="TRUE"
fi
+
+if [ "$NEED_UPDATE" = "TRUE" ]; then
+# pam_motd does not carry the environment
+[ -f /etc/default/locale ] && . /etc/default/locale
+export LANG
+cores=$(grep -c ^processor /proc/cpuinfo 2>/dev/null)
+[ "$cores" -eq "0" ] && cores=1
+threshold="${cores:-1}.0"
+if [ $(echo "`cut -f1 -d ' ' /proc/loadavg` < $threshold" | bc) -eq 1 ];
then
+ (
+echo
+echo -n " System information as of "
+/bin/date
+echo
+/usr/bin/landscape-sysinfo
+) > $stamp
+else
+ # do not replace a formerly good result due to load
+ if ! grep -q "System information as of" $stamp 2> /dev/null; then
+ (
+ echo
+ echo " System information disabled due to load higher than
$threshold"
+ ) > $stamp
+ fi
+fi
+fi
+
+[ ! -r "$stamp" ] || cat "$stamp"
# Info:
It might be worth to note, the optimizations to 95-hwe-eol and
91-release-upgrade save CPU cycles (which is good and worth on its own), but do
not improve the delay much.
The optimization to
P.S. I'll need some minor updates for style and avoiding errors (e.g.
the && exit 0 was working but could be bad)
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to pam in Ubuntu.
https://bugs.launchpad.net/bugs/1893716
Title:
scripts in /etc/update-motd.d/ run even on login via non-interactive
scp and sftp sessions
Status in landscape-client package in Ubuntu:
In Progress
Status in pam package in Ubuntu:
Confirmed
Status in ubuntu-release-upgrader package in Ubuntu:
In Progress
Status in update-motd package in Ubuntu:
Invalid
Status in update-notifier package in Ubuntu:
In Progress
Bug description:
My client has 200+ devices automatically uploading information via
sftp and scp to a server every few minutes. After a recent update, I
noticed the load on their server spiking through the roof. Upon
investigation, I discovered a horde of landscape-sysinfo and
/usr/bin/lsb_release processes running that correlated with login
session notifications in /var/log/syslog and the load spikes.
It appears that even in non-interactive sessions where this
information will never be seen, the configuration options below in
/etc/pam.d/sshd cause these items to be launched (in fact, probably
everything in /etc/update-motd.d). This only started on the system in
question after a recent set of system updates were installed.
The content of /etc/update-motd.d/* really, really, really shouldn't
be executed if the session in question is not interactive, as it
provides no value at all. Unfortunately, to disable it for these non-
interactive sessions, we also have to disable it for the interactive
ones as well where it has some value (though not enough to make
spiking the load on this server through the roof an acceptable
tradeoff).
# Print the message of the day upon successful login.
# This includes a dynamically generated part from /run/motd.dynamic
# and a static (admin-editable) part from /etc/motd.
#sessionoptional pam_motd.so motd=/run/motd.dynamic
#sessionoptional pam_motd.so noupdate
Also, looking at the script 00-header in /etc/update-motd.d/,
/usr/bin/lsb_release is being
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
#2 95-hwe-eol / update-motd-hwe-eol Sadly this already does some caching in update-motd-hwe-eol by checking if the last of these checks is older than an update to the source lists. But to do so it has already executed the - relatively - rather expensive apt-config calls. Since it comes down to checking new sources.lists there is no need to update this more often than once every few minutes or hours. Could be even slower without loosing most of its value, but we already achieve most of what we need with a delay of ~1h. We started with: 5.37% apt-config Now: -- no more present --- orig/95-hwe-eol 2022-03-30 07:53:18.396529018 + +++ /etc/update-motd.d/95-hwe-eol 2022-03-30 09:13:16.160985148 + @@ -1,5 +1,11 @@ #!/bin/sh +# this stamp is created and updated by /usr/lib/update-notifier/update-motd-hwe-eol +stamp="/var/lib/update-notifier/hwe-eol" + +# do not try to refresh this more than once per hour +find $stamp -newermt 'now-1 hours' 2> /dev/null | grep -m 1 '.' && exit 0 + if [ -x /usr/lib/update-notifier/update-motd-hwe-eol ]; then exec /usr/lib/update-notifier/update-motd-hwe-eol fi No big need left to reduce the apt-config usage in update-motd-hwe-eol after this change. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: New Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: New Status in update-motd package in Ubuntu: Confirmed Status in update-notifier package in Ubuntu: New Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
Collecting snippets: This one worked, proven by eliminating lsb_release on the consumption charts. #1 Caching for 91-release-upgrade: --- orig/91-release-upgrade 2022-03-30 07:53:26.560515795 + +++ /etc/update-motd.d/91-release-upgrade 2022-03-30 07:59:05.819971148 + @@ -1,7 +1,12 @@ #!/bin/sh # if the current release is under development there won't be a new one -if [ "$(lsb_release -sd | cut -d' ' -f4)" = "(development" ]; then +[ -r /etc/lsb-release ] && . /etc/lsb-release +if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then +DISTRIB_DESCRIPTION=$(lsb_release -s -d) +fi + +if [ "$(echo "$DISTRIB_DESCRIPTION" | cut -d' ' -f4)" = "(development" ]; then exit 0 fi -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: New Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: New Status in update-motd package in Ubuntu: Confirmed Status in update-notifier package in Ubuntu: New Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
Summarizing the low hanging fruits here: - Add caching to 50-landscape-sysinfo - Add caching to 95-hwe-eol - /usr/lib/update-notifier/update-motd-hwe-eol calls apt-config multiple times. consider reducing those calls - 91-release-upgrade unconditionally calls lsb_release which is expensive. Use the same check others use The rest already uses caching AND/OR is small, fast and simple. The follow on of making pam_motd truly not do anything on non- interactive can then be a follow on case and would no more be that important. For these fixes three packages need to be touched: Source: ubuntu-release-upgrader Source: update-notifier Source: landscape-client ** Also affects: ubuntu-release-upgrader (Ubuntu) Importance: Undecided Status: New ** Also affects: update-notifier (Ubuntu) Importance: Undecided Status: New ** Also affects: landscape-client (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in landscape-client package in Ubuntu: New Status in pam package in Ubuntu: Confirmed Status in ubuntu-release-upgrader package in Ubuntu: New Status in update-motd package in Ubuntu: Confirmed Status in update-notifier package in Ubuntu: New Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/landscape-client/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
Analysis has spotted 91-release-upgrade as the most likely expensive remainder. pam_motd enabled, but disabled: - 50-landscape-sysinfo - 91-release-upgrade - 95-hwe-eol disabled Bionic real0m18.669s us sy id wa st 22 23 55 0 0 Focal real0m23.821s us sy id wa st 40 39 21 0 0 Jammy real0m19.616s us sy id wa st 33 30 37 0 0 This is pretty close to "no-motd" and has no single spike left. The next ones I found in the list are now low and already use caching. The improvement for those would be a (slower and more complex) modification to pam_motd to detect and skip on non-interactive sessions. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in pam package in Ubuntu: Confirmed Status in update-motd package in Ubuntu: Confirmed Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pam/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
pam_motd enabled, but 50-landscape-sysinfo and 95-hwe-eol disabled Bionic real0m25.952s us sy id wa st 41 22 37 0 0 Focal real0m30.592s us sy id wa st 49 33 19 0 0 Jammy real0m25.395s us sy id wa st 44 28 29 0 0 That is still quite some overhead (~+60% to no motd) but clearly those are the two worst contributors. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in pam package in Ubuntu: Confirmed Status in update-motd package in Ubuntu: Confirmed Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pam/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
pam_motd completely disabled in /etc/pam.d/sshd Bionic real0m15.540s us sy id wa st 18 14 68 0 0 Focal real0m16.937s us sy id wa st 43 40 17 0 0 Jammy real0m16.260s us sy id wa st 36 19 45 0 0 The remaining difference of those is in the noise-range. Some difference in the cycles consumed though, but not too bad. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in pam package in Ubuntu: Confirmed Status in update-motd package in Ubuntu: Confirmed Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pam/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
Time and CPU consumption (results are rather consistent BTW): Bionic real1m11.714s user0m2.577s sys 0m0.410s procs ---memory-- ---swap-- -io -system-- cpu -timestamp- r b swpd free buffcache si sobibo in cs us sy 17 0 0 2022-03-29 13:13:15 0 00 22103628312 15698000 0 2124 4319 1330 67 19 14 0 0 2022-03-29 13:13:20 1 00 19677228320 15852400 011 4290 1274 68 19 13 0 0 2022-03-29 13:13:25 ... Focal real0m44.742s user0m2.489s sys 0m0.477s procs ---memory-- ---swap-- -io -system-- cpu -timestamp- r b swpd free buffcache si sobibo in cs us sy id wa st UTC 1 004278418448 26704800 011 4554 3561 67 24 8 0 0 2022-03-29 13:14:13 1 003773218456 26910000 012 4577 3851 65 25 10 0 0 2022-03-29 13:14:18 1 001840018464 27081200 011 4554 3547 67 24 9 0 0 2022-03-29 13:14:23 Jammy real1m8.010s user0m2.436s sys 0m0.484s --procs-- ---memory-- ---swap-- -io -system-- cpu -timestamp- rb swpd free buffcache si sobi bo in cs us sy id wa st UTC 0005826417760 25854400 0 9 4374 1953 68 20 11 0 0 2022-03-29 13:15:20 1003284417772 26033200 0 16 4352 1851 68 20 11 0 0 2022-03-29 13:15:25 1004942817784 26226800 0 17 4387 1985 67 22 12 0 0 2022-03-29 13:15:30 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in pam package in Ubuntu: Confirmed Status in update-motd package in Ubuntu: Confirmed Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pam/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
I wanted to get a better feeling about this before jumping to action. Therefore I have created three 1G/1vcpu KVM guests Bionic/Focal/Jammy to test and compare this on. I do not need hot-loop analysis or anything down to instructions, so no debug symbols needed. For now I only want to know: 1. how much time a bunch of low effort logins take (we measure only the overhead) 2. how much cpu is utilized while doing that 3. how that work is spread across programs (disable them one by one and look at data) The two I see most are: - /usr/lib/update-notifier/update-motd-hwe-eol apt-config shell SourceList Dir::Etc::sourcelist - /etc/update-motd.d/50-landscape-sysinfo /usr/bin/python3 /usr/bin/landscape-sysinfo Very non-pro data gathering: $ cat tracestart.sh #!/bin/bash sudo rm perf.data perf.log log.vmstat nohup vmstat -wt 5 &> log.vmstat & nohup perf record --event cpu-clock --all-cpus &> perf.log & sleep 5 $ cat traceend.sh #!/bin/bash killall perf killall vmstat cat perf.log cat log.vmstat #perf report --sort comm --stdio Most simple load involving those helpers. for sys in login-bionic login-focal login-jammy; do ssh $sys "sudo ~/tracestart.sh"; time for i in $(seq 1 100); do ssh $sys "/bin/true"; done; ssh $sys "sudo ~/traceend.sh"; done I'll get those numbers for Bionic/Focal/Jammy and enabling/disabling it all and/or individual elements. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in pam package in Ubuntu: Confirmed Status in update-motd package in Ubuntu: Confirmed Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pam/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
The only current interactivity detection code in pam is part of a pam.conf -> pam.d conversion tool that won't be useful here. The pam_motd code emits content via things like try_to_display_fd. A message is created and then printed via pam_info. Which is actually pam_prompt which wraps pam_vprompt This gets the conversation function via retval = pam_get_item (pamh, PAM_CONV, ); and on that it then emits the message retval = conv->conv (1, , _resp, conv->appdata_ptr); Either via this PAM_CONV and then attributes of that channel (as it is what we'd print on) OR via something like pam_get_item(pamh, PAM_TTY, ); we might get access from pam_motd to something that we can work out if it is interactive. I'm busy with other things now (for the rest of today), but I want ton continue tomorrow. I want this at least to get into a clear state that is sure if: a) this is as important as I think b) the steps needed from here are clear ** Also affects: pam (Ubuntu) Importance: Undecided Status: New ** Changed in: pam (Ubuntu) Status: New => Confirmed ** Changed in: update-motd (Ubuntu) Status: Triaged => Confirmed ** Changed in: pam (Ubuntu) Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to pam in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in pam package in Ubuntu: Confirmed Status in update-motd package in Ubuntu: Confirmed Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pam/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
Hello Thomas, Thanks for reporting this and making Ubuntu better. about your statement on lsb-release, in 00-header: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" the slowpath is only executed if the DISTRIB_DESCRIPTION variable is not set, at least in my tests. In regards to your statement: """ The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non-interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). """ Considering there IS a way of disabling the dynamic motd contents currently by commenting only the line containing the /run/motd.dynamic wording: # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. # sessionoptional pam_motd.so motd=/run/motd.dynamic sessionoptional pam_motd.so noupdate I'm flagging this bug as a whishlist priority. ** Also affects: update-motd (Ubuntu) Importance: Undecided Status: New ** No longer affects: openssh (Ubuntu) ** No longer affects: base-files (Ubuntu) ** Changed in: update-motd (Ubuntu) Status: New => Triaged ** Changed in: update-motd (Ubuntu) Importance: Undecided => Wishlist -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to base-files in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in update-motd package in Ubuntu: Triaged Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/update-motd/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
** Also affects: landscape-client (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to base-files in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in base-files package in Ubuntu: New Status in landscape-client package in Ubuntu: New Status in openssh package in Ubuntu: New Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/base-files/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
** Also affects: base-files (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to base-files in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in base-files package in Ubuntu: New Status in landscape-client package in Ubuntu: New Status in openssh package in Ubuntu: New Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc/update-motd.d). This only started on the system in question after a recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non- interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/base-files/+bug/1893716/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1893716] Re: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions
** Description changed: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in - /etc/syslog and the load spikes. + /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc /update-motd.d). This only started on the system in question after a recent set of system updates were in stalled. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non-interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then - # Fall back to using the very slow lsb_release utility - DISTRIB_DESCRIPTION=$(lsb_release -s -d) + # Fall back to using the very slow lsb_release utility + DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" ** Description changed: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login session notifications in /var/log/syslog and the load spikes. It appears that even in non-interactive sessions where this information will never be seen, the configuration options below in /etc/pam.d/sshd cause these items to be launched (in fact, probably everything in /etc /update-motd.d). This only started on the system in question after a - recent set of system updates were in stalled. + recent set of system updates were installed. The content of /etc/update-motd.d/* really, really, really shouldn't be executed if the session in question is not interactive, as it provides no value at all. Unfortunately, to disable it for these non-interactive sessions, we also have to disable it for the interactive ones as well where it has some value (though not enough to make spiking the load on this server through the roof an acceptable tradeoff). # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #sessionoptional pam_motd.so motd=/run/motd.dynamic #sessionoptional pam_motd.so noupdate Also, looking at the script 00-header in /etc/update-motd.d/, /usr/bin/lsb_release is being improperly launched, as /etc/lsb_release does include the necessary information: [ -r /etc/lsb-release ] && . /etc/lsb-release if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then # Fall back to using the very slow lsb_release utility DISTRIB_DESCRIPTION=$(lsb_release -s -d) fi # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS" -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1893716 Title: scripts in /etc/update-motd.d/ run even on login via non-interactive scp and sftp sessions Status in openssh package in Ubuntu: New Bug description: My client has 200+ devices automatically uploading information via sftp and scp to a server every few minutes. After a recent update, I noticed the load on their server spiking through the roof. Upon investigation, I discovered a horde of landscape-sysinfo and /usr/bin/lsb_release processes running that correlated with login
