[Touch-packages] [Bug 1966381] Re: applications crash that never crashed under Ubuntu-20.04
Well funny you should say that... When I installed 22.04 on my new Dell laptop with 16G RAM, Jammy still only allocated 976MB of swap. I think you have a problem there too. So after reporting this issue and continually having OOM crashes, I created a 20G swapfile - and ever since this problem has disappeared... Maybe running with such a starved swapspace triggers systemd-oom to do weird things? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1966381 Title: applications crash that never crashed under Ubuntu-20.04 Status in systemd package in Ubuntu: New Status in systemd source package in Jammy: New Bug description: Just now I was watching some video in Firefox. I popped over to another virtual workspace for a few minutes, and when I popped back to Firefox it had gone. The same thing had been happening all week (I installed fresh Ubuntu-22.04 last week) with Chrome, Firefox and Thunderbird. This time instead of shrugging it off I looked in the logs, and found this Mar 25 19:45:40 ubuntu systemd-oomd[960]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-firefox-6607.scope due to memory used (15940579328) / total (16153944064) and swap used (925564928) / total (1023406080) being more than 90.00% Mar 22 08:11:29 ubuntu systemd[5029]: app-gnome-google\x2dchrome-5412.scope: systemd-oomd killed 298 process(es) in this unit. Mar 23 11:09:28 ubuntu systemd-oomd[1055]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-thunderbird-5418.scope due to memory used (15591993344) / total (16149745664) and swap used (927760384) / total (1023406080) being more than 90.00% Mar 23 11:09:28 ubuntu systemd[5029]: app-gnome-thunderbird-5418.scope: systemd-oomd killed 173 process(es) in this unit. I know it's saying those three entirely unrelated applications had suddenly decided to swallow all the RAM+swap on this laptop of mine - but the very same apps didn't act like that last week under Ubuntu-20.04, so I suspect something else is going on I can't say they hadn't swallowed all the RAM, but there is ZERO sign of a system on the verge of collapsing - everything has been screaming along just nicely - no sign of the "staggering" you normally get when the OS is heavily into swap. However, now that I look I see my 16G laptop only has 1G swap??? I just let the Ubuntu installer do it's defaults - but it used to auto- choose 1xRAM or 2xRAM - what's with this 1G swap? Could that be related? ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: systemd-oomd 249.11-0ubuntu1 ProcVersionSignature: Ubuntu 5.15.0-23.23-generic 5.15.27 Uname: Linux 5.15.0-23-generic x86_64 ApportVersion: 2.20.11-0ubuntu79 Architecture: amd64 CasperMD5CheckResult: pass CurrentDesktop: ubuntu:GNOME Date: Fri Mar 25 19:47:44 2022 InstallationDate: Installed on 2022-03-13 (11 days ago) InstallationMedia: Ubuntu 22.04 LTS "Jammy Jellyfish" - Alpha amd64 (20220313) SourcePackage: systemd UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1966381/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1966381] Re: applications crash that never crashed under Ubuntu-20.04
Oh well, that latest systemd-oom didn't help. Chrome just crashed again - while I wasn't even using the computer. Here are all the syslogs at the time it crashed - nothing but the OOM Mar 28 19:30:56 ubuntu systemd-oomd[1121]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-google\x2dchrome-5640.scope due to memory used (15992975360) / total (16153948160) and swap used (921436160) / total (1023406080) being more than 90.00% Mar 28 19:30:56 ubuntu systemd[5083]: app-gnome-google\x2dchrome-5640.scope: systemd-oomd killed 311 process(es) in this unit. Mar 28 19:30:56 ubuntu systemd[5083]: app-gnome-google\x2dchrome-5640.scope: Consumed 7h 10min 56.132s CPU time. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1966381 Title: applications crash that never crashed under Ubuntu-20.04 Status in systemd package in Ubuntu: New Bug description: Just now I was watching some video in Firefox. I popped over to another virtual workspace for a few minutes, and when I popped back to Firefox it had gone. The same thing had been happening all week (I installed fresh Ubuntu-22.04 last week) with Chrome, Firefox and Thunderbird. This time instead of shrugging it off I looked in the logs, and found this Mar 25 19:45:40 ubuntu systemd-oomd[960]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-firefox-6607.scope due to memory used (15940579328) / total (16153944064) and swap used (925564928) / total (1023406080) being more than 90.00% Mar 22 08:11:29 ubuntu systemd[5029]: app-gnome-google\x2dchrome-5412.scope: systemd-oomd killed 298 process(es) in this unit. Mar 23 11:09:28 ubuntu systemd-oomd[1055]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-thunderbird-5418.scope due to memory used (15591993344) / total (16149745664) and swap used (927760384) / total (1023406080) being more than 90.00% Mar 23 11:09:28 ubuntu systemd[5029]: app-gnome-thunderbird-5418.scope: systemd-oomd killed 173 process(es) in this unit. I know it's saying those three entirely unrelated applications had suddenly decided to swallow all the RAM+swap on this laptop of mine - but the very same apps didn't act like that last week under Ubuntu-20.04, so I suspect something else is going on I can't say they hadn't swallowed all the RAM, but there is ZERO sign of a system on the verge of collapsing - everything has been screaming along just nicely - no sign of the "staggering" you normally get when the OS is heavily into swap. However, now that I look I see my 16G laptop only has 1G swap??? I just let the Ubuntu installer do it's defaults - but it used to auto- choose 1xRAM or 2xRAM - what's with this 1G swap? Could that be related? ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: systemd-oomd 249.11-0ubuntu1 ProcVersionSignature: Ubuntu 5.15.0-23.23-generic 5.15.27 Uname: Linux 5.15.0-23-generic x86_64 ApportVersion: 2.20.11-0ubuntu79 Architecture: amd64 CasperMD5CheckResult: pass CurrentDesktop: ubuntu:GNOME Date: Fri Mar 25 19:47:44 2022 InstallationDate: Installed on 2022-03-13 (11 days ago) InstallationMedia: Ubuntu 22.04 LTS "Jammy Jellyfish" - Alpha amd64 (20220313) SourcePackage: systemd UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1966381/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1966381] Re: applications crash that never crashed under Ubuntu-20.04
FYI About 6 hours ago I saw a new release of systemd-oom was released (249.11-0ubuntu2). I've upgraded the entire system and rebooted, so I'll report back if there's any change. I was getting these random OOM about every couple of days, so within a week should know if that changed anything -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1966381 Title: applications crash that never crashed under Ubuntu-20.04 Status in systemd package in Ubuntu: New Bug description: Just now I was watching some video in Firefox. I popped over to another virtual workspace for a few minutes, and when I popped back to Firefox it had gone. The same thing had been happening all week (I installed fresh Ubuntu-22.04 last week) with Chrome, Firefox and Thunderbird. This time instead of shrugging it off I looked in the logs, and found this Mar 25 19:45:40 ubuntu systemd-oomd[960]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-firefox-6607.scope due to memory used (15940579328) / total (16153944064) and swap used (925564928) / total (1023406080) being more than 90.00% Mar 22 08:11:29 ubuntu systemd[5029]: app-gnome-google\x2dchrome-5412.scope: systemd-oomd killed 298 process(es) in this unit. Mar 23 11:09:28 ubuntu systemd-oomd[1055]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-thunderbird-5418.scope due to memory used (15591993344) / total (16149745664) and swap used (927760384) / total (1023406080) being more than 90.00% Mar 23 11:09:28 ubuntu systemd[5029]: app-gnome-thunderbird-5418.scope: systemd-oomd killed 173 process(es) in this unit. I know it's saying those three entirely unrelated applications had suddenly decided to swallow all the RAM+swap on this laptop of mine - but the very same apps didn't act like that last week under Ubuntu-20.04, so I suspect something else is going on I can't say they hadn't swallowed all the RAM, but there is ZERO sign of a system on the verge of collapsing - everything has been screaming along just nicely - no sign of the "staggering" you normally get when the OS is heavily into swap. However, now that I look I see my 16G laptop only has 1G swap??? I just let the Ubuntu installer do it's defaults - but it used to auto- choose 1xRAM or 2xRAM - what's with this 1G swap? Could that be related? ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: systemd-oomd 249.11-0ubuntu1 ProcVersionSignature: Ubuntu 5.15.0-23.23-generic 5.15.27 Uname: Linux 5.15.0-23-generic x86_64 ApportVersion: 2.20.11-0ubuntu79 Architecture: amd64 CasperMD5CheckResult: pass CurrentDesktop: ubuntu:GNOME Date: Fri Mar 25 19:47:44 2022 InstallationDate: Installed on 2022-03-13 (11 days ago) InstallationMedia: Ubuntu 22.04 LTS "Jammy Jellyfish" - Alpha amd64 (20220313) SourcePackage: systemd UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1966381/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1966381] [NEW] applications crash that never crashed under Ubuntu-20.04
Public bug reported: Just now I was watching some video in Firefox. I popped over to another virtual workspace for a few minutes, and when I popped back to Firefox it had gone. The same thing had been happening all week (I installed fresh Ubuntu-22.04 last week) with Chrome, Firefox and Thunderbird. This time instead of shrugging it off I looked in the logs, and found this Mar 25 19:45:40 ubuntu systemd-oomd[960]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-firefox-6607.scope due to memory used (15940579328) / total (16153944064) and swap used (925564928) / total (1023406080) being more than 90.00% Mar 22 08:11:29 ubuntu systemd[5029]: app-gnome-google\x2dchrome-5412.scope: systemd-oomd killed 298 process(es) in this unit. Mar 23 11:09:28 ubuntu systemd-oomd[1055]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-thunderbird-5418.scope due to memory used (15591993344) / total (16149745664) and swap used (927760384) / total (1023406080) being more than 90.00% Mar 23 11:09:28 ubuntu systemd[5029]: app-gnome-thunderbird-5418.scope: systemd-oomd killed 173 process(es) in this unit. I know it's saying those three entirely unrelated applications had suddenly decided to swallow all the RAM+swap on this laptop of mine - but the very same apps didn't act like that last week under Ubuntu-20.04, so I suspect something else is going on I can't say they hadn't swallowed all the RAM, but there is ZERO sign of a system on the verge of collapsing - everything has been screaming along just nicely - no sign of the "staggering" you normally get when the OS is heavily into swap. However, now that I look I see my 16G laptop only has 1G swap??? I just let the Ubuntu installer do it's defaults - but it used to auto-choose 1xRAM or 2xRAM - what's with this 1G swap? Could that be related? ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: systemd-oomd 249.11-0ubuntu1 ProcVersionSignature: Ubuntu 5.15.0-23.23-generic 5.15.27 Uname: Linux 5.15.0-23-generic x86_64 ApportVersion: 2.20.11-0ubuntu79 Architecture: amd64 CasperMD5CheckResult: pass CurrentDesktop: ubuntu:GNOME Date: Fri Mar 25 19:47:44 2022 InstallationDate: Installed on 2022-03-13 (11 days ago) InstallationMedia: Ubuntu 22.04 LTS "Jammy Jellyfish" - Alpha amd64 (20220313) SourcePackage: systemd UpgradeStatus: No upgrade log present (probably fresh install) ** Affects: systemd (Ubuntu) Importance: Undecided Status: New ** Tags: amd64 apport-bug jammy wayland-session -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1966381 Title: applications crash that never crashed under Ubuntu-20.04 Status in systemd package in Ubuntu: New Bug description: Just now I was watching some video in Firefox. I popped over to another virtual workspace for a few minutes, and when I popped back to Firefox it had gone. The same thing had been happening all week (I installed fresh Ubuntu-22.04 last week) with Chrome, Firefox and Thunderbird. This time instead of shrugging it off I looked in the logs, and found this Mar 25 19:45:40 ubuntu systemd-oomd[960]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-firefox-6607.scope due to memory used (15940579328) / total (16153944064) and swap used (925564928) / total (1023406080) being more than 90.00% Mar 22 08:11:29 ubuntu systemd[5029]: app-gnome-google\x2dchrome-5412.scope: systemd-oomd killed 298 process(es) in this unit. Mar 23 11:09:28 ubuntu systemd-oomd[1055]: Killed /user.slice/user-1001.slice/user@1001.service/app.slice/app-gnome-thunderbird-5418.scope due to memory used (15591993344) / total (16149745664) and swap used (927760384) / total (1023406080) being more than 90.00% Mar 23 11:09:28 ubuntu systemd[5029]: app-gnome-thunderbird-5418.scope: systemd-oomd killed 173 process(es) in this unit. I know it's saying those three entirely unrelated applications had suddenly decided to swallow all the RAM+swap on this laptop of mine - but the very same apps didn't act like that last week under Ubuntu-20.04, so I suspect something else is going on I can't say they hadn't swallowed all the RAM, but there is ZERO sign of a system on the verge of collapsing - everything has been screaming along just nicely - no sign of the "staggering" you normally get when the OS is heavily into swap. However, now that I look I see my 16G laptop only has 1G swap??? I just let the Ubuntu installer do it's defaults - but it used to auto- choose 1xRAM or 2xRAM - what's with this 1G swap? Could that be related? ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: systemd-oomd 249.11-0ubuntu1 ProcVersionSignature: Ubuntu 5.15.0-23.23-generic 5.15.27 Uname: Linux 5.15.0-23-generic x86_64 ApportVersion: 2.20.11-0ubuntu79 Architecture: amd64 CasperMD5CheckResult:
[Touch-packages] [Bug 1966037] [NEW] some applications hang for up to 30sec due to /run/systemd/resolve/io.systemd.Resolve
Public bug reported: I just installed Ubuntu-22.04 last week and have noticed lots of command line apps seem to hang for 10-30sec before working. Eventually that got annoying enough that I looked into it. Anyway, apps like "telnet" (yep, I'm that old) and oddly enough sudo showed this issue strongly. In the end I ran them with strace and could see the pause occurred when /run/systemd/resolve/io.systemd.Resolve was called. So systemd-resolve was involved. Then I cranked up "tcpdump -n -i any port 53" and re-ran the commands. What I found was systemd-resolver was throwing DNS lookups for my hostname at all the DNS servers I had configured (default on wifi plus corporate always-on VPN link). Now the thing is our corporate VPN has SIXTEEN domains in the search domain field... So as systemd-resolver was doing one "A" plus one "" lookups for each "hostname.search-domain" - well there's the hang. In fact 30sec is quite a good response time ;-) To fix it I simply added my raw (ie non-dot) hostname to /etc/hosts - completely blocked this self-hostname lookup that must be going on. Now there are no DNS lookups at all for my hostname in this situation. All is good. But it does make me wonder if perhaps Ubuntu shouldn't always put the hostname into /etc/hosts? Even pointing at 127.0.0.99 (for example) might be enough (I haven't actually tested that - I'm pointing it at my static VPN IP) Anyway, this is a corner case for sure - but it might shave some delays off a bunch of folks. This also might seem cosmetic, but I would guess any application that calls gethostbyname or equivalent on itself is triggering this. PS: I had Ubuntu-20.04 before this and didn't have the problem. But looking at a backup I can see I also had my hostname in the /etc/hosts file - so this might have affected 20.04 too but I "accidentally" fixed it early on without realizing it ** Affects: systemd (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1966037 Title: some applications hang for up to 30sec due to /run/systemd/resolve/io.systemd.Resolve Status in systemd package in Ubuntu: New Bug description: I just installed Ubuntu-22.04 last week and have noticed lots of command line apps seem to hang for 10-30sec before working. Eventually that got annoying enough that I looked into it. Anyway, apps like "telnet" (yep, I'm that old) and oddly enough sudo showed this issue strongly. In the end I ran them with strace and could see the pause occurred when /run/systemd/resolve/io.systemd.Resolve was called. So systemd-resolve was involved. Then I cranked up "tcpdump -n -i any port 53" and re-ran the commands. What I found was systemd-resolver was throwing DNS lookups for my hostname at all the DNS servers I had configured (default on wifi plus corporate always-on VPN link). Now the thing is our corporate VPN has SIXTEEN domains in the search domain field... So as systemd-resolver was doing one "A" plus one "" lookups for each "hostname.search- domain" - well there's the hang. In fact 30sec is quite a good response time ;-) To fix it I simply added my raw (ie non-dot) hostname to /etc/hosts - completely blocked this self-hostname lookup that must be going on. Now there are no DNS lookups at all for my hostname in this situation. All is good. But it does make me wonder if perhaps Ubuntu shouldn't always put the hostname into /etc/hosts? Even pointing at 127.0.0.99 (for example) might be enough (I haven't actually tested that - I'm pointing it at my static VPN IP) Anyway, this is a corner case for sure - but it might shave some delays off a bunch of folks. This also might seem cosmetic, but I would guess any application that calls gethostbyname or equivalent on itself is triggering this. PS: I had Ubuntu-20.04 before this and didn't have the problem. But looking at a backup I can see I also had my hostname in the /etc/hosts file - so this might have affected 20.04 too but I "accidentally" fixed it early on without realizing it To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1966037/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
