Ah I see, that's subtle. thanks. On Sat, 1 Sep 2018 at 19:58, Brad Fitzpatrick <[email protected]> wrote: > > If the Go runtime can't get memory from the kernel, Go can crash itself with > a log message. > > If Go can allocate but the kernel overcommitted itself (on by default) then > the kennel picks a process to kill. > > > On Sat, Sep 1, 2018 at 7:52 PM Mathieu Lonjaret <[email protected]> > wrote: >> >> I make it allocate forever in a loop. >> Hold on, I'll reproduce it again in a minute. >> On Sat, 1 Sep 2018 at 19:50, Brad Fitzpatrick <[email protected]> wrote: >> > >> > What type of "OOM" are you referring to and how did you cause it? What do >> > your logs show when you do so? >> > >> > On Sat, Sep 1, 2018 at 7:48 PM Mathieu Lonjaret >> > <[email protected]> wrote: >> >> >> >> Hmm, so what is it that makes it behave that way? If I force Perkeep >> >> to OOM (not on systemd), it clearly shows up in the output here. Is my >> >> kernel not behaving the same way because Perkeep is not running >> >> through systemd? >> >> >> >> >> >> On Sat, 1 Sep 2018 at 19:44, Brad Fitzpatrick <[email protected]> wrote: >> >> > >> >> > The kernel is sending it an uncatchable SIGKILL. It has no chance to >> >> > log. >> >> > >> >> > On Sat, Sep 1, 2018, 7:18 PM Mathieu Lonjaret >> >> > <[email protected]> wrote: >> >> >> >> >> >> And you're maintaining that in 'journalctl -u perkeep', at the time of >> >> >> perkeep's death, there's no such sign of the OOM? I'm asking, because >> >> >> it usually is pretty obvious just by looking at perkeep's output. >> >> >> If that's the case, it's something to keep in mind for future >> >> >> debugging of Perkeep under systemd. >> >> >> >> >> >> On Sat, 1 Sep 2018 at 10:09, Niklas Merz <[email protected]> wrote: >> >> >> > >> >> >> > So it failed again. >> >> >> > >> >> >> > The exit code is: >> >> >> > Aug 28 20:21:16 stuff systemd[1]: Started Perkeep Server. >> >> >> > -- Subject: Unit perkeep.service has finished start-up >> >> >> > -- Defined-By: systemd >> >> >> > -- Support: https://www.debian.org/support >> >> >> > -- >> >> >> > -- Unit perkeep.service has finished starting up. >> >> >> > -- >> >> >> > -- The start-up result is done. >> >> >> > Aug 30 16:49:23 stuff systemd[1]: perkeep.service: Main process >> >> >> > exited, code=killed, status=9/KILL >> >> >> > Aug 30 16:49:23 stuff systemd[1]: perkeep.service: Unit entered >> >> >> > failed state. >> >> >> > Aug 30 16:49:23 stuff systemd[1]: perkeep.service: Failed with >> >> >> > result 'signal'. >> >> >> > >> >> >> > $ dmesg | grep perkeepd >> >> >> > >> >> >> > [9339544.204149] perkeepd invoked oom-killer: >> >> >> > gfp_mask=0x14000c0(GFP_KERNEL), nodemask=(null), order=0, >> >> >> > oom_score_adj=0 >> >> >> > [9339544.204153] perkeepd cpuset=ns mems_allowed=0 >> >> >> > [9339544.204156] CPU: 3 PID: 11500 Comm: perkeepd Tainted: P >> >> >> > O 4.13.16-2-pve #1 >> >> >> > [9339544.204447] [11488] 1000 11488 459372 282365 638 >> >> >> > 6 0 0 perkeepd >> >> >> > [9339544.204450] Memory cgroup out of memory: Kill process 11488 >> >> >> > (perkeepd) score 539 or sacrifice child >> >> >> > [9339544.205419] Killed process 11488 (perkeepd) total-vm:1837488kB, >> >> >> > anon-rss:1129460kB, file-rss:0kB, shmem-rss:0kB >> >> >> > [9339544.258384] oom_reaper: reaped process 11488 (perkeepd), now >> >> >> > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >> >> >> > [9340611.483660] [15711] 1000 15711 682515 276764 707 >> >> >> > 7 0 0 perkeepd >> >> >> > [9340611.483664] Memory cgroup out of memory: Kill process 15711 >> >> >> > (perkeepd) score 529 or sacrifice child >> >> >> > [9340611.484713] Killed process 15711 (perkeepd) total-vm:2730060kB, >> >> >> > anon-rss:1107056kB, file-rss:0kB, shmem-rss:0kB >> >> >> > [9341096.168476] perkeepd invoked oom-killer: >> >> >> > gfp_mask=0x14000c0(GFP_KERNEL), nodemask=(null), order=0, >> >> >> > oom_score_adj=0 >> >> >> > [9341096.168478] perkeepd cpuset=ns mems_allowed=0 >> >> >> > [9341096.168482] CPU: 3 PID: 18250 Comm: perkeepd Tainted: P >> >> >> > O 4.13.16-2-pve #1 >> >> >> > [9341096.168836] [18246] 1000 18246 421195 280089 619 >> >> >> > 6 0 0 perkeepd >> >> >> > [9341096.168842] Memory cgroup out of memory: Kill process 18246 >> >> >> > (perkeepd) score 535 or sacrifice child >> >> >> > [9341096.169788] Killed process 18246 (perkeepd) total-vm:1684780kB, >> >> >> > anon-rss:1120356kB, file-rss:0kB, shmem-rss:0kB >> >> >> > [9341096.228024] oom_reaper: reaped process 18246 (perkeepd), now >> >> >> > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >> >> >> > [9501543.900541] perkeepd invoked oom-killer: >> >> >> > gfp_mask=0x14000c0(GFP_KERNEL), nodemask=(null), order=0, >> >> >> > oom_score_adj=0 >> >> >> > [9501543.900543] perkeepd cpuset=ns mems_allowed=0 >> >> >> > [9501543.900547] CPU: 3 PID: 22466 Comm: perkeepd Tainted: P >> >> >> > O 4.13.16-2-pve #1 >> >> >> > [9501543.900832] [22465] 1000 22465 412239 265550 648 >> >> >> > 6 11826 0 perkeepd >> >> >> > [9501543.900845] Memory cgroup out of memory: Kill process 22465 >> >> >> > (perkeepd) score 530 or sacrifice child >> >> >> > [9501543.901793] Killed process 22465 (perkeepd) total-vm:1648956kB, >> >> >> > anon-rss:1062200kB, file-rss:0kB, shmem-rss:0kB >> >> >> > [9501543.956730] oom_reaper: reaped process 22465 (perkeepd), now >> >> >> > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >> >> >> > >> >> >> > So it runs out of memory right? I will allocate more RAM to this VM >> >> >> > and try running it again. >> >> >> > >> >> >> > -- >> >> >> > You received this message because you are subscribed to the Google >> >> >> > Groups "Perkeep" group. >> >> >> > To unsubscribe from this group and stop receiving emails from it, >> >> >> > send an email to [email protected]. >> >> >> > For more options, visit https://groups.google.com/d/optout. >> >> >> >> >> >> -- >> >> >> You received this message because you are subscribed to the Google >> >> >> Groups "Perkeep" group. >> >> >> To unsubscribe from this group and stop receiving emails from it, send >> >> >> an email to [email protected]. >> >> >> For more options, visit https://groups.google.com/d/optout. >> >> > >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups "Perkeep" group. >> >> > To unsubscribe from this group and stop receiving emails from it, send >> >> > an email to [email protected]. >> >> > For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> >> You received this message because you are subscribed to the Google Groups >> >> "Perkeep" group. >> >> To unsubscribe from this group and stop receiving emails from it, send an >> >> email to [email protected]. >> >> For more options, visit https://groups.google.com/d/optout. >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Perkeep" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to [email protected]. >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Perkeep" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Perkeep" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout.
-- You received this message because you are subscribed to the Google Groups "Perkeep" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
