Bug#940021: Re[2]: Bug#940021: systemd: socket activation leads to OOM situation due to slices not getting cleaned up

2019-10-04 Thread Michael Prokop
* Julian Hübenthal [Mon Sep 16, 2019 at 06:01:41PM +]:

> here is your requested output:
[...]

You might want to take a look at
https://checkmk.de/check_mk-werks.php?werk_id=10070 - I'm quoting
from there:

| We have seen cases in which the parameter "KillMode=process" of the
| unit file "check_mk@.service" causes a memory leak. This was first
| seen on Debian Buster but could occur in the future on other
| operating systems too.

So consider trying changing the parameter "KillMode=process" to
"Type=forking" and report whether this also fixes your issue.

regards
-mika-


signature.asc
Description: Digital signature


Bug#940021: Re[2]: Bug#940021: systemd: socket activation leads to OOM situation due to slices not getting cleaned up

2019-09-16 Thread Julian Hübenthal

Hi Michael,

here is your requested output:

 find /sys/fs/cgroup/system.slice/system-check_mk.slice/

produces:

root@debian:~# ls -la 
/sys/fs/cgroup/systemd/system.slice/system-check_mk.slice/

total 0
drwxr-xr-x  8 root root 0 Sep 16 19:53 .
drwxr-xr-x 18 root root 0 Sep 16 19:53 ..
-rw-r--r--  1 root root 0 Sep 16 19:53 cgroup.clone_children
-rw-r--r--  1 root root 0 Sep 16 19:53 cgroup.procs
drwxr-xr-x  2 root root 0 Sep 16 19:53 
check_mk@0-10.28.1.101:6556-10.28.1.99:61952.service
drwxr-xr-x  2 root root 0 Sep 16 19:53 
check_mk@1-10.28.1.101:6556-10.28.1.99:61957.service
drwxr-xr-x  2 root root 0 Sep 16 19:53 
check_mk@2-10.28.1.101:6556-10.28.1.99:61958.service
drwxr-xr-x  2 root root 0 Sep 16 19:53 
check_mk@3-10.28.1.101:6556-10.28.1.99:61960.service
drwxr-xr-x  2 root root 0 Sep 16 19:53 
check_mk@4-10.28.1.101:6556-10.28.1.99:61962.service
drwxr-xr-x  2 root root 0 Sep 16 19:53 
check_mk@5-10.28.1.101:6556-10.28.1.99:61964.service

-rw-r--r--  1 root root 0 Sep 16 19:53 notify_on_release
-rw-r--r--  1 root root 0 Sep 16 19:53 tasks

systemctl status check_mk@0-10.28.1.101:6556-10.28.1.99:61952.service

produces:


root@debian:~# systemctl status 
check_mk@0-10.28.1.101:6556-10.28.1.99:61952.service

* check_mk@0-10.28.1.101:6556-10.28.1.99:61952.service - Check_MK
   Loaded: loaded (/etc/systemd/system/check_mk@.service; static; vendor 
preset: enabled)

   Active: inactive (dead)

Sep 16 19:52:26 debian systemd[1]: Started Check_MK (10.28.1.99:61952).
Sep 16 19:52:26 debian systemd[1]: 
check_mk@0-10.28.1.101:6556-10.28.1.99:61952.service: Succeeded.



so systemctl status system-check_mk.slice

produces:

root@debian:~# systemctl status system-check_mk.slice
* system-check_mk.slice
   Loaded: loaded
   Active: active since Mon 2019-09-16 19:52:26 CEST; 4min 10s ago
Tasks: 0
   Memory: 6.0M
   CGroup: /system.slice/system-check_mk.slice
   Sep 16 19:52:26 debian systemd[1]: Created slice 
system-check_mk.slice.


While memory reported by "systemctl status system-check_mk.slice" grows 
with each execution by call  to the corresponding port (6556) of the 
service.



Kind Regards,
Julian

-- Original Message --
From: "Michael Biebl" 
To: "Julian Hübenthal" ; 
940...@bugs.debian.org

Sent: 13/09/2019 00:55:16
Subject: Re: Bug#940021: systemd: socket activation leads to OOM 
situation due to slices not getting cleaned up



Am 13.09.19 um 00:53 schrieb Michael Biebl:

 Am 12.09.19 um 14:29 schrieb Michael Biebl:

 Am 11.09.19 um 15:54 schrieb Julian Hübenthal:

 Just discovered something that may help to debug:



 It does not happen with a simple “Hello World” bash script instead of
 the Check MK Agent.

 It does not happen when the Encryption of the Check MK Agent is disabled.

 It happens when the Encryption of Check MK is enabled, which should be
 AES 128/256 output.



 I wonder if the MK agent does some tricks like locking the memory when
 encryption is on and then does not properly release its resources?


 Just to ask the obvious: The slice(s) themselves are empty, i.e. the
 check-mk agent process has exited (successfully)?

 What's the status of such a service that is not cleaned up?
 Taking your first email that would be
 systemctl status check_mk@10-10.28.5.6:6556-10.28.5.1:42844.service

 I'd be interested in the output of

 find /sys/fs/cgroup/system.slice/system-check_mk.slice/ as well




As well as systemctl status system-check_mk.slice




--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
Pkg-systemd-maintainers mailing list
Pkg-systemd-maintainers@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-systemd-maintainers