[systemd-devel] avoid unmounts in unprivileged containers

[Rodny Molina]

Hi,

As part of a prototype I'm working on to run systemd within an unprivileged
docker container, I would like to prevent mountpoints created at runtime
from being unmounted during the container shutdown process. I understand
that systemd creates ".mount" units dynamically for
these mountpoints as they show up in /proc/pid/mountinfo, but after reading
the docs + code, I don't see a way to avoid these unmounts during the
shutdown.target execution.

Interestingly, I see that there's code

that
skips the unmounting cycle attending to the ConditionVirtualization /
containeinarized settings, which is what I need, but I'm not able to see
that code being called during the container shutdown -- probably i'm not
understanding systemd's fsm unwinding logic well enough ...

Any suggestions?

Thanks!

PS: Last few logs obtained during my container shutdown process ...

---
Feb 20 03:00:23 08363a0a79ee umount[1273]: umount: /var/lib/kubelet: must
be superuser to unmount.
Feb 20 03:00:23 08363a0a79ee systemd[1]: Received SIGCHLD from PID 1273
(umount).
Feb 20 03:00:23 08363a0a79ee systemd[1]: Child 1273 (umount) died
(code=exited, status=32/n/a)
Feb 20 03:00:23 08363a0a79ee systemd[1]: var-lib-kubelet.mount: Child 1273
belongs to var-lib-kubelet.mount.
Feb 20 03:00:23 08363a0a79ee systemd[1]: var-lib-kubelet.mount: Mount
process exited, code=exited, status=32/n/a
Feb 20 03:00:23 08363a0a79ee systemd[1]: var-lib-kubelet.mount: Changed
unmounting -> mounted
Feb 20 03:00:23 08363a0a79ee systemd[1]: var-lib-kubelet.mount: Job 180
var-lib-kubelet.mount/stop finished, result=failed
Feb 20 03:00:23 08363a0a79ee systemd[1]: Failed unmounting /var/lib/kubelet.
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-journald.service: Received
EPOLLHUP on stored fd 47 (stored), closing.
Feb 20 03:00:23 08363a0a79ee systemd[1]: local-fs-pre.target changed active
-> dead
Feb 20 03:00:23 08363a0a79ee systemd[1]: local-fs-pre.target: Job 156
local-fs-pre.target/stop finished, result=done
Feb 20 03:00:23 08363a0a79ee systemd[1]: Stopped target Local File Systems
(Pre).
Feb 20 03:00:23 08363a0a79ee systemd[1]: umount.target changed dead ->
active
Feb 20 03:00:23 08363a0a79ee systemd[1]: umount.target: Job 168
umount.target/start finished, result=done
Feb 20 03:00:23 08363a0a79ee systemd[1]: Reached target Unmount All
Filesystems.
Feb 20 03:00:23 08363a0a79ee systemd[1]:
systemd-tmpfiles-setup-dev.service: Succeeded.
Feb 20 03:00:23 08363a0a79ee systemd[1]:
systemd-tmpfiles-setup-dev.service: Service restart not allowed.
Feb 20 03:00:23 08363a0a79ee systemd[1]:
systemd-tmpfiles-setup-dev.service: Changed exited -> dead
Feb 20 03:00:23 08363a0a79ee systemd[1]:
systemd-tmpfiles-setup-dev.service: Job 105
systemd-tmpfiles-setup-dev.service/stop finished, result=done
Feb 20 03:00:23 08363a0a79ee systemd[1]: Stopped Create Static Device Nodes
in /dev.
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-sysusers.service:
Succeeded.
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-sysusers.service: Service
restart not allowed.
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-sysusers.service: Changed
exited -> dead
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-sysusers.service: Job 164
systemd-sysusers.service/stop finished, result=done
Feb 20 03:00:23 08363a0a79ee systemd[1]: Stopped Create System Users.
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-remount-fs.service:
Succeeded.
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-remount-fs.service:
Service restart not allowed.
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-remount-fs.service:
Changed exited -> dead
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-remount-fs.service: Job
117 systemd-remount-fs.service/stop finished, result=done
Feb 20 03:00:23 08363a0a79ee systemd[1]: Stopped Remount Root and Kernel
File Systems.
Feb 20 03:00:23 08363a0a79ee systemd[1]: shutdown.target changed dead ->
active
Feb 20 03:00:23 08363a0a79ee systemd[1]: shutdown.target: Job 89
shutdown.target/start finished, result=done
Feb 20 03:00:23 08363a0a79ee systemd[1]: Reached target Shutdown.
Feb 20 03:00:23 08363a0a79ee systemd[1]: final.target changed dead -> active
Feb 20 03:00:23 08363a0a79ee systemd[1]: final.target: Job 167
final.target/start finished, result=done
Feb 20 03:00:23 08363a0a79ee systemd[1]: Reached target Final Step.
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-halt.service: Failed to
reset devices.allow/devices.deny: Operation not permitted
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-halt.service: Failed to
set invocation ID on control group /system.slice/systemd-halt.service,
ignoring: Operation not permitted
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-halt.service: Failed to
remove delegate flag on control group /system.slice/systemd-halt.service,
ignoring: Operation not permitted
Feb 20 03:00:23 08363a0a79ee systemd[1]: systemd-halt.service: Passing 0
fds to service
Feb 20 03:00:23 08363a0a79ee 

Re: [systemd-devel] systemd-resolved auto configure DNS server changed?

[Ed Greshko]

On 20/02/2021 00:30, Lennart Poettering wrote:

The fallback servers are only used as last resort, if there's nothing
else known. They are *fallback* as the name says.

Most likely the DNS servers were acquire by your network management
solution (NetworkManager or networkd) and set on the device. Maybe
theym come from IPv6 RA?


OK.  I have found that, using wireshark, there is a

Type: Router Solicitation (133)

followed by
Type: Router Advertisement (134)

which contains
ICMPv6 Option (Recursive DNS Server fe80::5054:ff:fe9a:e849)


Then, continuing my research I upgraded systemd to systemd-246.10-1.fc33.  In 
that version
there are no FallbackDNS servers defined by default.

Yeah, i think that's a bad change. I am not sure where the benefit of
having a non-working system is...


Scratching my head on that one as well.


Link 2 (enp1s0)
Current Scopes: LLMNR/IPv4 LLMNR/IPv6
  Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

So, now my question, why wasn't the dnsmasq server found/configured as had been 
the case?
An intentional change or unintentional change?

I am not sure which software manages that interface, but it would be
worth figuring that out, and then checking whether it propagated that
DNS info to resolved.



Well, I determined that in both the systemd-246.6-3 and systemd-246.10-1 cases 
(the only changes made)
the same Router Solicitation and Router Advertisement occur.

So, the only conclusion that I can come to is that something changed between 
the two versions of
systemd which results in the Recursive DNS Server option being ignored.

Would you consider this a candidate for a bug report against systemd?


___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Greg KH]

On Fri, Feb 19, 2021 at 06:09:16PM +0200, Mantas Mikulėnas wrote:
> On Fri, Feb 19, 2021 at 4:49 PM Lennart Poettering 
> wrote:
> 
> > On Fr, 19.02.21 09:28, Robert P. J. Day (rpj...@crashcourse.ca) wrote:
> >
> > > i guess i expected that the CVE identifier would be in the commit
> > > message. anyway, time to examine ...
> >
> > CVEs are assigned/published long after the commits to fix the issues
> > are made. We cannot retroactively change git commits, that's just not
> > how this works.
> >
> 
> This *could* work with git notes, it seems --grep searches them as well.

Git notes do not work for anything but a local repo, sorry.

if people really care about CVEs, they know how to use them.  But
really, they are mostly useless:

https://kernel-recipes.org/en/2019/talks/cves-are-dead-long-live-the-cve/

greg k-h
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Unprivileged user can kill root-owned processes by changing PID file and stopping service

[Reindl Harald]

Am 19.02.21 um 21:05 schrieb Frank Thommen:



Lennart Poettering  hat am 19.02.2021 15:44 geschrieben:

  
On Fr, 19.02.21 15:12, Frank Thommen (systemd-de...@lists.drosera.ch) wrote:



Dear all,

I am experiencing the issue, that an unprivileged user can kill
root-owned processes by changing a service's PIDFile.


The file referenced by PIDFile= should not be under control of an
unpriv user.

v219 is more than 5 years old. Since then we have tightened controls:


I am aware of this, but unfortunately for the time being we are stuck with this 
version (CentOS 7.4)


i yet need to see a real world usecase which needs "PIDFile=" at all - 
systemd kills everything in the cgroup anyways at stop


i even start mariadb with --pid-file=/dev/null and without "mysqlsafe" 
for years to get rid of all that shit


not a single service is using "PIDFile=" for years here and frankly i 
even forked systemd units only to get rid of that nosense from the 1990s

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Unprivileged user can kill root-owned processes by changing PID file and stopping service

[Frank Thommen]

> Lennart Poettering  hat am 19.02.2021 15:44 
> geschrieben:
> 
>  
> On Fr, 19.02.21 15:12, Frank Thommen (systemd-de...@lists.drosera.ch) wrote:
> 
> > Dear all,
> >
> > I am experiencing the issue, that an unprivileged user can kill
> > root-owned processes by changing a service's PIDFile.
> 
> The file referenced by PIDFile= should not be under control of an
> unpriv user.
> 
> v219 is more than 5 years old. Since then we have tightened controls:

I am aware of this, but unfortunately for the time being we are stuck with this 
version (CentOS 7.4).


> we now automatically detect wether the PID file is under control of
> unprivileged users either directly, or because a symlink is used in
> the path that is controlled by an unprivileged user, in which case
> we'll log abou this. We'll also ignore the PID file if the listed PID
> doesn't actually belong to the cgroup of the service.
> 
> See documentation about PIDFile= in current versions:
> 
> https://www.freedesktop.org/software/systemd/man/systemd.service.html#PIDFile=
> 
> But in general: don't do this! It's simply not safe, neither on
> systemd nor any other init system. The whole PID concept of UNIX is
> racy anyway but giving unprivileged users control on it is even worse.
> 
> PID files are mostly SysV construct. A better replacement is using
> Type=notify or Type=simple services that do not fork unnecessarily,
> and thus do not need to communicate their main PID explicitly.

I understand that, but whatever is required for the "notify" service type is 
probably not a core competence of those who wrote the current service :-). I 
have therefore created a special service account which shares the group with 
the developers' account, so that it can read all required data but the 
developers cannot modify any of the service's files like the PIDFile and the 
start script.  That works quite fine and we can probably use this setup as a 
template for future "mini services"

Cheers and thanks for your feedback,
Frank, Heidelberg


> 
> Lennart
> 
> --
> Lennart Poettering, Berlin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Robert P. J. Day]

On Fri, 19 Feb 2021, Lennart Poettering wrote:

> On Fr, 19.02.21 18:09, Mantas Mikulėnas (graw...@gmail.com) wrote:
>
> > On Fri, Feb 19, 2021 at 4:49 PM Lennart Poettering 
> > wrote:
> >
> > > On Fr, 19.02.21 09:28, Robert P. J. Day (rpj...@crashcourse.ca) wrote:
> > >
> > > > i guess i expected that the CVE identifier would be in the commit
> > > > message. anyway, time to examine ...
> > >
> > > CVEs are assigned/published long after the commits to fix the issues
> > > are made. We cannot retroactively change git commits, that's just not
> > > how this works.
> > >
> >
> > This *could* work with git notes, it seems --grep searches them as well.
>
> We used to attach security + backport info via git notes onto our
> commits, but github doesn#t show them/support them. They were
> basically invisible, noone knew they were there. Thus we eventually
> stopped doing them.
>
> If github would integrate git notes into their UI somehow this would
> be grand.

  i suspect that won't happen any time soon:

https://www.quora.com/Why-does-GitHub-no-longer-support-git-notes

rday___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] systemd-resolved auto configure DNS server changed?

[Lennart Poettering]

On Fr, 19.02.21 16:29, Ed Greshko (ed.gres...@greshko.com) wrote:

> Link 2 (enp1s0)
>   Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
> DefaultRoute setting: yes
>    LLMNR setting: yes
> MulticastDNS setting: no
>   DNSOverTLS setting: no
>   DNSSEC setting: no
>     DNSSEC supported: no
>   Current DNS Server: fe80::5054:ff:fe9a:e849%32767
>  DNS Servers: fe80::5054:ff:fe9a:e849%22096
>   DNS Domain: ~.
>
> The IPv6 address of fe80::5054:ff:fe9a:e849 is that of the Virtual Bridge and 
> wireshark does confirm
> DNS requests are being sent to that address' port 53 where dnsmasq is running.
>
> I have no idea how systemd-resolved discovered this server?  Why wasn't a 
> Fallback Server
> selected used?

The fallback servers are only used as last resort, if there's nothing
else known. They are *fallback* as the name says.

Most likely the DNS servers were acquire by your network management
solution (NetworkManager or networkd) and set on the device. Maybe
theym come from IPv6 RA?

> Then, continuing my research I upgraded systemd to systemd-246.10-1.fc33.  In 
> that version
> there are no FallbackDNS servers defined by default.

Yeah, i think that's a bad change. I am not sure where the benefit of
having a non-working system is...

> Link 2 (enp1s0)
> Current Scopes: LLMNR/IPv4 LLMNR/IPv6
>  Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
>
> So, now my question, why wasn't the dnsmasq server found/configured as had 
> been the case?
> An intentional change or unintentional change?

I am not sure which software manages that interface, but it would be
worth figuring that out, and then checking whether it propagated that
DNS info to resolved.

Lennart

--
Lennart Poettering, Berlin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Lennart Poettering]

On Fr, 19.02.21 18:09, Mantas Mikulėnas (graw...@gmail.com) wrote:

> On Fri, Feb 19, 2021 at 4:49 PM Lennart Poettering 
> wrote:
>
> > On Fr, 19.02.21 09:28, Robert P. J. Day (rpj...@crashcourse.ca) wrote:
> >
> > > i guess i expected that the CVE identifier would be in the commit
> > > message. anyway, time to examine ...
> >
> > CVEs are assigned/published long after the commits to fix the issues
> > are made. We cannot retroactively change git commits, that's just not
> > how this works.
> >
>
> This *could* work with git notes, it seems --grep searches them as well.

We used to attach security + backport info via git notes onto our
commits, but github doesn#t show them/support them. They were
basically invisible, noone knew they were there. Thus we eventually
stopped doing them.

If github would integrate git notes into their UI somehow this would
be grand.

Lennart

--
Lennart Poettering, Berlin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Mantas Mikulėnas]

On Fri, Feb 19, 2021 at 4:49 PM Lennart Poettering 
wrote:

> On Fr, 19.02.21 09:28, Robert P. J. Day (rpj...@crashcourse.ca) wrote:
>
> > i guess i expected that the CVE identifier would be in the commit
> > message. anyway, time to examine ...
>
> CVEs are assigned/published long after the commits to fix the issues
> are made. We cannot retroactively change git commits, that's just not
> how this works.
>

This *could* work with git notes, it seems --grep searches them as well.

-- 
Mantas Mikulėnas
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Robert P. J. Day]

On Fri, 19 Feb 2021, Lennart Poettering wrote:

> On Fr, 19.02.21 09:28, Robert P. J. Day (rpj...@crashcourse.ca) wrote:
>
> > i guess i expected that the CVE identifier would be in the commit
> > message. anyway, time to examine ...
>
> CVEs are assigned/published long after the commits to fix the issues
> are made. We cannot retroactively change git commits, that's just not
> how this works.

  oh, i get that. i have my commit so it's time to see if it fixes the
problem. thanks all for the assistance.

rday
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Lennart Poettering]

On Fr, 19.02.21 09:28, Robert P. J. Day (rpj...@crashcourse.ca) wrote:

> i guess i expected that the CVE identifier would be in the commit
> message. anyway, time to examine ...

CVEs are assigned/published long after the commits to fix the issues
are made. We cannot retroactively change git commits, that's just not
how this works.

Lennart

--
Lennart Poettering, Berlin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Antw: [EXT] Re: Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Lennart Poettering]

On Fr, 19.02.21 08:44, Ulrich Windl (ulrich.wi...@rz.uni-regensburg.de) wrote:

> >>> Lennart Poettering  schrieb am 18.02.2021 um 19:30
> in
> Nachricht :
> ...
> > entry instead of asking for new memory again. This allocation cache is
> > a bit quicker then going to malloc() all the time, but means if you
> > just watch the heap you'll assume there's a leak even though there
> > isn't really, the memory is not lost after all, and will be reused
> > eventually if we need it.
>
> That's an interesting point of view: If you save memory in case you might need
> it at some unspecified later time (which includes "never") it's "practically"
> (while not theoretically) a memory leak.

Allocation caches are a common technique. In systemd, but everywhere
else too. glibc's malloc() itself is one too actually (i.e. it's a
cache in front of kernel mmap()/sbrk()). Internally in the kernel
there are multiple different allocation caches in place as well.

If you have an issue with allocation caches, I am sorry, but modern
Linux kernel and userspace is not for you.

Lennart

--
Lennart Poettering, Berlin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Unprivileged user can kill root-owned processes by changing PID file and stopping service

[Lennart Poettering]

On Fr, 19.02.21 15:12, Frank Thommen (systemd-de...@lists.drosera.ch) wrote:

> Dear all,
>
> I am experiencing the issue, that an unprivileged user can kill
> root-owned processes by changing a service's PIDFile.

The file referenced by PIDFile= should not be under control of an
unpriv user.

v219 is more than 5 years old. Since then we have tightened controls:
we now automatically detect wether the PID file is under control of
unprivileged users either directly, or because a symlink is used in
the path that is controlled by an unprivileged user, in which case
we'll log abou this. We'll also ignore the PID file if the listed PID
doesn't actually belong to the cgroup of the service.

See documentation about PIDFile= in current versions:

https://www.freedesktop.org/software/systemd/man/systemd.service.html#PIDFile=

But in general: don't do this! It's simply not safe, neither on
systemd nor any other init system. The whole PID concept of UNIX is
racy anyway but giving unprivileged users control on it is even worse.

PID files are mostly SysV construct. A better replacement is using
Type=notify or Type=simple services that do not fork unnecessarily,
and thus do not need to communicate their main PID explicitly.

Lennart

--
Lennart Poettering, Berlin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Unprivileged user can kill root-owned processes by changing PID file and stopping service

[Frank Thommen]

Dear all,

I am experiencing the issue, that an unprivileged user can kill root-owned 
processes by changing a service's PIDFile.

Situation: We are running a web service based on a software which is maintained 
by "external" developers. The service is running as an unprivileged user and 
the maintaing developers have access to the server and to this user account to 
make updates and apply fixes themselves and independently from the system 
administration.


In a nutshell we have:

a) an unprivileged user "srvcusr", where "external" persons have access to

b) a start script /path1/to/startscript.sh which basically does
--
#!/bin/bash
PIDFILE=/path2/to/service.pid
[... initialize the environment ...]
run_service_script &
echo $! > $PIDFILE
--
"srvcusr" cannot modify this startscript!

c) a unit file with (in very short):
--
Type=simple
User=srvcusr
ExecStart=/path1/to/startscript.sh
PIDFile=/path2/to/service.pid
--

d) a `sudo` configuration which allows "srvcusr" to start and stop the service

Problem: To run the service as "srvcusr", this accounts needs write access to 
$PIDFILE.  However this also allows the user to write arbitrary PIDs to the 
file.  Once (s)he has done so and stops the service (`sudo systemctl stop 
myservice`), this process will be killed even if it doesn't belong to 
"srvcusr".  It doesn't work with PID=1 but it works with webservers, rootshell 
ecc. ecc.

This is either a hole in systemd (which I cannot imagine) or a wrong usage of 
running a service on behalf of an non-root UID.

This happens on CentOS 7.4.170 (for technical reasons we are currently bound to 
this version) with systemd version 219, release 42.

Any hint on how to fix this is very appreciated.
Thanks, Frank
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Robert P. J. Day]

On Fri, 19 Feb 2021, Reindl Harald wrote:

>
>
> Am 19.02.21 um 11:28 schrieb Robert P. J. Day:
> >I *may* have found the problem ... as one can read here:
> >
> > https://access.redhat.com/solutions/3840481
> >
> > "CVE-2019-3815 systemd: memory leak in journald-server.c introduced by
> > fix for CVE-2018-16864"
> >
> >So as I interpret that, a memory leak introduced by that earlier CVE
> > had to be corrected by that later CVE. I checked the state of
> > systemd_230 as shipped by WRL9, and it comes with an extensive set of
> > patches, which includes the earlier CVE, but *not* the later one.
> >
> >Hmmm ...
>
> that one should have been fixed long ago
> https://bugzilla.redhat.com/show_bug.cgi?id=1665931

  while i'm jumping back into this, a possibly silly question ...
where in the systemd commit log can i find the commit corresponding to
the "fix" associated with CVE-2019-3815?

  i wanted to see the original commit that introduced the bug, so i
naturally used:

  $ git log --grep="CVE-2018-16864"

and there it was. but i get nothing from:

  $ git log --grep="CVE-2019-3815"

after a bit of reading, i landed on:

  $ git log -p --grep="don't use overly large buffer"

  commit eb1ec489eef8a32918bbfc56a268c9d10464584d
  Author: Michal Sekletár 
  Date:   Tue Jan 22 14:29:50 2019 +0100

process-util: don't use overly large buffer to store process command line

Allocate new string as a return value and free our "scratch pad"
buffer that is potentially much larger than needed (up to
_SC_ARG_MAX).

Fixes #11502

... etc etc ...

i guess i expected that the CVE identifier would be in the commit
message. anyway, time to examine ...

rday

p.s. my first concern is whether this is a standalone patch that i can
shoehorn into systemd_230, or whether i can just drag the entire
version forward to the point where it incorporates that fix. that
might take a bit of work given the distance involved.___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Robert P. J. Day]

On Fri, 19 Feb 2021, Reindl Harald wrote:

>
>
> Am 19.02.21 um 11:28 schrieb Robert P. J. Day:
> >I *may* have found the problem ... as one can read here:
> >
> > https://access.redhat.com/solutions/3840481
> >
> > "CVE-2019-3815 systemd: memory leak in journald-server.c introduced by
> > fix for CVE-2018-16864"
> >
> >So as I interpret that, a memory leak introduced by that earlier CVE
> > had to be corrected by that later CVE. I checked the state of
> > systemd_230 as shipped by WRL9, and it comes with an extensive set of
> > patches, which includes the earlier CVE, but *not* the later one.
> >
> >Hmmm ...
>
> that one should have been fixed long ago
> https://bugzilla.redhat.com/show_bug.cgi?id=1665931

  yes, that fix is from a while ago, but the issue here is that it
wasn't incorporated in the patch set for wind river linux 9, which is
a few years old, so it's not at all surprising that WRL9 is not
keeping up with current patches.

rday
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Reindl Harald]

Am 19.02.21 um 11:28 schrieb Robert P. J. Day:

   I *may* have found the problem ... as one can read here:

https://access.redhat.com/solutions/3840481

"CVE-2019-3815 systemd: memory leak in journald-server.c introduced by
fix for CVE-2018-16864"

   So as I interpret that, a memory leak introduced by that earlier CVE
had to be corrected by that later CVE. I checked the state of
systemd_230 as shipped by WRL9, and it comes with an extensive set of
patches, which includes the earlier CVE, but *not* the later one.

   Hmmm ...


that one should have been fixed long ago
https://bugzilla.redhat.com/show_bug.cgi?id=1665931


but your original mail didn't talk about journald at all


 Weitergeleitete Nachricht 
Betreff: [systemd-devel] Looking for known memory leaks triggered by 
stress testing add/remove/up/down interfaces

Datum: Thu, 18 Feb 2021 11:48:58 -0500 (EST)
Von: Robert P. J. Day 
An: Systemd mailing list 


  A colleague has reported the following apparent issue in a fairly
old (v230) version of systemd -- this is in a Yocto Project Wind River
Linux 9 build, hence the age of the package.

  As reported to me (and I'm gathering more info), the system was
being put through some "longevity testing" by repeatedly adding,
removing, activating and de-activating network interfaces. According
to the report, the result was heap space slowly but inexorably being
consumed.

  While waiting for more info, I'm going to examine the commit log for
systemd from v230 moving forward to collect any commits that address
memory leaks, then peruse them more carefully to see if they might
resolve the problem.

  I realize it's asking a bit for folks here to remember that far
back, but does this issue sound at all familiar? Any pointers that
might save me some time? Thanks.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Robert P. J. Day]

On Thu, 18 Feb 2021, Lennart Poettering wrote:

> On Do, 18.02.21 11:48, Robert P. J. Day (rpj...@crashcourse.ca) wrote:
>
> >   A colleague has reported the following apparent issue in a fairly
> > old (v230) version of systemd -- this is in a Yocto Project Wind River
> > Linux 9 build, hence the age of the package.
> >
> >   As reported to me (and I'm gathering more info), the system was
> > being put through some "longevity testing" by repeatedly adding,
> > removing, activating and de-activating network interfaces. According
> > to the report, the result was heap space slowly but inexorably being
> > consumed.
> >
> >   While waiting for more info, I'm going to examine the commit log for
> > systemd from v230 moving forward to collect any commits that address
> > memory leaks, then peruse them more carefully to see if they might
> > resolve the problem.
> >
> >   I realize it's asking a bit for folks here to remember that far
> > back, but does this issue sound at all familiar? Any pointers that
> > might save me some time? Thanks.
>
> Note that our hash tables operate with an allocation cache: when
> adding entries to them and then removing them again the memory
> required for that is not returned to the OS but added to a local
> cache. When the next entry is then added again, we recycle the cached
> entry instead of asking for new memory again. This allocation cache is
> a bit quicker then going to malloc() all the time, but means if you
> just watch the heap you'll assume there's a leak even though there
> isn't really, the memory is not lost after all, and will be reused
> eventually if we need it.
>
> You may use the env var SYSTEMD_MEMPOOL=0 to turn this logic off, but
> not sure v230 already knew that env var.

  I *may* have found the problem ... as one can read here:

https://access.redhat.com/solutions/3840481

"CVE-2019-3815 systemd: memory leak in journald-server.c introduced by
fix for CVE-2018-16864"

  So as I interpret that, a memory leak introduced by that earlier CVE
had to be corrected by that later CVE. I checked the state of
systemd_230 as shipped by WRL9, and it comes with an extensive set of
patches, which includes the earlier CVE, but *not* the later one.

  Hmmm ...

rday
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Robert P. J. Day]

On Thu, 18 Feb 2021, Lennart Poettering wrote:

> On Do, 18.02.21 11:48, Robert P. J. Day (rpj...@crashcourse.ca) wrote:
>
> >   A colleague has reported the following apparent issue in a fairly
> > old (v230) version of systemd -- this is in a Yocto Project Wind River
> > Linux 9 build, hence the age of the package.
> >
> >   As reported to me (and I'm gathering more info), the system was
> > being put through some "longevity testing" by repeatedly adding,
> > removing, activating and de-activating network interfaces. According
> > to the report, the result was heap space slowly but inexorably being
> > consumed.
> >
> >   While waiting for more info, I'm going to examine the commit log for
> > systemd from v230 moving forward to collect any commits that address
> > memory leaks, then peruse them more carefully to see if they might
> > resolve the problem.
> >
> >   I realize it's asking a bit for folks here to remember that far
> > back, but does this issue sound at all familiar? Any pointers that
> > might save me some time? Thanks.
>
> Note that our hash tables operate with an allocation cache: when
> adding entries to them and then removing them again the memory
> required for that is not returned to the OS but added to a local
> cache. When the next entry is then added again, we recycle the cached
> entry instead of asking for new memory again. This allocation cache is
> a bit quicker then going to malloc() all the time, but means if you
> just watch the heap you'll assume there's a leak even though there
> isn't really, the memory is not lost after all, and will be reused
> eventually if we need it.
>
> You may use the env var SYSTEMD_MEMPOOL=0 to turn this logic off, but
> not sure v230 already knew that env var.

  One more observation before I dive head-first into debugging this --
I just logged into the embedded system in question after several
hours, and "top" shows systemd, RES = 2.706g ... 2.707g ... 2.708g,
bumping up every 30 seconds or so, so something is definitely eating
memory.

rday
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] systemd-resolved auto configure DNS server changed?

[Ed Greshko]

First a little background.  I'm using a Fedora 33 system in a qemu VM.  I was 
doing some research
on a question which arose on a Fedora mailing list regarding changes to 
FallbackDNS.  I don't know
if this change was universal or Fedora only.  But a recent update changed the 
default to have no
FallbackDNS servers defined.

In doing my research I used the default install of Fedora 33 which is running 
systemd-246.6-3.fc33.
I did not supply a DNS server in the static IP settings and I purposely created 
a broken
/etc/systemd/resolved.conf file with the bad entry of

DNS=192.168.1.142,192.168.1.1

DNS resolution works and I fully expected that one of the defined FallbackDNS 
servers would be used.
However, resolvectl shows

Global
   LLMNR setting: resolve
MulticastDNS setting: no
  DNSOverTLS setting: no
  DNSSEC setting: no
    DNSSEC supported: no
Fallback DNS Servers: 1.1.1.1
  8.8.8.8
  1.0.0.1
  8.8.4.4
  2606:4700:4700::
  2001:4860:4860::
  2606:4700:4700::1001
  2001:4860:4860::8844
  DNS Domain: greshko.com

Link 2 (enp1s0)
  Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
DefaultRoute setting: yes
   LLMNR setting: yes
MulticastDNS setting: no
  DNSOverTLS setting: no
  DNSSEC setting: no
    DNSSEC supported: no
  Current DNS Server: fe80::5054:ff:fe9a:e849%32767
 DNS Servers: fe80::5054:ff:fe9a:e849%22096
  DNS Domain: ~.

The IPv6 address of fe80::5054:ff:fe9a:e849 is that of the Virtual Bridge and 
wireshark does confirm
DNS requests are being sent to that address' port 53 where dnsmasq is running.

I have no idea how systemd-resolved discovered this server?  Why wasn't a 
Fallback Server
selected used?

Then, continuing my research I upgraded systemd to systemd-246.10-1.fc33.  In 
that version
there are no FallbackDNS servers defined by default.

Owing to previous behavior I was expecting DNS resolution to still work.  (Not 
that I really wanted it to)
But it didn't.

[egreshko@f33T ~]$ host cnn.com
Host cnn.com not found: 2(SERVFAIL)

and

[egreshko@f33T ~]$ resolvectl
Global
   Protocols: LLMNR=resolve -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
  DNS Domain: greshko.com

Link 2 (enp1s0)
Current Scopes: LLMNR/IPv4 LLMNR/IPv6
 Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

So, now my question, why wasn't the dnsmasq server found/configured as had been 
the case?
An intentional change or unintentional change?
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Antw: Re: Antw: [EXT] Re: Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Ulrich Windl]

>>> Reindl Harald  schrieb am 19.02.2021 um 09:13 in
Nachricht <068b561d-677f-1abf-ef9f-0f43571e1...@thelounge.net>:

> 
> Am 19.02.21 um 08:44 schrieb Ulrich Windl:
> Lennart Poettering  schrieb am 18.02.2021 um 19:30
>> in
>> Nachricht :
>> ...
>>> entry instead of asking for new memory again. This allocation cache is
>>> a bit quicker then going to malloc() all the time, but means if you
>>> just watch the heap you'll assume there's a leak even though there
>>> isn't really, the memory is not lost after all, and will be reused
>>> eventually if we need it.
>> 
>> That's an interesting point of view: If you save memory in case you might 
> need
>> it at some unspecified later time (which includes "never") it's 
> "practically"
>> (while not theoretically) a memory leak
> 
> following that logic every memory pool and mysql buffer would be a memleak

Database memory pools typically have a settable upper-bound.

> 
> a memleak is *unintended* and never freed memory allocation, practically 
> as well as theoretically

So for every memory leak the programmer just has to say "that's intentional" 
and it's no longer a memory leak ;-)

> 
> you can argue about the size of such cases but it don't make them a 
> memleak to begin with

With Lennart's explanation there are two unbounded dimensions: 1) the amount of 
"cache" 2) the time until actual re-use
(With 2) leading to 1) it seems)

Regards,
Ulrich





___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Antw: [EXT] Re: Looking for known memory leaks triggered by stress testing add/remove/up/down interfaces

[Reindl Harald]

Am 19.02.21 um 08:44 schrieb Ulrich Windl:

Lennart Poettering  schrieb am 18.02.2021 um 19:30

in
Nachricht :
...

entry instead of asking for new memory again. This allocation cache is
a bit quicker then going to malloc() all the time, but means if you
just watch the heap you'll assume there's a leak even though there
isn't really, the memory is not lost after all, and will be reused
eventually if we need it.


That's an interesting point of view: If you save memory in case you might need
it at some unspecified later time (which includes "never") it's "practically"
(while not theoretically) a memory leak


following that logic every memory pool and mysql buffer would be a memleak

a memleak is *unintended* and never freed memory allocation, practically 
as well as theoretically


you can argue about the size of such cases but it don't make them a 
memleak to begin with

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel