Re: [systemd-devel] Serial getty on USB dongle insertion

[Andrei Borzenkov]

13.07.2017 03:38, Paul D. DeRocco пишет:
> I've created a serial-getty@ttyUSB2.service file, which is basically a
> copy of serial-getty@.service with a specific device name and a fixed baud
> rate. It works fine when I plug in a serial dongle that gets assigned that
> name. (I have permanent ttyUSB0 and ttyUSB1 devices which don't use
> getty.)
> 
> If I logout, it just offers me a new login prompt. If I unplug the dongle,
> the automatic retrying fails, and plugging it in again doesn't bring it
> back to life.
> 
> If I use Restart=no, then I can terminate it either by logging out or
> yanking the dongle, but it naturally won't restart if I plug the dongle in
> again. Is there a way of configuring the service to start every time the
> dongle is inserted, whether I logged out before yanking the dongle or not?
> Or does this require some special udev rule?
> 

Yes, set ENV{SYSTEMD_WANTS}="serial-getty@ttyUSB2.service" property on
your device in udev rule; this will trigger starting of this service
when device appears. You probably also want to add
"BindsTo=dev-ttyUSB2.device" (or whatever name of your device unit is)
so systemd stops it when dongle is unplugged and does not attempt to
restart.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Serial getty on USB dongle insertion

[Paul D. DeRocco]

I've created a serial-getty@ttyUSB2.service file, which is basically a
copy of serial-getty@.service with a specific device name and a fixed baud
rate. It works fine when I plug in a serial dongle that gets assigned that
name. (I have permanent ttyUSB0 and ttyUSB1 devices which don't use
getty.)

If I logout, it just offers me a new login prompt. If I unplug the dongle,
the automatic retrying fails, and plugging it in again doesn't bring it
back to life.

If I use Restart=no, then I can terminate it either by logging out or
yanking the dongle, but it naturally won't restart if I plug the dongle in
again. Is there a way of configuring the service to start every time the
dongle is inserted, whether I logged out before yanking the dongle or not?
Or does this require some special udev rule?

-- 

Ciao,   Paul D. DeRocco
Paulmailto:pdero...@ix.netcom.com

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] [ANNOUNCE] systemd 234

[Lennart Poettering]

Heya!

Here we go:

https://github.com/systemd/systemd/releases/tag/v234

CHANGES WITH 234:

* Meson is now supported as build system in addition to Automake. It is
  our plan to remove Automake in one of our next releases, so that
  Meson becomes our exclusive build system. Hence, please start using
  the Meson build system in your downstream packaging. There's plenty
  of documentation around how to use Meson, the extremely brief
  summary:

  ./autogen.sh && ./configure && make && sudo make install

  becomes:

  meson build && ninja -C build && sudo ninja -C build install

* Unit files gained support for a new JobRunningTimeoutUSec= setting,
  which permits configuring a timeout on the time a job is
  running. This is particularly useful for setting timeouts on jobs for
  .device units.

* Unit files gained two new options ConditionUser= and ConditionGroup=
  for conditionalizing units based on the identity of the user/group
  running a systemd user instance.

* systemd-networkd now understands a new FlowLabel= setting in the
  [VXLAN] section of .network files, as well as a Priority= in
  [Bridge], GVRP= + MVRP= + LooseBinding= + ReorderHeader= in [VLAN]
  and GatewayOnlink= + IPv6Preference= + Protocol= in [Route]. It also
  gained support for configuration of GENEVE links, and IPv6 address
  labels. The [Network] section gained the new IPv6ProxyNDP= setting.

* .link files now understand a new Port= setting.

* systemd-networkd's DHCP support gained support for DHCP option 119
  (domain search list).

* systemd-networkd gained support for serving IPv6 address ranges using
  the Router Advertisment protocol. The new .network configuration
  section [IPv6Prefix] may be used to configure the ranges to
  serve. This is implemented based on a new, minimal, native server
  implementation of RA.

* journalctl's --output= switch gained support for a new parameter
  "short-iso-precise" for a mode where timestamps are shown as precise
  ISO date values.

* systemd-udevd's "net_id" builtin may now generate stable network
  interface names from IBM PowerVM VIO devices as well as ACPI platform
  devices.

* MulticastDNS support in systemd-resolved may now be explicitly
  enabled/disabled using the new MulticastDNS= configuration file
  option.

* systemd-resolved may now optionally use libidn2 instead of the libidn
  for processing internationalized domain names. Support for libidn2
  should be considered experimental and should not be enabled by
  default yet.

* "machinectl pull-tar" and related call may now do verification of
  downloaded images using SUSE-style .sha256 checksum files in addition
  to the already existing support for validating using Ubuntu-style
  SHA256SUMS files.

* sd-bus gained support for a new sd_bus_message_appendv() call which
  is va_list equivalent of sd_bus_message_append().

* sd-boot gained support for validating images using SHIM/MOK.

* The SMACK code learnt support for "onlycap".

* systemd-mount --umount is now much smarter in figuring out how to
  properly unmount a device given its mount or device path.

* The code to call libnss_dns as a fallback from libnss_resolve when
  the communication with systemd-resolved fails was removed. This
  fallback was redundant and interfered with the [!UNAVAIL=return]
  suffix. See nss-resolve(8) for the recommended configuration.

* systemd-logind may now be restarted without losing state. It stores
  the file descriptors for devices it manages in the system manager
  using the FDSTORE= mechanism. Please note that further changes in
  other components may be required to make use of this (for example
  Xorg has code to listen for stops of systemd-logind and terminate
  itself when logind is stopped or restarted, in order to avoid using
  stale file descriptors for graphical devices, which is now
  counterproductive and must be reverted in order for restarts of
  systemd-logind to be safe. See
  https://cgit.freedesktop.org/xorg/xserver/commit/?id=dc48bd653c7e101.)

* All kernel install plugins are called with the environment variable
  KERNEL_INSTALL_MACHINE_ID which is set to the machine ID given by
  /etc/machine-id. If the file is missing or empty, the variable is
  empty and BOOT_DIR_ABS is the path of a temporary directory which is
  removed after all the plugins exit. So, if KERNEL_INSTALL_MACHINE_ID
  is empty, all plugins should not put anything in 

Re: [systemd-devel] Trying to come up with udev rule for USB geographic naming

[Paul D. DeRocco]

> From: Andrei Borzenkov
> 
> Please copy-paste actual command invocation and its output.

No need. I had used = instead of == on the DRIVERS. Works fine now. Thanks for 
your assistance.

-- 

Ciao,   Paul D. DeRocco
Paulmailto:pdero...@ix.netcom.com

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Realtime scheduling with CONFIG_RT_GROUP_SCHED=y

[Lars Kellogg-Stedman]

On Thu, Jul 6, 2017 at 10:56 AM, Lars Kellogg-Stedman 
wrote:

> In older versions of systemd one could handle this using the directives
> described in https://www.freedesktop.org/wiki/Software/systemd/
> MyServiceCantGetRealtime/, but unfortunately that document, despite being
> the number 1 search result for pretty much anything involving "systemd" and
> "realtime", is obsolete and those directives no longer exist.
>
> Is there a way to make this work correctly with modern versions of
> systemd?  I've hacked around it for now by creating
> /etc/systemd/system/myservice.service.d/realtime.conf that moves the
> service back to the root cgroup and then uses chrt to set the scheduling
> policy:
>

It looks like systemd sets up cgroups before calling ExecStartPre, which
means I can emulate the behavior of those obsolete directives by running:

ExecStartPre=/bin/sh -c 'echo 55 >
/sys/fs/cgroup/cpu,cpuacct/cpu.rt_runtime_us'
ExecStartPre=/bin/sh -c 'echo 20 >
/sys/fs/cgroup/cpu,cpuacct/system.slice/cpu.rt_runtime_us'
ExecStartPre=/bin/sh -c 'echo 20 >
/sys/fs/cgroup/cpu,cpuacct/system.slice/myservice.service/cpu.rt_runtime_us'

In an environment where CONFIG_RT_GROUP_SCHED is set, is this the best way
of addressing the problem?

-- 
Lars Kellogg-Stedman 
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] systemd-networkd not sending DHCP v6 requests

[Ian Pilcher]

On 07/12/2017 12:06 AM, Mantas Mikulėnas wrote:
What global flags do each network's RAs have? If I remember correctly, 
there are two, "Managed Addresses" and "Managed Other", which trigger 
DHCPv6 – if neither of them is set, that is supposed to mean DHCPv6 is 
unneeded.


The managed and other config flags are both set:

 09:07:36.859265 00:01:5c:77:bc:46 > 33:33:00:00:00:01, ethertype IPv6 
(0x86dd), length 70: (hlim 255, next-header ICMPv6 (58) payload length: 
16) fe80::201:5cff:fe77:bc46 > ff02::1: [icmp6 sum ok] ICMP6, router 
advertisement, length 16
 hop limit 0, Flags [managed, other stateful], pref medium, 
router lifetime 1800s, reachable time 360ms, retrans time 0ms


--

Ian Pilcher arequip...@gmail.com
 "I grew up before Mark Zuckerberg invented friendship" 

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Does Journal conf Max Use configs applies to remote journal as well

[P.R.Dinesh]

Does the journal configurations like "SystemMaxUse=, SystemKeepFree=,
SystemMaxFileSize=, SystemMaxFiles=, RuntimeMaxUse=, RuntimeKeepFree=,
RuntimeMaxFileSize=, RuntimeMaxFiles=" applies to the remote journal files
(created by systemd-journal-remote in the /var/log/journal/remote or
/run/log/journal/remote folders ) as well.

If not, how could we size limit or log rotate these remote journal files

Thank you
Regards,
Dinesh
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Florian Weimer]

* Lennart Poettering:

> On Wed, 12.07.17 09:53, Florian Weimer (f...@deneb.enyo.de) wrote:
>
>> * Lennart Poettering:
>> 
>> > On Tue, 11.07.17 21:24, Florian Weimer (f...@deneb.enyo.de) wrote:
>> >
>> >> * Lennart Poettering:
>> >> 
>> >> > This all stems from my experiences with PulseAudio back in the day:
>> >> > People do not grok the effect of fork(): it only duplicates the
>> >> > invoking thread, not any other threads of the process, moreover all
>> >> > data structures are copied as they are, and that's a time bomb really:
>> >> 
>> >> These days, the PID can change even without a fork, so the story is a
>> >> bit different.
>> >
>> > Can you elaborate?
>> 
>> I'm no longer sure that the PID can change with the current kernel,
>> but I cannot rule it out, either.  But other weirdness is possible:
>> for example, after a fork, getpid and getppid could be equal.
>
> Please be less vague. So I understand correctly that the PID cannot
> change without fork after all?

I don't know.  I don't see such a guarantee provided by the kernel.

I think the important aspect is that as far as glibc is concerned, the
PID *can* change even if its fork and vfork functions are never
called, so we cannot reliably invalidate a cache.

We could decide that it is undefined to call the fork/vfork/clone
system calls directly, but that would break systemd functionality, I
think.

>> > Are you talking about cases where you invoke clone() directly, instead
>> > of via glibc's wrappers? We do that too in systemd, but I am not sure
>> > this is really reason enough to introduce this regression in glibc:
>> > this is easily worked around (which we do in systemd), and given that
>> > the time between clone() and execve() should be short, and the
>> > code between the two minimal this isn't really much of a problem.
>> 
>> There are other uses of clone which do not immediately lead to an
>> execve.
>
> Can you elaborate? Have a real-life user of this?

src/nspawn/nspawn.c?  A lot of things happen in the inner_child
function.

>> > Where was this discussed in detail? Do you have any links about the
>> > discussions about this?
>> 
>> It was on libc-alpha and the glibc bug tracker.
>
> Link?

https://sourceware.org/ml/libc-alpha/2016-10/msg00136.html
https://sourceware.org/ml/libc-alpha/2016-10/msg00233.html
https://sourceware.org/ml/libc-alpha/2016-11/msg00247.html
https://sourceware.org/ml/libc-alpha/2016-11/msg00303.html
https://www.postgresql.org/message-id/e1ukzbn-0006c6...@gemulon.postgresql.org

The last reason explains why using getpid for fork detection does not
work in the way one would expect.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Florian Weimer]

* Lennart Poettering:

> On Wed, 12.07.17 09:51, Florian Weimer (f...@deneb.enyo.de) wrote:
>
>> * Lennart Poettering:
>> 
>> > On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote:
>> >
>> >> * Lennart Poettering:
>> >> 
>> >> > Apparently, this regressed between this version and
>> >> > glibc-2.24-9.fc25.x86_64 hence.
>> >> 
>> >> Yes, I backported the fork cache removal to Fedora 25.  There is no
>> >> longer a good way to main such a cache in userspace because glibc
>> >> cannot intercept anymore all the ways that can change the PID of the
>> >> current process because the kernel interfaces for process management
>> >> are incredibly rich these days.
>> >
>> > Please be more specific here. What is this all about?
>> 
>> We got many bug reports over the years about sandboxes and other heavy
>> users of namespaces and clone that the glibc PID cache got out of
>> sync, both in child and parent (!) processes.
>
> have any links?



You guys ran into this as well and wrote a raw_getpid function which
calls the system call.  (You should have reported the bug instead.)

>> > What triggered this specifically? is this about docker? docker is
>> > written in golang anyway, iirc, which doesn't bother with linking to
>> > libc anyway?
>> 
>> It needs glibc for access to the host and user databases.
>
> can you elaborate? I fail to see any relationship between
> unshare()/fork()/getpid() and NSS?

You asked why docker links against glibc.

>> > Is this a glibc upstream choice primarily? Were the regressions this
>> > causes considered?
>> 
>> I raised the problem of applications calling getpid frequently and
>> named OpenSSL as an example.
>
> Link?

See the collection of links in the other message.

> And I am pretty sure the usecase is very valid... And yes,
> even if checking getpid() misses some theoretical corner cases,
> pthread_atfork() or whatever else you propose will miss others too,
> and is much uglier codewise, introduces deps, yadda yadda...

We actually increased the accuracy of your fork detection logic (even
though it's still broken), so I'm puzzled why you keep calling this a
regression.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Reindl Harald]

Am 12.07.2017 um 10:23 schrieb Lennart Poettering:

On Wed, 12.07.17 09:51, Florian Weimer (f...@deneb.enyo.de) wrote:

Fork detection using getpid is not reliable.  It gives false negatives
in the case of double-forks, where the process can be different but
the PID is the same due to reuse.  Considering that this use case is
broken, I don't think it's worthwhile to jump through hoops to support
code which is fundamentally broken anyway.


Uh, that's a bit non-chalant, no? Yes, the UNIX PID concept is awfully
designed, but if you argue on that level, you#d have to remove kill(),
and half of the other syscalls that take a PID from glibc too...


don't get me wrong: this is exactly how you argue to all others over 
years "fundamentally broken anyway, fix it"

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Florian Weimer]

* Lennart Poettering:

> On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote:
>
>> * Lennart Poettering:
>> 
>> > Apparently, this regressed between this version and
>> > glibc-2.24-9.fc25.x86_64 hence.
>> 
>> Yes, I backported the fork cache removal to Fedora 25.  There is no
>> longer a good way to main such a cache in userspace because glibc
>> cannot intercept anymore all the ways that can change the PID of the
>> current process because the kernel interfaces for process management
>> are incredibly rich these days.
>
> BTW, with this change you are breaking expressly documented behaviour:
>
> http://man7.org/linux/man-pages/man2/getpid.2.html
>
> "Since glibc version 2.3.4, the glibc wrapper function for
>  getpid() caches PIDs, so as to avoid additional system calls when
>  a process calls getpid() repeatedly."

Note that the manpages project, while extremely valuable, is not
official glibc documentation and sometimes explains library internals
which can change over time.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Michael Chapman]

On Wed, 12 Jul 2017, Lennart Poettering wrote:

On Wed, 12.07.17 09:53, Florian Weimer (f...@deneb.enyo.de) wrote:

* Lennart Poettering:

Where was this discussed in detail? Do you have any links about the
discussions about this?


It was on libc-alpha and the glibc bug tracker.


Link?

Lennart


These seem quite relevant:

  https://sourceware.org/ml/libc-alpha/2016-10/msg00136.html
  https://sourceware.org/bugzilla/show_bug.cgi?id=19957

There's a bunch of other related bugs coming up in my searches too... it 
seems that this one change was able to fix many things simultaneously.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Lennart Poettering]

On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote:

> * Lennart Poettering:
> 
> > Apparently, this regressed between this version and
> > glibc-2.24-9.fc25.x86_64 hence.
> 
> Yes, I backported the fork cache removal to Fedora 25.  There is no
> longer a good way to main such a cache in userspace because glibc
> cannot intercept anymore all the ways that can change the PID of the
> current process because the kernel interfaces for process management
> are incredibly rich these days.

BTW, with this change you are breaking expressly documented behaviour:

http://man7.org/linux/man-pages/man2/getpid.2.html

"Since glibc version 2.3.4, the glibc wrapper function for
 getpid() caches PIDs, so as to avoid additional system calls when
 a process calls getpid() repeatedly."

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] systemd-networkd not sending DHCP v6 requests

[Mantas Mikulėnas]

On Wed, Jul 12, 2017 at 9:32 AM, Andrei Borzenkov 
wrote:

> On Wed, Jul 12, 2017 at 8:06 AM, Mantas Mikulėnas 
> wrote:
> > On Tue, Jul 11, 2017, 22:24 Ian Pilcher  wrote:
> >>
> >> On 07/11/2017 02:58 AM, Lennart Poettering wrote:
> >> > Note that DHCPv6 is not done unless IPv6 RA packets tell networkd to
> >> > do so. Hence, areyou sure the RA spoken on your network properly
> >> > indicates that?
> >>
> >> Interesting.  I am seeing somewhat different behavior (but note that
> >> this is systemd-networkd 219 on CentOS 7, which is pretty old).
> >>
> >> * On networks with no router advertisements at all, systemd-networkd 219
> >>will eventually send out dhcp6 solicit packets.
> >>
> >> * On a network with router advertisements that include prefix info
> >>(option 3), systemd-networkd 219 will send dhcp6 solicit packets.
> >>
> >> * If the router advertisements on a network do not include any prefix
> >>information, however, systemd-networkd 219 will never send any dhcp6
> >>solicit packets and never configure an IPv6 address.
> >>
> >> Unfortunately, my ISP's router sends RAs without prefix information.
> >> (Clients get their addresses via DHCPv6, and are presumably expected to
> >> simply assume a 64 bit prefix length.)
> >>
> >> So it looks like I won't be able to use systemd-networkd to get around
> >> the dhclient wall clock problem, at least until RHEL/CentOS see an
> >> updated version of systemd (systemd-networkd 231 does seem to behave
> >> differently).
> >
> >
> > What global flags do each network's RAs have? If I remember correctly,
> there
> > are two, "Managed Addresses" and "Managed Other", which trigger DHCPv6 –
> if
> > neither of them is set, that is supposed to mean DHCPv6 is unneeded.
> >
>
> Citation please. RFC 2462 says:
>
>On receipt of a valid Router Advertisement (as defined in
>[DISCOVERY]), a host copies the value of the advertisement's M bit
>into ManagedFlag. If the value of ManagedFlag changes from FALSE to
>TRUE, and the host is not already running the stateful address
>autoconfiguration protocol, the host should invoke the stateful
>address autoconfiguration protocol, requesting both address
>information and other information.  If the value of the ManagedFlag
>changes from TRUE to FALSE, the host should continue running the
>stateful address autoconfiguration, i.e., the change in the value of
>the ManagedFlag has no effect.  If the value of the flag stays
>unchanged, no special action takes place. In particular, a host MUST
>NOT reinvoke stateful address configuration if it is already
>participating in the stateful protocol as a result of an earlier
>advertisement.
>
>
> There are no statements anywhere (I can find) that host MUST NOT
> invoke stateful autocnfiguration if RA has M flag set to false. It is
> entirely up to host to use stateful autocnfioguration in addition (or
> for that matter instead of) SLAAC.
>
> If you have reference to RFC that says otherwise, please provide it.
>

I see that 2462 was obsoleted by 4862, which no longer defines either flag
at all.

I hereby apologize for being wrong and for offending you with dissemination
of my inaccurate knowledge.

-- 
Mantas Mikulėnas 
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Mantas Mikulėnas]

On Tue, Jul 11, 2017 at 7:06 PM, Tomasz Torcz  wrote:

> On Tue, Jul 11, 2017 at 05:20:10PM +0200, Lennart Poettering wrote:
> > On Tue, 11.07.17 16:55, Tomasz Torcz (to...@pipebreaker.pl) wrote:
> > > > Forgot to mention:
> > > >
> > > > $ rpm -qa glibc
> > > > glibc-2.24-4.fc25.x86_64
> > > >
> > > > Apparently, this regressed between this version and
> > > > glibc-2.24-9.fc25.x86_64 hence.
> > > >
> > >
> > >   From glibc changelog:
> > >
> > > * Wed Jun 07 2017 Arjun Shankar  - 2.24-6
> > > - Auto-sync with upstream release/2.24/master,
> > >   commit 7b60553e360731338631ccdda71590ac5deca137, fixing:
> > > - Remove the PID cache  (#1443976)
> >
> > I commented on that bug now. It doesn#t really have a proper
> > explanation, all it says is that "The glibc PID cache negatively
> > interacts with setting up containers and namespaces."...
>
>   Upstream commit has longer rationale:
> http://repo.or.cz/glibc.git/commit/c579f48edba88380635ab98cb61203
> 0e3ed8691e


Looks like they're basically saying that the same could be done via VDSO in
a much simpler way.

-- 
Mantas Mikulėnas 
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Lennart Poettering]

On Wed, 12.07.17 09:53, Florian Weimer (f...@deneb.enyo.de) wrote:

> * Lennart Poettering:
> 
> > On Tue, 11.07.17 21:24, Florian Weimer (f...@deneb.enyo.de) wrote:
> >
> >> * Lennart Poettering:
> >> 
> >> > This all stems from my experiences with PulseAudio back in the day:
> >> > People do not grok the effect of fork(): it only duplicates the
> >> > invoking thread, not any other threads of the process, moreover all
> >> > data structures are copied as they are, and that's a time bomb really:
> >> 
> >> These days, the PID can change even without a fork, so the story is a
> >> bit different.
> >
> > Can you elaborate?
> 
> I'm no longer sure that the PID can change with the current kernel,
> but I cannot rule it out, either.  But other weirdness is possible:
> for example, after a fork, getpid and getppid could be equal.

Please be less vague. So I understand correctly that the PID cannot
change without fork after all? So why then regress so badly? And what
does getppid have to do with all this?

> > Are you talking about cases where you invoke clone() directly, instead
> > of via glibc's wrappers? We do that too in systemd, but I am not sure
> > this is really reason enough to introduce this regression in glibc:
> > this is easily worked around (which we do in systemd), and given that
> > the time between clone() and execve() should be short, and the
> > code between the two minimal this isn't really much of a problem.
> 
> There are other uses of clone which do not immediately lead to an
> execve.

Can you elaborate? Have a real-life user of this?

> > Where was this discussed in detail? Do you have any links about the
> > discussions about this?
> 
> It was on libc-alpha and the glibc bug tracker.

Link?

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Lennart Poettering]

On Wed, 12.07.17 09:51, Florian Weimer (f...@deneb.enyo.de) wrote:

> * Lennart Poettering:
> 
> > On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote:
> >
> >> * Lennart Poettering:
> >> 
> >> > Apparently, this regressed between this version and
> >> > glibc-2.24-9.fc25.x86_64 hence.
> >> 
> >> Yes, I backported the fork cache removal to Fedora 25.  There is no
> >> longer a good way to main such a cache in userspace because glibc
> >> cannot intercept anymore all the ways that can change the PID of the
> >> current process because the kernel interfaces for process management
> >> are incredibly rich these days.
> >
> > Please be more specific here. What is this all about?
> 
> We got many bug reports over the years about sandboxes and other heavy
> users of namespaces and clone that the glibc PID cache got out of
> sync, both in child and parent (!) processes.

have any links?

> > What triggered this specifically? is this about docker? docker is
> > written in golang anyway, iirc, which doesn't bother with linking to
> > libc anyway?
> 
> It needs glibc for access to the host and user databases.

can you elaborate? I fail to see any relationship between
unshare()/fork()/getpid() and NSS?

> > Is this a glibc upstream choice primarily? Were the regressions this
> > causes considered?
> 
> I raised the problem of applications calling getpid frequently and
> named OpenSSL as an example.

Link?

> > I mean, the getpid() checking code is not only in use in systemd, but
> > in various other bits, in particular PulseAudio, where I started
> > adding these checks for a good reason. It sounds pretty strange to me
> > to just regress all that...
> 
> Fork detection using getpid is not reliable.  It gives false negatives
> in the case of double-forks, where the process can be different but
> the PID is the same due to reuse.  Considering that this use case is
> broken, I don't think it's worthwhile to jump through hoops to support
> code which is fundamentally broken anyway.

Uh, that's a bit non-chalant, no? Yes, the UNIX PID concept is awfully
designed, but if you argue on that level, you#d have to remove kill(),
and half of the other syscalls that take a PID from glibc too...

The primary intention of checking the PID in our calls is to filter
out cases where people assume they can use our context objects across
fork()s: a clean, early error code is a ton better than memory
corruption. And I am pretty sure the usecase is very valid... And yes,
even if checking getpid() misses some theoretical corner cases,
pthread_atfork() or whatever else you propose will miss others too,
and is much uglier codewise, introduces deps, yadda yadda...

Gah, this is all so ugly. I understand systemd is not a program you
are particularly interested in, but making us chase around your
regressions is just mean...

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Florian Weimer]

* Lennart Poettering:

> On Tue, 11.07.17 21:24, Florian Weimer (f...@deneb.enyo.de) wrote:
>
>> * Lennart Poettering:
>> 
>> > This all stems from my experiences with PulseAudio back in the day:
>> > People do not grok the effect of fork(): it only duplicates the
>> > invoking thread, not any other threads of the process, moreover all
>> > data structures are copied as they are, and that's a time bomb really:
>> 
>> These days, the PID can change even without a fork, so the story is a
>> bit different.
>
> Can you elaborate?

I'm no longer sure that the PID can change with the current kernel,
but I cannot rule it out, either.  But other weirdness is possible:
for example, after a fork, getpid and getppid could be equal.

> Are you talking about cases where you invoke clone() directly, instead
> of via glibc's wrappers? We do that too in systemd, but I am not sure
> this is really reason enough to introduce this regression in glibc:
> this is easily worked around (which we do in systemd), and given that
> the time between clone() and execve() should be short, and the
> code between the two minimal this isn't really much of a problem.

There are other uses of clone which do not immediately lead to an
execve.

> Where was this discussed in detail? Do you have any links about the
> discussions about this?

It was on libc-alpha and the glibc bug tracker.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Florian Weimer]

* Lennart Poettering:

> On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote:
>
>> * Lennart Poettering:
>> 
>> > Apparently, this regressed between this version and
>> > glibc-2.24-9.fc25.x86_64 hence.
>> 
>> Yes, I backported the fork cache removal to Fedora 25.  There is no
>> longer a good way to main such a cache in userspace because glibc
>> cannot intercept anymore all the ways that can change the PID of the
>> current process because the kernel interfaces for process management
>> are incredibly rich these days.
>
> Please be more specific here. What is this all about?

We got many bug reports over the years about sandboxes and other heavy
users of namespaces and clone that the glibc PID cache got out of
sync, both in child and parent (!) processes.

> What triggered this specifically? is this about docker? docker is
> written in golang anyway, iirc, which doesn't bother with linking to
> libc anyway?

It needs glibc for access to the host and user databases.

> Is this a glibc upstream choice primarily? Were the regressions this
> causes considered?

I raised the problem of applications calling getpid frequently and
named OpenSSL as an example.

> I mean, the getpid() checking code is not only in use in systemd, but
> in various other bits, in particular PulseAudio, where I started
> adding these checks for a good reason. It sounds pretty strange to me
> to just regress all that...

Fork detection using getpid is not reliable.  It gives false negatives
in the case of double-forks, where the process can be different but
the PID is the same due to reuse.  Considering that this use case is
broken, I don't think it's worthwhile to jump through hoops to support
code which is fundamentally broken anyway.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Lennart Poettering]

On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote:

> * Lennart Poettering:
> 
> > Apparently, this regressed between this version and
> > glibc-2.24-9.fc25.x86_64 hence.
> 
> Yes, I backported the fork cache removal to Fedora 25.  There is no
> longer a good way to main such a cache in userspace because glibc
> cannot intercept anymore all the ways that can change the PID of the
> current process because the kernel interfaces for process management
> are incredibly rich these days.

Please be more specific here. What is this all about? What triggered
this specifically? is this about docker? docker is written in golang
anyway, iirc, which doesn't bother with linking to libc anyway?

Is this a glibc upstream choice primarily? Were the regressions this
causes considered?

I mean, the getpid() checking code is not only in use in systemd, but
in various other bits, in particular PulseAudio, where I started
adding these checks for a good reason. It sounds pretty strange to me
to just regress all that...

Were there considerations to solve this differently? (for example,
expose some concept how getpid() caching can be turned out
specifically for apps needing that? it appears to me that after all
the number of apps calling clone() directly are very much numbered,
while a ton of apps will benefit from getpid() caching...)

Please elaborate!

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Lennart Poettering]

On Tue, 11.07.17 21:24, Florian Weimer (f...@deneb.enyo.de) wrote:

> * Lennart Poettering:
> 
> > This all stems from my experiences with PulseAudio back in the day:
> > People do not grok the effect of fork(): it only duplicates the
> > invoking thread, not any other threads of the process, moreover all
> > data structures are copied as they are, and that's a time bomb really:
> 
> These days, the PID can change even without a fork, so the story is a
> bit different.

Can you elaborate?

Are you talking about cases where you invoke clone() directly, instead
of via glibc's wrappers? We do that too in systemd, but I am not sure
this is really reason enough to introduce this regression in glibc:
this is easily worked around (which we do in systemd), and given that
the time between clone() and execve() should be short, and the
code between the two minimal this isn't really much of a problem.

Where was this discussed in detail? Do you have any links about the
discussions about this?

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] systemd-networkd not sending DHCP v6 requests

[Andrei Borzenkov]

On Wed, Jul 12, 2017 at 8:06 AM, Mantas Mikulėnas  wrote:
> On Tue, Jul 11, 2017, 22:24 Ian Pilcher  wrote:
>>
>> On 07/11/2017 02:58 AM, Lennart Poettering wrote:
>> > Note that DHCPv6 is not done unless IPv6 RA packets tell networkd to
>> > do so. Hence, areyou sure the RA spoken on your network properly
>> > indicates that?
>>
>> Interesting.  I am seeing somewhat different behavior (but note that
>> this is systemd-networkd 219 on CentOS 7, which is pretty old).
>>
>> * On networks with no router advertisements at all, systemd-networkd 219
>>will eventually send out dhcp6 solicit packets.
>>
>> * On a network with router advertisements that include prefix info
>>(option 3), systemd-networkd 219 will send dhcp6 solicit packets.
>>
>> * If the router advertisements on a network do not include any prefix
>>information, however, systemd-networkd 219 will never send any dhcp6
>>solicit packets and never configure an IPv6 address.
>>
>> Unfortunately, my ISP's router sends RAs without prefix information.
>> (Clients get their addresses via DHCPv6, and are presumably expected to
>> simply assume a 64 bit prefix length.)
>>
>> So it looks like I won't be able to use systemd-networkd to get around
>> the dhclient wall clock problem, at least until RHEL/CentOS see an
>> updated version of systemd (systemd-networkd 231 does seem to behave
>> differently).
>
>
> What global flags do each network's RAs have? If I remember correctly, there
> are two, "Managed Addresses" and "Managed Other", which trigger DHCPv6 – if
> neither of them is set, that is supposed to mean DHCPv6 is unneeded.
>

Citation please. RFC 2462 says:

   On receipt of a valid Router Advertisement (as defined in
   [DISCOVERY]), a host copies the value of the advertisement's M bit
   into ManagedFlag. If the value of ManagedFlag changes from FALSE to
   TRUE, and the host is not already running the stateful address
   autoconfiguration protocol, the host should invoke the stateful
   address autoconfiguration protocol, requesting both address
   information and other information.  If the value of the ManagedFlag
   changes from TRUE to FALSE, the host should continue running the
   stateful address autoconfiguration, i.e., the change in the value of
   the ManagedFlag has no effect.  If the value of the flag stays
   unchanged, no special action takes place. In particular, a host MUST
   NOT reinvoke stateful address configuration if it is already
   participating in the stateful protocol as a result of an earlier
   advertisement.


There are no statements anywhere (I can find) that host MUST NOT
invoke stateful autocnfiguration if RA has M flag set to false. It is
entirely up to host to use stateful autocnfioguration in addition (or
for that matter instead of) SLAAC.

If you have reference to RFC that says otherwise, please provide it.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel