[systemd-devel] Journal message timestamps
Hello I am working on time synchronisation issues at boot for systems without an RTC (using balenaOS on a Raspberry Pi 3) and have some questions about how journald assigns timestamps to log messages. When I boot my system and look at the journal I see an initial date/time for kernel messages, e.g. '1 June 2020 10:00:00' followed by messages with the 'correct' date/time once the system time has been set from another source, e.g. build time, NTP, etc. This means that over several reboots I have lots of sets of log messages from 1 June 2020 which understandably confuses the 'journalctl --list-boots' command. I found an issue that describes the problem here https://github.com/systemd/systemd/issues/662 and had assumed that there wasn't anything I could do about this. However, when running some tests on a board with Raspberry Pi OS I found that it didn't suffer from the same problem. RPI OS uses a 'fake-hwclock' service to restore the previous boot time from a file and this time gets applied to all messages in the journal prior to this point. I added the 'fake-hwclock' service to my system which resulted in the timestamps being correct the majority of the time, but I was still seeing some boots where the initial messages were showing '1 June 2020'. I eventually tracked the intermittent behaviour down to whether the 'fake-hwclock' time setting occurred in the same system log file as the initial kernel boot messages. On my RPI OS board the runtime journal was set to 8MB, so the date/time setting always occurred in the first journal file. On my system the runtime journal was set to 4MB, so the date/time setting was sometimes happening in the second journal file leaving the messages in the first file with a date of '1 June 2020'. So my questions are... It seems that journald is using the date/time from the 'fake-hwclock' service to update the timestamps of earlier log messages within the same file. Is this correct? What would be the best technique for ensuring that my journal logs always display the 'best' time for log messages (either 'fake-hwclock' or NTP)? Do I always have to ensure that the journal is large enough to capture my initial time setting event in the first file? Any general details about how journald applies timestamps would also be greatly appreciated. Thanks Mark ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] A sh -c '${name} and $name' are treated inconsistently within a .service unit
On Thu, 27 Aug 2020 at 16:11:37 +, u...@net9.ga wrote:
> ExecStart=/bin/bash -c 'set -x; declare -r str="1 2"; echo ${str}; echo $str;
> exit 0;'
This seems to be behaving as documented:
Basic environment variable substitution is supported. Use "${FOO}" as
part of a word, or as a word of its own, on the command line, in which
case it will be erased and replaced by the exact value of the
environment variable (if any) including all whitespace it contains,
always resulting in exactly a single argument. Use "$FOO" as a separate
word on the command line, in which case it will be replaced by the
value of the environment variable split at whitespace, resulting in
zero or more arguments.
— systemd.service(5)
The single-quoted string is considered to be a "word" here.
The string is parsed by systemd, and the result of parsing is inserted
into bash's argv and parsed again by bash. It's the same as the way
you have to write shell variables as $$foo if you are embedding a shell
script in a Makefile, or the way you have to escape backslashes multiple
times if you want to write a regular expression that matches backslashes,
inside a double-quoted shell string, inside a C string (add as many more
levels of embedding as you want, or see https://xkcd.com/1638/).
If you want to write shell or Perl scripts, or other constructs that
*also* use $ as special syntax, then you will need to either escape all
occurrences of $ that you don't want systemd to interpret (write them
as $$, like in make), or put your shell script in a separate file and
reference it from the systemd unit.
smcv
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] A sh -c '${name} and $name' are treated inconsistently within a .service unit
27.08.2020 19:11, u...@net9.ga пишет:
> Consider
>
> [Unit]
> Description=Is it looking for ${} construct in the wrong place?
> [Service]
> Type=oneshot
> ExecStart=/bin/bash -c 'set -x; declare -r str="1 2"; echo ${str}; echo $str;
> exit 0;'
>
> When ran, the journal has:
>
> bash[14190]: + declare -r 'str=1 2'
> bash[14190]: + echo
> bash[14190]: + echo 1 2
> bash[14190]: 1 2
> bash[14190]: + exit 0
>
> Note the top bash[14190]: + echo line. It has no arguments. Since the other
> echo line
> has its 1 2 arguments, I find this behaviour inconsistent. And surprising. As
> such, I
> think this is a bug.
> It could be that the ${NAME} construct is looked for in the environment. But
> then, why
> $NAME works in the script, but not ${NAME}?
>
As documented $NAME is recognized only when used as separate word. The
string in quotes is single word; so ${str} inside it is recognized as
environment substitution and replaced by empty string but $str inside
the same word is left as is and later processed by shell.
> In addition, for the top bash[14190]: + echo line, isn't there a missing
> empty, just
> [bash[14190]:, line?
>
> Same results were obtained with systemd 241-7 and 246.2.
>
> u34.
> ___
> systemd-devel mailing list
> systemd-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/systemd-devel
>
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] A sh -c '${name} and $name' are treated inconsistently within a .service unit
Am 27.08.20 um 18:11 schrieb u...@net9.ga:
> [Unit]
> Description=Is it looking for ${} construct in the wrong place?
> [Service]
> Type=oneshot
> ExecStart=/bin/bash -c 'set -x; declare -r str="1 2"; echo ${str}; echo $str;
> exit 0;'
>
> When ran, the journal has:
>
> bash[14190]: + declare -r 'str=1 2'
> bash[14190]: + echo
> bash[14190]: + echo 1 2
> bash[14190]: 1 2
> bash[14190]: + exit 0
>
> Note the top bash[14190]: + echo line. It has no arguments. Since the other
> echo line
> has its 1 2 arguments, I find this behaviour inconsistent. And surprising. As
> such, I
> think this is a bug.
> It could be that the ${NAME} construct is looked for in the environment. But
> then, why
> $NAME works in the script, but not ${NAME}?
if you want a script just use a script and call it with ExecStart
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] A sh -c '${name} and $name' are treated inconsistently within a .service unit
Consider
[Unit]
Description=Is it looking for ${} construct in the wrong place?
[Service]
Type=oneshot
ExecStart=/bin/bash -c 'set -x; declare -r str="1 2"; echo ${str}; echo $str;
exit 0;'
When ran, the journal has:
bash[14190]: + declare -r 'str=1 2'
bash[14190]: + echo
bash[14190]: + echo 1 2
bash[14190]: 1 2
bash[14190]: + exit 0
Note the top bash[14190]: + echo line. It has no arguments. Since the other
echo line
has its 1 2 arguments, I find this behaviour inconsistent. And surprising. As
such, I
think this is a bug.
It could be that the ${NAME} construct is looked for in the environment. But
then, why
$NAME works in the script, but not ${NAME}?
In addition, for the top bash[14190]: + echo line, isn't there a missing empty,
just
[bash[14190]:, line?
Same results were obtained with systemd 241-7 and 246.2.
u34.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Antw: [EXT] Re: tmpfiles chicken-egg problem
Am 26.08.20 um 16:01 schrieb Ulrich Windl: Lennart Poettering schrieb am 26.08.2020 um 15:40 > in > Nachricht <20200826134031.GA257903@gardel-login>: >> On Mi, 26.08.20 08:37, Ulrich Windl (ulrich.wi...@rz.uni‑regensburg.de) > wrote: >> >>> Hi! >>> >>> I see this problem in SLES12 (systemd‑228‑157.12.5.x86_64): On boot systemd > >> tries to use LDAP to resolve user names, resulting in an error like this: >>> systemd‑tmpfiles: nss‑ldap: do_open: do_start_tls failed:stat=‑1 >> >> Files and directories managed by systemd‑tmpfiles have to be owned by >> *system* users and groups. If you declare files/dirs that are owned by >> non‑system users, then you are on your own, and things will fall apart. >> >> A system user must be resolvable during the entire runtime of the >> system, i.e. managed in /etc/passwd and /etc/group, not in LDAP. >> >> This is extensively documented in tmpfiles.d(5) or here: >> >> https://systemd.io/UIDS‑GIDS/#notes‑on‑resolvability‑of‑user‑and‑group‑names > >> >> Hence, if this happens your setup is borked in some way: some entries >> in tmpfiles.d/ drop‑ins are owned by users/groups managed by LDAP. Fix >> that, and everything should be fine. > > It's all transitional in some way. In the past a system user was a user with a > UID below the UIDs given to interactive users. Directories existed right from > the beginning of the boot, and the user had to be known when a corresponding > process had to be started. Not earlier. Systemd redefined the world, so don't > point at the world if things are broken now... did you skip the follow-up paragraph by intention to repeat "Systemd redefined the world, so don't point at the world"? just create directories when the process has to be startet eiter by ExecStartPre or RuntimeDirectory/StateDirectory and you are done And besides that, we actually push people towards using RuntimeDirectory=, StateDirectory=, … and stuff so that these dirs are created when the service is started and not earlier, for services where that works. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Antw: [EXT] Re: tmpfiles chicken-egg problem
>>> Lennart Poettering schrieb am 26.08.2020 um 15:40 in Nachricht <20200826134031.GA257903@gardel-login>: > On Mi, 26.08.20 08:37, Ulrich Windl (ulrich.wi...@rz.uni‑regensburg.de) wrote: > >> Hi! >> >> I see this problem in SLES12 (systemd‑228‑157.12.5.x86_64): On boot systemd > tries to use LDAP to resolve user names, resulting in an error like this: >> systemd‑tmpfiles: nss‑ldap: do_open: do_start_tls failed:stat=‑1 > > Files and directories managed by systemd‑tmpfiles have to be owned by > *system* users and groups. If you declare files/dirs that are owned by > non‑system users, then you are on your own, and things will fall apart. > > A system user must be resolvable during the entire runtime of the > system, i.e. managed in /etc/passwd and /etc/group, not in LDAP. > > This is extensively documented in tmpfiles.d(5) or here: > > https://systemd.io/UIDS‑GIDS/#notes‑on‑resolvability‑of‑user‑and‑group‑names > > Hence, if this happens your setup is borked in some way: some entries > in tmpfiles.d/ drop‑ins are owned by users/groups managed by LDAP. Fix > that, and everything should be fine. It's all transitional in some way. In the past a system user was a user with a UID below the UIDs given to interactive users. Directories existed right from the beginning of the boot, and the user had to be known when a corresponding process had to be started. Not earlier. Systemd redefined the world, so don't point at the world if things are broken now... I know that it's not all perfect, and I'm working on it... wondering: if I'd un-tar the temporary directorires on boot, the UDIs would be stored correctly in the tar... That would add compatibility to pre-systemd times... [...] Regards, Ulrich ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
