[systemd-devel] udev and probing of eMMC partition devices
Hi, I am trying to understand behavior that I am seeing with udev and eMMC partition devices and was hoping that someone here could help. The system that I am running has an eMMC device with something like 7-8 partitions. The kernel does add block events for the device and each of it partitions and the logs show them being queued up by udevd. Then things get interesting. Sometimes processing of the add event for the device itself gets to a probe step and, in the time frame of the logs that I have, never gets further in that processing and the add block device events for the partitions are never processed. However, usually, it will process the add block device event for the entire device and after that (as per the implementation of is_devpath_busy()) start processing the add block events for the partition devices. Often times, the processing of the partition device add events will get stuck at the probe step. The partition devices stuck at the probe are then "waiting" and the services that depend on them blocked. Can anyone here give me some insight into what is going on, in particular, how there is such difference in behavior between test runs on the same system? Thanks, alan perry Microsoft ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] correct usage for file-only & (sub)dir-only shell globs in unit tmpfiles?
I want to chmod all files and dirs/subdirs in a tree using tmpfiles.
Starting with an example tree
tree -pF /home/test/TEST
/home/test/TEST
└── [dr] topdir/
├── [-r] A.txt
└── [dr] subdir1/
├── [-r] B.txt
└── [dr] subdir2/
└── [-r] C.txt
I want to chmod
all files -> 0640
all dirs -> 0750
i.e., ending up with
/home/test/TEST
└── [drwxr-x---] topdir/
├── [-rw-r-] A.txt
└── [drwxr-x---] subdir1/
├── [-rw-r-] B.txt
└── [drwxr-x---] subdir2/
└── [-rw-r-] C.txt
for this, it seems shell globs are appropriate.
but, with
/etc/tmpfiles.d/test.conf
(1) Z /home/test/TEST/{.,**/} 0750 nobody nobody -
(2A)Z /home/test/TEST/** 0640 nobody nobody -
or
(2B)Z /home/test/TEST/{**,!.,!**/} 0640 nobody nobody -
, where the intention was that shell glob (1) matches only dirs/subdirs, and
glob (2A) or (2B) matches files,
after exec of
systemd-tmpfiles --create /etc/tmpfiles.d/dovecot.conf
I end up, instead, with
tree -pF /home/test/TEST
/home/test/TEST
└── [drw-r-] topdir/
├── [-rw-r-] A.txt
└── [drw-r-] subdir1/
├── [-rw-r-] B.txt
└── [drw-r-] subdir2/
└── [-rw-r-] C.txt
... with all my intended 0750 exec'able dir modes are overwritten with
intended-for-files modes.
either my globs are wrong &/or I'm bumping into recursion conflicts with "Z" in
tmpfiles.
What's the right usage/syntax in a tmpfile to efficiently/correctly get that^
done?
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] journald forwarding to rsyslogd. Huge (350 times) performance degradation. What am I doing wrong???
On So, 20.09.20 16:09, Vitaly Repin (vitaly_re...@fsfe.org) wrote: > Hello, > > Thanks for the answer! > > However, it looks like that imjournal module is not an option for me. > > rsyslog imjournal module documentation > (https://www.rsyslog.com/doc/v8-stable/configuration/modules/imjournal.html) > clearly recommends to use imuxsock module in my case: > > [begin quote] > As such, the performance of a configuration utilizing this module may > be notably slower than when using imuxsock. > The journal provides imuxsock with a copy of all “classical” syslog > messages, however, it does not provide structured data. ' > Only if that structured data is needed, imjournal must be used. > Otherwise, imjournal may simply be replaced by imuxsock, and we highly > suggest doing so. > [end quote] > > I do not need structural data and I need performance. My system > generates a substantial amount of logs. I want to be able to store > messages with an arrival rate of 10 000 messages per second at > least. The rsyslog maintainer doesn't like systemd very much. Much of what he puts in the docs as big fat warnings is quite a bit FUDdy. Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] journald forwarding to rsyslogd. Huge (350 times) performance degradation. What am I doing wrong???
On So, 20.09.20 12:07, Vitaly Repin (vitaly_re...@fsfe.org) wrote: > Any ideas are appreciated. Note that you compare apples with oranges. journald spends much time on collecting log metadata (i.e. process identity, user identity, cgroup stuff, …), which the others simply don't. That said, for starters, try running something a bit newer than a version from 215. 5 years is an eternity in software. The metadata colleciton code gained support for caching much of the metadata, which makes a major difference in particular if you have only a few log sources which log a lot (in contrast to many different log sources that log at the same time). Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] journald forwarding to rsyslogd. Huge (350 times) performance degradation. What am I doing wrong???
Hm, some performance penalty is certainly expected but 350times slower looks like something is odd. Would be interesting to know, if you can reproduce the performance issue with a more recent version. Afaik, using imuxsock and syslog forwarding should be more performant then imjournal (which was suggested earlier). You could configure systemd, to not do syslog forwarding and let rsyslog handle syslog messages directly. This has a few downsides though: - syslog messages don't end up in the journal - non-syslog messages from journald don't end up in your syslog, say messages from systemd or services logging to stdout/stderr. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] journald on a system without real-time clock
On Mon, 21 Sep 2020 10:16:38 +0200 Magnus Berglund wrote: > Hello, > > I have an embedded system that does not have a real-time clock. I was > hoping to run journald on it, but have run into some problems. > > My problem is that my system currently does not guarantee a > monotonically increasing time, and that seems to confuse journald a > bit. > > Btw: I've tested this on v243-stable. > > (The system syncs over NTP, but there are some issues with this as > well, more on that later) > > I originally found the problem since there were boots missing. E.g) > > # journalctl --list-boots > -4 44cabeed86e34d09a4eca0dbff43b19f Mon 2020-08-03 14:27:59 UTC—Tue > 2020-09-15 09:13:52 UTC > -3 71164e7d83a9448c9e70bb59b6190a45 Tue 2020-09-15 09:13:52 UTC—Tue > 2020-09-15 09:14:48 UTC > -2 2e565a1c8dc84d4e95055e4cb4d0cc25 Tue 2020-09-15 09:14:48 UTC—Tue > 2020-09-15 09:16:11 UTC > -1 3c672d6fb8084f5fa5923a1ae5e0e53d Tue 2020-09-15 09:16:11 UTC—Tue > 2020-09-15 09:31:52 UTC > 0 afc0d98f275e4999aa061c7bb61b85d2 Tue 2020-09-15 09:33:35 UTC—Tue > 2020-09-15 09:56:12 UTC > > # journalctl -F _BOOT_ID > 44cabeed86e34d09a4eca0dbff43b19f > 71164e7d83a9448c9e70bb59b6190a45 > 1465a36c753f43df92bcc0a76d8e763e > 2e565a1c8dc84d4e95055e4cb4d0cc25 > 3c672d6fb8084f5fa5923a1ae5e0e53d > afc0d98f275e4999aa061c7bb61b85d2 > 302cc41b146c4b3c88f06df102913c3f > > I've poked around in the source code think I found the reason for > this: journal_file_compare_locations() (in journal-file.c) compares > seqnum if within the the same seqnum_id, but uses current_realtime if > not within the same seqnum_id. Since my realtime can't be trusted I > sometimes have journalfiles within the same seqnum_id which "jumps" > back in time. That combined with the traversal in real_journal_next() > gives a stochastic behaviour. > > I have tried fixing my clock to give a monotonically increasing > clock. But: I want my system to boot as fast as possible, even without > a NTP-syncronized time. And I've noticed that if I shutdown before > getting a NTP time timesyncd will not have touched the "clock" file, > and I end up in the situation above. > > Now to my question: What is the best practice in using journald on > systems without RTC? It it even "supported"? > > /Regards, > Magnus Berglund There was recently a long thread about some of these issues, but specifically as they apply to Raspberry Pis. I think it's worth reading as a first step, though. The thread starts at https://lists.freedesktop.org/archives/systemd-devel/2020-August/045118.html ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] journald on a system without real-time clock
Hello, I have an embedded system that does not have a real-time clock. I was hoping to run journald on it, but have run into some problems. My problem is that my system currently does not guarantee a monotonically increasing time, and that seems to confuse journald a bit. Btw: I've tested this on v243-stable. (The system syncs over NTP, but there are some issues with this as well, more on that later) I originally found the problem since there were boots missing. E.g) # journalctl --list-boots -4 44cabeed86e34d09a4eca0dbff43b19f Mon 2020-08-03 14:27:59 UTC—Tue 2020-09-15 09:13:52 UTC -3 71164e7d83a9448c9e70bb59b6190a45 Tue 2020-09-15 09:13:52 UTC—Tue 2020-09-15 09:14:48 UTC -2 2e565a1c8dc84d4e95055e4cb4d0cc25 Tue 2020-09-15 09:14:48 UTC—Tue 2020-09-15 09:16:11 UTC -1 3c672d6fb8084f5fa5923a1ae5e0e53d Tue 2020-09-15 09:16:11 UTC—Tue 2020-09-15 09:31:52 UTC 0 afc0d98f275e4999aa061c7bb61b85d2 Tue 2020-09-15 09:33:35 UTC—Tue 2020-09-15 09:56:12 UTC # journalctl -F _BOOT_ID 44cabeed86e34d09a4eca0dbff43b19f 71164e7d83a9448c9e70bb59b6190a45 1465a36c753f43df92bcc0a76d8e763e 2e565a1c8dc84d4e95055e4cb4d0cc25 3c672d6fb8084f5fa5923a1ae5e0e53d afc0d98f275e4999aa061c7bb61b85d2 302cc41b146c4b3c88f06df102913c3f I've poked around in the source code think I found the reason for this: journal_file_compare_locations() (in journal-file.c) compares seqnum if within the the same seqnum_id, but uses current_realtime if not within the same seqnum_id. Since my realtime can't be trusted I sometimes have journalfiles within the same seqnum_id which "jumps" back in time. That combined with the traversal in real_journal_next() gives a stochastic behaviour. I have tried fixing my clock to give a monotonically increasing clock. But: I want my system to boot as fast as possible, even without a NTP-syncronized time. And I've noticed that if I shutdown before getting a NTP time timesyncd will not have touched the "clock" file, and I end up in the situation above. Now to my question: What is the best practice in using journald on systems without RTC? It it even "supported"? /Regards, Magnus Berglund ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
