[systemd-devel] systemd prerelease 248-rc4
A new systemd ☠️ pre-release ☠️ has just been tagged. Please download the
tarball here:
https://github.com/systemd/systemd/archive/v248-rc4.tar.gz
NOTE: This is ☠️ pre-release ☠️ software. Do not run this on production
systems, but please test this and report any issues you find to GitHub:
https://github.com/systemd/systemd/issues/new?template=Bug_report.md
Changes since the previous release:
* A concept of system extension images is introduced. Such images may
be used to extend the /usr/ and /opt/ directory hierarchies at
runtime with additional files (even if the file system is read-only).
When a system extension image is activated, its /usr/ and /opt/
hierarchies and os-release information are combined via overlayfs
with the file system hierarchy of the host OS.
A new systemd-sysext tool can be used to merge, unmerge, list, and
refresh system extension hierarchies. See
https://www.freedesktop.org/software/systemd/man/systemd-sysext.html.
The systemd-sysext.service automatically merges installed system
extensions during boot (before basic.target, but not in very early
boot, since various file systems have to be mounted first).
The SYSEXT_LEVEL= field in os-release(5) may be used to specify the
supported system extension level.
* A new ExtensionImages= unit setting can be used to apply the same
system extension image concept from systemd-sysext to the namespaced
file hierarchy of specific services, following the same rules and
constraints.
* Support for a new special "root=tmpfs" kernel command-line option has
been added. When specified, a tmpfs is mounted on /, and mount.usr=
should be used to point to the operating system implementation.
* A new configuration file /etc/veritytab may be used to configure
dm-verity integrity protection for block devices. Each line is in the
format "volume-name data-device hash-device roothash options",
similar to /etc/crypttab.
* A new kernel command-line option systemd.verity.root_options= may be
used to configure dm-verity behaviour for the root device.
* The key file specified in /etc/crypttab (the third field) may now
refer to an AF_UNIX/SOCK_STREAM socket in the file system. The key is
acquired by connecting to that socket and reading from it. This
allows the implementation of a service to provide key information
dynamically, at the moment when it is needed.
* When the hostname is set explicitly to "localhost", systemd-hostnamed
will respect this. Previously such a setting would be mostly silently
ignored. The goal is to honour configuration as specified by the
user.
* The fallback hostname that will be used by the system manager and
systemd-hostnamed can now be configured in two new ways: by setting
DEFAULT_HOSTNAME= in os-release(5), or by setting
$SYSTEMD_DEFAULT_HOSTNAME in the environment block. As before, it can
also be configured during compilation. The environment variable is
intended for testing and local overrides, the os-release(5) field is
intended to allow customization by different variants of a
distribution that share the same compiled packages.
* The environment block of the manager itself may be configured through
a new ManagerEnvironment= setting in system.conf or user.conf. This
complements existing ways to set the environment block (the kernel
command line for the system manager, the inherited environment and
user@.service unit file settings for the user manager).
* systemd-hostnamed now exports the default hostname and the source of
the configured hostname ("static", "transient", or "default") as
D-Bus properties.
* systemd-hostnamed now exports the "HardwareVendor" and
"HardwareModel" D-Bus properties, which are supposed to contain a
pair of cleaned up, human readable strings describing the system's
vendor and model. It's typically sourced from the firmware's DMI
tables, but may be augmented from a new hwdb database. hostnamectl
shows this in the status output.
* Support has been added to systemd-cryptsetup for extracting the
PKCS#11 token URI and encrypted key from the LUKS2 JSON embedded
metadata header. This allows the information how to open the
encrypted device to be embedded directly in the device and obviates
the need for configuration in an external file.
* systemd-cryptsetup gained support for unlocking LUKS2 volumes using
TPM2 hardware, as well as FIDO2 security tokens (in addition to
[systemd-devel] [PATCH] usb-storage: Add quirk to defeat Kindle's automatic unload
Matthias reports that the Amazon Kindle automatically removes its
emulated media if it doesn't receive another SCSI command within about
one second after a SYNCHRONIZE CACHE. It does so even when the host
has sent a PREVENT MEDIUM REMOVAL command. The reason for this
behavior isn't clear, although it's not hard to make some guesses.
At any rate, the results can be unexpected for anyone who tries to
access the Kindle in an unusual fashion, and in theory they can lead
to data loss (for example, if one file is closed and synchronized
while other files are still in the middle of being written).
To avoid such problems, this patch creates a new usb-storage quirks
flag telling the driver always to issue a REQUEST SENSE following a
SYNCHRONIZE CACHE command, and adds an unusual_devs entry for the
Kindle with the flag set. This is sufficient to prevent the Kindle
from doing its automatic unload, without interfering with proper
operation.
Another possible way to deal with this would be to increase the
frequency of TEST UNIT READY polling that the kernel normally carries
out for removable-media storage devices. However that would increase
the overall load on the system and it is not as reliable, because the
user can override the polling interval. Changing the driver's
behavior is safer and has minimal overhead.
Reported-and-tested-by: Matthias Schwarzott
Signed-off-by: Alan Stern
CC:
---
[as1953]
drivers/usb/storage/transport.c|7 +++
drivers/usb/storage/unusual_devs.h | 12
include/linux/usb_usual.h |2 ++
3 files changed, 21 insertions(+)
Index: usb-devel/drivers/usb/storage/transport.c
===
--- usb-devel.orig/drivers/usb/storage/transport.c
+++ usb-devel/drivers/usb/storage/transport.c
@@ -656,6 +656,13 @@ void usb_stor_invoke_transport(struct sc
need_auto_sense = 1;
}
+ /* Some devices (Kindle) require another command after SYNC CACHE */
+ if ((us->fflags & US_FL_SENSE_AFTER_SYNC) &&
+ srb->cmnd[0] == SYNCHRONIZE_CACHE) {
+ usb_stor_dbg(us, "-- sense after SYNC CACHE\n");
+ need_auto_sense = 1;
+ }
+
/*
* If we have a failure, we're going to do a REQUEST_SENSE
* automatically. Note that we differentiate between a command
Index: usb-devel/drivers/usb/storage/unusual_devs.h
===
--- usb-devel.orig/drivers/usb/storage/unusual_devs.h
+++ usb-devel/drivers/usb/storage/unusual_devs.h
@@ -2212,6 +2212,18 @@ UNUSUAL_DEV( 0x1908, 0x3335, 0x0200, 0x0
US_FL_NO_READ_DISC_INFO ),
/*
+ * Reported by Matthias Schwarzott
+ * The Amazon Kindle treats SYNCHRONIZE CACHE as an indication that
+ * the host may be finished with it, and automatically ejects its
+ * emulated media unless it receives another command within one second.
+ */
+UNUSUAL_DEV( 0x1949, 0x0004, 0x, 0x,
+ "Amazon",
+ "Kindle",
+ USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+ US_FL_SENSE_AFTER_SYNC ),
+
+/*
* Reported by Oliver Neukum
* This device morphes spontaneously into another device if the access
* pattern of Windows isn't followed. Thus writable media would be dirty
Index: usb-devel/include/linux/usb_usual.h
===
--- usb-devel.orig/include/linux/usb_usual.h
+++ usb-devel/include/linux/usb_usual.h
@@ -86,6 +86,8 @@
/* lies about caching, so always sync */\
US_FLAG(NO_SAME, 0x4000)\
/* Cannot handle WRITE_SAME */ \
+ US_FLAG(SENSE_AFTER_SYNC, 0x8000) \
+ /* Do REQUEST_SENSE after SYNCHRONIZE_CACHE */ \
#define US_FLAG(name, value) US_FL_##name = value ,
enum { US_DO_ALL_FLAGS };
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [usb-storage] Re: Amazon Kindle disconnect after Synchronize Cache
Hi, On 3/17/21 6:56 PM, Matthias Schwarzott wrote: > Am 17.03.21 um 16:17 schrieb Alan Stern: >> On Wed, Mar 17, 2021 at 01:21:50PM +0100, Hans de Goede wrote: >>> Hi, >>> >>> On 3/16/21 6:04 PM, Alan Stern wrote: I think it would be mildly better, but not a whole lot. Since the Kindle describes itself as having removable media, the kernel normally probes it periodically to make sure the media remains present. The default probing interval is 2 seconds. Reducing it to 0.9 seconds doesn't represent an exorbitant additional load IMO -- especially since Kindles don't tend to spend huge amounts of time connected to computers. >>> >>> Ah, I did not know that the default polling interval was that low(ish), >>> given that the default indeed is already that low, then changing it to >>> 0.8 seconds would not be a big change. And we probably have a lot of >>> lower hanging fruit for unnecessary wakeups then that. >> >> So we need to make a decision: Should the patch be merged, or should we >> punt the issue to userspace tools? >> >> On the plus side, the patch is rather small and non-invasive (although >> it does allocate the last remaining bit in the 32-bit fflags field). >> There's also the advantage of sending the extra command only when it is >> needed, as opposed to increasing the overall frequency of TUR polling. >> >> Any opinions? > > I would vote to do in kernel as that is a cleaner solution: > > 1. It will work out of the box. > 2. It only sends one extra command when needed. > 3. It makes the block-device not break if user-space adjusts the poll > interval to higher values. FWIW my vote goes to the in kernel fix too. Regards, Hans ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [usb-storage] Re: Amazon Kindle disconnect after Synchronize Cache
Am 17.03.21 um 16:17 schrieb Alan Stern: On Wed, Mar 17, 2021 at 01:21:50PM +0100, Hans de Goede wrote: Hi, On 3/16/21 6:04 PM, Alan Stern wrote: I think it would be mildly better, but not a whole lot. Since the Kindle describes itself as having removable media, the kernel normally probes it periodically to make sure the media remains present. The default probing interval is 2 seconds. Reducing it to 0.9 seconds doesn't represent an exorbitant additional load IMO -- especially since Kindles don't tend to spend huge amounts of time connected to computers. Ah, I did not know that the default polling interval was that low(ish), given that the default indeed is already that low, then changing it to 0.8 seconds would not be a big change. And we probably have a lot of lower hanging fruit for unnecessary wakeups then that. So we need to make a decision: Should the patch be merged, or should we punt the issue to userspace tools? On the plus side, the patch is rather small and non-invasive (although it does allocate the last remaining bit in the 32-bit fflags field). There's also the advantage of sending the extra command only when it is needed, as opposed to increasing the overall frequency of TUR polling. Any opinions? I would vote to do in kernel as that is a cleaner solution: 1. It will work out of the box. 2. It only sends one extra command when needed. 3. It makes the block-device not break if user-space adjusts the poll interval to higher values. Matthias ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [usb-storage] Re: Amazon Kindle disconnect after Synchronize Cache
On Mi, 17.03.21 11:17, Alan Stern (st...@rowland.harvard.edu) wrote: > On Wed, Mar 17, 2021 at 01:21:50PM +0100, Hans de Goede wrote: > > Hi, > > > > On 3/16/21 6:04 PM, Alan Stern wrote: > > > I think it would be mildly better, but not a whole lot. Since the > > > Kindle describes itself as having removable media, the kernel normally > > > probes it periodically to make sure the media remains present. The > > > default probing interval is 2 seconds. Reducing it to 0.9 seconds > > > doesn't represent an exorbitant additional load IMO -- especially since > > > Kindles don't tend to spend huge amounts of time connected to computers. > > > > Ah, I did not know that the default polling interval was that low(ish), > > given that the default indeed is already that low, then changing it to > > 0.8 seconds would not be a big change. And we probably have a lot of > > lower hanging fruit for unnecessary wakeups then that. > > So we need to make a decision: Should the patch be merged, or should we > punt the issue to userspace tools? > > On the plus side, the patch is rather small and non-invasive (although > it does allocate the last remaining bit in the 32-bit fflags field). > There's also the advantage of sending the extra command only when it is > needed, as opposed to increasing the overall frequency of TUR polling. > > Any opinions? I'd argue that this should be done in the kernel. IIUC the issue can actually lead to data corruption, no? i.e. some program writes 25 different files to different places on such an fs, then calls fsync() on one of them but not the others. Then quite possibly the fsync() will trigger a device disconnect sooner or later at which point the one file surely hit the disk, but there's no guarantee for the other 24, they might remain cached in memory and are never written out. I'd say quirks that are necessary to avoid data corruption should better be done in the kernel and udev's hwdb stuff is only for stuff that "fills in gaps", i.e. adds additional tweaks that make things prettier, cleaner, nicer, more efficient but not things that make the basic things work, and data integrity sounds pretty basic to me. Or to give a counter example: the device advertises it can do media change, but actually cannot, right, it's not a floppy drive or cdrom driver after all? maybe hwdb would thus actually be the place for the opposite of the suggested fix: turn off the media change polling to reduce needless wakeups. I mean, I'd be more open to this if this was a frequent thing and the database for devices like this was just tooo large for the kernel to carry, but that's not the case here either: it's two devices afaik, and such an issue wasn't seen elswhere. Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [usb-storage] Re: Amazon Kindle disconnect after Synchronize Cache
On Wed, Mar 17, 2021 at 01:21:50PM +0100, Hans de Goede wrote: > Hi, > > On 3/16/21 6:04 PM, Alan Stern wrote: > > I think it would be mildly better, but not a whole lot. Since the > > Kindle describes itself as having removable media, the kernel normally > > probes it periodically to make sure the media remains present. The > > default probing interval is 2 seconds. Reducing it to 0.9 seconds > > doesn't represent an exorbitant additional load IMO -- especially since > > Kindles don't tend to spend huge amounts of time connected to computers. > > Ah, I did not know that the default polling interval was that low(ish), > given that the default indeed is already that low, then changing it to > 0.8 seconds would not be a big change. And we probably have a lot of > lower hanging fruit for unnecessary wakeups then that. So we need to make a decision: Should the patch be merged, or should we punt the issue to userspace tools? On the plus side, the patch is rather small and non-invasive (although it does allocate the last remaining bit in the 32-bit fflags field). There's also the advantage of sending the extra command only when it is needed, as opposed to increasing the overall frequency of TUR polling. Any opinions? Alan Stern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [usb-storage] Re: Amazon Kindle disconnect after Synchronize Cache
Hi, On 3/16/21 6:04 PM, Alan Stern wrote: > On Tue, Mar 16, 2021 at 05:43:34PM +0100, Hans de Goede wrote: >> Hi, >> >> On 3/16/21 5:26 PM, Alan Stern wrote: >>> On Tue, Mar 16, 2021 at 06:26:30AM +0100, Matthias Schwarzott wrote: I implemented solution 3b. This is the pullrequest for udev (systemd repository): https://github.com/systemd/systemd/pull/19002 Now Lennart asks if udev is the best place for such hacks/work-arounds? Well, I implemented it as suggested by Alan (see above). This was the simplest of the considered alternatives. Different quirks in kernel has been considered, but are more effort to be implemented. >>> >>> Lennart probably isn't aware how the usb-storage driver works. It does >>> not create commands on its own; it merely sends the commands that it >>> gets from higher SCSI layers. >>> >>> It may be possible to modify the SCSI core, to make it send a TEST UNIT >>> READY command immediately following any SYNCHRONIZE CACHE to a Kindle. >>> >>> However, there may be an easier solution. usb-storage does indeed send >>> a command of its own, REQUEST SENSE, to get error data when a command >>> fails. The patch below will make it do the same thing whenever it sends >>> a SYNCHRONIZE CACHE to a Kindle, failure or not. >>> >>> The only question is whether the Kindle will regard REQUEST SENSE as a >>> sufficient indication that it shouldn't do an eject. The only way to >>> find out is by testing the patch. >>> >>> Alan Stern >> >> Thank you for this patch, yes if this works it would IMHO be >> a much better solution then the udev rule. > > I think it would be mildly better, but not a whole lot. Since the > Kindle describes itself as having removable media, the kernel normally > probes it periodically to make sure the media remains present. The > default probing interval is 2 seconds. Reducing it to 0.9 seconds > doesn't represent an exorbitant additional load IMO -- especially since > Kindles don't tend to spend huge amounts of time connected to computers. Ah, I did not know that the default polling interval was that low(ish), given that the default indeed is already that low, then changing it to 0.8 seconds would not be a big change. And we probably have a lot of lower hanging fruit for unnecessary wakeups then that. Regards, Hans ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Antw: [EXT] Re: Odd effect when using systemd-analyze verify in RPM %check
>>> Lennart Poettering schrieb am 15.03.2021 um 17:04 in Nachricht : > On Mo, 15.03.21 16:54, Ulrich Windl (ulrich.wi...@rz.uni‑regensburg.de) wrote: > >> Hi! >> >> While constructing a new RPM package I tried "systemd‑analyze verify" as > %check, resulting in: >> ... >> Failed to open /dev/tty0: Permission denied >> ... >> >> However: >> > ll /dev/tty /dev/tty0 >> crw‑rw‑rw‑ 1 root tty 5, 0 Mar 15 16:26 /dev/tty >> crw‑‑w 1 root tty 4, 0 Mar 11 10:11 /dev/tty0 >> >> What's the issue with running "systemd‑analyze verify" and non‑root? > > This has been fixed a long time ago. > > /dev/tty0 is the foreground console, so it changes identity whenever > you switch VT. > > "systemd‑analyze verify" ultimately just runs the service manager with > different parameters, and there was a bug once upon a time, where the > service manager would use /dev/tty0 as it does when running PID1, but > of course it should not when running on a console. > > Maybe update to something less ancient, or ask your distro to backport > the fix. Maybe if you could either provide the version when it was fixed or maybe even the commit that fixed it, I'd be happy to report this bug to my distribution. > > Lennart > > ‑‑ > Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
