Re: [systemd-devel] more verbose debug info than systemd.log_level=debug?

[Tomasz Torcz]

On Sun, Apr 09, 2017 at 10:37:36PM -0600, Chris Murphy wrote:
> On Sun, Apr 9, 2017 at 5:17 AM, Lennart Poettering
>  wrote:
> 
> > That said, are you sure FIFREEZE is really what we want there? it
> > appears to also pause any further writes to disk (until FITHAW is
> > called).
> 
> > So, I am still puzzled why the file system people think that "sync()"
> > isn't supposed to actually sync things to disk...
> 
> https://www.spinics.net/lists/linux-xfs/msg05113.html
> 

  So the “solution” seems to be adding FIFREEZE/FITHAW ioctls after sync()?

-- 
Tomasz Torcz   "Never underestimate the bandwidth of a station
xmpp: zdzich...@chrome.plwagon filled with backup tapes." -- Jim Gray

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] more verbose debug info than systemd.log_level=debug?

[Chris Murphy]

On Sun, Apr 9, 2017 at 5:17 AM, Lennart Poettering
 wrote:

> That said, are you sure FIFREEZE is really what we want there? it
> appears to also pause any further writes to disk (until FITHAW is
> called).

> So, I am still puzzled why the file system people think that "sync()"
> isn't supposed to actually sync things to disk...

https://www.spinics.net/lists/linux-xfs/msg05113.html

The question isn't directly answered in there (it is part of the
thread on this very subject though). My guess at is that sync()
predates journaled file systems, and the expectations of sync() for a
journaled file system are basically just crash consistency, not all
metadata is on disk. Fully writing all metadata is expensive; as is
checking fixing it with an offline fsck. Both of those are reasons why
we have journaled filesystems. If sync() required all fs metadata to
commit to stable media it would make file systems dog slow. Every damn
thing is doing fsync's now. Before Btrfs had a log tree, workloads
with many fsyncs would hang the file system and the entire workfload
as well, so my guess is sync() meaning all fs metadata is committed on
ext4 and XFS would mean massive performance hits that no one would be
happy about.



>
> Why bother with sync() at all, if it implies no guarantees? This is
> quite frankly bullshit...
>
> It appears to me that using /boot on a file system whith such broken
> sync() semantics is really not a safe thing to do, and people should
> probably only use something more reliable, i.e. ext2 or vfat where
> sync() actually works correctly...

Oh god - that's the opposite direction to go in. There's not even
pretend crash safety with those file systems. If they're dirty, you
must use an fsck to get them back to consistency. Even if the toy fs
support found in firmware will tolerate the inconsistency, who knows
what blocks it actually ends up loading into memory, you can just get
a crash later at the bootloader, or the kernel, or initramfs. That so
much consumer hardware routinely lies about having committed data to
stable media following sync() makes those file systems even less
reliable for this purpose. Once corrupt, the file system has no fail
safe or fallback like a journaled or COW file system. It's busted
until fixed with fsck.

-- 
Chris Murphy
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] ExecStartPost in a systemd unit file does not start docker service

[Mantas Mikulėnas]

Nested jobs are problematic as they usually result in a deadlock – you'll
have to use "/bin/systemctl --no-block restart docker" there.

Though the whole setup in general seems suspicious...

On Sat, Apr 8, 2017, 18:29 Alex Chistyakov  wrote:

> Hello,
>
> I am trying to establish a connection between firewalld and docker
> services. I extended the default firewalld.service unit file by adding
> the following:
>
> [Service]
> ExecStartPost=-/bin/bash -c '/usr/bin/test -f /etc/default/docker &&
> /bin/systemctl stop docker && /bin/systemctl start docker'
>
> to /etc/systemd/system/firewalld.service.d/docker.conf.
>
> But this did not work, firewalld service timed out on start:
>
> root@ubuntu-xenial:~# systemctl status firewalld
> ● firewalld.service - firewalld - dynamic firewall daemon
>Loaded: loaded (/lib/systemd/system/firewalld.service; enabled;
> vendor preset: enabled)
>   Drop-In: /etc/systemd/system/firewalld.service.d
>└─docker.conf
>Active: failed (Result: timeout) since Sat 2017-04-08 14:39:45 UTC;
> 1min 35s ago
>   Process: 26050 ExecStartPost=/bin/bash -c /usr/bin/test -f
> /etc/default/docker && /bin/systemctl stop docker && /bin/systemctl
> start docker (code=killed, signal=TERM)
>   Process: 26000 ExecStart=/usr/sbin/firewalld --nofork --nopid
> (code=exited, status=0/SUCCESS)
>  Main PID: 26000 (code=exited, status=0/SUCCESS)
>
> Apr 08 14:38:10 ubuntu-xenial systemd[1]: Starting firewalld - dynamic
> firewall daemon...
> Apr 08 14:39:41 ubuntu-xenial systemd[1]: firewalld.service:
> Start-post operation timed out. Stopping.
> Apr 08 14:39:45 ubuntu-xenial systemd[1]: Failed to start firewalld -
> dynamic firewall daemon.
> Apr 08 14:39:45 ubuntu-xenial systemd[1]: firewalld.service: Unit
> entered failed state.
> Apr 08 14:39:45 ubuntu-xenial systemd[1]: firewalld.service: Failed
> with result 'timeout'.
>
> I am aware of BindTo and Requires but I would like to restart the
> docker service on every state change of firewalld so these directives
> do not solve my problem.
>
> Thank you,
>
> --
> SY,
> Alex
> ___
> systemd-devel mailing list
> systemd-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/systemd-devel
>
-- 

Mantas Mikulėnas 
Sent from my phone
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] high latency on systemd-resolved repsonses of LLMNR names

[Paul Freeman]

Hi,
 We are seeing high latency (>100ms) when resolving local names via 
LLMNR.


systemd v233 / Gentoo

There appears to be a large delay between systemd-resolved receiving 
the udp query over 127.0.0.53:53 and when it dispatches the LLMNR query:


18:29:38.752644 clock_gettime(CLOCK_BOOTTIME, {18217, 767968786}) = 0
18:29:38.752854 recvfrom(12, NULL, 0, MSG_PEEK|MSG_TRUNC, NULL, NULL) = 
46
18:29:38.753119 recvmsg(12, {msg_name={sa_family=AF_INET, 
sin_port=htons(52277), sin_addr=inet_addr("127.0.0.1")}, 
msg_namelen=128->16, msg_iov=[{iov_base="9m\1 
\0\1\0\0\0\0\0\1\5zeusv\0\0\1\0\1\0\0)\20\0\0\0\0\0"..., iov_len=3936}], 
msg_iovlen=1, msg_control=[{cmsg_len=28, cmsg_level=SOL_IP, 
cmsg_type=IP_PKTINFO, cmsg_data={ipi_ifindex=if_nametoindex("lo"), 
ipi_spec_dst=inet_addr("127.0.0.53"), 
ipi_addr=inet_addr("127.0.0.53")}}, {cmsg_len=20, cmsg_level=SOL_IP, 
cmsg_type=IP_TTL, cmsg_data=[64]}], msg_controllen=56, msg_flags=0}, 0) 
= 46
18:29:38.753680 stat("/etc/hosts", {st_dev=makedev(8, 4), 
st_ino=17019407, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, 
st_blksize=4096, st_blocks=8, st_size=1213, 
st_atime=2017/01/23-05:40:35.417021678, st_mtime=2016/09/01-21:55:25, 
st_ctime=2017/03/20-01:21:31.905862840}) = 0
18:29:38.754087 stat("/etc/resolv.conf", {st_dev=makedev(8, 4), 
st_ino=1667316, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, 
st_blksize=4096, st_blocks=8, st_size=484, 
st_atime=2017/04/07-02:39:49.250057911, 
st_mtime=2017/04/07-02:26:21.445885318, 
st_ctime=2017/04/07-02:39:55.170044459}) = 0

18:29:38.754466 getrandom("G\355", 2, GRND_NONBLOCK) = 2
18:29:38.754665 stat("/etc/resolv.conf", {st_dev=makedev(8, 4), 
st_ino=1667316, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, 
st_blksize=4096, st_blocks=8, st_size=484, 
st_atime=2017/04/07-02:39:49.250057911, 
st_mtime=2017/04/07-02:26:21.445885318, 
st_ctime=2017/04/07-02:39:55.170044459}) = 0

18:29:38.755028 clock_gettime(CLOCK_BOOTTIME, {18217, 770345701}) = 0
18:29:38.755224 clock_gettime(CLOCK_BOOTTIME, {18217, 770532540}) = 0
18:29:38.755399 getrandom("$\367\2557x\301\261!", 8, GRND_NONBLOCK) = 8
18:29:38.755581 timerfd_settime(19, TFD_TIMER_ABSTIME, {it_interval={0, 
0}, it_value={18217, 960132000}}, NULL) = 0
18:29:38.755766 epoll_wait(4, [{EPOLLIN, {u32=44602104, 
u64=292102378232}}], 20, -1) = 1




18:29:38.950146 clock_gettime(CLOCK_BOOTTIME, {18217, 965532171}) = 0
18:29:38.950445 read(19, "\1\0\0\0\0\0\0\0", 8) = 8
18:29:38.950760 sendmsg(13, {msg_name={sa_family=AF_INET, 
sin_port=htons(5355), sin_addr=inet_addr("224.0.0.252")}, 
msg_namelen=16, 
msg_iov=[{iov_base="G\355\0\0\0\1\0\0\0\0\0\0\5zeusv\0\0\1\0\1", 
iov_len=23}], msg_iovlen=1, msg_control=[{cmsg_len=28, 
cmsg_level=SOL_IP, cmsg_type=IP_PKTINFO, 
cmsg_data={ipi_ifindex=if_nametoindex("eth0"), 
ipi_spec_dst=inet_addr("0.0.0.0"), ipi_addr=inet_addr("0.0.0.0")}}], 
msg_controllen=28, msg_flags=0}, 0) = 23
18:29:38.951422 timerfd_settime(19, TFD_TIMER_ABSTIME, {it_interval={0, 
0}, it_value={18218, 278485000}}, NULL) = 0
18:29:38.951757 epoll_wait(4, [{EPOLLIN, {u32=44630672, 
u64=292102406800}}], 20, -1) = 1

18:29:38.952078 clock_gettime(CLOCK_BOOTTIME, {18217, 967408726}) = 0
18:29:38.952388 recvfrom(13, NULL, 0, MSG_PEEK|MSG_TRUNC, NULL, NULL) = 
23
18:29:38.952782 recvmsg(13, {msg_name={sa_family=AF_INET, 
sin_port=htons(5355), sin_addr=inet_addr("192.168.166.49")}, 
msg_namelen=128->16, 
msg_iov=[{iov_base="G\355\0\0\0\1\0\0\0\0\0\0\5zeusv\0\0\1\0\1", 
iov_len=3936}], msg_iovlen=1, msg_control=[{cmsg_len=28, 
cmsg_level=SOL_IP, cmsg_type=IP_PKTINFO, 
cmsg_data={ipi_ifindex=if_nametoindex("eth0"), 
ipi_spec_dst=inet_addr("192.168.166.49"), 
ipi_addr=inet_addr("224.0.0.252")}}, {cmsg_len=20, cmsg_level=SOL_IP, 
cmsg_type=IP_TTL, cmsg_data=[255]}], msg_controllen=56, msg_flags=0}, 0) 
= 23
18:29:38.953365 epoll_wait(4, [{EPOLLIN, {u32=44630672, 
u64=292102406800}}], 20, -1) = 1

18:29:38.953616 clock_gettime(CLOCK_BOOTTIME, {18217, 968941015}) = 0
18:29:38.953831 recvfrom(13, NULL, 0, MSG_PEEK|MSG_TRUNC, NULL, NULL) = 
39
18:29:38.954057 recvmsg(13, {msg_name={sa_family=AF_INET, 
sin_port=htons(5355), sin_addr=inet_addr("192.168.166.52")}, 
msg_namelen=128->16, 
msg_iov=[{iov_base="G\355\200\0\0\1\0\1\0\0\0\0\5zeusv\0\0\1\0\1\300\f\0\1\0\1\0\0\0"..., 
iov_len=3936}], msg_iovlen=1, msg_control=[{cmsg_len=28, 
cmsg_level=SOL_IP, cmsg_type=IP_PKTINFO, 
cmsg_data={ipi_ifindex=if_nametoindex("eth0"), 
ipi_spec_dst=inet_addr("192.168.166.49"), 
ipi_addr=inet_addr("192.168.166.49")}}, {cmsg_len=20, cmsg_level=SOL_IP, 
cmsg_type=IP_TTL, cmsg_data=[255]}], msg_controllen=56, msg_flags=0}, 0) 
= 39

18:29:38.954738 clock_gettime(CLOCK_BOOTTIME, {18217, 970057860}) = 0
18:29:38.954965 sendmsg(12, {msg_name={sa_family=AF_INET, 
sin_port=htons(52277), sin_addr=inet_addr("127.0.0.1")}, msg_namelen=16, 
msg_iov=[{iov_base="9m\201\200\0\1\0\1\0\0\0\1\5zeusv\0\0\1\0\1\300\f\0\1\0\1\0\0\0"..., 
iov_len=50}], msg_iovlen=1, msg_control=[{cmsg_len=28, 

Re: [systemd-devel] more verbose debug info than systemd.log_level=debug?

[Holger Kiehl]

On Sat, 8 Apr 2017, Chris Murphy wrote:

> On Tue, Apr 4, 2017 at 11:55 AM, Andrei Borzenkov  wrote:
> > grub2 is not limited to 640KiB. Actually it will actively avoid using
> > low memory. It switches to protected mode as the very first thing and
> > can use up to 4GiB (and even this probably can be lifted on 64 bit
> > platform). The real problem is the fact that grub is read-only so every
> > time you access file on journaled partition it will need to replay
> > journal again from scratch. This will likely be painfully slow (I
> > remember that grub legacy on reiser needed couple of minutes to read
> > kernel and much more to read initrd, and that was when both were smaller
> > than now).
> 
> OK well that makes more sense; but yeah it still sounds like journal
> replay is a non-starter. The entire fs metadata would have to be read
> into memory and create something like a RAM based rw snapshot which is
> backed by the ro disk version as origin, and then play the log against
> the RAM snapshot. That could be faster than constantly replaying the
> journal from scratch for each file access. But still - sounds overly
> complicated.
> 
> I think this qualifies as "Doctor, it hurt when I do this." And the
> doctor says, "So don't do that." And I'm referring to Plymouth
> exempting itself from kill while also not running from initramfs. So
> I'll kindly make the case with Plymouth folks to stop pressing this
> particular hurt me button.
> 
> But hey, pretty cool bug. Not often is it the case you find such an
> old bug so easily reproducible but near as I can tell only one person
> was hitting it until I tried to reproduce it.
> 
I too was hit by this bug on one of my systems. But what I did is that
I just removed all plymouth rpms and everything was good form that moment
on.

Holger
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] more verbose debug info than systemd.log_level=debug?

[Lennart Poettering]

On Sun, 09.04.17 10:11, Michael Chapman (m...@very.puzzling.org) wrote:

> Don't forget, they've provided an interface for software to use if it needs
> more than the guarantees provided by sync. Informally speaking, the FIFREEZE
> ioctl is intended to place a filesystem into a "fully consistent" state, not
> just a "fully recoverable" state. (Formally it's all a bit hazy: POSIX
> really doesn't guarantee anything with sync.)

If FIFREEZE is a generic ioctl() supported by a number of different
file systems I figure it would be much more OK with calling it.

That said, are you sure FIFREEZE is really what we want there? it
appears to also pause any further writes to disk (until FITHAW is
called). Which isn't really what we are interested in here (note that
we return back to the initrd after the umount spree and it shall be
able to do the rest, and if it actually can do that, then the file
systems should be able to unmount and that usually results in writes
to disk...)

So, I am still puzzled why the file system people think that "sync()"
isn't supposed to actually sync things to disk... I mean, it appears
the call is pretty much useless and it's traditional usage (which
prominently is in sysvinit before reboot()) appears to be broken by
their behaviour.

Why bother with sync() at all, if it implies no guarantees? This is
quite frankly bullshit...

It appears to me that using /boot on a file system whith such broken
sync() semantics is really not a safe thing to do, and people should
probably only use something more reliable, i.e. ext2 or vfat where
sync() actually works correctly...

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel