** Description changed:
Instead of killing something with a high niceness low memory situations
seem to hang something super-ultra-so-very vital, writes fail and the
root partition is remounted read-only. After that systemd-journald
starts stupidly spamming and earlier logs are rendered unusable.
I'd post more logs but https://github.com/systemd/systemd/issues/2339
hasn't been fixed in years.
This has happened at least four times, across many kernel versions. With
Ubuntu 18.10, 19.04 and 19.10.
-
```
[827788.409138] sd 5:0:0:0: [sdb] tag#12 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_TIMEOUT
[827788.409141] sd 5:0:0:0: [sdb] tag#12 CDB: Write(10) 2a 00 01 e0 b8 f0 00
00 10 00
[827788.409143] blk_update_request: I/O error, dev sdb, sector 31504624 op
0x1:(WRITE) flags 0x0 phys_seg 2 prio class 0
[827788.409151] EXT4-fs warning (device dm-1): ext4_end_bio:309: I/O error 10
writing to inode 1577543 (offset 0 size 0 starting block 3871518)
[827788.409154] Buffer I/O error on device dm-1, logical block 3871518
[827788.409163] EXT4-fs warning (device dm-1): ext4_end_bio:309: I/O error 10
writing to inode 1577543 (offset 0 size 0 starting block 3871519)
[827788.409164] Buffer I/O error on device dm-1, logical block 3871519
[827788.409172] sd 5:0:0:0: [sdb] tag#11 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_TIMEOUT
[827788.409174] sd 5:0:0:0: [sdb] tag#11 CDB: Write(10) 2a 00 01 e0 bb 70 00
00 08 00
[827788.409175] blk_update_request: I/O error, dev sdb, sector 31505264 op
0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0
[827788.409179] EXT4-fs warning (device dm-1): ext4_end_bio:309: I/O error 10
writing to inode 1577543 (offset 0 size 0 starting block 3871598)
[827788.409180] Buffer I/O error on device dm-1, logical block 3871598
[827788.409188] sd 5:0:0:0: [sdb] tag#10 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_TIMEOUT
[827788.409189] sd 5:0:0:0: [sdb] tag#10 CDB: Write(10) 2a 00 01 e0 ba 60 00
00 38 00
[827788.409190] blk_update_request: I/O error, dev sdb, sector 31504992 op
0x1:(WRITE) flags 0x0 phys_seg 7 prio class 0
[827788.409194] EXT4-fs warning (device dm-1): ext4_end_bio:309: I/O error 10
writing to inode 1577543 (offset 0 size 0 starting block 3871564)
[827788.409195] Buffer I/O error on device dm-1, logical block 3871564
[827788.409196] Buffer I/O error on device dm-1, logical block 3871565
[827788.409198] Buffer I/O error on device dm-1, logical block 3871566
[827788.409199] Buffer I/O error on device dm-1, logical block 3871567
[827788.409201] Buffer I/O error on device dm-1, logical block 3871568
[827788.409202] Buffer I/O error on device dm-1, logical block 3871569
[827788.409203] Buffer I/O error on device dm-1, logical block 3871570
[827788.409210] sd 5:0:0:0: [sdb] tag#9 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_TIMEOUT
[827788.409211] sd 5:0:0:0: [sdb] tag#9 CDB: Write(10) 2a 00 01 e0 ba 30 00
00 08 00
[827788.409212] blk_update_request: I/O error, dev sdb, sector 31504944 op
0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0
[827788.409216] EXT4-fs warning (device dm-1): ext4_end_bio:309: I/O error 10
writing to inode 1577543 (offset 0 size 0 starting block 3871558)
[827788.409220] sd 5:0:0:0: [sdb] tag#8 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_TIMEOUT
[827788.409221] sd 5:0:0:0: [sdb] tag#8 CDB: Write(10) 2a 00 01 e0 ba 50 00
00 08 00
[827788.409222] blk_update_request: I/O error, dev sdb, sector 31504976 op
0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0
[827788.409225] EXT4-fs warning (device dm-1): ext4_end_bio:309: I/O error 10
writing to inode 1577543 (offset 0 size 0 starting block 3871562)
[827788.409228] sd 5:0:0:0: [sdb] tag#4 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_TIMEOUT
[827788.409229] sd 5:0:0:0: [sdb] tag#4 CDB: Write(10) 2a 00 01 e0 bb 88 00
00 08 00
[827788.409230] blk_update_request: I/O error, dev sdb, sector 31505288 op
0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0
[827788.409233] EXT4-fs warning (device dm-1): ext4_end_bio:309: I/O error 10
writing to inode 1577543 (offset 0 size 0 starting block 3871601)
[827818.628273] sd 5:0:0:0: [sdb] tag#31 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_TIMEOUT
[827818.628276] sd 5:0:0:0: [sdb] tag#31 CDB: Write(10) 2a 00 01 cf 12 38 00
00 30 00
[827818.628279] blk_update_request: I/O error, dev sdb, sector 30347832 op
0x1:(WRITE) flags 0x800 phys_seg 6 prio class 0
[827818.628347] sd 5:0:0:0: [sdb] tag#28 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_TIMEOUT
[827818.628351] sd 5:0:0:0: [sdb] tag#28 CDB: Write(10) 2a 00 01 93 2f b0 00
00 08 00
[827818.628353] blk_update_request: I/O error, dev sdb, sector 26423216 op
0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0
[827818.628358] Aborting journal on device dm-1-8.
[827818.628374] EXT4-fs warning (device dm-1): ext4_end_bio:309: I/O error 10
writing to inode 1577543 (offset 0 size 0 starting block