[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Joshua Diamant]

I am having this issue using IMSM / VROC 6.2 on Ubuntu 18.04 LTS, Kernel
5.3.0-28, mdadm - v4.1-rc1 - 2018-03-22.

I am running a bcache cache device on one of the RAID 1 IMSM (VROC)
arrays. At the very minimum, every reboot the device needs to resync
(not shutting down clean).

What other information is required to help us debug this issue?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: [Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[John Center]

Hi Dimitri,

> On Feb 22, 2018, at 8:32 AM, Dimitri John Ledkov  
> wrote:
> 
> john-center - I believe mdadm is used by default, instead of dmraid.
> However, I'm not sure if we do correctly activate Intel Raids with
> mdadm. I have not done an install with either 16.04 or 18.04. Or I guess
> at least start the install up to the partitioning screen. My expectation
> is, if one has raid arrays pre-setup in BIOS (ctrl+I on boot) already,
> they should show up as assembled and offered to be autopartitioned. With
> both 16.04 and 18.04.
> 
I know with 16.04 it wasn’t used by default. I had to do a lot of manipulation 
to set up the raid array before I could do the install. It used dmraid once it 
detected the imsm raid. I removed dmraid completely, then installed mdadm, 
assembled the array & ran the installation program again. I was hoping that 
mdadm would just do it all instead. 

-John

> -- 
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1587142
> 
> Title:
>  Shutdown hangs in md kworker after "Reached target Shutdown."
> 
> Status in systemd package in Ubuntu:
>  Confirmed
> 
> Bug description:
>  I'm booting a fully patched 16.04 from an Intel Rapid Storage
>  Technology enterprise RAID1 volume (ThinkServer TS140 with two SATA
>  ST1000NM0033-9ZM drives, ext4 root partition, no LVM, UEFI mode).
> 
>  If the RAID volume is recovering or resyncing for whatever reason, then 
> `sudo systemctl reboot` and `sudo systemctl poweroff` work fine (I had to 
> `sudo systemctl --now disable lvm2-lvmetad lvm2-lvmpolld lvm2-monitor` in 
> order to consistently get that). However, once the recovery/resync is 
> complete and clean, the reboot and poweroff commands above hang forever after 
> "Reached target Shutdown.". Note that issuing `sudo swapoff -a` beforehand 
> (suggested in the bug #1464917) does not help.
>  [EDIT]Actually, the shutdown also hangs from time to time during a resync. 
> But I've never seen it succeed once the resync is complete.[/EDIT]
> 
>  Then, if the server has been forcibly restarted with the power button,
>  the Intel Matrix Storage Manager indicates a "Normal" status for the
>  RAID1 volume, but Ubuntu then resyncs the volume anyway:
> 
>  [1.223649] md: bind
>  [1.228426] md: bind
>  [1.230030] md: bind
>  [1.230738] md: bind
>  [1.232985] usbcore: registered new interface driver usbhid
>  [1.233494] usbhid: USB HID core driver
>  [1.234022] md: raid1 personality registered for level 1
>  [1.234876] md/raid1:md126: not clean -- starting background 
> reconstruction
>  [1.234956] input: CHESEN USB Keyboard as 
> /devices/pci:00/:00:14.0/usb3/3-10/3-10:1.0/0003:0A81:0101.0001/input/input5
>  [1.236273] md/raid1:md126: active with 2 out of 2 mirrors
>  [1.236797] md126: detected capacity change from 0 to 1000202043392
>  [1.246271] md: md126 switched to read-write mode.
>  [1.246834] md: resync of RAID array md126
>  [1.247325] md: minimum _guaranteed_  speed: 1000 KB/sec/disk.
>  [1.247503]  md126: p1 p2 p3 p4
>  [1.248269] md: using maximum available idle IO bandwidth (but not more 
> than 20 KB/sec) for resync.
>  [1.248774] md: using 128k window, over a total of 976759940k.
> 
>  Note that the pain of "resync upon every (re)boot" cannot even be a
>  bit relieved thanks to bitmaps because mdadm does not support them for
>  IMSM containers:
> 
>  $ sudo mdadm --grow --bitmap=internal /dev/md126
>  mdadm: Cannot add bitmaps to sub-arrays yet
> 
>  I also get this in syslog during boot when the individual drives are
>  detected, but this seems to be harmless:
> 
>  May 30 17:26:07 wssrv1 systemd-udevd[608]: Process '/sbin/mdadm 
> --incremental /dev/sdb --offroot' failed with exit code 1.
>  May 30 17:26:07 wssrv1 systemd-udevd[608]: Process '/lib/udev/hdparm' failed 
> with exit code 1.
> 
>  May 30 17:26:07 wssrv1 systemd-udevd[606]: Process '/sbin/mdadm 
> --incremental /dev/sda --offroot' failed with exit code 1.
>  May 30 17:26:07 wssrv1 systemd-udevd[606]: Process '/lib/udev/hdparm' failed 
> with exit code 1.
> 
>  During a resync, `sudo sh -c 'echo idle >
>  /sys/block/md126/md/sync_action'` actually stops it as expected, but
>  it restarts immediately though nothing seems to have triggered it:
> 
>  May 30 18:17:02 wssrv1 kernel: [ 3106.826710] md: md126: resync interrupted.
>  May 30 18:17:02 wssrv1 kernel: [ 3106.836320] md: checkpointing resync of 
> md126.
>  May 30 18:17:02 wssrv1 kernel: [ 3106.836623] md: resync of RAID array md126
>  May 30 18:17:02 wssrv1 kernel: [ 3106.836625] md: minimum _guaranteed_  
> speed: 1000 KB/sec/disk.
>  May 30 18:17:02 wssrv1 kernel: [ 3106.836626] md: using maximum available 
> idle IO bandwidth (but not more than 20 KB/sec) for resync.
>  May 30 18:17:02 wssrv1 kernel: [ 3106.836627] md: using 128k window, over a 
> total of 976759940k.
>  May 30 18:17:02 

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dimitri John Ledkov]

john-center - I believe mdadm is used by default, instead of dmraid.
However, I'm not sure if we do correctly activate Intel Raids with
mdadm. I have not done an install with either 16.04 or 18.04. Or I guess
at least start the install up to the partitioning screen. My expectation
is, if one has raid arrays pre-setup in BIOS (ctrl+I on boot) already,
they should show up as assembled and offered to be autopartitioned. With
both 16.04 and 18.04.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Jens]

I followed the steps from #27 and everything works fine now on 16.04.3
with a SuperMicro Board X11SSH-F with Intel C236 Chipsets RST and two
sata disks as Raid 1.

I can also confirm #28, the first reboot still needed a hard reset and a
rebuild of the Raid 1, but after this, every (>10) shutdown and reboot
was OK.

And the solution still is alive after an update on the newest 4.4.0-116
kernel today.

Sorry for testing delay, but after missing a solution for too long time,
we installed our servers different way, only this week I got a new
testing machine until 18.04 is out.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[John Center]

I added a comment about this new fix on
https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/1608495.  One thing
I wanted to add here, for the first time shutdown completed
successfully.  In fact, I was a little startled when it happened.  :-)
It powered off at the end of shutdown, instead of hanging at the
"Reached target Shutdown" message.

One question: Does this mean that Ubuntu 18.04 will use mdadm by default
during installation instead of dmraid?  I really hope so.  It's a real
pain to do it manually.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dimitri John Ledkov]

Just a quick clarification, first boot/shutdown will still not be clean,
but subsequent ones (those that are booted with the updated mdadm
package) should be clean.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dimitri John Ledkov]

Hello,

as part of https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/1722491
there is currently mdadm update available from xenial-proposed and
zesty-proposed that might resolve this issue.

To test that solution please perform the following:

1) Install mdadm from xenial-proposed/zesty-proposed
   - See https://wiki.ubuntu.com/Testing/EnableProposed
   - Or download & install packages from
xenial https://launchpad.net/ubuntu/+source/mdadm/3.4-4ubuntu0.1/+build/13596415
zesty  https://launchpad.net/ubuntu/+source/mdadm/3.3-2ubuntu7.5/+build/13596431

2) $ sudo apt install dracut-core

3) $ sudo systemctl enable mdadm-shutdown.service

4) $ sudo systemctl start mdadm-shutdown.service

After this the expectation is for shutdown/reboots to perform clean a
shutdown, maintaining the raid array in a synced state, such that it
comes up clean.

Please let me know if above resolves shutdown/reboot issues for you.

Regards,

Dimitri.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dr. Thomas Orgis]

Oh … this is _another_ bug. We are dealing here with the situation that
mdmon controlling rootfs on RAID is not handled at all with Ubuntu
initrd, while this new CentOS issue is a bug in said handling in the
CentOS initrd …

The reference is valuable nevertheless … not least because I have
systems about to be upgraded to CentOS 7.4 that might get hit by this
bug!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Marek]

I would just like to inform you that this bug appear since last CentOS 7.4 
update, 7.3 was ok.
https://bugs.centos.org/view.php?id=13916

Maybe it should be reported to Systemd Issues ?

** Bug watch added: bugs.centos.org/ #13916
   https://bugs.centos.org/view.php?id=13916

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dr. Thomas Orgis]

Can we get any reaction from Ubuntu on this? Is the needed reworking of
the initrd about to happen for the LTS or is this a WONTFIX and root-on-
Intel-Matrix-RAID is simply not supported? It is clear what has to
happen to make things work again with systemd and mdadm/mdmon. Will it
happen?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[max moro]

as I don't want to rebuild now again, did anyone try
eg.

chmod +x ~/myshutdown.sh
sudo ~/myshutdown.sh

# with

echo r > /proc/sysrq-trigger
echo e > /proc/sysrq-trigger
echo s > /proc/sysrq-trigger
echo i > /proc/sysrq-trigger

echo u > /proc/sysrq-trigger
echo o > /proc/sysrq-trigger

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[max moro]

a BIOS reconfig. from "RAID" to "AHCI" did not change anything.
m.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[max moro]

Shutting down with a "R E S U I O" Workaround
was ok. The Raid was not rebuilt.
m.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dr. Thomas Orgis]

Is anyone working on this? I see that the bug is assigned, but apart
from that only messages from affected users.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dr. Thomas Orgis]

Well, of course effects are a pain. The sync is only 256G SSDs im my
case.

Main point is that the fix is known since many years and should
hopefully be quickly adaptible to Ubuntu. Or not … if it is really
necessary to switch to a return to the initrd like Fedora.

I wonder if 17.04 has this fixed. Can someone comment on that? Is the
proper initrd buildup/teardown implemented there? I cannot test this on
my servers as they are needed in production.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[max moro]

... interesting, but the pain is the new resync ... (writing 4(8) TB on each 
"hanging" reboot, ehm ...)

I used SysReq to sync before power-off

echo 1 > /proc/sys/kernel/sysrq

ALT + "DRUCK" ( or "Print Screen") + R E I S U O

as a workaround. as I have to wait for the sync, I can't report now.
of course that's no fun @ for remote machines.

ps. the servers were on centos before, I did not have any probs there ?!

m.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dr. Thomas Orgis]

So I got a simple fix for being able to reboot again:

```
udpate-rc.d mdadm disable
```

With that, the mdmon instance from the initrd persists (with @sbin/mdmon
as argv[0]) and is not killed by systemd. So systemd-reboot does not
hang. But: Since mdmon is not killed at all now, the array always gets a
fresh resync on each boot.

We really, really need to implement the return-to-initrd practice to be
able to bring the system down in the proper reverse order of bringing it
up.

(So initrd filesystem has to be kept in memory for the whole time …
wasting some memory with the huge initrds of today's time. It would also
be nice if mdmon was just part of the kernel as it should be.)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dr. Thomas Orgis]

This very much looks like this ancient issue in RedHat:

https://bugzilla.redhat.com/show_bug.cgi?id=752593

It got fixed at some point in 2012 …

https://bugzilla.redhat.com/show_bug.cgi?id=785739

Is Ubuntu still running mdmon from inside the root fs and killing it
before r/o remount?

I am now pondering what to do with a set of file servers running 16.04
and root on an IMSM RAID. I got a number of systems with similar
hardware running CentOS 7 just fine with no issue on reboot.

I see elaborate handling of mdmon in systemd units on CentOS … including
the --offroot command line parameter that I don't see documented
anywhere. KillMode=none might also play a role in the mdmon@.service
file.

** Bug watch added: Red Hat Bugzilla #752593
   https://bugzilla.redhat.com/show_bug.cgi?id=752593

** Bug watch added: Red Hat Bugzilla #785739
   https://bugzilla.redhat.com/show_bug.cgi?id=785739

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[max moro]

** Attachment added: "screenshot"
   
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+attachment/4945763/+files/reached_target.png

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[max moro]

confirmed on 
HP ProLiant ML10 Gen9, Xeon E3-1225 v5
Intel Raid Bios set to RAID1 (4 drives, 2xraid1)

Shutdowen/reboot hangs (kworkers) and Raid is rebuilt on every "reset"


Linux w11 4.4.0-93-generic #116-Ubuntu SMP Fri Aug 11 21:17:51 UTC 2017 x86_64 
x86_64 x86_64 GNU/Linux
Description:Ubuntu 16.04.3 LTS
Release:16.04

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] 
[raid10]
md124 : active raid1 sdc[1] sdd[0]
  2930264064 blocks super external:/md125/0 [2/2] [UU]
  [=>...]  resync = 27.1% (795895744/2930264196) 
finish=190.3min speed=186838K/sec

md125 : inactive sdd[1](S) sdc[0](S)
  4776 blocks super external:imsm

md126 : active raid1 sda[1] sdb[0]
  976759808 blocks super external:/md127/0 [2/2] [UU]
  [===>.]  resync = 75.0% (733104704/976759940) 
finish=28.2min speed=143955K/sec

md127 : inactive sda[1](S) sdb[0](S)
  5288 blocks super external:imsm

unused devices: 

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Adam Lansky]

Confirm on HPE ML10 gen9.
Ubuntu 16.04.2 LTS
Intel SATA RAID - RAID 1. Intel SSD.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Roman Ledovskiy]

Confirm this on:
Supermicro X10DAi board and ASUS ESC4000 G3 server
Ubuntu 16.04.2 LTS

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Jim Nijkamp]

Confirm on this bug with hardware:

HP Z230, Intel Raid bios, Ubuntu 16.04.2.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Jens]

Same problem with a SuperMicro Board X11SSH-F with Intel C236 Chipset
and two sata disks.

Installing the server with bios sata mode "RAID" and a RAID1 configured
in the Intel Raid bios (Ctrl+I), on a fresh Ubuntu install and also on
updated Ubuntu 16.04.2 the server doesn't restart or shutdown, it hangs
at "Reached target shutdown".

Installing the server with bios sata mode "AHCI", so without configuring
something in the Intel Raid bios, but configuring a RAID1 only direct at
Ubuntu as mdraid, restart and shutdown work fine with fresh install and
also with all updates to 16.04.2 as expected.

May I ask, if this bug will be solved in the future, and if there is a
timeline? We planned to buy a lot of this servers, install Ubuntu and
put them in a data centre miley away, but we can't wait another year.

It someone in Germany really is able and willing to solve this bug, I
would be willing to lend our server for some weeks.

Regards, Jens

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Dimitri John Ledkov]

** Changed in: systemd (Ubuntu)
   Importance: High => Critical

** Changed in: systemd (Ubuntu)
 Assignee: (unassigned) => Dimitri John Ledkov (xnox)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Michael Cain]

Confirming this same issue for me on a clean install of Ubuntu 16.04.2
X64 server on a HP 8300 Elite SFF pc.  Anything I can collect to
troubleshoot?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Matt Wear]

I have a similar problem with a super micro X10DRH system board.  If I have the 
intel sata controller set to ACHI and install a fresh instance of 16.04 the 
server works without issue.  If I have the intel sata controller in RSTe RAID 
mode with a 2 drive RAID1 configuration the server will hang on reboot at the 
'Reached target shutdown" message.  Are there any logs that I can provide to 
assist in troubleshooting?
Thanks,
-Matt

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Benoît Thébaudeau]

With upstart-sysv, I have now observed a spurious RAID1 verify following
a reboot after an unrelated apt upgrade (libc stuff). It's the only
spurious RAID1 verify that I have observed so far with upstart-sysv. So
upstart-sysv is not perfect, but still better than systemd-sysv
regarding these issues.

Also, after one reboot, I got my ext4 rootfs mounted ro without any
apparent reason (no errors in the logs). Everything came back to normal
after another reboot.

I think I will try Debian, CentOS, and maybe Windows.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Benoît Thébaudeau]

I still get the same issues with a fully updated Ubuntu Server 16.04.1
LTS (Linux 4.4.0-45-generic).

The patch from
https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/1320402/comments/13
does not seem to have any effect on these issues.

I have also tried the latest mainline kernel build from 
http://kernel.ubuntu.com/~kernel-ppa/mainline/?C=N;O=D , as suggested in 
https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/1320402 :
Linux wssrv1 4.8.4-040804-generic #201610220733 SMP Sat Oct 22 11:35:18 UTC 
2016 x86_64 x86_64 x86_64 GNU/Linux
With or without the patch indicated above, this kernel seems to fix the 
spurious verify issue (so far...), but not the hanging reboot and poweroff. 
This is an unmaintained kernel intended for debug purposes and not for 
production anyway.

I attach a hanging reboot log with Linux 4.8.4-040804-generic and the
patch indicated above, with dynamic debug enabled in drivers/md/*. It
hangs when remounting '/' read-only, still in the md kworker.

Then, I tried to install upstart-sysv to see if getting rid of systemd
helps, and it does. With upstart-sysv, all the issues described here
seem to be fixed with Linux 4.4.0-45-generic and no patches. This seems
to confirm that these issues are caused by systemd. However, I don't
know how reliable and tested upstart-sysv is on Ubuntu Server 16.04.1
LTS, so this might possibly be an issue for a production server. I would
prefer to keep working with systemd.

** Attachment added: "reboot-md-dyn-debug-patch-4.8.4.png"
   
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+attachment/4767751/+files/reboot-md-dyn-debug-patch-4.8.4.png

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Sergio Callegari]

Possibly also related to 1320402 (as the unnecessary resync probably
follows the incorrect stopping of the array).

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Alberto Salvia Novella]

** Changed in: systemd (Ubuntu)
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1587142] Re: Shutdown hangs in md kworker after "Reached target Shutdown."

[Launchpad Bug Tracker]

Status changed to 'Confirmed' because the bug affects multiple users.

** Changed in: systemd (Ubuntu)
   Status: New => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1587142

Title:
  Shutdown hangs in md kworker after "Reached target Shutdown."

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1587142/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs