This is fixed in 0.7.9.
** Changed in: cloud-init
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of नेपाली
भाषा समायोजकहरुको समूह, which is subscribed to Xenial.
Matching subscriptions: Ubuntu 16.04 Bugs
https://bugs.launchpad.net/bugs/1611074
Title:
Reformatting of ephemeral drive fails on resize of Azure VM
Status in cloud-init:
Fix Released
Status in cloud-init package in Ubuntu:
Fix Released
Status in cloud-init source package in Xenial:
Fix Released
Status in cloud-init source package in Yakkety:
Fix Released
Bug description:
=== Begin SRU Template ===
[Impact]
In some cases, cloud-init writes entries to /etc/fstab, and on azure it will
even format a disk for mounting and then write the entry for that 'ephemeral'
disk there.
A supported operation on Azure is to "resize" the system. When you do this
the system is shut down, resized (given larger/faster disks and more CPU) and
then brought back up. In that process, the "ephemeral" disk re-initialized
to its original NTFS format. The designed goal is for cloud-init to recognize
this situation and re-format the disk to ext4.
The problem is that the mount of that disk happens before cloud-init can
reformat. Thats because the entry in fstab has 'auto' and is automatically
mounted. The end result is that after resize operation the user will be left
with the ephemeral disk mounted at /mnt and having a ntfs filesystem rather
than ext4.
[Test Case]
The text in comment 3 describes how to recreate by the original reporter.
Another way to do this is to just re-format the ephemeral disk as
ntfs and then reboot. The result *should* be that after reboot it
comes back up and has an ext4 filesystem on it.
1.) boot system on azure
(for this, i use https://gist.github.com/smoser/5806147, but you can
use web ui or any other way).
Save output of
journalctl --no-pager > journalctl.orig
systemctl status --no-pager > systemctl-status.orig
systemctl --no-pager > systemctl.orig
2.) unmount the ephemeral disk
$ umount /mnt
3.) repartition it so that mkfs.ntfs does less and is faster
This is not strictly necessary, but mkfs.ntfs can take upwards of
20 minutes. shrinking /dev/sdb2 to be 200M means it will finish
in < 1 minute.
$ disk=/dev/disk/cloud/azure_resource
$ part=/dev/disk/cloud/azure_resource-part1
$ echo "2048,$((2*1024*100)),7" | sudo sfdisk "$disk"
$ time mkfs.ntfs --quick "$part"
4.) reboot
5.) expect that /proc/mounts has /dev/disk/cloud/azure_resource-part1 as ext4
and that fstab has x-systemd.requires in it.
$ awk '$2 == "/mnt" { print $0 }' /proc/mounts
/dev/sdb1 /mnt ext4 rw,relatime,data=ordered 0 0
$ awk '$2 == "/mnt" { print $0 }' /etc/fstab
/dev/sdb1 /mnt auto
defaults,nofail,x-systemd.requires=cloud-init.service,comment=cloudconfig 0 2
6.) collect journal and systemctl information as described in step 1 above.
Compare output, specifically looking for case insensitve "breaks"
[Regression Potential]
Regression is unlikely. Likely failure case is just that the problem is not
correctly fixed, and the user ends up with either an NTFS formated disk that
is mounted at /mnt or there is nothing mounted at /mnt.
=== End SRU Template ===
After resizing a 16.04 VM on Azure, the VM is presented with a new
ephemeral drive (of a different size), which initially is NTFS
formatted. Cloud-init tries to format the appropriate partition ext4,
but fails because it is mounted. Cloud-init has unmount logic for
exactly this case in the get_data call on the Azure data source, but
this is never called because fresh cache is found.
Jun 27 19:07:47 azubuntu1604arm [CLOUDINIT] handlers.py[DEBUG]: start:
init-network/check-cache: attempting to read from cache [trust]
Jun 27 19:07:47 azubuntu1604arm [CLOUDINIT] util.py[DEBUG]: Reading from
/var/lib/cloud/instance/obj.pkl (quiet=False)
Jun 27 19:07:47 azubuntu1604arm [CLOUDINIT] util.py[DEBUG]: Read 5950 bytes
from /var/lib/cloud/instance/obj.pkl
Jun 27 19:07:47 azubuntu1604arm [CLOUDINIT] stages.py[DEBUG]: restored from
cache: DataSourceAzureNet [seed=/dev/sr0]
Jun 27 19:07:47 azubuntu1604arm [CLOUDINIT] handlers.py[DEBUG]: finish:
init-network/check-cache: SUCCESS: restored from cache: DataSourceAzureNet
[seed=/dev/sr0]
...
Jun 27 19:07:48 azubuntu1604arm [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating
file system None on /dev/sdb1
Jun 27 19:07:48 azubuntu1604arm [CLOUDINIT] cc_disk_setup.py[DEBUG]:
Using cmd: /sbin/mkfs.ext4 /dev/sdb1
Jun 27 19:07:48 azubuntu1604arm [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/mkfs.ext4', '/dev/sdb1'] with allowed return codes [0] (shell=False,
capture=True)
Jun 27 19:07:48 azubuntu1604arm [CLOUDINIT] util.py[DEBUG]: Creating fs for
/dev/disk/cloud/azure_resource took 0.052 seconds
Jun 27 19:07:48 azubuntu1604arm [CLOUDINIT] util.py[WARNING]: Failed during
filesystem operation#012Failed to exec of '['/sbin/mkfs.ext4',
'/dev/sdb1']':#012Unexpected error while running command.#012Command:
['/sbin/mkfs.ext4', '/dev/sdb1']#012Exit code: 1#012Reason: -#012Stdout:
''#012Stderr: 'mke2fs 1.42.13 (17-May-2015)\n/dev/sdb1 is mounted; will not
make a filesystem here!\n'
$ lsb_release -rd
Description: Ubuntu 16.04.1 LTS
Release: 16.04
$ cat /etc/cloud/build.info
build_name: server
serial: 20160721
~$ dpkg -l cloud-init
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture
Description
+++-================================-=====================-=====================-=====================================================================
ii cloud-init 0.7.7~bzr1256-0ubuntu all
Init scripts for cloud instances
We're seeing ~100% repro of this bug on resize, where the only success
cases are caused by another bug that messes up fstab and prevents
mounting of the drive.
Related bugs:
bug 1629868: cloud-init times out because of no dbus
bug 1603222: Azure: incorrect entry in fstab for ephemeral disk
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1611074/+subscriptions
_______________________________________________
Mailing list: https://launchpad.net/~group.of.nepali.translators
Post to : group.of.nepali.translators@lists.launchpad.net
Unsubscribe : https://launchpad.net/~group.of.nepali.translators
More help : https://help.launchpad.net/ListHelp
