[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

This bug is believed to be fixed in cloud-init in 17.1. If this is still
a problem for you, please make a comment and set the state back to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Launchpad Bug Tracker]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-
0ubuntu1~16.04.1

---
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~16.04.1) xenial-proposed; 
urgency=medium

  * New upstream snapshot.
- net: fix reading and rendering addresses in cidr format.
  [Dimitri John Ledkov] (LP: #1689346, #1684349)
- disk_setup: udev settle before attempting partitioning or fs creation.
  (LP: #1692093)
- GCE: Update the attribute used to find instance SSH keys.
  [Daniel Watkins] (LP: #1693582)
- nplan: For bonds, allow dashed or underscore names of keys.
  [Dimitri John Ledkov] (LP: #1690480)
- tests: python2.6: fix unit tests usage of assertNone and format.
- tests: update docstring on test_configured_list_with_none
- fix tools/ds-identify to not write None twice.
- tox/build: do not package depend on style requirements.
- tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
- flake8: move the pinned version of flake8 up to 3.3.0
- tests: Apply workaround for snapd bug in test case. [Joshua Powers]
- RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration.  [Andreas Karis]
- disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
- function spelling & docstring update [Joshua Powers]
- tests: Fix unittest bug in ntp tests. [Joshua Powers]
- tox: move pylint target to 1.7.1
- Fix get_interfaces_by_mac for empty macs (LP: #1692028)
- DigitalOcean: remove routes except for the public interface.
  [Ben Howard] (LP: #1681531.)
- netplan: pass macaddress, when specified, for vlans
  [Dimitri John Ledkov] (LP: #1690388)
- doc: various improvements for the docs on cc_users_groups.
  [Felix Dreissig]
- cc_ntp: write template before installing and add service restart
  [Ryan Harper] (LP: #1645644)
- tests: fix cloudstack unit tests to avoid accessing
  /var/lib/NetworkManager [Lars Kellogg-Stedman]
- tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
- Actually skip warnings when .skip file is present.
  [Chris Brinker] (LP: #1691551)
- netplan: fix netplan render_network_state signature.
  [Dimitri John Ledkov] (LP: #1685944)
- Azure: fix reformatting of ephemeral disks on resize to large types.
  (LP: #1686514)
- make deb: Add devscripts dependency for make deb.
  Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
- openstack: fix log message copy/paste typo in _get_url_settings
  [Lars Kellogg-Stedman]
- unittests: fix unittests run on centos [Joshua Powers]
- Improve detection of snappy to include os-release and kernel cmdline.
  (LP: #1689944)
- Add address to config entry generated by _klibc_to_config_entry.
  [Julien Castets] (LP: #1691135)
- sysconfig: Raise ValueError when multiple default gateways are present.
  [Chad Smith] (LP: #1687485)
- FreeBSD: improvements and fixes for use on Azure
  [Hongjiang Zhang] (LP: #1636345)
- Add unit tests for ds-identify, fix Ec2 bug found.
- fs_setup: if cmd is specified, use shell interpretation.
  [Paul Meyer] (LP: #1687712)
- doc: document network configuration defaults policy and formats.
  [Ryan Harper]
- doc: Fix name of "uri" key in docs for "cc_apt_configure" module
  [Felix Dreissig]
- tests: Enable artful in integration tests [Joshua Powers]

 -- Scott Moser   Fri, 26 May 2017 15:58:48 -0400

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Launchpad Bug Tracker]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-
0ubuntu1~16.10.1

---
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~16.10.1) yakkety-proposed; 
urgency=medium

  * New upstream snapshot.
- net: fix reading and rendering addresses in cidr format.
  [Dimitri John Ledkov] (LP: #1689346, #1684349)
- disk_setup: udev settle before attempting partitioning or fs creation.
  (LP: #1692093)
- GCE: Update the attribute used to find instance SSH keys.
  [Daniel Watkins] (LP: #1693582)
- nplan: For bonds, allow dashed or underscore names of keys.
  [Dimitri John Ledkov] (LP: #1690480)
- tests: python2.6: fix unit tests usage of assertNone and format.
- tests: update docstring on test_configured_list_with_none
- fix tools/ds-identify to not write None twice.
- tox/build: do not package depend on style requirements.
- tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
- flake8: move the pinned version of flake8 up to 3.3.0
- tests: Apply workaround for snapd bug in test case. [Joshua Powers]
- RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration.  [Andreas Karis]
- disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
- function spelling & docstring update [Joshua Powers]
- tests: Fix unittest bug in ntp tests. [Joshua Powers]
- tox: move pylint target to 1.7.1
- Fix get_interfaces_by_mac for empty macs (LP: #1692028)
- DigitalOcean: remove routes except for the public interface.
  [Ben Howard] (LP: #1681531.)
- netplan: pass macaddress, when specified, for vlans
  [Dimitri John Ledkov] (LP: #1690388)
- doc: various improvements for the docs on cc_users_groups.
  [Felix Dreissig]
- cc_ntp: write template before installing and add service restart
  [Ryan Harper] (LP: #1645644)
- tests: fix cloudstack unit tests to avoid accessing
  /var/lib/NetworkManager [Lars Kellogg-Stedman]
- tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
- Actually skip warnings when .skip file is present.
  [Chris Brinker] (LP: #1691551)
- netplan: fix netplan render_network_state signature.
  [Dimitri John Ledkov] (LP: #1685944)
- Azure: fix reformatting of ephemeral disks on resize to large types.
  (LP: #1686514)
- make deb: Add devscripts dependency for make deb.
  Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
- openstack: fix log message copy/paste typo in _get_url_settings
  [Lars Kellogg-Stedman]
- unittests: fix unittests run on centos [Joshua Powers]
- Improve detection of snappy to include os-release and kernel cmdline.
  (LP: #1689944)
- Add address to config entry generated by _klibc_to_config_entry.
  [Julien Castets] (LP: #1691135)
- sysconfig: Raise ValueError when multiple default gateways are present.
  [Chad Smith] (LP: #1687485)
- FreeBSD: improvements and fixes for use on Azure
  [Hongjiang Zhang] (LP: #1636345)
- Add unit tests for ds-identify, fix Ec2 bug found.
- fs_setup: if cmd is specified, use shell interpretation.
  [Paul Meyer] (LP: #1687712)
- doc: document network configuration defaults policy and formats.
  [Ryan Harper]
- doc: Fix name of "uri" key in docs for "cc_apt_configure" module
  [Felix Dreissig]
- tests: Enable artful in integration tests [Joshua Powers]

 -- Scott Moser   Fri, 26 May 2017 16:08:21 -0400

** Changed in: cloud-init (Ubuntu Yakkety)
   Status: Fix Committed => Fix Released

** Changed in: cloud-init (Ubuntu Xenial)
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Launchpad Bug Tracker]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-
0ubuntu1~17.04.1

---
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~17.04.1) zesty-proposed; urgency=medium

  * New upstream snapshot.
- net: fix reading and rendering addresses in cidr format.
  [Dimitri John Ledkov] (LP: #1689346, #1684349)
- disk_setup: udev settle before attempting partitioning or fs creation.
  (LP: #1692093)
- GCE: Update the attribute used to find instance SSH keys.
  [Daniel Watkins] (LP: #1693582)
- nplan: For bonds, allow dashed or underscore names of keys.
  [Dimitri John Ledkov] (LP: #1690480)
- tests: python2.6: fix unit tests usage of assertNone and format.
- tests: update docstring on test_configured_list_with_none
- fix tools/ds-identify to not write None twice.
- tox/build: do not package depend on style requirements.
- tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
- flake8: move the pinned version of flake8 up to 3.3.0
- tests: Apply workaround for snapd bug in test case. [Joshua Powers]
- RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration.  [Andreas Karis]
- disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
- function spelling & docstring update [Joshua Powers]
- tests: Fix unittest bug in ntp tests. [Joshua Powers]
- tox: move pylint target to 1.7.1
- Fix get_interfaces_by_mac for empty macs (LP: #1692028)
- DigitalOcean: remove routes except for the public interface.
  [Ben Howard] (LP: #1681531.)
- netplan: pass macaddress, when specified, for vlans
  [Dimitri John Ledkov] (LP: #1690388)
- doc: various improvements for the docs on cc_users_groups.
  [Felix Dreissig]
- cc_ntp: write template before installing and add service restart
  [Ryan Harper] (LP: #1645644)
- tests: fix cloudstack unit tests to avoid accessing
  /var/lib/NetworkManager [Lars Kellogg-Stedman]
- tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
- Actually skip warnings when .skip file is present.
  [Chris Brinker] (LP: #1691551)
- netplan: fix netplan render_network_state signature.
  [Dimitri John Ledkov] (LP: #1685944)
- Azure: fix reformatting of ephemeral disks on resize to large types.
  (LP: #1686514)
- make deb: Add devscripts dependency for make deb.
  Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
- openstack: fix log message copy/paste typo in _get_url_settings
  [Lars Kellogg-Stedman]
- unittests: fix unittests run on centos [Joshua Powers]
- Improve detection of snappy to include os-release and kernel cmdline.
  (LP: #1689944)
- Add address to config entry generated by _klibc_to_config_entry.
  [Julien Castets] (LP: #1691135)
- sysconfig: Raise ValueError when multiple default gateways are present.
  [Chad Smith] (LP: #1687485)
- FreeBSD: improvements and fixes for use on Azure
  [Hongjiang Zhang] (LP: #1636345)
- Add unit tests for ds-identify, fix Ec2 bug found.
- fs_setup: if cmd is specified, use shell interpretation.
  [Paul Meyer] (LP: #1687712)
- doc: document network configuration defaults policy and formats.
  [Ryan Harper]
- doc: Fix name of "uri" key in docs for "cc_apt_configure" module
  [Felix Dreissig]
- tests: Enable artful in integration tests [Joshua Powers]

 -- Scott Moser   Fri, 26 May 2017 16:14:09 -0400

** Changed in: cloud-init (Ubuntu Zesty)
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Joshua Powers]

Because this resolves the original issues and the new issue is covered
by bug 1691489 I am marking this as verification-done-zesty. Thanks
Scott!

** Tags removed: verification-failed-zesty
** Tags added: verification-done-zesty

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

Josh,
The bug you see when it does not work properly is bug 1691489.
The issue as seen in your cloud-init.log is that when mkfs is run its check to 
see if the device is used fails.  That is because the device is in use by the 
fsck process that is being run on filesystem that is already present there. 
systemd decides to run fsck on /dev/sdb1 because there is an entry 
(comment=cloudconfig) from the previous run.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Joshua Powers]

Zesty Steps to reproduce
1. Launch an L32s instance with 17.04 (Zesty)
2. Verify /mnt is mounted (mount or lsblk)
3. Update to proposed
4. From Azure console, redeploy the instance
5. Login and look if /mnt is mounted. If it is mounted, GOTO 4

I have seen it work properly before, so a second redeploy got the issue
to appear.

cloud-init.log http://paste.ubuntu.com/24926325/
cloud-init-output.log http://paste.ubuntu.com/24926327/
journal http://paste.ubuntu.com/24926328/
syslog http://paste.ubuntu.com/24926329/

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Stephen A. Zarkos]

I tested Xenial as well and verified that the cloud-init package in
proposed resolves this bug.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Joshua Powers]

I am marking this verification-done-xenial and verification-done-yakkety
as those tests passed.

Zesty is being marked verification-failed-zesty due to the above issue.
I am working on reproducing and collecting additional details.

** Tags removed: verification-needed
** Tags added: verification-done-xenial verification-done-yakkety 
verification-failed-zesty

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Joshua Powers]

This failed verification on an Azure L32s Zesty instance.

Steps to reproduce:
1. Booted an L32s instance. 
2. Redeployed and verified that /mnt was not mounted. 
3. Upgraded to the version of cloud-init to proposed, deleted /var/lib/cloud/, 
and rebooted.
4. Upon reboot /mnt was not mounted as expected and the cloud-init logs below 
show tracebacks.

cloud-init.log: https://paste.ubuntu.com/24920403/
cloud-init-output.log https://paste.ubuntu.com/24920406/

This did, however, work with Yakkety and Xenial.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Brian Murray]

Hello Stephen, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and
be available at https://launchpad.net/ubuntu/+source/cloud-
init/0.7.9-153-g16a7302f-0ubuntu1~16.04.1 in a few hours, and then in
the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: cloud-init (Ubuntu Xenial)
   Status: Confirmed => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Brian Murray]

Hello Stephen, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now
and be available at https://launchpad.net/ubuntu/+source/cloud-
init/0.7.9-153-g16a7302f-0ubuntu1~16.10.1 in a few hours, and then in
the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: cloud-init (Ubuntu Yakkety)
   Status: Confirmed => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Brian Murray]

Hello Stephen, or anyone else affected,

Accepted cloud-init into zesty-proposed. The package will build now and
be available at https://launchpad.net/ubuntu/+source/cloud-
init/0.7.9-153-g16a7302f-0ubuntu1~17.04.1 in a few hours, and then in
the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: cloud-init (Ubuntu Zesty)
   Status: Confirmed => Fix Committed

** Tags added: verification-needed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

** Description changed:

- Some Azure instances such as L32 or G5 have very large ephemeral disks
- which are partitioned via GPT vs. smaller ephemeral disks that have dos
- disklabels.
+ === Begin SRU Template ===
+ [Impact]
+ On Azure, cloud-init handles re-formatting the ephemeral disk.
+ The contents of the ephemeral disk for a system will be replaced with
+ a stock ephemeral disk in the following scenarios:
+  a.) first boot
+  b.) after a resize.
+  c.) after a VM has been migrated from one host to another.
+ 
+ That ephemeral disk is either
+  1. mbr partitioned with 1 ntfs partition
+  2. gpt partitioned with 2 partitions, a msft reserved partition and a
+ ntfs partition.  This scenario is newer, and only occurs on
+ larger instance types that have large ephemeral disks.
+ 
+ cloud-init previously did not handle '2' above.
+ 
+ [Test Case]
+ Generically this is re-creatable by:
+  1.) launch an instance on Azure
+  2.) resize it to a L32 or G5 size
+  3.) check to see that the ephemeral disk (/dev/disk/cloud/azure_resource)
+  has been formatted to ext4.
+ 
+ It is more easily recreated for testing and verification by:
+ 
+ 1. launch instance on azure
+ 2. re-partition the ephemeral disk to look like a "clean" disk above
+ 3. remove old logs, reboot
+$ dir=logs-$(date +"%Y%m%d-%H%M%S");
+$ mkdir -p $dir; mv /var/log/cloud-init* $dir
+ 4. ssh back in, expect that this the disk has an ext4 filesystem on it.
+And that it is mounted on /mnt.
+ 
+$ grep reformattable= /var/log/cloud-init.log
+2017-05-12 15:14:57,125 - DataSourceAzure.py[DEBUG]: reformattable=False: 
partition 1 (/dev/sdb1) on device /dev/disk/cloud/azure_resource was not ntfs 
formatted
+ 
+Or, if it was formatted, you'll see something like:
+2017-05-12 15:17:47,021 - DataSourceAzure.py[DEBUG]: reformattable=True: 
partition 2 (/dev/sdb2) on device /dev/disk/cloud/azure_resource was ntfs 
formatted and had no important files. Safe for reformatting.
+ 
+$ grep /mnt /proc/mounts
+/dev/sdb1 /mnt ext4 rw,relatime,data=ordered 0 0
+ 
+ 
+ [Regression Potential]
+ The change makes cloud-init accept another situation when it decides
+ to be reformat a disk.  Reformatting of a disk could result in loss of
+ customer data if the decision to do so results in a false positive.
+ 
+ The fix came with some fairly extensive unit tests (TestCanDevBeReformatted)
+ on the 'can_dev_be_reformatted' method.
+ 
+ [Other Info]
+ Upstream commit at
+   https://git.launchpad.net/cloud-init/commit/?id=31b6f1732
+ 
+ === End SRU Template ===
+ 
+ 
+ Some Azure instances such as L32 or G5 have very large ephemeral disks which 
are partitioned via GPT vs. smaller ephemeral disks that have dos disklabels.
  
  At first boot of an instance the ephemeral disk is prepared and
  formatted properly. But if the instance is deallocated and then
  reallocated (thus receiving a new ephemeral disk) then cloud-init does
  not handle reformatting GPT partition ephemeral disks properly.
  Therefore /mnt is never mounted again.
  
  Test cases:
   1. Deploy an L32(s) VM on Azure
   2. Log in and ensure that the ephemeral disk is formatted and mounted to /mnt
   3. Via the portal you can "Redeploy" the VM to a new Azure Host (or 
alternatively stop and deallocate the VM for some time, and then 
restart/reallocate the VM).
  
  Expected Results:
   - After reallocation we expect the ephemeral disk to be formatted and 
mounted to /mnt.
  
  Actual Results:
   - After reallocation /mnt is not mounted and there are errors in the 
cloud-init log.
  
  *This was tested on Ubuntu 16.04 - but may affect other releases.
  
  Note: This bug a regression from previous cloud-init releases. GPT
  support for Azure ephemeral disk handling was added to cloud-init via
  this bug: https://bugs.launchpad.net/ubuntu/+source/cloud-
  init/+bug/1422919.
  
  Related bugs:
-  * bug 1691489: fstab entries written by cloud-config may not be mounted
+  * bug 1691489: fstab entries written by cloud-config may not be mounted

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

Bug 1692087
I expect that this should be functional for you now. There are still some 
issues left that Paul has filed in bug 1692093.
I'll propose a merge for that tomorrow.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Stephen A. Zarkos]

Hi Scott,

Do you have a new bug number for the issue in comment #13?

Thanks,
Steve

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Launchpad Bug Tracker]

This bug was fixed in the package cloud-init -
0.7.9-144-g2825a917-0ubuntu1

---
cloud-init (0.7.9-144-g2825a917-0ubuntu1) artful; urgency=medium

  * New upstream snapshot.
- flake8: move the pinned version of flake8 up to 3.3.0
- tests: Apply workaround for snapd bug in test case. [Joshua Powers]
- RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration. [Andreas Karis]
  (LP: #1679817, #1685534, #1685532)
- disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
- function spelling & docstring update [Joshua Powers]
- Fixing wrong file name regression. [Joshua Powers]
- tox: move pylint target to 1.7.1
- Fix get_interfaces_by_mac for empty macs (LP: #1692028)
- DigitalOcean: remove routes except for the public interface.
  [Ben Howard] (LP: #1681531.)
- netplan: pass macaddress, when specified, for vlans
  [Dimitri John Ledkov] (LP: #1690388)
- doc: various improvements for the docs on cc_users_groups.
  [Felix Dreissig]
- cc_ntp: write template before installing and add service restart
  [Ryan Harper] (LP: #1645644)
- cloudstack: fix tests to avoid accessing /var/lib/NetworkManager
  [Lars Kellogg-Stedman]
- tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
- Actually skip warnings when .skip file is present. [Chris Brinker]
  (LP: #1691551)
- netplan: fix netplan render_network_state signature.
  [Dimitri John Ledkov] (LP: #1685944)
- Azure: fix reformatting of ephemeral disks on resize to large types.
  (LP: #1686514)
- Revert "tools/net-convert: fix argument order for render_network_state"
- make deb: Add devscripts dependency for make deb. Cleanup packages/bddeb.
  [Chad Smith] (LP: #1685935)
- tools/net-convert: fix argument order for render_network_state
  [Ryan Harper] (LP: #1685944)
- openstack: fix log message copy/paste typo in _get_url_settings
  [Lars Kellogg-Stedman]
- unittests: fix unittests run on centos [Joshua Powers]
- Improve detection of snappy to include os-release and kernel cmdline.
  (LP: #1689944)
- Add address to config entry generated by _klibc_to_config_entry.
  [Julien Castets] (LP: #1691135)
- sysconfig: Raise ValueError when multiple default gateways are present.
  [Chad Smith] (LP: #1687485)
- FreeBSD: improvements and fixes for use on Azure
  [Hongjiang Zhang] (LP: #1636345)
- Add unit tests for ds-identify, fix Ec2 bug found.
- fs_setup: if cmd is specified, use shell interpretation.
  [Paul Meyer] (LP: #1687712)
- doc: document network configuration defaults policy and formats.
  [Ryan Harper]
- Fix name of "uri" key in docs for "cc_apt_configure" module
  [Felix Dreissig]
- tests: Enable artful [Joshua Powers]

 -- Scott Moser   Tue, 23 May 2017 17:04:40 -0400

** Changed in: cloud-init (Ubuntu Artful)
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

Well, now I'm not so sure. I can reproduce this I think.
I'll open a separate bug tomorrow.  The issue I think now is that 
'check_partition_layout' is deciding that it does not need to repartition.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

Stephen,
I think the issue in your comment above is that cloud-init on the first boot 
wrote an entry into /etc/fstab for the swap partition.  Then you upgraded 
rebooted.  The /etc/fstab entry for swap was still present and systemd mounted 
swap. Then, when cloud-init went to repartition, the disk was busy (with swap) 
and could not get it done.

Could you attach *that* /var/log/cloud-init.log ?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Stephen A. Zarkos]

Hi Scott,

As an additional test case I've tried to use the steps as described here
to re-partition the ephemeral disk:
https://wiki.ubuntu.com/AzureSwapPartitions.

Using your branch from this bug the sult is that it appears to fail to
repartition but it does format/mount the disks. Here are my repro steps

 $ azure group create ubtest 'east us'

 $ azure vm create ubtest ubtest-1604 Linux --image-urn
canonical:UbuntuServer:16.04-LTS:latest --admin-username azureuser
--admin-password '' --nic-name ubtest-1604 --location "East
US" --vnet-name "ubtest" --vnet-address-prefix "10.0.0.0/24" --vnet-
subnet-name "ubtest" --vnet-subnet-address-prefix "10.0.0.0/24"
--public-ip-name "ubtest" --public-ip-domain-name "ubtest" --custom-data
./customdata.txt -z 'Standard_L32s'

The customdata.txt file contains something like the following:

 #cloud-config
 disk_setup:
 ephemeral0:
 table_type: gpt
 layout: [66, [33, 82]]
 overwrite: True
 fs_setup:
 - device: ephemeral0.1
   filesystem: ext4
 - device: ephemeral0.2
   filesystem: swap
 mounts:
 - ["ephemeral0.1", "/mnt"]
 - ["ephemeral0.2", "none", "swap", "sw", "0", "0"]

.
.
Result with default 16.04 image: /mnt is not mounted and we see a failure in 
the cloud-init.log (attached).

Results with branch from this bug: /mnt and swap space are created, but
the partition layout is not applied. So part1 is still type "Microsoft
reserved" and only 120MB, and part2 is type "Microsoft basic data" and
is a 5.5TB swap partition :)

Maybe there's something wrong with my cloud-config?  Do I need to use a
different layout format when table_type is 'gpt'?


** Attachment added: "cloud-init.log"
   
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1686514/+attachment/4878669/+files/cloud-init.log

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

** Also affects: cloud-init (Ubuntu Xenial)
   Importance: Undecided
   Status: New

** Also affects: cloud-init (Ubuntu Yakkety)
   Importance: Undecided
   Status: New

** Also affects: cloud-init (Ubuntu Artful)
   Importance: High
 Assignee: Scott Moser (smoser)
   Status: In Progress

** Also affects: cloud-init (Ubuntu Zesty)
   Importance: Undecided
   Status: New

** Changed in: cloud-init (Ubuntu Xenial)
   Status: New => Confirmed

** Changed in: cloud-init (Ubuntu Yakkety)
   Status: New => Confirmed

** Changed in: cloud-init (Ubuntu Zesty)
   Status: New => Confirmed

** Changed in: cloud-init (Ubuntu Xenial)
   Importance: Undecided => Medium

** Changed in: cloud-init (Ubuntu Yakkety)
   Importance: Undecided => Medium

** Changed in: cloud-init (Ubuntu Zesty)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

** Changed in: cloud-init
   Status: In Progress => Fix Committed

** Changed in: cloud-init
 Assignee: (unassigned) => Scott Moser (smoser)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

I've opened up bug 1691489 to address the mount/fsck race, and have been
able to reproduce it and it looks like the fix i had in mind will work
(more info there).

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

** Description changed:

  Some Azure instances such as L32 or G5 have very large ephemeral disks
  which are partitioned via GPT vs. smaller ephemeral disks that have dos
  disklabels.
  
  At first boot of an instance the ephemeral disk is prepared and
  formatted properly. But if the instance is deallocated and then
  reallocated (thus receiving a new ephemeral disk) then cloud-init does
  not handle reformatting GPT partition ephemeral disks properly.
  Therefore /mnt is never mounted again.
  
  Test cases:
   1. Deploy an L32(s) VM on Azure
   2. Log in and ensure that the ephemeral disk is formatted and mounted to /mnt
   3. Via the portal you can "Redeploy" the VM to a new Azure Host (or 
alternatively stop and deallocate the VM for some time, and then 
restart/reallocate the VM).
  
  Expected Results:
   - After reallocation we expect the ephemeral disk to be formatted and 
mounted to /mnt.
  
  Actual Results:
   - After reallocation /mnt is not mounted and there are errors in the 
cloud-init log.
  
  *This was tested on Ubuntu 16.04 - but may affect other releases.
  
  Note: This bug a regression from previous cloud-init releases. GPT
  support for Azure ephemeral disk handling was added to cloud-init via
  this bug: https://bugs.launchpad.net/ubuntu/+source/cloud-
  init/+bug/1422919.
+ 
+ Related bugs:
+  * bug 1691489: fstab entries written by cloud-config may not be mounted

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

Stephen,
I think this could be an issue that I saw as a possible issue a while ago, but 
never reproduced.
 
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/bug/before-fsck
has some details.

For what is normally responsible for mounting the ephemeral disk... its
supposed to be just mounted by systemd's normal mounts.  cc_mounts.py
writes or cleans the entries in /etc/fstab, and then does a 'systemctl
daemon-reload' with the intent of having systemd recognize realize the
new entries and then handle everything.

I think we should probably treat that bug as separate than this one as
it could happen anywhere independent of the disk format.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Stephen A. Zarkos]

Hi Scott,

The linked branch seems to reliably format the ephemeral disk on large
instances. However, after redeploying the VM I don't see the ephemeral
disk mounted after boot. The cloud-init log does not indicate any error,
and if I type 'mount -a' (or do a simple reboot) then the disk is
mounted properly with an ext4 filesystem.

To test this I'm creating an L32s VM on Azure with Ubuntu 16.04,
updating cloud-init manually from your git branch, and then I click the
'redeploy' option via the portal.  I see that the ephemeral disk is
partitioned/formatted correctly, but just not mounted.  Am I missing
some configuration?  What is normally responsible for mounting the
ephemeral disk once it is formatted?

Steve

** Attachment added: "cloud-init.log"
   
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1686514/+attachment/4877814/+files/cloud-init.log

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

Stephen,
I believe the linked branch has a suitable fix for this issue now.

In response to my comment #6, I've realized that the 'replace_fs' in
fs_setup configuration was always ignored, because if a partition is
provided (in the "dotted" format there), then replace_fs is not used.

I've tested the general path on azure by re-formatting/partitioning the
disk into something that looks like a newly partitioned disk and then
rebooting.

** Changed in: cloud-init
   Status: Confirmed => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

I've linked an in-progress branch 
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/323420
more work to do, but I think i have acceptable fixes there for the 
address_ephemeral_resize().

I think we still will need to modify somehow the builtin 'fs_setup' so
that the disk_setup module also realizes that it can overwrite the ntfs
partitions.

it currently shows:
'fs_setup': [{'filesystem': 'ext4',
  'device': 'ephemeral0.1',
  'replace_fs': 'ntfs'}],


** Merge proposal linked:
   https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/323420

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

Hi,
Steven, Thanks for the good analysis.
I think we have a good understanding of the problem.

I'm working on getting a fix for the issue and also the TypeError that
you pointed out.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

** Changed in: cloud-init (Ubuntu)
   Importance: Medium => High

** Changed in: cloud-init (Ubuntu)
   Status: Confirmed => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Scott Moser]

** Changed in: cloud-init (Ubuntu)
   Status: New => Confirmed

** Changed in: cloud-init (Ubuntu)
   Importance: Undecided => Medium

** Also affects: cloud-init
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Status: New => Confirmed

** Changed in: cloud-init
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[David Britton]

** Changed in: cloud-init (Ubuntu)
 Assignee: (unassigned) => Scott Moser (smoser)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[saim shrestha]

the issue has been reproduced for Ubuntu 14.04LTS images in azure L32S
instance.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Stephen A. Zarkos]

GPT formatted ephemeral disks have two partitions (unfortunately). The
first is the Microsoft reserved partition
(https://en.wikipedia.org/wiki/Microsoft_Reserved_Partition). This
appears to be tripping up can_dev_be_reformatted().

Function address_ephemeral_resize() will first check if
is_new_instance=True, and if so it will happily format the ephemeral
disk.  That's why this works at first boot.

For subsequent boots where is_new_instance=False,
address_ephemeral_resize() will call can_dev_be_reformatted(). This
function sees that the ephemeral disk has two partitions and returns
false:

if os.path.exists(devpath + suff + "2"):
msg = ('device %s had more than 1 partition: %s, %s' %
   devpath, cand, devpath + suff + "2")
return False, msg

We also end up with the following TypeError in the logs:

 Traceback (most recent call last):
   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 647, in 
status_wrapper
 ret = functor(name, args)
   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 415, in 
main_init
 init.activate_datasource()
   File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 369, in 
activate_datasource
 is_new_instance=self.is_new_instance())
   File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", 
line 251, in activate
 address_ephemeral_resize(is_new_instance=is_new_instance)
   File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", 
line 329, in address_ephemeral_resize
 result, msg = can_dev_be_reformatted(devpath)
   File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", 
line 279, in can_dev_be_reformatted
 devpath, cand, devpath + suff + "2")
 TypeError: not enough arguments for format string
.
.
Aside from the TypeError, even if we comment this part there is a later check 
to ensure that this partition is NTFS formatted (which it isn't). This will 
also return false and you will see the following error in the logs:

 "reformattable=False: partition 1 (/dev/disk/cloud/azure_resource-part1
-> /dev/sdb1) on device /dev/disk/cloud/azure_resource was not ntfs
formatted"

I think the correct behavior should be for cloud-init to detect that
this is a GPT formatted ephemeral disk, or check that azure_resource-
part1 is a "Microsoft reserved partition".  And if so skip
azure_resource-part1 and focus on azure_resource-part2 instead to
determine if it is safe to reformat.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Stephen A. Zarkos]

Adding cloud-init logs.  The logs show a number of boots (sorry), but
you should just be able to look at the first two boots to see the issue.

** Attachment added: "cloud-init-output.log"
   
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1686514/+attachment/4868360/+files/cloud-init-output.log

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1686514] Re: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Stephen A. Zarkos]

** Attachment added: "cloud-init.log"
   
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1686514/+attachment/4868359/+files/cloud-init.log

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1686514

Title:
  Azure: cloud-init does not handle reformatting GPT partition ephemeral
  disks

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1686514/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs