Public bug reported: Original bug report with detailed description was created for Xen Orchestra here: https://github.com/vatesfr/xen-orchestra/issues/4449
Brief description: On systems with util-linux-2.33-rc1 or younger (e.g. Debian 10 Buster), ds-identify fails to detect when a disk of a NoCloud datasource has a label written to the boot sector of the disk. Before util- linux-2.33-rc1, blkid showed "LABEL=cidata". With the change, blkid shows "LABEL_FATBOOT=cidata" (newly introduced / additional label). Longer description: I ran into this when using cloud-init together with Xen Orchestra v5.48 (Xen Orchestra is a management interface for xen; in my case xcp-ng v8.0.0). I created a VM template based on the recently released Debian 10.0 Buster, which uses util-linux 2.33.1. Upon boot, ds-identify fails to detect the NoCloud datasource / virtual disk which Xen Orchestra generated (the disk is created with this code from https://github.com/natevw/fatfs. With an older Debian 8 (util- linux-2.25.0) based template, ds-identify detects the NoCloud datasource disk fine. Likely explanation: Xen Orchestra creates the NoCloud as a partition-less disk with a FAT16 filesystem which has the NoCloud user-data and meta-data files. The label "cidata" is written into the boot sector of the virtual disk. With the same disk, oder versions of blkid report "LABEL=cidata" whereas newer versions detect "LABEL_FATBOOT=cidata". The ds-identify shell script checks only for the presence of the field called "LABEL" and not for "LABEL_FATBOOT". Relevant commit message from the util-linux-2.33-rc1 changelog (commit f0ca7e80d7a171701d0d04a3eae22d97f15d0683): libblkid: vfat: Change parsing label in special cases * Use only label from the root directory and do not fallback to the label stored in boot sector. This is how MS-DOS 6.22, MS-DOS 7.10, Windows 98, Windows XP and also Windows 10 behave. Moreover Windows XP and Windows 10 do not touch label in boot sector anymore, so removing FAT label on those Windowses leads to having old label still stored in boot sector (which MS-DOS and Windows fully ignore). * Label entry "NO NAME" in root directory is treated as label "NO NAME" instead of empty label. In root directory it has no special meaning. String "NO NAME" has a special meaning (empty label) only for label stored in boot sector. * Label from the boot sector is now stored into LABEL_FATBOOT field. So if there are applications which depends or needs to read this label, they have ability. * After this change LABEL always correspondent to the label from the root directory and LABEL_FATBOOT to the label stored in the boot sector. If some of those labels is missing or is not present (e.g. "NO LABEL" in boot sector) then particular field is not set. Possible fix: I did a trivial change of 2 lines to ds-identify to check for LABEL_FATBOOT after the check for LABEL. For me this solves the problem, as in: the cloud-init enabled VM boots up, ds-identify finds "LABEL_FATBOOT=cidata" and cloud-init correctly executes. In cases where both labels are written, the latter over-writes the former, which could be a theoretical problem if the values differ, but I am not sure how likely this case is. Further debug information as requested by @rharper on IRC: - cloud-init.tar.gz (Debian 10 / ds-identify fail) - Debian version: debian@cloudbuster:~$ lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 10 (buster) Release: 10 Codename: buster - util-linux version: debian@cloudbuster:~$ sudo blkid -V blkid from util-linux 2.33.1 (libblkid 2.33.1, 09-Jan-2019) - blkid output: debian@cloudbuster:~$ sudo blkid /dev/xvdb /dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT="cidata" UUID="355A-4FC2" TYPE="vfat" - udevadm outout: debian@cloudbuster:~$ udevadm info --query=all /sys/class/block/xvdb P: /devices/vbd-832/block/xvdb N: xvdb L: 0 S: disk/by-uuid/355A-4FC2 E: DEVPATH=/devices/vbd-832/block/xvdb E: DEVNAME=/dev/xvdb E: DEVTYPE=disk E: MAJOR=202 E: MINOR=16 E: SUBSYSTEM=block E: USEC_INITIALIZED=4239917 E: ID_FS_UUID=355A-4FC2 E: ID_FS_UUID_ENC=355A-4FC2 E: ID_FS_VERSION=FAT16 E: ID_FS_TYPE=vfat E: ID_FS_USAGE=filesystem E: DEVLINKS=/dev/disk/by-uuid/355A-4FC2 E: TAGS=:systemd: # Some experiments: - This is interesting - dosfslabel incorrectly reports the label, while blkid (above) clearly shows the field is empty / not set: debian@cloudbuster:~$ sudo dosfslabel /dev/xvdb cidata - Here I am first setting the label with dosfslabel to see what happens and then check blkid again: debian@cloudbuster:~$ sudo dosfslabel /dev/xvdb cidata fatlabel: warning - lowercase labels might not work properly with DOS or Windows debian@cloudbuster:~$ sudo blkid /dev/xvdb /dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT="cidata" LABEL="cidata" UUID="355A-4FC2" TYPE="vfat" # Now blkid reports both labels ** Affects: cloud-init Importance: Undecided Status: New ** Attachment added: "Debian 10 cloud-init collect-logs" https://bugs.launchpad.net/bugs/1841466/+attachment/5284763/+files/cloud-init.tar.gz ** Description changed: Original bug report with detailed description was created for Xen Orchestra here: https://github.com/vatesfr/xen-orchestra/issues/4449 - Brief description: On systems with util-linux-2.33-rc1 or younger (e.g. Debian 10 Buster), ds-identify fails to detect when a disk of a NoCloud datasource has a label written to the boot sector of the disk. Before util- linux-2.33-rc1, blkid showed "LABEL=cidata". With the change, blkid shows "LABEL_FATBOOT=cidata" (newly introduced / additional label). - Longer description: I ran into this when using cloud-init together with Xen Orchestra v5.48 (Xen Orchestra is a management interface for xen; in my case xcp-ng v8.0.0). I created a VM template based on the recently released Debian 10.0 Buster, which uses util-linux 2.33.1. Upon boot, ds-identify fails to detect the NoCloud datasource / virtual disk which Xen Orchestra generated (the disk is created with this code from https://github.com/natevw/fatfs. With an older Debian 8 (util- linux-2.25.0) based template, ds-identify detects the NoCloud datasource disk fine. - Likely explanation: Xen Orchestra creates the NoCloud as a partition-less disk with a FAT16 filesystem which has the NoCloud user-data and meta-data files. The label "cidata" is written into the boot sector of the virtual disk. With the same disk, oder versions of blkid report "LABEL=cidata" whereas newer versions detect "LABEL_FATBOOT=cidata". The ds-identify shell script checks only for the presence of the field called "LABEL" and not for "LABEL_FATBOOT". Relevant commit message from the util-linux-2.33-rc1 changelog (commit f0ca7e80d7a171701d0d04a3eae22d97f15d0683): libblkid: vfat: Change parsing label in special cases - - * Use only label from the root directory and do not fallback to the label - stored in boot sector. This is how MS-DOS 6.22, MS-DOS 7.10, Windows 98, - Windows XP and also Windows 10 behave. Moreover Windows XP and Windows 10 - do not touch label in boot sector anymore, so removing FAT label on those - Windowses leads to having old label still stored in boot sector (which - MS-DOS and Windows fully ignore). - - * Label entry "NO NAME" in root directory is treated as label "NO NAME" - instead of empty label. In root directory it has no special meaning. - String "NO NAME" has a special meaning (empty label) only for label - stored in boot sector. - - * Label from the boot sector is now stored into LABEL_FATBOOT field. So if - there are applications which depends or needs to read this label, they - have ability. - - After this change LABEL always correspondent to the label from the root - directory and LABEL_FATBOOT to the label stored in the boot sector. If some - of those labels is missing or is not present (e.g. "NO LABEL" in boot - sector) then particular field is not set. + * Use only label from the root directory and do not fallback to the + label stored in boot sector. This is how MS-DOS 6.22, MS-DOS 7.10, + Windows 98, Windows XP and also Windows 10 behave. Moreover Windows XP + and Windows 10 do not touch label in boot sector anymore, so removing + FAT label on those Windowses leads to having old label still stored in + boot sector (which MS-DOS and Windows fully ignore). + + * Label entry "NO NAME" in root directory is treated as label "NO NAME" + instead of empty label. In root directory it has no special meaning. + String "NO NAME" has a special meaning (empty label) only for label + stored in boot sector. + + * Label from the boot sector is now stored into LABEL_FATBOOT field. So + if there are applications which depends or needs to read this label, + they have ability. + + * After this change LABEL always correspondent to the label from the + root directory and LABEL_FATBOOT to the label stored in the boot sector. + If some of those labels is missing or is not present (e.g. "NO LABEL" in + boot sector) then particular field is not set. Possible fix: I did a trivial change of 2 lines to ds-identify to check for LABEL_FATBOOT after the check for LABEL. For me this solves the problem, as in: the cloud-init enabled VM boots up, ds-identify finds "LABEL_FATBOOT=cidata" and cloud-init correctly executes. In cases where both labels are written, the latter over-writes the former, which could be a theoretical problem if the values differ, but I am not sure how likely this case is. - Further debug information as requested by @rharper on IRC: - cloud-init.tar.gz (Debian 10 / ds-identify fail) - Debian version: debian@cloudbuster:~$ lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 10 (buster) Release: 10 Codename: buster - - util-linux version: debian@cloudbuster:~$ sudo blkid -V blkid from util-linux 2.33.1 (libblkid 2.33.1, 09-Jan-2019) - - blkid output: debian@cloudbuster:~$ sudo blkid /dev/xvdb /dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT="cidata" UUID="355A-4FC2" TYPE="vfat" - - udevadm outout: debian@cloudbuster:~$ udevadm info --query=all /sys/class/block/xvdb P: /devices/vbd-832/block/xvdb N: xvdb L: 0 S: disk/by-uuid/355A-4FC2 E: DEVPATH=/devices/vbd-832/block/xvdb E: DEVNAME=/dev/xvdb E: DEVTYPE=disk E: MAJOR=202 E: MINOR=16 E: SUBSYSTEM=block E: USEC_INITIALIZED=4239917 E: ID_FS_UUID=355A-4FC2 E: ID_FS_UUID_ENC=355A-4FC2 E: ID_FS_VERSION=FAT16 E: ID_FS_TYPE=vfat E: ID_FS_USAGE=filesystem E: DEVLINKS=/dev/disk/by-uuid/355A-4FC2 E: TAGS=:systemd: - # Some experiments: - This is interesting - dosfslabel incorrectly reports the label, while blkid (above) clearly shows the field is empty / not set: debian@cloudbuster:~$ sudo dosfslabel /dev/xvdb cidata - - - Here I am first setting the label with dosfslabel to see what happens and then check blkid again: + - Here I am first setting the label with dosfslabel to see what happens + and then check blkid again: debian@cloudbuster:~$ sudo dosfslabel /dev/xvdb cidata fatlabel: warning - lowercase labels might not work properly with DOS or Windows debian@cloudbuster:~$ sudo blkid /dev/xvdb /dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT="cidata" LABEL="cidata" UUID="355A-4FC2" TYPE="vfat" # Now blkid reports both labels -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1841466 Title: ds-identify fails to detect NoCloud datastore with LABEL_FATBOOT instead of LABEL (change introduced recently in util-linux-2.33-rc1) Status in cloud-init: New Bug description: Original bug report with detailed description was created for Xen Orchestra here: https://github.com/vatesfr/xen-orchestra/issues/4449 Brief description: On systems with util-linux-2.33-rc1 or younger (e.g. Debian 10 Buster), ds-identify fails to detect when a disk of a NoCloud datasource has a label written to the boot sector of the disk. Before util-linux-2.33-rc1, blkid showed "LABEL=cidata". With the change, blkid shows "LABEL_FATBOOT=cidata" (newly introduced / additional label). Longer description: I ran into this when using cloud-init together with Xen Orchestra v5.48 (Xen Orchestra is a management interface for xen; in my case xcp-ng v8.0.0). I created a VM template based on the recently released Debian 10.0 Buster, which uses util-linux 2.33.1. Upon boot, ds- identify fails to detect the NoCloud datasource / virtual disk which Xen Orchestra generated (the disk is created with this code from https://github.com/natevw/fatfs. With an older Debian 8 (util- linux-2.25.0) based template, ds-identify detects the NoCloud datasource disk fine. Likely explanation: Xen Orchestra creates the NoCloud as a partition-less disk with a FAT16 filesystem which has the NoCloud user-data and meta-data files. The label "cidata" is written into the boot sector of the virtual disk. With the same disk, oder versions of blkid report "LABEL=cidata" whereas newer versions detect "LABEL_FATBOOT=cidata". The ds-identify shell script checks only for the presence of the field called "LABEL" and not for "LABEL_FATBOOT". Relevant commit message from the util-linux-2.33-rc1 changelog (commit f0ca7e80d7a171701d0d04a3eae22d97f15d0683): libblkid: vfat: Change parsing label in special cases * Use only label from the root directory and do not fallback to the label stored in boot sector. This is how MS-DOS 6.22, MS-DOS 7.10, Windows 98, Windows XP and also Windows 10 behave. Moreover Windows XP and Windows 10 do not touch label in boot sector anymore, so removing FAT label on those Windowses leads to having old label still stored in boot sector (which MS-DOS and Windows fully ignore). * Label entry "NO NAME" in root directory is treated as label "NO NAME" instead of empty label. In root directory it has no special meaning. String "NO NAME" has a special meaning (empty label) only for label stored in boot sector. * Label from the boot sector is now stored into LABEL_FATBOOT field. So if there are applications which depends or needs to read this label, they have ability. * After this change LABEL always correspondent to the label from the root directory and LABEL_FATBOOT to the label stored in the boot sector. If some of those labels is missing or is not present (e.g. "NO LABEL" in boot sector) then particular field is not set. Possible fix: I did a trivial change of 2 lines to ds-identify to check for LABEL_FATBOOT after the check for LABEL. For me this solves the problem, as in: the cloud-init enabled VM boots up, ds-identify finds "LABEL_FATBOOT=cidata" and cloud-init correctly executes. In cases where both labels are written, the latter over-writes the former, which could be a theoretical problem if the values differ, but I am not sure how likely this case is. Further debug information as requested by @rharper on IRC: - cloud-init.tar.gz (Debian 10 / ds-identify fail) - Debian version: debian@cloudbuster:~$ lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 10 (buster) Release: 10 Codename: buster - util-linux version: debian@cloudbuster:~$ sudo blkid -V blkid from util-linux 2.33.1 (libblkid 2.33.1, 09-Jan-2019) - blkid output: debian@cloudbuster:~$ sudo blkid /dev/xvdb /dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT="cidata" UUID="355A-4FC2" TYPE="vfat" - udevadm outout: debian@cloudbuster:~$ udevadm info --query=all /sys/class/block/xvdb P: /devices/vbd-832/block/xvdb N: xvdb L: 0 S: disk/by-uuid/355A-4FC2 E: DEVPATH=/devices/vbd-832/block/xvdb E: DEVNAME=/dev/xvdb E: DEVTYPE=disk E: MAJOR=202 E: MINOR=16 E: SUBSYSTEM=block E: USEC_INITIALIZED=4239917 E: ID_FS_UUID=355A-4FC2 E: ID_FS_UUID_ENC=355A-4FC2 E: ID_FS_VERSION=FAT16 E: ID_FS_TYPE=vfat E: ID_FS_USAGE=filesystem E: DEVLINKS=/dev/disk/by-uuid/355A-4FC2 E: TAGS=:systemd: # Some experiments: - This is interesting - dosfslabel incorrectly reports the label, while blkid (above) clearly shows the field is empty / not set: debian@cloudbuster:~$ sudo dosfslabel /dev/xvdb cidata - Here I am first setting the label with dosfslabel to see what happens and then check blkid again: debian@cloudbuster:~$ sudo dosfslabel /dev/xvdb cidata fatlabel: warning - lowercase labels might not work properly with DOS or Windows debian@cloudbuster:~$ sudo blkid /dev/xvdb /dev/xvdb: SEC_TYPE="msdos" LABEL_FATBOOT="cidata" LABEL="cidata" UUID="355A-4FC2" TYPE="vfat" # Now blkid reports both labels To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1841466/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
