[Bug 1410835] Re: Azure ephemeral disk and custom mount points
Hi trialotto, cloud-init has default settings for Azure instances that ensure that the first partition of the ephemeral disk is formatted as ext4. It does this by providing default values for disk_setup and fs_setup; you can see these at http://bazaar.launchpad.net/~cloud-init-dev/cloud- init/trunk/view/head:/cloudinit/sources/DataSourceAzure.py#L54. cloud-init, by default, mounts the first partition of the first ephemeral disk at /mnt. You can see the code that does this at http://bazaar.launchpad.net/~cloud-init-dev/cloud- init/trunk/view/head:/cloudinit/config/cc_mounts.py#L236. If you are changing the layout of the disk, you will also need to provide mount configuration (using the `mounts` configuration key) so that cloud-init knows what it should mount where. We could, potentially, look at making cloud-init a little smarter about automatically working out mounts, but this would require a fair bit of testing across different platforms. Please let me know if this addresses your issues. Thanks, Dan -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1410835 Title: Azure ephemeral disk and custom mount points To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1410835/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1410835] Re: Azure ephemeral disk and custom mount points
Steve, I have just filed https://bugs.launchpad.net/ubuntu/+source/cloud- init/+bug/1411582 regarding the inconstancy of device naming; could you confirm that I've captured the issue accurately? Thanks, Dan -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1410835 Title: Azure ephemeral disk and custom mount points To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1410835/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1410835] Re: Azure ephemeral disk and custom mount points
trialotto, I've just been doing some testing, and I've realised my above answer is incorrect; this is definitely a problem in how cloud-init's Azure data source works. I'm looking in to it now. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1410835 Title: Azure ephemeral disk and custom mount points To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1410835/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1410835] Re: Azure ephemeral disk and custom mount points
Actually, I've realised that I was passing in invalid cloud configuration data. If I have the following in azure-disk.cfg: #cloud-config disk_setup: ephemeral0: table_type: mbr layout: [66, 33] overwrite: True fs_setup: - device: ephemeral0.1 filesystem: ext4 - device: ephemeral0.2 filesystem: ext4 mounts: - [ephemeral0.1, /mnt, auto, defaults,nobootwait, 0, 0] - [ephemeral0.2, /media, auto, defaults,nobootwait, 0, 0] and I run: $ azure vm create ... --custom-data azure-disk.cfg then I see the following: $ ssh ubu...@oddbloke-disk-150116-1555.cloudapp.net df -h Filesystem Size Used Avail Use% Mounted on ... /dev/sdb1 393G 71M 373G 1% /mnt /dev/sdb2 203G 60M 192G 1% /media I think there _is_ a problem with specifying swap partitions (they don't seem to get `mkswap` run on them before a mount is attempted). That will be handled in bug 1374166, so I'm going to mark this as Invalid and we can track the swap issues over there. ** Changed in: cloud-init (Ubuntu) Status: New = Invalid -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1410835 Title: Azure ephemeral disk and custom mount points To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1410835/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1410835] Re: Azure ephemeral disk and custom mount points
Daniel,
I was aware of DataSourceAzure.py and cc_mounts.py and the fact that
built-in config for Azure requires declaration of sdb (and not
ephemeral0). The before mentioned fact is only an inconvenience for
users that would like to declare custom mount points and make the
mistake to use an in incorrect reference to the ephemeral disk (i.e. not
declaring sdb). The misdeclaration will only result in the ephemeral
disk not being mounted, that should not be a real issue.
However, the above reveals a (minor) bug in cc_mounts.py.
The cfg mnt array in the cc_mounts.py script is sanitized, in such a
fashion that default mounts are only added if other entries do not have
the same device name (see lines 285 - 307).
A declaration of [ sdb, mount point ] or [ sdbX, mount point ] (in
case of partitions) in custom cloud config settings will not result in
an append of the ephemeral0 default mount (since the device name remains
the same) and that is good.
After all, when declaring [ sdb, mount point ] or [ sdbX, mount
point ] (in case of partitions), cc_mounts.py will append device
/dev/sdb or /dev/sdbx (in case of partitions) to the cfg mnt arry,
instead of the default mount declarations for ephemeral0.
This is really cumbersome, when using partitions and certainly when
using swap (dedicated) partitions.
The bug is (in essence) that the sanitizing in cc_mounts.py does not
prevent that the standard alias /dev/sdb, as specificied in
DataSourceAzure.py, or any other declared aliases will be appended to
the cfg mnt array, which should not occur, if these aliases are pointing
to one and the same ephemeral disk.
The (minor) bug can be easily resolved by sanitizing for aliases in
cc_mounts.py.
I was (also) aware that cc_mounts.py contains the declaration [swap, none,
swap, sw, 0, 0] in the default set of mounts and that
DataSourceAzure.py does not have to contain any reference to swap or mount
points thereof.
However, the accidental declaration of [ swap, mount point or None
] reveals a bug, that occurs even when a declaration of the form [ sdb,
mount point ] is provided in cloud-config settings.
The bug is, in my humble opinion, (solely) related to the following
lines of code in cc_mounts.py:
for line in actlist:
# write 'comment' in the fs_mntops, entry, claiming this
line[3] = %s,%s % (line[3], comment)
if line[2] == swap:
needswap = True
if line[1].startswith(/):
dirs.append(line[1])
cc_lines.append('\t'.join(line))
with these lines of code stating that any device called swap (or
/dev/swap, since that is the same in cloud-init)
a) will mount, as a result of
if line[1].startswith(/):
dirs.append(line[1])
if only if
- the declaration is of form [ swap, mount point ] AND
- the device called swap actually exists, for instance as a result of the
existence of swap partitions AND
- the swap partition is declared first in the disk_setup cloud-config settings,
b) will not mount, as a result of (taking into account defvals = [None,
None, auto, defaults,nobootwait, 0, 2])
if line[2] == swap:
needswap = True
if the declaration is of form [ swap, None ], with the missing
fields hence being replaced by appropriate values from defvals, hence
implying that line[2] == swap never applies,
c) will not mount properly, if the declaration is of form [ swap,
mount point or None, swap ],
d) will not mount at all, if the declaration is of form [swap, none,
swap, sw, 0, 0], due to sanitizing in cc_mounts.py,
and one could therefore come to the obvious conclusion that the bug is
related to the definition of defvals.
However, the obvious conclusion is not appropriate in this case.
In essence, one should not be able to declare [ swap, mount point or
None ] (or similar) at all (!).
Furthermore, a declaration for swap only makes sense when using swap
partitions, but swap partitions are not desirable at all.
Moreover, the cloud-init versions 0.7.6 and 0.7.7 already allow for the
creation of swap files.
In short, the bug can and should be resolved by sanitizing (i.e.
ignoring) any declarations of the form [ swap, mount point ] and [
swap, None ] or similar, hence also implying that even the [ swap ]
declaration has no function in case of sanitizing.
In my humble opinion, the swap bug can and should be resolved quickly.
Kind regards.
--
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to cloud-init in Ubuntu.
https://bugs.launchpad.net/bugs/1410835
Title:
Azure ephemeral disk and custom mount points
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1410835/+subscriptions
--
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1410835] Re: Azure ephemeral disk and custom mount points
@Daniel, I just read your latest (two) posts, I was not able to read them while commenting, my apologies. The makeswap bug (when using partitions) is very different from the swap bug and the alias bug. In short: - the swap bug can be resolved by sanitizing declarations of form [ swap, x, x, ... ] - the alias bug can be resolved by adding some code to check for double entries in the cfg mgt array, due to aliases Kind regards. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1410835 Title: Azure ephemeral disk and custom mount points To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1410835/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1410835] Re: Azure ephemeral disk and custom mount points
@Stephen, The non-persistence of device names is indeed a bug, even though device names can be declared by cloud-config settings. With respect to the Azure Linux Agent, it should be noted that one has the option to use a standard declaration of device names (and partitions, if desired), instead of taking the autodetection approach. The above option has the advantage that it allows for and/or can allow for a - (persistent) fixation of device names, b - (persistent) configuration of device names (and even partitions, if desired) in waagent.conf, c - removal of multiple lines of code in the waagent script, and it must be emphasized that the above implies that, in essence, the approach of using CustomData for injecting cloud-config settings becomes less relevant or even obsolete, if and only if the waagent script allows for using waagent.conf variables in the cloud-init sequence. In short, there are some possibilities for the improvement of the waagent script and the proper functioning thereof. Kind regards -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1410835 Title: Azure ephemeral disk and custom mount points To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1410835/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1410835] Re: Azure ephemeral disk and custom mount points
+1 :) The [ sdb, mount point ] syntax is a bug as well. Device names are not persistent in Linux and could change, so it is not guaranteed that the ephemeral disk will be called /dev/sdb. Ideally this should be auto-detected in cloud-init at runtime (for example, see DeviceForIdePort() in the Azure Linux agent). -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1410835 Title: Azure ephemeral disk and custom mount points To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1410835/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
