Public bug reported:
Hi there,
I'm trying to set cloudinit config to mount EBS with data to EC2
instance. Unfortunately, I'm not able to achieve that, maybe docs is
unclear or I do something wrong, but in each try (I try a lot of
possibilities of config) EBS is formatted and I'm loosing the data put
on it. Could you please provide the desired config for this setup or
check it this is a bug?
To summarize: mounting is working pretty well, but I'm not able to mount EBS
without formatting before mounting.
Current version of cloudinit config:
Cloudinit config:
#cloud-config
---
disk_setup:
/dev/xvdf:
layout: true
overwrite: false
table_type: mbr
fs_setup:
- device: /dev/xvdf1
filesystem: ext4
label: munin
overwrite: false
mounts:
- - /dev/xvdf1
- /var/lib/munin
Logs handled during mounting new and clear EBS at the first time:
Creating EBS:
Nov 21 10:46:17 ip-10-0-30-30 [CLOUDINIT] url_helper.py[DEBUG]: [0/1] open
'http://169.254.169.254/2009-04-04/meta-data/instance-id' with {'timeout':
50.0, 'method': 'GET', 'headers': {'User-Agent': 'Cloud-Init/0.7.7'}, 'url':
'http://169.254.169.254/2009-04-04/meta-data/instance-id', 'allow_redirects':
True} configuration
...skipping...
Nov 21 10:46:23 ip-10-0-30-30 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Using
cmd: /sbin/mkfs.ext4 /dev/xvdf1 -L munin
Nov 21 10:46:23 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/mkfs.ext4', '/dev/xvdf1', '-L', 'munin'] with allowed return codes [0]
(shell=False, capture=True)
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Creating fs for
/dev/xvdf1 took 11.388 seconds
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] handlers.py[DEBUG]: finish:
modules-config/config-disk_setup: SUCCESS: config-disk_setup ran successfully
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] stages.py[DEBUG]: Running module
mounts (<module 'cloudinit.config.cc_mounts' from
'/usr/lib/python3/dist-packages/cloudinit/config/cc_mounts.py'>) with frequency
once-per-instance
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] handlers.py[DEBUG]: start:
modules-config/config-mounts: running config-mounts with frequency
once-per-instance
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Writing to
/var/lib/cloud/instances/i-01123e824b16160fe/sem/config_mounts - wb: [420] 23
bytes
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] helpers.py[DEBUG]: Running
config-mounts using lock (<FileLock using file
'/var/lib/cloud/instances/i-01123e824b16160fe/sem/config_mounts'>)
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Attempting to
determine the real name of /dev/xvdf1
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Ignorming
nonexistant named mount /dev/xvdf1
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Attempting to
determine the real name of ephemeral0
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] DataSourceEc2.py[DEBUG]: Unable to
convert ephemeral0 to a device
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Ignoring
nonexistant default named mount ephemeral0
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Attempting to
determine the real name of swap
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] DataSourceEc2.py[DEBUG]: Unable to
convert swap to a device
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Ignoring
nonexistant default named mount swap
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: no need to setup
swap
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Reading from
/etc/fstab (quiet=False)
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Read 62 bytes from
/etc/fstab
Trying to mount EBS on the recreated instance:
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] stages.py[DEBUG]: Running module
disk_setup (<module 'cloudinit.config.cc_dis
k_setup' from
'/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py'>) with
frequency once-per-instance
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] handlers.py[DEBUG]: start:
modules-config/config-disk_setup: running config-d
isk_setup with frequency once-per-instance
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Writing to
/var/lib/cloud/instances/i-04ebf02c7eb73ecc6/sem/c
onfig_disk_setup - wb: [420] 24 bytes
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] helpers.py[DEBUG]: Running
config-disk_setup using lock (<FileLock using file
'/var/lib/cloud/instances/i-04ebf02c7eb73ecc6/sem/config_disk_setup'>)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] DataSourceEc2.py[DEBUG]: Unable to
convert /dev/xvdf to a device
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Partitioning
disks: {'/dev/xvdf': {'overwrite': Fals
e, 'table_type': 'mbr', 'layout': True}}
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating new
partition table/disk
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking
values for /dev/xvdf definition
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking
against default devices
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking if
device /dev/xvdf is a valid device
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/bin/lsblk', '--pairs', '--output', 'NAME,T
YPE,FSTYPE,LABEL', '/dev/xvdf', '--nodeps'] with allowed return codes [0]
(shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking if
device layout matches
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/udevadm', 'settle'] with allowed retu
rn codes [0] (shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/blockdev', '--rereadpt', '/dev/xvdf']
with allowed return codes [0] (shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/udevadm', 'settle'] with allowed retu
rn codes [0] (shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/sfdisk', '-l', '/dev/xvdf'] with allo
wed return codes [0] (shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Device
partitioning layout matches
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Creating partition on
/dev/xvdf took 0.238 seconds
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: setting up
filesystems: [{'overwrite': False, 'device': '/dev/xvdf1', 'filesystem':
'ext4', 'label': 'munin'}]
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] DataSourceEc2.py[DEBUG]: Unable to
convert /dev/xvdf1 to a device
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating new
filesystem.
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking
/dev/xvdf1 against default devices
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Using the
raw device to place filesystem munin on
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: File system
munin will be created on /dev/xvdf1
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating
file system munin on /dev/xvdf1
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Using
cmd: /sbin/mkfs.ext4 /dev/xvdf1 -L munin
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/mkfs.ext4', '/dev/xvdf1', '-L', 'munin'] with allowed return codes [0]
(shell=False, capture=True)
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Creating fs for
/dev/xvdf1 took 4.744 seconds
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] handlers.py[DEBUG]: finish:
modules-config/config-disk_setup: SUCCESS: config-disk_setup ran successfully
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] stages.py[DEBUG]: Running module
mounts (<module 'cloudinit.config.cc_mounts' from
'/usr/lib/python3/dist-packages/cloudinit/config/cc_mounts.py'>) with frequency
once-per-instance
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] handlers.py[DEBUG]: start:
modules-config/config-mounts: running config-mounts with frequency
once-per-instance
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Writing to
/var/lib/cloud/instances/i-04ebf02c7eb73ecc6/sem/config_mounts - wb: [420
** Affects: cloud-init
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1643531
Title:
Lost data with disk_setup on EBS
Status in cloud-init:
New
Bug description:
Hi there,
I'm trying to set cloudinit config to mount EBS with data to EC2
instance. Unfortunately, I'm not able to achieve that, maybe docs is
unclear or I do something wrong, but in each try (I try a lot of
possibilities of config) EBS is formatted and I'm loosing the data put
on it. Could you please provide the desired config for this setup or
check it this is a bug?
To summarize: mounting is working pretty well, but I'm not able to mount EBS
without formatting before mounting.
Current version of cloudinit config:
Cloudinit config:
#cloud-config
---
disk_setup:
/dev/xvdf:
layout: true
overwrite: false
table_type: mbr
fs_setup:
- device: /dev/xvdf1
filesystem: ext4
label: munin
overwrite: false
mounts:
- - /dev/xvdf1
- /var/lib/munin
Logs handled during mounting new and clear EBS at the first time:
Creating EBS:
Nov 21 10:46:17 ip-10-0-30-30 [CLOUDINIT] url_helper.py[DEBUG]: [0/1] open
'http://169.254.169.254/2009-04-04/meta-data/instance-id' with {'timeout':
50.0, 'method': 'GET', 'headers': {'User-Agent': 'Cloud-Init/0.7.7'}, 'url':
'http://169.254.169.254/2009-04-04/meta-data/instance-id', 'allow_redirects':
True} configuration
...skipping...
Nov 21 10:46:23 ip-10-0-30-30 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Using
cmd: /sbin/mkfs.ext4 /dev/xvdf1 -L munin
Nov 21 10:46:23 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/mkfs.ext4', '/dev/xvdf1', '-L', 'munin'] with allowed return codes [0]
(shell=False, capture=True)
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Creating fs for
/dev/xvdf1 took 11.388 seconds
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] handlers.py[DEBUG]: finish:
modules-config/config-disk_setup: SUCCESS: config-disk_setup ran successfully
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] stages.py[DEBUG]: Running module
mounts (<module 'cloudinit.config.cc_mounts' from
'/usr/lib/python3/dist-packages/cloudinit/config/cc_mounts.py'>) with frequency
once-per-instance
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] handlers.py[DEBUG]: start:
modules-config/config-mounts: running config-mounts with frequency
once-per-instance
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Writing to
/var/lib/cloud/instances/i-01123e824b16160fe/sem/config_mounts - wb: [420] 23
bytes
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] helpers.py[DEBUG]: Running
config-mounts using lock (<FileLock using file
'/var/lib/cloud/instances/i-01123e824b16160fe/sem/config_mounts'>)
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Attempting to
determine the real name of /dev/xvdf1
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Ignorming
nonexistant named mount /dev/xvdf1
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Attempting to
determine the real name of ephemeral0
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] DataSourceEc2.py[DEBUG]: Unable to
convert ephemeral0 to a device
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Ignoring
nonexistant default named mount ephemeral0
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Attempting to
determine the real name of swap
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] DataSourceEc2.py[DEBUG]: Unable to
convert swap to a device
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: Ignoring
nonexistant default named mount swap
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] cc_mounts.py[DEBUG]: no need to
setup swap
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Reading from
/etc/fstab (quiet=False)
Nov 21 10:46:35 ip-10-0-30-30 [CLOUDINIT] util.py[DEBUG]: Read 62 bytes from
/etc/fstab
Trying to mount EBS on the recreated instance:
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] stages.py[DEBUG]: Running module
disk_setup (<module 'cloudinit.config.cc_dis
k_setup' from
'/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py'>) with
frequency once-per-instance
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] handlers.py[DEBUG]: start:
modules-config/config-disk_setup: running config-d
isk_setup with frequency once-per-instance
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Writing to
/var/lib/cloud/instances/i-04ebf02c7eb73ecc6/sem/c
onfig_disk_setup - wb: [420] 24 bytes
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] helpers.py[DEBUG]: Running
config-disk_setup using lock (<FileLock using file
'/var/lib/cloud/instances/i-04ebf02c7eb73ecc6/sem/config_disk_setup'>)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] DataSourceEc2.py[DEBUG]: Unable to
convert /dev/xvdf to a device
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]:
Partitioning disks: {'/dev/xvdf': {'overwrite': Fals
e, 'table_type': 'mbr', 'layout': True}}
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating
new partition table/disk
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking
values for /dev/xvdf definition
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking
against default devices
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking
if device /dev/xvdf is a valid device
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/bin/lsblk', '--pairs', '--output', 'NAME,T
YPE,FSTYPE,LABEL', '/dev/xvdf', '--nodeps'] with allowed return codes [0]
(shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking
if device layout matches
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/udevadm', 'settle'] with allowed retu
rn codes [0] (shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/blockdev', '--rereadpt', '/dev/xvdf']
with allowed return codes [0] (shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/udevadm', 'settle'] with allowed retu
rn codes [0] (shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/sfdisk', '-l', '/dev/xvdf'] with allo
wed return codes [0] (shell=False, capture=True)
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Device
partitioning layout matches
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Creating partition
on /dev/xvdf took 0.238 seconds
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: setting up
filesystems: [{'overwrite': False, 'device': '/dev/xvdf1', 'filesystem':
'ext4', 'label': 'munin'}]
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] DataSourceEc2.py[DEBUG]: Unable to
convert /dev/xvdf1 to a device
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating
new filesystem.
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking
/dev/xvdf1 against default devices
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Using the
raw device to place filesystem munin on
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: File
system munin will be created on /dev/xvdf1
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating
file system munin on /dev/xvdf1
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] cc_disk_setup.py[DEBUG]: Using
cmd: /sbin/mkfs.ext4 /dev/xvdf1 -L munin
Nov 21 10:51:57 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Running command
['/sbin/mkfs.ext4', '/dev/xvdf1', '-L', 'munin'] with allowed return codes [0]
(shell=False, capture=True)
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Creating fs for
/dev/xvdf1 took 4.744 seconds
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] handlers.py[DEBUG]: finish:
modules-config/config-disk_setup: SUCCESS: config-disk_setup ran successfully
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] stages.py[DEBUG]: Running module
mounts (<module 'cloudinit.config.cc_mounts' from
'/usr/lib/python3/dist-packages/cloudinit/config/cc_mounts.py'>) with frequency
once-per-instance
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] handlers.py[DEBUG]: start:
modules-config/config-mounts: running config-mounts with frequency
once-per-instance
Nov 21 10:52:02 ip-10-0-30-26 [CLOUDINIT] util.py[DEBUG]: Writing to
/var/lib/cloud/instances/i-04ebf02c7eb73ecc6/sem/config_mounts - wb: [420
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1643531/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
