Re: [Openstack] File injection support
On Tue, Jun 12, 2012 at 8:20 AM, Pádraig Brady p...@draigbrady.com wrote:
On 06/12/2012 04:07 PM, Fredric Morenius wrote:
From: Scott Moser [mailto:ssmos...@gmail.com] On Behalf Of Scott Moser
Sent: den 11 juni 2012 23:16
...Without digging around on older versions of OS's and their included
kernel/udev/kpartx i'd have to say that from your message above, and my
testing that running 'kpartx' is no longer necessary. That can reasonably
be expected to be created by the kernel or some other plumbing bits
automatically...
I just now tested to remove the kpartx invocation from
/nova/virt/disk/mount.py, like this:
diff --git a/nova/virt/disk/mount.py b/nova/virt/disk/mount.py
index 11959b2..4d9527b 100644
--- a/nova/virt/disk/mount.py
+++ b/nova/virt/disk/mount.py
@@ -61,25 +61,9 @@ class Mount(object):
if self.partition == -1:
self.error = _('partition search unsupported with %s') %
self.mode
elif self.partition:
-map_path = '/dev/mapper/%sp%s' % (os.path.basename(self.device),
- self.partition)
-assert(not os.path.exists(map_path))
-
-# Note kpartx can output warnings to stderr and succeed
-# Also it can output failures to stderr and succeed
-# So we just go on the existence of the mapped device
-_out, err = utils.trycmd('kpartx', '-a', self.device,
- run_as_root=True,
discard_warnings=True)
-
-# Note kpartx does nothing when presented with a raw image,
-# so given we only use it when we expect a partitioned image,
fail
-if not os.path.exists(map_path):
-if not err:
-err = _('partition %s not found') % self.partition
-self.error = _('Failed to map partitions: %s') % err
-else:
-self.mapped_device = map_path
-self.mapped = True
+#qemu-nbd already has mapped the partition for mounting
+self.mapped_device = '%sp%s' % (self.device, self.partition)
+self.mapped = True
Looks good. That should also support raw through the /dev/loop%s devices.
It might be safer to fallback to kpartx if not exists ...
'%sp%s' % (self.device, self.partition)
That would support kernels before 3.2
I'd also add a comment in the code, that nbd must be mounted with param
max_part=16 or whatever.
Might be totally unrelated to the above issues but I figured I'd throw
it out there. I was messing with file injection today and was getting
odd errors.
Some were due to hitting errors on a previous boot and partitions not
getting cleaned up (e.g. kpartx -d never ran and the /dev/mapper/nbd*
entries were still there, or qemu-nbd was still running). Some of
those were likely a result of my own circumstances.
More interesting though, and what might be of use to other people, the
kpartx -a calls get run and then the code in nova/virt/disk/mount.py
immediately checks for whether or not the newly created
/dev/mapper/nbdXXpX partitions exist. They'd actually get created but
the os.exists call would fail. Apparently the os.exists call was
getting run too soon. I added a time.sleep() after both the 'kpartx
-a' and 'kpartx -d' calls, to give things time to catch up before the
os.exists calls and things worked much better.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On 07/14/2012 12:09 AM, Mark Moseley wrote:
On Tue, Jun 12, 2012 at 8:20 AM, Pádraig Brady p...@draigbrady.com wrote:
On 06/12/2012 04:07 PM, Fredric Morenius wrote:
From: Scott Moser [mailto:ssmos...@gmail.com] On Behalf Of Scott Moser
Sent: den 11 juni 2012 23:16
...Without digging around on older versions of OS's and their included
kernel/udev/kpartx i'd have to say that from your message above, and
my testing that running 'kpartx' is no longer necessary. That can
reasonably be expected to be created by the kernel or some other plumbing
bits automatically...
I just now tested to remove the kpartx invocation from
/nova/virt/disk/mount.py, like this:
diff --git a/nova/virt/disk/mount.py b/nova/virt/disk/mount.py
index 11959b2..4d9527b 100644
--- a/nova/virt/disk/mount.py
+++ b/nova/virt/disk/mount.py
@@ -61,25 +61,9 @@ class Mount(object):
if self.partition == -1:
self.error = _('partition search unsupported with %s') %
self.mode
elif self.partition:
-map_path = '/dev/mapper/%sp%s' %
(os.path.basename(self.device),
- self.partition)
-assert(not os.path.exists(map_path))
-
-# Note kpartx can output warnings to stderr and succeed
-# Also it can output failures to stderr and succeed
-# So we just go on the existence of the mapped device
-_out, err = utils.trycmd('kpartx', '-a', self.device,
- run_as_root=True,
discard_warnings=True)
-
-# Note kpartx does nothing when presented with a raw image,
-# so given we only use it when we expect a partitioned image,
fail
-if not os.path.exists(map_path):
-if not err:
-err = _('partition %s not found') % self.partition
-self.error = _('Failed to map partitions: %s') % err
-else:
-self.mapped_device = map_path
-self.mapped = True
+#qemu-nbd already has mapped the partition for mounting
+self.mapped_device = '%sp%s' % (self.device, self.partition)
+self.mapped = True
Looks good. That should also support raw through the /dev/loop%s devices.
It might be safer to fallback to kpartx if not exists ...
'%sp%s' % (self.device, self.partition)
That would support kernels before 3.2
I'd also add a comment in the code, that nbd must be mounted with param
max_part=16 or whatever.
Might be totally unrelated to the above issues but I figured I'd throw
it out there. I was messing with file injection today and was getting
odd errors.
Some were due to hitting errors on a previous boot and partitions not
getting cleaned up (e.g. kpartx -d never ran and the /dev/mapper/nbd*
entries were still there, or qemu-nbd was still running). Some of
those were likely a result of my own circumstances.
More interesting though, and what might be of use to other people, the
kpartx -a calls get run and then the code in nova/virt/disk/mount.py
immediately checks for whether or not the newly created
/dev/mapper/nbdXXpX partitions exist. They'd actually get created but
the os.exists call would fail. Apparently the os.exists call was
getting run too soon. I added a time.sleep() after both the 'kpartx
-a' and 'kpartx -d' calls, to give things time to catch up before the
os.exists calls and things worked much better.
Sigh.
The amount of synchronization bugs I've noticed in lower
Linux layers lately is worrying.
Anyway I've created a bug for this:
https://bugs.launchpad.net/nova/+bug/1024586
cheers,
Pádraig
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Friday, July 13, 2012 at 20:35 PM, Pádraig Brady wrote: More interesting though, and what might be of use to other people, the kpartx -a calls get run and then the code in nova/virt/disk/mount.py immediately checks for whether or not the newly created /dev/mapper/nbdXXpX partitions exist. They'd actually get created but the os.exists call would fail. Apparently the os.exists call was getting run too soon. I added a time.sleep() after both the 'kpartx -a' and 'kpartx -d' calls, to give things time to catch up before the os.exists calls and things worked much better. Sigh. The amount of synchronization bugs I've noticed in lower Linux layers lately is worrying. The kernel tools can certainly be asynchronous at times. Perhaps try to see if using inotify would solve this. It is platform-dependent, but it is the best way to solve these problems without race conditions. If we're calling kpartx, platform independence is unlikely to be an issue anyway. However, if compatibility is desired, BSD/MacOS provide similar functionality via kqueue, and Windows FindFirstChangeNotification. Regards, Eric Windisch ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On 06/11/2012 10:16 PM, Scott Moser wrote: Without digging around on older versions of OS's and their included kernel/udev/kpartx i'd have to say that from your message above, and my testing that running 'kpartx' is no longer necessary. That can reasonably be expected to be created by the kernel or some other plumbing bits automatically. It would be good to get away from kpartx. I've noticed these issues: 1. The git repo on kernel.org is no longer available. 2. kpartx -l had side effects: $ kpartx -l /bin/ls $ ls text file busy To fix you need to run losetup -a to find the assigned loopback device and then losetup -d /dev/loop... 3. On an unconnected loop device we get warnings, but an EXIT_SUCCESS ? # kpartx -a /dev/loop1 echo EXIT_SUCCESS read error, sector 0 llseek error llseek error llseek error EXIT_SUCCESS 4. Also for a loop device that is connected, I get a failed warning, but the EXIT_SUCCESS is appropriate in that case as the mapped device is present and usable # kpartx -a /dev/loop0 /dev/mapper/loop0p1: mknod for loop0p1 failed: File exists That last item is related to the new code for auto parsing partitions. That's only available since kernel 3.2 I think so we'll have to be wary on relying on it. cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On 06/12/2012 04:07 PM, Fredric Morenius wrote:
From: Scott Moser [mailto:ssmos...@gmail.com] On Behalf Of Scott Moser
Sent: den 11 juni 2012 23:16
...Without digging around on older versions of OS's and their included
kernel/udev/kpartx i'd have to say that from your message above, and my
testing that running 'kpartx' is no longer necessary. That can reasonably
be expected to be created by the kernel or some other plumbing bits
automatically...
I just now tested to remove the kpartx invocation from
/nova/virt/disk/mount.py, like this:
diff --git a/nova/virt/disk/mount.py b/nova/virt/disk/mount.py
index 11959b2..4d9527b 100644
--- a/nova/virt/disk/mount.py
+++ b/nova/virt/disk/mount.py
@@ -61,25 +61,9 @@ class Mount(object):
if self.partition == -1:
self.error = _('partition search unsupported with %s') %
self.mode
elif self.partition:
-map_path = '/dev/mapper/%sp%s' % (os.path.basename(self.device),
- self.partition)
-assert(not os.path.exists(map_path))
-
-# Note kpartx can output warnings to stderr and succeed
-# Also it can output failures to stderr and succeed
-# So we just go on the existence of the mapped device
-_out, err = utils.trycmd('kpartx', '-a', self.device,
- run_as_root=True, discard_warnings=True)
-
-# Note kpartx does nothing when presented with a raw image,
-# so given we only use it when we expect a partitioned image,
fail
-if not os.path.exists(map_path):
-if not err:
-err = _('partition %s not found') % self.partition
-self.error = _('Failed to map partitions: %s') % err
-else:
-self.mapped_device = map_path
-self.mapped = True
+#qemu-nbd already has mapped the partition for mounting
+self.mapped_device = '%sp%s' % (self.device, self.partition)
+self.mapped = True
Looks good. That should also support raw through the /dev/loop%s devices.
It might be safer to fallback to kpartx if not exists ...
'%sp%s' % (self.device, self.partition)
That would support kernels before 3.2
I'd also add a comment in the code, that nbd must be mounted with param
max_part=16 or whatever.
cheers,
Pádraig.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
Hello again, I did some further tests on this issue with the qcow2 image, the qcow2 image converted to raw and the _blank (raw) image from the UEC tar archive. I loaded nbd with $ sudo modprobe nbd max_part=8 *** qcow2 image *** $ k$ md5sum cirros-0.3.0-x86_64-disk.img 50bdc35edb03a38d91b1b071afb20a3c cirros-0.3.0-x86_64-disk.img $ qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 9.3M cluster_size: 65536 $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img That creates the device /dev/nbd15p1 kpartx on top of that: $ sudo kpartx -a /dev/nbd15 device-mapper: resume ioctl failed: Invalid argument create/reload failed on nbd15p1 $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected *** UEC image *** $ qemu-img info cirros-0.3.0-x86_64-blank.img image: cirros-0.3.0-x86_64-blank.img file format: raw virtual size: 24M (25165824 bytes) disk size: 24M $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-blank.img That DOES NOT create the device /dev/nbd15p1 kpartx on top of that: $ sudo kpartx -a /dev/nbd15 kpartx executes without error $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected *** raw image *** $ qemu-img convert -p -O raw cirros-0.3.0-x86_64-disk.img cirros-0.3.0-x86_64-disk_raw.img $ qemu-img info cirros-0.3.0-x86_64-disk_raw.img image: cirros-0.3.0-x86_64-disk_raw.img file format: raw virtual size: 39M (41126400 bytes) disk size: 12M $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk_raw.img That creates the device /dev/nbd15p1 kpartx on top of that: $ sudo kpartx -a /dev/nbd15 device-mapper: resume ioctl failed: Invalid argument create/reload failed on nbd15p1 $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected So, the issue is not with the qcow2 vs raw image formats. Instead kpartx only executes without error provided that the device /dev/nbd15p1 is not created. When p1 is created by qemu-nbd, kpartx is not needed to mount the image device, but the problem is qemu-nbd does not create p1 in all cases. I have not checked the details of the OpenStack code, but an (ugly) work around could be to only execute kpartx when it is needed. Best Regards, Fredric -Original Message- From: openstack-bounces+fredric.morenius=ericsson@lists.launchpad.net [mailto:openstack-bounces+fredric.morenius=ericsson@lists.launchpad.net] On Behalf Of Vishvananda Ishaya Sent: den 10 juni 2012 22:22 To: Pádraig Brady Cc: igor.lask...@gmail.com; openstack Subject: Re: [Openstack] File injection support On Jun 8, 2012, at 9:03 AM, Pádraig Brady wrote: $ qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 11M cluster_size: 65536 $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img $ ls -l /dev/nbd15* brw-rw 1 root disk 43, 240 Jun 8 11:32 /dev/nbd15 brw-rw 1 root disk 43, 241 Jun 8 11:32 /dev/nbd15p1 Very interesting. I don't get the ...p1 device here on a 3.3.7.fc17.x86_64 kernel did you modrpobe nbd with max_part=8 or some such? It won't detect partitions automatically unless you do so. We really should stop using kpartx and force people to modprobe nbd with max_part=X There seem to be various issues with kpartx Vish cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Mon, 11 Jun 2012, Fredric Morenius wrote: Hello again, I did some further tests on this issue with the qcow2 image, the qcow2 image converted to raw and the _blank (raw) image from the UEC tar archive. I loaded nbd with $ sudo modprobe nbd max_part=8 *** qcow2 image *** $ k$ md5sum cirros-0.3.0-x86_64-disk.img 50bdc35edb03a38d91b1b071afb20a3c cirros-0.3.0-x86_64-disk.img this image is a qcow compressed disk image. The disk contains a partition table, grub (0.97) installed on the MBR, and a root filesystem on the first partition. That creates the device /dev/nbd15p1 kpartx on top of that: $ sudo kpartx -a /dev/nbd15 device-mapper: resume ioctl failed: Invalid argument create/reload failed on nbd15p1 $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected That seems like working as designed. at least on ubuntu 12.04, the kpartx is no longer necessary. I'd have to dig more to figure out exactly why, but some combination of kernel partition scanner and udev do the right thing when this new network block device is set up. *** UEC image *** $ qemu-img info cirros-0.3.0-x86_64-blank.img image: cirros-0.3.0-x86_64-blank.img file format: raw virtual size: 24M (25165824 bytes) disk size: 24M $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-blank.img This image is essentially the same as you'd get if you did: truncate --size 24M disk.img mkfs.ext3 -F disk.img -L cirros-rootfs Ie, it is a unpartitioned disk with an (empty) filesystem on it. There are no partitions, or data in the filesystem. On boot, cirros initramfs copies itself to that disk, and then pivot_roots to it. Cirros will run perfectly fine as an initramfs with no root filesystem too, fwiw. That DOES NOT create the device /dev/nbd15p1 kpartx on top of that: $ sudo kpartx -a /dev/nbd15 kpartx executes without error $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected This also seems like working as designed. *** raw image *** $ qemu-img convert -p -O raw cirros-0.3.0-x86_64-disk.img cirros-0.3.0-x86_64-disk_raw.img $ qemu-img info cirros-0.3.0-x86_64-disk_raw.img image: cirros-0.3.0-x86_64-disk_raw.img file format: raw virtual size: 39M (41126400 bytes) disk size: 12M $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk_raw.img That creates the device /dev/nbd15p1 kpartx on top of that: $ sudo kpartx -a /dev/nbd15 device-mapper: resume ioctl failed: Invalid argument create/reload failed on nbd15p1 $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected So, the issue is not with the qcow2 vs raw image formats. Instead kpartx only executes without error provided that the device /dev/nbd15p1 is not created. When p1 is created by qemu-nbd, kpartx is not needed to mount the image device, but the problem is qemu-nbd does not create p1 in all cases. I have not checked the details of the OpenStack code, but an (ugly) work around could be to only execute kpartx when it is needed. Without digging around on older versions of OS's and their included kernel/udev/kpartx i'd have to say that from your message above, and my testing that running 'kpartx' is no longer necessary. That can reasonably be expected to be created by the kernel or some other plumbing bits automatically. If it just happens magically,though, I'm not sure how you would know if it hasn't happened yet. That would imply that this might be racey. I ran into similar bugs http://pad.lv/937352 and http://pad.lv/942788 that affected the growing of the root partition from an initramfs. The thing we ended up settling on was that after re-writing the partition table, we had to run 'udevadm --settle'. its quite possible that 'qemu-nbd --connect' sets off a similar flurry of events that would need to be waited on before a /dev/nbdXpY would exist. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Jun 8, 2012, at 9:03 AM, Pádraig Brady wrote: $ qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 11M cluster_size: 65536 $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img $ ls -l /dev/nbd15* brw-rw 1 root disk 43, 240 Jun 8 11:32 /dev/nbd15 brw-rw 1 root disk 43, 241 Jun 8 11:32 /dev/nbd15p1 Very interesting. I don't get the ...p1 device here on a 3.3.7.fc17.x86_64 kernel did you modrpobe nbd with max_part=8 or some such? It won't detect partitions automatically unless you do so. We really should stop using kpartx and force people to modprobe nbd with max_part=X There seem to be various issues with kpartx Vish cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
Thank for this Igor Laskovy Kiev, Ukraine On Jun 8, 2012 11:20 AM, Fredric Morenius fredric.moren...@ericsson.com wrote: Hello All, An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images: After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools. BR Fredric -Original Message- From: Pádraig Brady [mailto:p...@draigbrady.com] Sent: den 30 maj 2012 18:03 To: Fredric Morenius Cc: openstack; igor.lask...@gmail.com Subject: Re: [Openstack] File injection support On 05/30/2012 03:47 PM, Fredric Morenius wrote: Hello Pádraig, I am also trying to get file injection to work in Essex, but have run into some issues, as stated here: https://answers.launchpad.net/nova/+question/198878 Igor Laskovy also had that device-mapper: resume ioctl failed: issue with the qemu-nbd on a cirros image too, though he didn't need the injection though and just avoided it. The image I am launching is a simple bare container qcow2 image (CirrOS, this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64 -disk.img ) That image is a simple single partition image, and so should not need the patch referenced below. What system are you using? If you installed libguestfs then that should be tried as a method without need for the patch below. Would it be possible to backport this: https://github.com/openstack/nova/commit/2b3a1e7 So that file injection as I am trying to do it will work? Or is there any other way to make it work? The backport is trivial and already done in the Fedora/EPEL Essex packages. I was thinking though that this was extra functionality and so not appropriate for the official stable branch? * NOT A CONTRIBUTION * The content of this email shall not be considered as a contribution to OpenStack :) cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On 06/08/2012 09:20 AM, Fredric Morenius wrote: Hello All, An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images: After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools. Yes it's seems that qemu-nbd has issues with this image? # rpm -qf $(which qemu-nbd) qemu-common-1.0-17.fc17.x86_64 # qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 9.3M cluster_size: 65536 # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img # ls -la /sys/block/nbd15/pid -r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid # kpartx -a /dev/nbd15 device-mapper: resume ioctl on nbd15p1 failed: Invalid argument create/reload failed on nbd15p1 If I convert to raw with qemu-img, I can mount loopback fine. cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Fri, 8 Jun 2012, Pádraig Brady wrote: On 06/08/2012 09:20 AM, Fredric Morenius wrote: Hello All, An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images: After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools. Yes it's seems that qemu-nbd has issues with this image? # rpm -qf $(which qemu-nbd) qemu-common-1.0-17.fc17.x86_64 # qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 9.3M cluster_size: 65536 # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img # ls -la /sys/block/nbd15/pid -r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid # kpartx -a /dev/nbd15 device-mapper: resume ioctl on nbd15p1 failed: Invalid argument create/reload failed on nbd15p1 If I convert to raw with qemu-img, I can mount loopback fine. Well, interesting. It does seem to work here: $ dpkg-query --show qemu-utils qemu-utils 1.0+noroms-0ubuntu13 $ cd /tmp $ wget -q https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img $ md5sum cirros-0.3.0-x86_64-disk.img 6654705afc4b74fda3e1f4330559d066 cirros-0.3.0-x86_64-disk.img $ qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 11M cluster_size: 65536 $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img $ ls -l /dev/nbd15* brw-rw 1 root disk 43, 240 Jun 8 11:32 /dev/nbd15 brw-rw 1 root disk 43, 241 Jun 8 11:32 /dev/nbd15p1 $ sudo mount /dev/nbd15p1 /mnt $ ls /mnt bin etc initrd.img lost+found opt run tmp vmlinuz boot home lib mnt proc sbin usr dev init linuxrc old-rootroot sys var $ sudo umount /mnt $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Fri, 8 Jun 2012, Scott Moser wrote: On Fri, 8 Jun 2012, Pádraig Brady wrote: On 06/08/2012 09:20 AM, Fredric Morenius wrote: Hello All, An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images: After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools. Yes it's seems that qemu-nbd has issues with this image? # rpm -qf $(which qemu-nbd) qemu-common-1.0-17.fc17.x86_64 # qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 9.3M cluster_size: 65536 # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img # ls -la /sys/block/nbd15/pid -r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid # kpartx -a /dev/nbd15 device-mapper: resume ioctl on nbd15p1 failed: Invalid argument create/reload failed on nbd15p1 If I convert to raw with qemu-img, I can mount loopback fine. Well, interesting. It does seem to work here: $ dpkg-query --show qemu-utils qemu-utils 1.0+noroms-0ubuntu13 $ cd /tmp $ wget -q https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img $ md5sum cirros-0.3.0-x86_64-disk.img 6654705afc4b74fda3e1f4330559d066 cirros-0.3.0-x86_64-disk.img Whoops. I had a local image that was different (must have been laying around from long ago, the wget saved to .1). It does still work though. $ md5sum cirros-0.3.0-x86_64-disk.img 50bdc35edb03a38d91b1b071afb20a3c cirros-0.3.0-x86_64-disk.img $ qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 9.3M cluster_size: 65536 The same nbd -c... mount work as shown.___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On 06/08/2012 04:35 PM, Scott Moser wrote: On Fri, 8 Jun 2012, Pádraig Brady wrote: On 06/08/2012 09:20 AM, Fredric Morenius wrote: Hello All, An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images: After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools. Yes it's seems that qemu-nbd has issues with this image? # rpm -qf $(which qemu-nbd) qemu-common-1.0-17.fc17.x86_64 # qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 9.3M cluster_size: 65536 # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img # ls -la /sys/block/nbd15/pid -r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid # kpartx -a /dev/nbd15 device-mapper: resume ioctl on nbd15p1 failed: Invalid argument create/reload failed on nbd15p1 If I convert to raw with qemu-img, I can mount loopback fine. Well, interesting. It does seem to work here: $ dpkg-query --show qemu-utils qemu-utils 1.0+noroms-0ubuntu13 $ cd /tmp $ wget -q https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img $ md5sum cirros-0.3.0-x86_64-disk.img 6654705afc4b74fda3e1f4330559d066 cirros-0.3.0-x86_64-disk.img I downloaded again and get 50bdc35edb03a38d91b1b071afb20a3c I presume the above md5sum is for a mounted/modified image? $ qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 11M cluster_size: 65536 $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img $ ls -l /dev/nbd15* brw-rw 1 root disk 43, 240 Jun 8 11:32 /dev/nbd15 brw-rw 1 root disk 43, 241 Jun 8 11:32 /dev/nbd15p1 Very interesting. I don't get the ...p1 device here on a 3.3.7.fc17.x86_64 kernel cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
Hello Pádraig, I am also trying to get file injection to work in Essex, but have run into some issues, as stated here: https://answers.launchpad.net/nova/+question/198878 The image I am launching is a simple bare container qcow2 image (CirrOS, this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img ) Would it be possible to backport this: https://github.com/openstack/nova/commit/2b3a1e7 So that file injection as I am trying to do it will work? Or is there any other way to make it work? Thanks and Best Regards, Fredric * NOT A CONTRIBUTION * The content of this email shall not be considered as a contribution to OpenStack -Original Message- From: openstack-bounces+fredric.morenius=ericsson@lists.launchpad.net [mailto:openstack-bounces+fredric.morenius=ericsson@lists.launchpad.net] On Behalf Of Pádraig Brady Sent: den 30 maj 2012 01:22 To: Nicolae Paladi Cc: openstack Subject: Re: [Openstack] File injection support On 05/29/2012 07:35 PM, Nicolae Paladi wrote: Hello, I am looking for more information about file injection support in OpenStack and find it increasingly inconsistent and incomplete. I have several questions: * This wiki article (http://wiki.openstack.org/HypervisorSupportMatrix) claims that file injection is supported in XenServer, is NOT supported inn KVM it's not clear about the others. is that still the case? One of the key Essex Features was Hypervisor Feature Parity, including: Disk Config, resize and file injection. I've updated the wiki accordingly. So there is support in kvm for injecting ssh keys, net info, root password, or arbitrary files. By default there are 3 methods tried in turn to do this: loopback mount, qemu-nbd, libguestfs. In Essex injection is supported to the first partition of guest images. Folsom enables further libguestfs functionality to inspect arbitrarily complex guest images and find the root partition to inject to. At least there is some mentioning of file injection in the openstack xenserver wiki: http://wiki.openstack.org/XenServer/Development * Is there any guide or man page where file injection is explained? in this use case I would like to inject a file to e.g. /etc/security/.conf at launch time -- is that possible, and if yes how can it be done in the Essex release. The API docs explain the personality parameter to achieve this: http://docs.openstack.org/api/openstack-compute/2/content/CreateServers.html This is exposed on the nova command line tool though the --injected-files option. * Are there any discussions about the future of file injection? There are other options, like cloud-init or using a config drive. You enable a config drive with the nova --config-drive option, with value as a boolean or a volume ID. See also: https://blueprints.launchpad.net/nova/+spec/config-drive-v2 cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Wed, 30 May 2012, Fredric Morenius wrote: Hello Pádraig, I am also trying to get file injection to work in Essex, but have run into some issues, as stated here: https://answers.launchpad.net/nova/+question/198878 The image I am launching is a simple bare container qcow2 image (CirrOS, this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img ) I generally think that file injection is significantly less preferable to use of user-data or config-drive (an api/convention for providing information to an instance). cirros supports executing user-data starting with '#!', so you can probably accomplish what you're after via passing data like this: #!/bin/sh cat /path/to/file EOF file contents here EOF wget http://some.url/my.tar | ( cd / tar xvzf - ) base64 -d /usr/bin/my-exe EOF IyEvYmluL3NoCmVjaG8gImhpIHdvcmxkIgo= EOF chmod 755 /usr/bin/my-exe end user data ### You could also use 'shar' to do this.___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On 05/30/2012 03:47 PM, Fredric Morenius wrote: Hello Pádraig, I am also trying to get file injection to work in Essex, but have run into some issues, as stated here: https://answers.launchpad.net/nova/+question/198878 Igor Laskovy also had that device-mapper: resume ioctl failed: issue with the qemu-nbd on a cirros image too, though he didn't need the injection though and just avoided it. The image I am launching is a simple bare container qcow2 image (CirrOS, this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img ) That image is a simple single partition image, and so should not need the patch referenced below. What system are you using? If you installed libguestfs then that should be tried as a method without need for the patch below. Would it be possible to backport this: https://github.com/openstack/nova/commit/2b3a1e7 So that file injection as I am trying to do it will work? Or is there any other way to make it work? The backport is trivial and already done in the Fedora/EPEL Essex packages. I was thinking though that this was extra functionality and so not appropriate for the official stable branch? * NOT A CONTRIBUTION * The content of this email shall not be considered as a contribution to OpenStack :) cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
https://help.ubuntu.com/community/CloudInit Might want to check this little beastie out. Folks using kvm seem to end up here for certain use cases. -Matt On Tue, May 29, 2012 at 11:35 AM, Nicolae Paladi n.pal...@gmail.com wrote: Hello, I am looking for more information about file injection support in OpenStack and find it increasingly inconsistent and incomplete. I have several questions: * This wiki article (http://wiki.openstack.org/HypervisorSupportMatrix) claims that file injection is supported in XenServer, is NOT supported inn KVM it's not clear about the others. is that still the case? At least there is some mentioning of file injection in the openstack xenserver wiki: http://wiki.openstack.org/XenServer/Development * Is there any guide or man page where file injection is explained? in this use case I would like to inject a file to e.g. /etc/security/.conf at launch time -- is that possible, and if yes how can it be done in the Essex release. * Are there any discussions about the future of file injection? Thanks a lot! /Nico. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On 05/29/2012 07:35 PM, Nicolae Paladi wrote: Hello, I am looking for more information about file injection support in OpenStack and find it increasingly inconsistent and incomplete. I have several questions: * This wiki article (http://wiki.openstack.org/HypervisorSupportMatrix) claims that file injection is supported in XenServer, is NOT supported inn KVM it's not clear about the others. is that still the case? One of the key Essex Features was Hypervisor Feature Parity, including: Disk Config, resize and file injection. I've updated the wiki accordingly. So there is support in kvm for injecting ssh keys, net info, root password, or arbitrary files. By default there are 3 methods tried in turn to do this: loopback mount, qemu-nbd, libguestfs. In Essex injection is supported to the first partition of guest images. Folsom enables further libguestfs functionality to inspect arbitrarily complex guest images and find the root partition to inject to. At least there is some mentioning of file injection in the openstack xenserver wiki: http://wiki.openstack.org/XenServer/Development * Is there any guide or man page where file injection is explained? in this use case I would like to inject a file to e.g. /etc/security/.conf at launch time -- is that possible, and if yes how can it be done in the Essex release. The API docs explain the personality parameter to achieve this: http://docs.openstack.org/api/openstack-compute/2/content/CreateServers.html This is exposed on the nova command line tool though the --injected-files option. * Are there any discussions about the future of file injection? There are other options, like cloud-init or using a config drive. You enable a config drive with the nova --config-drive option, with value as a boolean or a volume ID. See also: https://blueprints.launchpad.net/nova/+spec/config-drive-v2 cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
