Re: [Libguestfs] [PATCH nbdkit] s3: Only run the tests if boto3 is installed

2022-05-17 Thread Nikolaus Rath
On May 17 2022, Laszlo Ersek wrote: > I think so, yes: from your description, "test-S3-unit.sh" seems "less > demanding" than "test-S3.sh", so if the mocked module satisfies the > latter, it should be good enough for the former too. Not quite. test-S3-unit.sh exercises the write path and the

Re: [Libguestfs] S3 plugin test case breaks test suite / development workflow...

2022-05-17 Thread Nikolaus Rath
On May 17 2022, Laszlo Ersek wrote: > Hi, > > I'm writing this about a specific problem and about a general problem. > > * The specific problem is that commit 5130c43bc1f9 ("S3 plugin: add > support for accessing multiple objects", 2022-05-12) introduced a > dependency on the "botocore" python

Re: [Libguestfs] Communication issues between NBD driver and NBDKit server

2022-05-17 Thread Nikolaus Rath
On May 16 2022, "yukuai (C)" wrote: > On 2022/05/16 3:25, Richard W.M. Jones wrote: >> On Sun, May 15, 2022 at 08:12:59PM +0100, Nikolaus Rath wrote: >>> Do you see any way for this to happen? >> I think it's impossible. A more likely explanation follows. >> If you look at the kernel code, the

Re: [Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Richard W.M. Jones
Thanks - upstream in: https://github.com/libguestfs/virt-v2v/commit/4ad661b545f476f8dfa00f3cdc28be01f6cc0510 https://github.com/libguestfs/virt-v2v/commit/e06cf5b5dcbfd3f0a798eaf75c8778502c9e3bc6 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my

Re: [Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Laszlo Ersek
On 05/17/22 15:09, Richard W.M. Jones wrote: > On Tue, May 17, 2022 at 01:10:32PM +0200, Laszlo Ersek wrote: >> On 05/17/22 12:59, Richard W.M. Jones wrote: >>> These devices can be left around by grub2 when it runs the osprober >>> tool after we run “/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg”.

Re: [Libguestfs] [nbdkit PATCH] vddk: advise user on obscure thumbprint mismatch error condition

2022-05-17 Thread Laszlo Ersek
On 05/17/22 10:09, Richard W.M. Jones wrote: > On Tue, May 17, 2022 at 10:02:28AM +0200, Laszlo Ersek wrote: >> If the thumbprint parameter is wrong, it's only reported in >> VixDiskLib_Open(), and then with the non-descript VIX_E_FAIL error code. >> If the user typed or cut-and-pasted the

Re: [Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 01:10:32PM +0200, Laszlo Ersek wrote: > On 05/17/22 12:59, Richard W.M. Jones wrote: > > These devices can be left around by grub2 when it runs the osprober > > tool after we run “/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg”. > > They are read-only mirrors of existing

Re: [Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Laszlo Ersek
On 05/17/22 14:07, Richard W.M. Jones wrote: > On Tue, May 17, 2022 at 11:59:26AM +0100, Richard W.M. Jones wrote: >> Still testing this one as well ... > > The version (with rm) fixes the bug for me. Awesome, I suggest that you please go ahead and merge this set! Laszlo

Re: [Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 11:59:26AM +0100, Richard W.M. Jones wrote: > Still testing this one as well ... The version (with rm) fixes the bug for me. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog:

Re: [Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Laszlo Ersek
On 05/17/22 13:18, Richard W.M. Jones wrote: > On Tue, May 17, 2022 at 01:10:32PM +0200, Laszlo Ersek wrote: >> On 05/17/22 12:59, Richard W.M. Jones wrote: >>> These devices can be left around by grub2 when it runs the osprober >>> tool after we run “/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg”.

Re: [Libguestfs] [PATCH] daemon: In list_filesystems ignore /dev/mapper/osprober-* devices

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 11:20:32AM +0100, Richard W.M. Jones wrote: > Still running virt-v2v to test this one ... Whether or not we still want to go with this, I have now tested this version and it works for me. Rich. -- Richard Jones, Virtualization Group, Red Hat

Re: [Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 01:10:32PM +0200, Laszlo Ersek wrote: > On 05/17/22 12:59, Richard W.M. Jones wrote: > > These devices can be left around by grub2 when it runs the osprober > > tool after we run “/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg”. > > They are read-only mirrors of existing

Re: [Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Laszlo Ersek
On 05/17/22 12:59, Richard W.M. Jones wrote: > These devices can be left around by grub2 when it runs the osprober > tool after we run “/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg”. > They are read-only mirrors of existing filesystems. These confuse > later steps in conversion, specifically

Re: [Libguestfs] [PATCH v2v 1/2] convert: Don't duplicate grub2-mkconfig invocations

2022-05-17 Thread Laszlo Ersek
On 05/17/22 12:59, Richard W.M. Jones wrote: > We were invoking grub2-mkconfig in the same way in two different > places. This just refactors one of the calls so it uses the #update > method. It should make no difference. > --- > convert/linux_bootloaders.ml | 2 +- > 1 file changed, 1

Re: [Libguestfs] [PATCH nbdkit] s3: Only run the tests if boto3 is installed

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 12:50:53PM +0200, Laszlo Ersek wrote: > On 05/17/22 12:11, Richard W.M. Jones wrote: > > > > OK I see what's going on. > > > > test-S3.sh uses the mocked boto3 in tests/test-S3/ to do an end-to-end > > test (nbdcopy). > > > > test-S3-unit.sh runs the unit tests within

[Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Richard W.M. Jones
These devices can be left around by grub2 when it runs the osprober tool after we run “/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg”. They are read-only mirrors of existing filesystems. These confuse later steps in conversion, specifically fstrim. Reported-by: Ming Xie Fixes:

[Libguestfs] [PATCH v2v 2/2] convert: Remove /dev/mapper/osprober-* devices left around by grub2

2022-05-17 Thread Richard W.M. Jones
Still testing this one as well ... Rich. ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

[Libguestfs] [PATCH v2v 1/2] convert: Don't duplicate grub2-mkconfig invocations

2022-05-17 Thread Richard W.M. Jones
We were invoking grub2-mkconfig in the same way in two different places. This just refactors one of the calls so it uses the #update method. It should make no difference. --- convert/linux_bootloaders.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

Re: [Libguestfs] [PATCH] daemon: In list_filesystems ignore /dev/mapper/osprober-* devices

2022-05-17 Thread Laszlo Ersek
On 05/17/22 12:20, Richard W.M. Jones wrote: > After virt-v2v runs grub2, which runs osprober, the osprober tool > leaves around read-only linear DM maps covering existing filesystems. > These should be ignored since they duplicate the existing filesystems. > >

Re: [Libguestfs] [PATCH nbdkit] s3: Only run the tests if boto3 is installed

2022-05-17 Thread Laszlo Ersek
On 05/17/22 12:11, Richard W.M. Jones wrote: > > OK I see what's going on. > > test-S3.sh uses the mocked boto3 in tests/test-S3/ to do an end-to-end > test (nbdcopy). > > test-S3-unit.sh runs the unit tests within the plugin. It's basically > testing the plugin as if it was a standalone

Re: [Libguestfs] [PATCH v2v] convert: Ignore /dev/mapper/osprober-* devices when trimming

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 12:29:23PM +0200, Laszlo Ersek wrote: > On 05/17/22 11:48, Laszlo Ersek wrote: > > > Do you think it's feasible to *move* the name-based filtering from > > virt-v2v's do_fstrim to the libguestfs daemon's list-filesystems API > > implementation? I can't see a reason why the

Re: [Libguestfs] [PATCH v2v] convert: Ignore /dev/mapper/osprober-* devices when trimming

2022-05-17 Thread Laszlo Ersek
On 05/17/22 11:48, Laszlo Ersek wrote: > Do you think it's feasible to *move* the name-based filtering from > virt-v2v's do_fstrim to the libguestfs daemon's list-filesystems API > implementation? I can't see a reason why the list-filesystems API should > *ever* return these

[Libguestfs] [PATCH] daemon: In list_filesystems ignore /dev/mapper/osprober-* devices

2022-05-17 Thread Richard W.M. Jones
After virt-v2v runs grub2, which runs osprober, the osprober tool leaves around read-only linear DM maps covering existing filesystems. These should be ignored since they duplicate the existing filesystems. https://listman.redhat.com/archives/libguestfs/2022-May/028871.html

[Libguestfs] [PATCH] daemon: In list_filesystems ignore /dev/mapper/osprober-* devices

2022-05-17 Thread Richard W.M. Jones
Still running virt-v2v to test this one ... Rich. ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

Re: [Libguestfs] [PATCH nbdkit] s3: Only run the tests if boto3 is installed

2022-05-17 Thread Richard W.M. Jones
OK I see what's going on. test-S3.sh uses the mocked boto3 in tests/test-S3/ to do an end-to-end test (nbdcopy). test-S3-unit.sh runs the unit tests within the plugin. It's basically testing the plugin as if it was a standalone Python script (without nbdkit being involved). This uses an

Re: [Libguestfs] [PATCH nbdkit] s3: Only run the tests if boto3 is installed

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 11:53:44AM +0200, Laszlo Ersek wrote: > On 05/17/22 10:55, Richard W.M. Jones wrote: > > ERROR: S3 (unittest.loader._FailedTest) > > -- > > ImportError: Failed to import test module: S3 > > Traceback (most

Re: [Libguestfs] [PATCH v2v] convert: Ignore /dev/mapper/osprober-* devices when trimming

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 11:48:34AM +0200, Laszlo Ersek wrote: > On 05/17/22 09:53, Richard W.M. Jones wrote: > > On Tue, May 17, 2022 at 07:54:01AM +0200, Laszlo Ersek wrote: > >> On 05/16/22 17:46, Richard W.M. Jones wrote: > >>> These devices can be left around by either grub2 or the osprober

Re: [Libguestfs] [PATCH nbdkit] s3: Only run the tests if boto3 is installed

2022-05-17 Thread Laszlo Ersek
On 05/17/22 10:55, Richard W.M. Jones wrote: > ERROR: S3 (unittest.loader._FailedTest) > -- > ImportError: Failed to import test module: S3 > Traceback (most recent call last): > File "/usr/lib64/python3.10/unittest/loader.py",

Re: [Libguestfs] [PATCH v2v] convert: Ignore /dev/mapper/osprober-* devices when trimming

2022-05-17 Thread Laszlo Ersek
On 05/17/22 09:53, Richard W.M. Jones wrote: > On Tue, May 17, 2022 at 07:54:01AM +0200, Laszlo Ersek wrote: >> On 05/16/22 17:46, Richard W.M. Jones wrote: >>> These devices can be left around by either grub2 or the osprober tool. >>> They are read-only mirrors of existing filesystems and it

[Libguestfs] [PATCH nbdkit] s3: Only run the tests if boto3 is installed

2022-05-17 Thread Richard W.M. Jones
ERROR: S3 (unittest.loader._FailedTest) -- ImportError: Failed to import test module: S3 Traceback (most recent call last): File "/usr/lib64/python3.10/unittest/loader.py", line 154, in loadTestsFromName module =

Re: [Libguestfs] S3 plugin test case breaks test suite / development workflow...

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 10:31:59AM +0200, Laszlo Ersek wrote: > Hi, > > I'm writing this about a specific problem and about a general problem. > > * The specific problem is that commit 5130c43bc1f9 ("S3 plugin: add > support for accessing multiple objects", 2022-05-12) introduced a > dependency

[Libguestfs] S3 plugin test case breaks test suite / development workflow...

2022-05-17 Thread Laszlo Ersek
Hi, I'm writing this about a specific problem and about a general problem. * The specific problem is that commit 5130c43bc1f9 ("S3 plugin: add support for accessing multiple objects", 2022-05-12) introduced a dependency on the "botocore" python module, and now "make check" fails for me, because

Re: [Libguestfs] [nbdkit PATCH] vddk: advise user on obscure thumbprint mismatch error condition

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 10:02:28AM +0200, Laszlo Ersek wrote: > If the thumbprint parameter is wrong, it's only reported in > VixDiskLib_Open(), and then with the non-descript VIX_E_FAIL error code. > If the user typed or cut-and-pasted the thumbprint incorrectly, said > "Unkown error" message is

[Libguestfs] [nbdkit PATCH] vddk: advise user on obscure thumbprint mismatch error condition

2022-05-17 Thread Laszlo Ersek
If the thumbprint parameter is wrong, it's only reported in VixDiskLib_Open(), and then with the non-descript VIX_E_FAIL error code. If the user typed or cut-and-pasted the thumbprint incorrectly, said "Unkown error" message is not helpful for fixing the nbkit command line. Hint at the thumbprint

Re: [Libguestfs] [PATCH v2v] convert: Ignore /dev/mapper/osprober-* devices when trimming

2022-05-17 Thread Richard W.M. Jones
On Tue, May 17, 2022 at 07:54:01AM +0200, Laszlo Ersek wrote: > On 05/16/22 17:46, Richard W.M. Jones wrote: > > These devices can be left around by either grub2 or the osprober tool. > > They are read-only mirrors of existing filesystems and it appears we > > can safely ignore them. > > > >