Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-16 Thread Martin Kletzander
On Fri, Jan 13, 2023 at 08:23:05PM +, Richard W.M. Jones wrote: On Fri, Jan 13, 2023 at 01:06:09PM -0700, alan somers wrote: Which files get overwritten by libvirt-ci ? I think it's everything except ci/manifest.yml, possibly. Eric and Martin (CC'd) know more about this. The libvirt-ci

Re: [Libguestfs] [PATCH nbdkit] ci: Expire artifacts after 1 week

2022-08-10 Thread Martin Kletzander
On Tue, Aug 09, 2022 at 02:01:18PM +0100, Richard W.M. Jones wrote: On Tue, Aug 09, 2022 at 10:07:37AM +0100, Richard W.M. Jones wrote: On Tue, Aug 09, 2022 at 09:23:57AM +0200, Martin Kletzander wrote: > On Mon, Aug 08, 2022 at 09:51:32PM +0100, Richard W.M. Jones wrote: > >nbdkit u

Re: [Libguestfs] [PATCH nbdkit] ci: Expire artifacts after 1 week

2022-08-09 Thread Martin Kletzander
On Mon, Aug 08, 2022 at 09:51:32PM +0100, Richard W.M. Jones wrote: nbdkit uses about 1.1G of storage for artifacts. I have personally never looked at anything other than the artifacts of the latest failed job, so I don't believe we need to keep these around for long. This commit proposes

Re: [Libguestfs] libnbd: Failed to update to Fedora 36 & OpenSUSE Leap 15.3

2022-05-30 Thread Martin Kletzander
On Sun, May 29, 2022 at 11:22:05AM +0100, Richard W.M. Jones wrote: I added this commit and regenerated the CI files: https://gitlab.com/nbdkit/libnbd/-/commit/b6a98aacbe22d599f000d4d1c84c27081ec06957 https://gitlab.com/nbdkit/libnbd/-/commit/2439fd5c7a07b314ce47728c6fbae16b9a26dcdb However

Re: [Libguestfs] Running lcitool in nbdkit

2022-03-14 Thread Martin Kletzander
On Mon, Mar 14, 2022 at 11:54:57AM +, Richard W.M. Jones wrote: On Mon, Mar 14, 2022 at 11:33:16AM +, Richard W.M. Jones wrote: Pipeline is currently running: https://gitlab.com/nbdkit/nbdkit/-/pipelines/491516164 I had previously fixed this failure in Alpine: perl.c:45:10: fatal

Re: [Libguestfs] Running lcitool in nbdkit

2022-03-14 Thread Martin Kletzander
On Mon, Mar 14, 2022 at 11:33:16AM +, Richard W.M. Jones wrote: To sum it up, I added these commits to nbdkit which converge the generated files with what lcitool now generates: https://gitlab.com/nbdkit/nbdkit/-/commit/8e99576b89b58ad09c634b96cff579dd9a352fc6

Re: [Libguestfs] libnbd | Failed pipeline for master | 70da51e5

2022-01-04 Thread Martin Kletzander
On Tue, Jan 04, 2022 at 08:42:14PM +, Richard W.M. Jones wrote: On Tue, Jan 04, 2022 at 09:35:18PM +0100, Martin Kletzander wrote: On Tue, Jan 04, 2022 at 03:55:08PM +, Richard W.M. Jones wrote: > >I created a merge request on the theory that this will run the CI >pipeline, thu

Re: [Libguestfs] libnbd | Failed pipeline for master | 70da51e5

2022-01-04 Thread Martin Kletzander
On Tue, Jan 04, 2022 at 03:55:08PM +, Richard W.M. Jones wrote: I created a merge request on the theory that this will run the CI pipeline, thus testing my patch on OpenSUSE ... https://gitlab.com/nbdkit/libnbd/-/merge_requests/2 It did, OpenSUSE is fine, Ubuntu 18.04 deadlocks because

Re: [Libguestfs] libnbd | Failed pipeline for master | 70da51e5

2022-01-04 Thread Martin Kletzander
On Tue, Jan 04, 2022 at 01:56:08PM +, Richard W.M. Jones wrote: On Thu, Sep 02, 2021 at 05:12:35PM +0100, Richard W.M. Jones wrote: Hi Martin, On Thu, Sep 02, 2021 at 04:05:56PM +, GitLab wrote: > GitLab >✖ Pipeline #364204388 has failed! > > Project nbdkit /

Re: [Libguestfs] [PATCH libnbd CI] ci: Drop Fedora 33, add Fedora 35

2021-12-13 Thread Martin Kletzander
On Mon, Dec 13, 2021 at 02:40:40PM +, Richard W.M. Jones wrote: I pushed this (commit 202d0ecad) in the hope it might fix the tests. However Martin indicates that the error message might be nothing to do with the Fedora version. Let's see. It looks like everything went swimmingly the

Re: [Libguestfs] [PATCH libnbd CI] ci: Drop Fedora 33, add Fedora 35

2021-12-11 Thread Martin Kletzander
On Sat, Dec 11, 2021 at 02:49:29PM +, Richard W.M. Jones wrote: Hi Martin, The pipeline failed, and I'm assuming this is because Fedora 33 is out of support. The log is not very helpful: That should not have anything related to it. https://gitlab.com/nbdkit/libnbd/-/jobs/1874368914

Re: [Libguestfs] libnbd | Failed pipeline for master | 1af77750

2021-11-01 Thread Martin Kletzander
It looks like there is a wrong API token set in for the group. I created a new one and updated it there to make sure they are in sync, but retrying the build could've used the old one, so I guess I need to hope for the next push to trigger a fresh rebuild, which should work. On Sun, Oct 31,

Re: [Libguestfs] Publishing the libnbd go module

2021-10-25 Thread Martin Kletzander
Couple of months back I was playing with GitHub Actions so that the libnbd-go is regenerated after each commit and the changes pushed into a separate repository, tagging it if tag was pushed as well. Unfortunately more pressing things came my way and I did not finish this work. Now I am making

[Libguestfs] [libnbd PATCH] ci: Adopt libvirt-ci's manifest approach

2021-09-23 Thread Martin Kletzander
This makes it a bit easier to write .gitlab-ci.yml as it inherits most of what can be generated from the ci/gitlab.yml and deprecates the ci/refresh script. Signed-off-by: Martin Kletzander --- .gitlab-ci.yml| 270 +--- ci/README.rst

[Libguestfs] [libnbd PATCH 0/3] CI and configure fixes for cross-builds

2021-09-22 Thread Martin Kletzander
Pipeline is green here: https://gitlab.com/nertpinx/libnbd/-/pipelines/375185133 although the patches it is built from are not cleaned up. However the tree is identical. Martin Kletzander (3): ci: Simplify and fix .gitlab.yml ci: Fix build script configure: Do not misuse enable_ocaml

[Libguestfs] [libnbd PATCH 3/3] configure: Do not misuse enable_ocaml

2021-09-22 Thread Martin Kletzander
That switch is documented to control the building of bindings, not running the generator. For that reason the conditional should disregard the value of enable_ocaml otherwise the generator cannot be run without building the bindings. Signed-off-by: Martin Kletzander --- configure.ac | 5

[Libguestfs] [libnbd PATCH 1/3] ci: Simplify and fix .gitlab.yml

2021-09-22 Thread Martin Kletzander
No need for .native_build_job, that is decided with the env var CROSS which is set when running the container. And cross builds should use the proper image name. Signed-off-by: Martin Kletzander --- .gitlab-ci.yml | 43 --- 1 file changed, 20 insertions

[Libguestfs] [libnbd PATCH 2/3] ci: Fix build script

2021-09-22 Thread Martin Kletzander
The test for cross builds was written wrong and we should explicitly disable building of OCaml bindings for cross-builds. Signed-off-by: Martin Kletzander --- ci/build.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/ci/build.sh b/ci/build.sh index fc43c3008952

Re: [Libguestfs] [PATCH nbdinfo v2 1/3] common/utils: Add function to convert sizes to human-readable

2021-09-20 Thread Martin Kletzander
ACK to these three. On Mon, Sep 20, 2021 at 12:04:46PM +0100, Richard W.M. Jones wrote: For example 1024 is returned as "1K". This does not attempt to handle decimals or SI units. If the number isn't some multiple of a power of 1024 then it is returned as bytes (a flag is available to

Re: [Libguestfs] [PATCH nbdinfo 1/2] common/utils: Add function to convert sizes to human-readable

2021-09-20 Thread Martin Kletzander
On Sat, Sep 18, 2021 at 03:38:52PM +0100, Richard W.M. Jones wrote: For example 1024 is returned as "1K". This does not attempt to handle decimals or SI units. If the number isn't some multiple of a power of 1024 then it is returned as bytes (a flag is available to indicate this). I looked at

Re: [Libguestfs] [libnbd PATCH] ci: Add i686 builds

2021-09-03 Thread Martin Kletzander
On Fri, Sep 03, 2021 at 10:19:32AM +0100, Richard W.M. Jones wrote: On Fri, Sep 03, 2021 at 11:05:02AM +0200, Martin Kletzander wrote: - ci/refresh: add generation for i686 connections - ci/build.sh: indentation fix and i686-related changes - ci/{containers,cirrus}: regenerate files - .gitlab

[Libguestfs] [libnbd PATCH] ci: Add i686 builds

2021-09-03 Thread Martin Kletzander
- ci/refresh: add generation for i686 connections - ci/build.sh: indentation fix and i686-related changes - ci/{containers,cirrus}: regenerate files - .gitlab-ci.yaml: add i686 builds Signed-off-by: Martin Kletzander --- The pipeline for this commit can be seen here: https://gitlab.com/nertpinx

Re: [Libguestfs] libnbd | Failed pipeline for master | 70da51e5

2021-09-03 Thread Martin Kletzander
On Thu, Sep 02, 2021 at 05:12:35PM +0100, Richard W.M. Jones wrote: Hi Martin, On Thu, Sep 02, 2021 at 04:05:56PM +, GitLab wrote: GitLab ✖ Pipeline #364204388 has failed! Project nbdkit / libnbd Branch● master Commit● 70da51e5 interop:

Re: [Libguestfs] Fwd: libnbd | Failed pipeline for master | 2e381ac2

2021-08-31 Thread Martin Kletzander
On Tue, Aug 31, 2021 at 09:02:52AM +0100, Richard W.M. Jones wrote: On Mon, Aug 30, 2021 at 10:29:09AM +0200, Martin Kletzander wrote: On Fri, Aug 27, 2021 at 11:16:25AM +0100, Daniel P. Berrangé wrote: >On Fri, Aug 27, 2021 at 11:09:13AM +0100, Richard W.M. Jones wrote: >>>

Re: [Libguestfs] Fwd: libnbd | Failed pipeline for master | 2e381ac2

2021-08-30 Thread Martin Kletzander
On Fri, Aug 27, 2021 at 11:16:25AM +0100, Daniel P. Berrangé wrote: On Fri, Aug 27, 2021 at 11:09:13AM +0100, Richard W.M. Jones wrote: >From the log: https://gitlab.com/nbdkit/libnbd/-/jobs/1540375264 opensuse zypper seems to be failing with a recoverable error: Resolving package

[Libguestfs] [libnbd PATCH] ci: Do not run valgrind checks

2021-08-23 Thread Martin Kletzander
Since valgrind checks find unrelated issues it spoils the CI effort and makes us ignore the output in the long run. If we want we can turn on valgrind checks for one or two particular platforms by setting "CHECK_VALGRIND" to "force". Signed-off-by: Martin Kletzander ---

Re: [Libguestfs] [nbdkit PATCH] tests: Check that cache-min-block-size works properly

2021-08-09 Thread Martin Kletzander
On Fri, Aug 06, 2021 at 01:19:23PM +0100, Richard W.M. Jones wrote: On Fri, Aug 06, 2021 at 02:07:37PM +0200, Martin Kletzander wrote: On Wed, Aug 04, 2021 at 04:48:08PM +0100, Richard W.M. Jones wrote: >On Wed, Aug 04, 2021 at 05:08:32PM +0200, Martin Kletzander wrote: >>Signed-off-b

[Libguestfs] [nbdkit PATCH v2] tests: Check that cache-min-block-size works properly

2021-08-09 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- tests/test-cache-block-size.sh | 41 +- 1 file changed, 26 insertions(+), 15 deletions(-) diff --git a/tests/test-cache-block-size.sh b/tests/test-cache-block-size.sh index d20cc94002b6..403c295e57c9 100755 --- a/tests/test

Re: [Libguestfs] [nbdkit PATCH] tests: Check that cache-min-block-size works properly

2021-08-06 Thread Martin Kletzander
On Wed, Aug 04, 2021 at 04:48:08PM +0100, Richard W.M. Jones wrote: On Wed, Aug 04, 2021 at 05:08:32PM +0200, Martin Kletzander wrote: Signed-off-by: Martin Kletzander --- tests/test-cache-block-size.sh | 33 ++--- 1 file changed, 22 insertions(+), 11 deletions

[Libguestfs] [nbdkit PATCH] tests: Check that cache-min-block-size works properly

2021-08-04 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- tests/test-cache-block-size.sh | 33 ++--- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/tests/test-cache-block-size.sh b/tests/test-cache-block-size.sh index d20cc94002b6..5e83ebc1cfaa 100755 --- a/tests/test

Re: [Libguestfs] [PATCH nbdkit 6/7] cache: Add cache-min-block-size parameter

2021-08-04 Thread Martin Kletzander
On Wed, Aug 04, 2021 at 11:07:32AM +0100, Richard W.M. Jones wrote: On Wed, Aug 04, 2021 at 10:47:34AM +0200, Martin Kletzander wrote: >+# Run nbdkit with the caching filter. >+start_nbdkit -P cache-block-size.pid -U $sock --filter=cache \ >+ file cache-block-size.img

Re: [Libguestfs] [PATCH nbdkit 0/7] Miscellaneous virt-v2v changes to cache and cow filters

2021-08-04 Thread Martin Kletzander
On Mon, Jul 26, 2021 at 03:56:38PM -0500, Eric Blake wrote: On Mon, Jul 26, 2021 at 06:28:53PM +0100, Richard W.M. Jones wrote: I've spent some time today trying to optimize modular virt-v2v a bit, and these are some suggested changes to nbdkit which should help. Series looks good to me.

Re: [Libguestfs] [PATCH nbdkit 6/7] cache: Add cache-min-block-size parameter

2021-08-04 Thread Martin Kletzander
On Mon, Jul 26, 2021 at 06:28:59PM +0100, Richard W.M. Jones wrote: This allows you to choose a larger block size. I found experimentally that this improves performance because of locality in access patterns. The idea came from qcow2 which implicitly does the same thing because of the

Re: [Libguestfs] [PATCH nbdkit 4/7] cache, cow: Use full pread/pwrite operations

2021-08-04 Thread Martin Kletzander
On Mon, Jul 26, 2021 at 06:28:57PM +0100, Richard W.M. Jones wrote: Although it probably cannot happen on Linux, POSIX allows pread/pwrite to return or write fewer bytes than requested. The cache and cow filters didn't handle this situation. Replace the raw pread(2)/pwrite(2) syscalls with

Re: [Libguestfs] [PATCH] point users to Libera Chat rather than FreeNode

2021-07-28 Thread Martin Kletzander
On Wed, Jul 28, 2021 at 09:47:49AM +0100, Daniel P. Berrangé wrote: On Thu, May 27, 2021 at 01:00:55PM +0100, Richard W.M. Jones wrote: On Thu, May 27, 2021 at 09:52:58AM +0100, Daniel P. Berrangé wrote: > Signed-off-by: Daniel P. Berrangé > --- > docs/guestfs-faq.pod | 2 +- >

[Libguestfs] [libnbd PATCH 3/3] Update CI files once more

2021-07-27 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- ci/cirrus/freebsd-12.vars| 4 ++-- ci/cirrus/freebsd-13.vars| 4 ++-- ci/cirrus/freebsd-current.vars | 4 ++-- ci/cirrus/macos-11.vars | 2 +- ci/containers/alpine-314.Dockerfile

[Libguestfs] [libnbd PATCH 0/3] ci: Few clean-ups

2021-07-27 Thread Martin Kletzander
Really just to make my head clear. Martin Kletzander (3): ci: Consolidate refresh scripts ci: Allow failure when building fedora rawhide container Update CI files once more .gitlab-ci.yml | 1 + ci/cirrus/freebsd-12.vars| 4 +-- ci

[Libguestfs] [libnbd PATCH 1/3] ci: Consolidate refresh scripts

2021-07-27 Thread Martin Kletzander
And make it cleaner. Signed-off-by: Martin Kletzander --- ci/cirrus/refresh | 22 -- ci/containers/refresh | 22 -- ci/refresh| 27 +++ 3 files changed, 27 insertions(+), 44 deletions(-) delete mode 100755 ci

[Libguestfs] [libnbd PATCH 2/3] ci: Allow failure when building fedora rawhide container

2021-07-27 Thread Martin Kletzander
Rawhide is now not being able to build and it is not because of our wrongdoing and libvirt switched the container building phase to allow failures, which I concur and hence have no reason for keeping it as it was. Signed-off-by: Martin Kletzander --- .gitlab-ci.yml | 1 + 1 file changed, 1

Re: [Libguestfs] [libnbd PATCH v2 2/2] macOS: Simple cloexec/nonblock fix

2021-07-27 Thread Martin Kletzander
On Tue, Jul 27, 2021 at 01:52:34PM +0100, Richard W.M. Jones wrote: On Tue, Jul 27, 2021 at 01:31:05PM +0200, Martin Kletzander wrote: This is the most trivial way to fix the issue with macOS not having SOCK_CLOEXEC and SOCK_NONBLOCK. This is the only way to make it work on such platform(s

Re: [Libguestfs] Some questions about nbdkit vs qemu performance affecting virt-v2v

2021-07-27 Thread Martin Kletzander
On Tue, Jul 27, 2021 at 12:16:59PM +0100, Richard W.M. Jones wrote: Hi Eric, a couple of questions below about nbdkit performance. Modular virt-v2v will use disk pipelines everywhere. The input pipeline looks something like this: socket <- cow filter <- cache filter <- nbdkit

[Libguestfs] [libnbd PATCH v2 0/2] Paint the pipeline complete green

2021-07-27 Thread Martin Kletzander
Last two fixes missing ;) Martin Kletzander (2): info: Require can_cache for info-can.sh macOS: Simple cloexec/nonblock fix lib/internal.h | 7 ++ generator/states-connect-socket-activation.c | 2 +- generator/states-connect.c | 11 +-- lib

[Libguestfs] [libnbd PATCH v2 1/2] info: Require can_cache for info-can.sh

2021-07-27 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- info/info-can.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/info/info-can.sh b/info/info-can.sh index 4154e38e85ba..00ba98107a29 100755 --- a/info/info-can.sh +++ b/info/info-can.sh @@ -22,7 +22,7 @@ set -e set -x requires nbdkit

[Libguestfs] [libnbd PATCH v2 2/2] macOS: Simple cloexec/nonblock fix

2021-07-27 Thread Martin Kletzander
This is the most trivial way to fix the issue with macOS not having SOCK_CLOEXEC and SOCK_NONBLOCK. This is the only way to make it work on such platform(s) unless they are fixed. Signed-off-by: Martin Kletzander Signed-off-by: Martin Kletzander --- lib/internal.h

Re: [Libguestfs] nbdkit: cow.c:591: cow_extents: Assertion `count > 0' failed.

2021-07-26 Thread Martin Kletzander
On Mon, Jul 26, 2021 at 01:45:19PM +0200, Martin Kletzander wrote: On Sat, Jul 24, 2021 at 11:40:49AM +0100, Richard W.M. Jones wrote: I wonder if this is an overflow: $ ./nbdkit sparse-random 3G --filter=cow --run 'nbdinfo --map $uri' # OK $ ./nbdkit sparse-random 4G --filter=cow --run

Re: [Libguestfs] nbdkit: cow.c:591: cow_extents: Assertion `count > 0' failed.

2021-07-26 Thread Martin Kletzander
On Sat, Jul 24, 2021 at 11:40:49AM +0100, Richard W.M. Jones wrote: I wonder if this is an overflow: $ ./nbdkit sparse-random 3G --filter=cow --run 'nbdinfo --map $uri' # OK $ ./nbdkit sparse-random 4G --filter=cow --run 'nbdinfo --map $uri' nbdkit: cow.c:591: cow_extents: Assertion `count >

Re: [Libguestfs] [libnbd PATCH 0/6] Paint the pipeline complete green, finally

2021-07-14 Thread Martin Kletzander
On Tue, Jul 13, 2021 at 11:26:02PM +0200, Martin Kletzander wrote: I hated the fact that it was not finished and it was (is) keeping me awake even though I do not really have tome to do this, but it is finally finished. We can even enable CI notifications to go public if that's something we'd

[Libguestfs] [libnbd PATCH 4/6] macOS: Do not use --version_script

2021-07-13 Thread Martin Kletzander
The linker does not support this option. Signed-off-by: Martin Kletzander --- configure.ac| 10 ++ lib/Makefile.am | 2 +- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index 9f6ba03a4773..18a66ee9114f 100644 --- a/configure.ac +++ b

[Libguestfs] [libnbd PATCH 5/6] Update CI files

2021-07-13 Thread Martin Kletzander
- removes valgrind from macOS but makes the package installation not fail because of its installation which cannot be done on macOS. - fixes python update from 3.7 to 3.8 on FreeBSDs and the followup renaming of packages. Signed-off-by: Martin Kletzander --- ci/cirrus/freebsd-12.vars

[Libguestfs] [libnbd PATCH 6/6] CI: Add testing on Alpine

2021-07-13 Thread Martin Kletzander
It is the favourite small container distribution and it just works. Signed-off-by: Martin Kletzander --- .gitlab-ci.yml | 23 +++ ci/containers/alpine-314.Dockerfile | 57 ci/containers/alpine-edge.Dockerfile | 57

[Libguestfs] [libnbd PATCH 2/6] One more VSOCK include fix

2021-07-13 Thread Martin Kletzander
This file was forgotten in commit e8ed016c34e1. Signed-off-by: Martin Kletzander --- lib/uri.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/uri.c b/lib/uri.c index bcecbad5005a..56f47376ed98 100644 --- a/lib/uri.c +++ b/lib/uri.c @@ -32,6 +32,8 @@ #ifdef HAVE_LINUX_VM_SOCKETS_H

[Libguestfs] [libnbd PATCH 1/6] info: Require nbdkit >= 1.14 for info-can.sh

2021-07-13 Thread Martin Kletzander
The can_cache flag was introduced in 1.13.4, so the tests might fail on an older one. To make the check easier, only run this test with nbdkit >= 1.14 Signed-off-by: Martin Kletzander --- info/info-can.sh | 4 1 file changed, 4 insertions(+) diff --git a/info/info-can.sh b/info/i

[Libguestfs] [libnbd PATCH 0/6] Paint the pipeline complete green, finally

2021-07-13 Thread Martin Kletzander
of my mind and hopefully move to CI for other projects ASAP. Night night ;) Martin Kletzander (6): info: Require nbdkit >= 1.14 for info-can.sh One more VSOCK include fix macOS: Simple cloexec/nonblock fix macOS: Do not use --version_script Update CI files CI: Add testing on Alp

[Libguestfs] [libnbd PATCH 3/6] macOS: Simple cloexec/nonblock fix

2021-07-13 Thread Martin Kletzander
This is the most trivial way to fix the issue with macOS not having SOCK_CLOEXEC and SOCK_NONBLOCK. There is not much better way, so this is the only way to make it work on such platform(s). Signed-off-by: Martin Kletzander --- lib/internal.h | 3 + generator

Re: [Libguestfs] Figuring out some failing tests for libnbd

2021-07-07 Thread Martin Kletzander
On Wed, Jul 07, 2021 at 12:01:51PM +0100, Richard W.M. Jones wrote: On Wed, Jul 07, 2021 at 12:23:28PM +0200, Martin Kletzander wrote: On Tue, Jul 06, 2021 at 09:06:13AM +0100, Richard W.M. Jones wrote: >On Wed, Jun 30, 2021 at 05:07:31PM +0100, Richard W.M. Jones wrote: >>https://g

Re: [Libguestfs] Figuring out some failing tests for libnbd

2021-07-07 Thread Martin Kletzander
On Tue, Jul 06, 2021 at 09:06:13AM +0100, Richard W.M. Jones wrote: On Wed, Jun 30, 2021 at 05:07:31PM +0100, Richard W.M. Jones wrote: https://gitlab.com/nertpinx/libnbd/-/jobs/1389193577 This has dozens of failures in the OCaml tests. Most but not all of them are like this: ==20650== 16

Re: [Libguestfs] Figuring out some failing tests for libnbd

2021-06-30 Thread Martin Kletzander
On Wed, Jun 30, 2021 at 05:11:45PM +0200, Martin Kletzander wrote: I am preparing more patches for CI to run check-valgrind and fix ongoing errors but there are two issues I can not identify the reason why they are failing. - On debian-10 the info/info-can.sh started failing and the error

[Libguestfs] [libnbd PATCH 0/3] CI updates

2021-06-30 Thread Martin Kletzander
Some fixes plus check-valgrind enablement. Martin Kletzander (3): qemu-storage-daemon 5.2.0 is still broken fuse: move check-valgrind out from condition ci: Rework the build script to run check-valgrind properly interop/interop-qemu-storage-daemon.sh | 3 +- .gitlab-ci.yml

[Libguestfs] [libnbd PATCH 1/3] qemu-storage-daemon 5.2.0 is still broken

2021-06-30 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- interop/interop-qemu-storage-daemon.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/interop/interop-qemu-storage-daemon.sh b/interop/interop-qemu-storage-daemon.sh index 16aafab9e65a..d3ae0320063f 100755 --- a/interop/interop-qemu

[Libguestfs] [libnbd PATCH 2/3] fuse: move check-valgrind out from condition

2021-06-30 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- fuse/Makefile.am | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fuse/Makefile.am b/fuse/Makefile.am index 8c66193402ea..d8691862ad40 100644 --- a/fuse/Makefile.am +++ b/fuse/Makefile.am @@ -95,7 +95,7 @@ TESTS += \ test

[Libguestfs] [libnbd PATCH 3/3] ci: Rework the build script to run check-valgrind properly

2021-06-30 Thread Martin Kletzander
. Also regenerate the container and cirrus files as the libnbd patches for libvirt-ci were finally pushed to master. Signed-off-by: Martin Kletzander --- .gitlab-ci.yml | 3 + ci/build.sh | 96 +++- ci/cirrus/freebsd-12

[Libguestfs] Figuring out some failing tests for libnbd

2021-06-30 Thread Martin Kletzander
I am preparing more patches for CI to run check-valgrind and fix ongoing errors but there are two issues I can not identify the reason why they are failing. - On debian-10 the info/info-can.sh started failing and the error message is just one of those I saw earlier in other places:

Re: [Libguestfs] [libnbd PATCH] ci: Enable python code style

2021-06-23 Thread Martin Kletzander
On Tue, Jun 22, 2021 at 10:36:54PM +0300, Nir Soffer wrote: On Tue, Jun 22, 2021 at 9:20 PM Martin Kletzander wrote: Signed-off-by: Martin Kletzander --- ci/build.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/ci/build.sh b/ci/build.sh index 7d62a84a5d4b..4ea3fec7d512 100755

Re: [Libguestfs] [libnbd PATCH] ci: Enable python code style

2021-06-23 Thread Martin Kletzander
On Wed, Jun 23, 2021 at 12:36:00PM +0100, Daniel P. Berrangé wrote: On Wed, Jun 23, 2021 at 12:31:14PM +0100, Richard W.M. Jones wrote: On Tue, Jun 22, 2021 at 03:38:13PM +0200, Martin Kletzander wrote: > Signed-off-by: Martin Kletzander > --- > ci/build.sh | 1 + > 1 fil

Re: [Libguestfs] [libnbd PATCH] nbdsh: Simplify the script

2021-06-23 Thread Martin Kletzander
On Wed, Jun 23, 2021 at 12:29:38PM +0100, Richard W.M. Jones wrote: On Tue, Jun 22, 2021 at 01:33:27PM +0200, Martin Kletzander wrote: There is no need for any hacks if we just do what execution of the module would have done. Signed-off-by: Martin Kletzander --- sh/nbdsh.in | 12

[Libguestfs] [libnbd PATCH] nbdsh: Simplify the script

2021-06-22 Thread Martin Kletzander
There is no need for any hacks if we just do what execution of the module would have done. Signed-off-by: Martin Kletzander --- sh/nbdsh.in | 12 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/sh/nbdsh.in b/sh/nbdsh.in index d10f0c1b6b26..f66e2918d304 100644 --- a/sh

[Libguestfs] [libnbd PATCH] ci: Enable python code style

2021-06-22 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- ci/build.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/ci/build.sh b/ci/build.sh index 7d62a84a5d4b..4ea3fec7d512 100755 --- a/ci/build.sh +++ b/ci/build.sh @@ -8,6 +8,7 @@ main() { autoreconf -if CONFIG_ARGS="\ +--enable-python

[Libguestfs] [libnbd PATCH v2] bash: Generate completion files during build

2021-06-22 Thread Martin Kletzander
At that point the dependencies are more relevant and we will also avoid shipping symlinks. Signed-off-by: Martin Kletzander --- bash/Makefile.am | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bash/Makefile.am b/bash/Makefile.am index 4245eca485a2..41d7b137baf2 100644

Re: [Libguestfs] virt-builder fails in fedora 34 container:

2021-06-14 Thread Martin Kletzander
On Tue, Jun 15, 2021 at 02:12:00AM +0300, Nir Soffer wrote: On Tue, Jun 15, 2021 at 1:26 AM Martin Kletzander wrote: [...] I looked at GitHub and saw this: https://github.com/oVirt/ovirt-imageio/tree/d163d80f04e5ae64f8e0138463aae5e867fd1fa8/docker if that is where the images are built

Re: [Libguestfs] virt-builder fails in fedora 34 container:

2021-06-14 Thread Martin Kletzander
On Tue, Jun 15, 2021 at 12:15:35AM +0300, Nir Soffer wrote: On Mon, Jun 14, 2021 at 11:50 PM Nir Soffer wrote: On Mon, Jun 14, 2021 at 7:47 PM Martin Kletzander wrote: > > On Mon, Jun 14, 2021 at 01:01:37PM +0100, Richard W.M. Jones wrote: > >On Mon, Jun 14, 2021 at 02:20:40P

Re: [Libguestfs] virt-builder fails in fedora 34 container:

2021-06-14 Thread Martin Kletzander
On Mon, Jun 14, 2021 at 01:01:37PM +0100, Richard W.M. Jones wrote: On Mon, Jun 14, 2021 at 02:20:40PM +0300, Nir Soffer wrote: On Mon, Jun 14, 2021 at 11:57 AM Martin Kletzander wrote: > I've got a tiny hunch, so I must ask. Is someone, by any stroke > of misfortune, using

Re: [Libguestfs] virt-builder fails in fedora 34 container:

2021-06-14 Thread Martin Kletzander
On Sun, Jun 13, 2021 at 11:15:31PM +0300, Nir Soffer wrote: On Sun, Jun 13, 2021 at 1:38 PM Richard W.M. Jones wrote: On Sat, Jun 12, 2021 at 11:11:22PM +0300, Nir Soffer wrote: > I'm running this inside a fedora 34 container in ovirt-imageio CI: > > 'virt-builder', '-v', '-x',

Re: [Libguestfs] [libnbd PATCH v2 23/23] ci: Update build files

2021-06-11 Thread Martin Kletzander
On Fri, Jun 11, 2021 at 09:31:12AM +0100, Richard W.M. Jones wrote: All good except patch 2 where I have questions. You might as well push the other patches, especially as I would like to do a 1.8.1 release soon. Thanks, I pushed it without the second patch, although it does mean that the

Re: [Libguestfs] [libnbd PATCH v2 02/23] bash: Generate completion files during build

2021-06-11 Thread Martin Kletzander
On Fri, Jun 11, 2021 at 09:27:22AM +0100, Richard W.M. Jones wrote: On Fri, Jun 11, 2021 at 01:12:06AM +0200, Martin Kletzander wrote: nbdcopy: nbdsh - rm -f $@ - $(LN_S) $(srcdir)/nbdsh $@ + $(LN_S) nbdsh $@ nbdfuse: nbdsh - rm -f $@ - $(LN_S) $(srcdir)/nbdsh

[Libguestfs] [libnbd PATCH v2 23/23] ci: Update build files

2021-06-10 Thread Martin Kletzander
The upstream patch for libvirt-ci is still not completed yet, but there are some improvements that need to be pulled in to clean up some of the builds. Signed-off-by: Martin Kletzander --- ci/cirrus/freebsd-12.vars| 5 ++--- ci/cirrus/freebsd-13.vars| 2

[Libguestfs] [libnbd PATCH v2 13/23] ci: Remove cross-compilation targets

2021-06-10 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- .gitlab-ci.yml| 203 -- .../debian-10-cross-aarch64.Dockerfile| 86 .../debian-10-cross-armv6l.Dockerfile | 86 .../debian-10-cross-armv7l.Dockerfile | 86

[Libguestfs] [libnbd PATCH v2 05/23] tests: Fix hexdump probing

2021-06-10 Thread Martin Kletzander
It does not support `--version` parameter (at least on some platforms). Signed-off-by: Martin Kletzander --- copy/copy-nbd-to-file.sh | 2 +- copy/copy-nbd-to-nbd.sh | 2 +- copy/copy-tls.sh | 2 +- fuse/test-pattern.sh | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff

[Libguestfs] [libnbd PATCH v2 04/23] interop: Do not test against broken qemu-storage-daemon

2021-06-10 Thread Martin Kletzander
The version 6.0.0 has very often a bug that makes the test fail, but which should be fixed in later versions. Version 5.1.0 has an issue as well. The version check is not the nicest way to check for this, but is enough. Signed-off-by: Martin Kletzander --- interop/interop-qemu-storage

[Libguestfs] [libnbd PATCH v2 07/23] tests: Enhance fuse probing

2021-06-10 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- tests/functions.sh.in | 10 ++ fuse/test-errors.sh | 2 +- fuse/test-file-mode.sh| 3 +-- fuse/test-nbdkit-command.sh | 2 +- fuse/test-nbdkit-file-null.sh | 2 +- fuse/test-nbdkit.sh | 3 +-- fuse/test

[Libguestfs] [libnbd PATCH v2 22/23] ci: Also perform `brew upgrade` on MacOS

2021-06-10 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- .gitlab-ci.yml | 2 ++ ci/cirrus/build.yml | 1 + 2 files changed, 3 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 2993ca987c4d..99896d471475 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -79,6 +79,7 @@ stages: -e &q

[Libguestfs] [libnbd PATCH v2 11/23] tests: Rename version to get-version

2021-06-10 Thread Martin Kletzander
version` binary name to `get-version` to avoid the issue. Signed-off-by: Martin Kletzander --- tests/Makefile.am | 12 ++-- tests/{version.c => get-version.c} | 0 .gitignore | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) rename

[Libguestfs] [libnbd PATCH v2 12/23] ci: Change docker driver and archive logs

2021-06-10 Thread Martin Kletzander
Logs are by default kept for 1 week, and when we output them directly, we sometimes exceed the limit for the job. This also makes the output cleaner. Docker driver overlayfs2 should be faster for docker dind builds. Signed-off-by: Martin Kletzander --- .gitlab-ci.yml | 30

[Libguestfs] [libnbd PATCH v2 09/23] tests: Require cap_sys_admin where root is required

2021-06-10 Thread Martin Kletzander
capability that is requested. Signed-off-by: Martin Kletzander --- tests/functions.sh.in | 29 +++ copy/copy-block-to-nbd.sh | 1 + copy/copy-nbd-to-block.sh | 1 + copy/copy-nbd-to-small-block-error.sh | 1 + 4 files changed, 32

[Libguestfs] [libnbd PATCH v2 10/23] tests: Do not use magic parameter for some nbdkit plugins

2021-06-10 Thread Martin Kletzander
The 'memory' and 'pattern' plugins gained support for that some time ago, however on some systems such nbdkit is not in the official repositories. This makes sure that the tests don't fail unnecessarily on such systems. Signed-off-by: Martin Kletzander --- copy/copy-block-to-nbd.sh

[Libguestfs] [libnbd PATCH v2 00/23] Many CI fixes

2021-06-10 Thread Martin Kletzander
completion fixed - no need to skip dist on opensuse - fixed check for qemu-storage-daemon version - also run "brew upgrade" on MacOS - incorporated reviews Martin Kletzander (23): Include sys/vsock.h if linux/vm_sockets.h is not available bash: Generate completion files during bu

[Libguestfs] [libnbd PATCH v2 15/23] ci: Add support for FreeBSD-13.0

2021-06-10 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- .gitlab-ci.yml| 10 ++ ci/cirrus/freebsd-13.vars | 13 + 2 files changed, 23 insertions(+) create mode 100644 ci/cirrus/freebsd-13.vars diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index ad73daa9c01f..4dd3d55791ca 100644

[Libguestfs] [libnbd PATCH v2 20/23] ci: Skip some broken tests/distro combinations

2021-06-10 Thread Martin Kletzander
This cleans up most of the mess in GitLab CI. Signed-off-by: Martin Kletzander --- ci/skipped_tests | 12 1 file changed, 12 insertions(+) create mode 100644 ci/skipped_tests diff --git a/ci/skipped_tests b/ci/skipped_tests new file mode 100644 index ..e2de9330baff

[Libguestfs] [libnbd PATCH v2 21/23] ci: Only run `make check` on Linux

2021-06-10 Thread Martin Kletzander
This is a temporary test suspension, it does not completely work on other systems as of now. Signed-off-by: Martin Kletzander --- ci/build.sh | 6 ++ 1 file changed, 6 insertions(+) diff --git a/ci/build.sh b/ci/build.sh index ed3646ea969e..7d62a84a5d4b 100755 --- a/ci/build.sh +++ b/ci

[Libguestfs] [libnbd PATCH v2 19/23] ci: Rename build_script.sh to just build.sh

2021-06-10 Thread Martin Kletzander
The "script" is kind of implied and a bother to write all the time when testing. Signed-off-by: Martin Kletzander --- .gitlab-ci.yml | 2 +- ci/README.rst| 4 ++-- ci/{build_script.sh => build.sh} | 0 ci/cirrus/build.yml |

[Libguestfs] [libnbd PATCH v2 16/23] ci: Fix run commands in README

2021-06-10 Thread Martin Kletzander
Mention how to run the script and keep the same UID, podman and docker use a little bit different arguments for that. While on it also fix the -v parameter for docker which, on some machines, only accepts absolute paths. Signed-off-by: Martin Kletzander --- ci/README.rst | 12 ++-- 1

[Libguestfs] [libnbd PATCH v2 14/23] ci: Remove debian-10-clang build

2021-06-10 Thread Martin Kletzander
Signed-off-by: Martin Kletzander --- .gitlab-ci.yml | 8 1 file changed, 8 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index cd2167c52ab8..ad73daa9c01f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -159,14 +159,6 @@ x64-debian-10: variables: NAME: debian-10

[Libguestfs] [libnbd PATCH v2 06/23] tests: Clean up qemu-nbd detection

2021-06-10 Thread Martin Kletzander
It was already different in copy/ and info/, so it is nicer to merge them together. Signed-off-by: Martin Kletzander --- tests/functions.sh.in | 19 +++ copy/copy-file-to-qcow2.sh | 17 +++-- info/Makefile.am | 1

[Libguestfs] [libnbd PATCH v2 18/23] ci: Add support for test skipping

2021-06-10 Thread Martin Kletzander
Some tests cannot be ran successfully and/or cannot properly probe for all their dependency configurations. This allows for skipping of individual test cases based on the OS and its version. Signed-off-by: Martin Kletzander --- ci/build_script.sh | 59

[Libguestfs] [libnbd PATCH v2 17/23] ci: Some build script fixes

2021-06-10 Thread Martin Kletzander
of a script we have the easier the adaptation to other projects could be. Signed-off-by: Martin Kletzander --- ci/build_script.sh | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/ci/build_script.sh b/ci/build_script.sh index 8e2cb5b986ca..1439df365cec 100755 --- a/ci

[Libguestfs] [libnbd PATCH v2 03/23] fuse: Only support defined fallocate modes

2021-06-10 Thread Martin Kletzander
None of them is defined on FreeBSD, for example, so by setting them to 0 they will be skipped as they are used as bit flags. Signed-off-by: Martin Kletzander --- fuse/operations.c | 8 1 file changed, 8 insertions(+) diff --git a/fuse/operations.c b/fuse/operations.c index

[Libguestfs] [libnbd PATCH v2 01/23] Include sys/vsock.h if linux/vm_sockets.h is not available

2021-06-10 Thread Martin Kletzander
This is the case on MacOS for example. The issue is that AF_VSOCK is defined in sys/socket.h, but struct sockaddr_vm is not, so the safeguard lower in the file does not work. Thankfully sys/vsock.h has the sockaddr_vm definition, so including that fixes the build issue. Signed-off-by: Martin

[Libguestfs] [libnbd PATCH v2 08/23] tests: Fix port randomisation

2021-06-10 Thread Martin Kletzander
The shell/environment variable `$RANDOM` is not defined POSIX, so let bash evaluate the whole expression by changing double quotes to single quotes. Signed-off-by: Martin Kletzander --- tests/Makefile.am | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/Makefile.am

[Libguestfs] [libnbd PATCH v2 02/23] bash: Generate completion files during build

2021-06-10 Thread Martin Kletzander
At that point the dependencies are more relevant and we will also avoid shipping symlinks. Signed-off-by: Martin Kletzander --- bash/Makefile.am | 16 +--- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/bash/Makefile.am b/bash/Makefile.am index 4245eca485a2

Re: [Libguestfs] distributing bash completion links [was: [libnbd PATCH 17/20] ci: Add support for test skipping]

2021-06-10 Thread Martin Kletzander
On Wed, Jun 09, 2021 at 06:17:39PM +0100, Richard W.M. Jones wrote: On Wed, Jun 09, 2021 at 11:00:26AM -0500, Eric Blake wrote: On Tue, Jun 08, 2021 at 09:09:32PM +0200, Martin Kletzander wrote: > I still have one more thing that bothers me, though. The symlinks in > bash/ a

Re: [Libguestfs] [libnbd PATCH 17/20] ci: Add support for test skipping

2021-06-08 Thread Martin Kletzander
On Tue, Jun 08, 2021 at 05:22:06PM +0100, Richard W.M. Jones wrote: On Tue, Jun 08, 2021 at 03:14:21PM +0200, Martin Kletzander wrote: On Tue, Jun 08, 2021 at 08:03:57AM -0500, Eric Blake wrote: >On Tue, Jun 08, 2021 at 09:53:58AM +0200, Martin Kletzander wrote: >>Some tests cann

  1   2   3   4   >