Re: [Libguestfs] libguestfs failure "mkdir: command not found ..."

2023-10-23 Thread Richard W.M. Jones
On Mon, Oct 23, 2023 at 03:53:37PM +, Maximilian Kirschner wrote:
> Is it possible that this error occcurs because my root partition is encryped
> with LUKS. If the virtual file system tries to mount the /root of my host
> system, this will not work.
> Is this the case? Does libguestfs mount the /root of the host system?
> And if so, do you know a workaround for systems with encrypted disks?

It's not to do with LUKS.  Just about everyone uses LUKS and
libguestfs works fine.  It's a bug in the Ubuntu package, try:

> 
> I'm afraid looks like however libguestfs was built, it is
> severly broken.  You might want to file an Ubuntu bug if it's
> the official package, or else try rebuilding the dpkg from
> source and see if that works.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] libguestfs failure "mkdir: command not found ..."

2023-10-23 Thread Richard W.M. Jones


On Mon, Oct 23, 2023 at 07:38:28AM +, Maximilian Kirschner wrote:
> Hello,
> 
> 
> I'm trying to run virt-make-fs, which fails.
> 
> I have installed libguestfs-tools via apt on a Ubuntu 22.04 system.

I'm afraid looks like however libguestfs was built, it is
severly broken.  You might want to file an Ubuntu bug if it's
the official package, or else try rebuilding the dpkg from
source and see if that works.

Rich.

> When I run libguestfs-test-tool I get the output attached at the end of the 
> mail. Can you please help me with this?
> 
> 
> Kind regards
> 
> Maximilian Kirschner
> 
> 
> libguestfs: trace: set_verbose true
> libguestfs: trace: set_verbose = 0
> libguestfs: trace: set_verbose true
> libguestfs: trace: set_verbose = 0
> LIBGUESTFS_TRACE=1
> LIBGUESTFS_DEBUG=1
> LIBVIRT_DEFAULT_URI=qemu:///system
> PATH=/home/kirschner/.modular/pkg/packages.modular.com_mojo/bin:/home/kirschner
> /.local/bin:/home/kirschner/.modular/pkg/packages.modular.com_mojo/bin:/usr/
> local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/
> games:/snap/bin:/home/kirschner/.local/share/coursier/bin
> XDG_RUNTIME_DIR=/run/user/1000
> SELinux: sh: 1: getenforce: not found
> libguestfs: trace: add_drive_scratch 104857600
> libguestfs: trace: get_tmpdir
> libguestfs: trace: get_tmpdir = "/tmp"
> libguestfs: trace: disk_create "/tmp/libguestfsX16XPh/scratch1.img" "raw"
> 104857600
> libguestfs: trace: disk_create = 0
> libguestfs: trace: add_drive "/tmp/libguestfsX16XPh/scratch1.img" "format:raw"
> "cachemode:unsafe"
> libguestfs: trace: add_drive = 0
> libguestfs: trace: add_drive_scratch = 0
> libguestfs: trace: get_append
> libguestfs: trace: get_append = "NULL"
> guestfs_get_append: (null)
> libguestfs: trace: get_autosync
> libguestfs: trace: get_autosync = 1
> guestfs_get_autosync: 1
> libguestfs: trace: get_backend
> libguestfs: trace: get_backend = "direct"
> guestfs_get_backend: direct
> libguestfs: trace: get_backend_settings
> libguestfs: trace: get_backend_settings = []
> guestfs_get_backend_settings: []
> libguestfs: trace: get_cachedir
> libguestfs: trace: get_cachedir = "/var/tmp"
> guestfs_get_cachedir: /var/tmp
> libguestfs: trace: get_hv
> libguestfs: trace: get_hv = "/usr/bin/qemu-system-x86_64"
> guestfs_get_hv: /usr/bin/qemu-system-x86_64
> libguestfs: trace: get_memsize
> libguestfs: trace: get_memsize = 1280
> guestfs_get_memsize: 1280
> libguestfs: trace: get_network
> libguestfs: trace: get_network = 0
> guestfs_get_network: 0
> libguestfs: trace: get_path
> libguestfs: trace: get_path = "/usr/lib/x86_64-linux-gnu/guestfs"
> guestfs_get_path: /usr/lib/x86_64-linux-gnu/guestfs
> libguestfs: trace: get_pgroup
> libguestfs: trace: get_pgroup = 0
> guestfs_get_pgroup: 0
> libguestfs: trace: get_program
> libguestfs: trace: get_program = "libguestfs-test-tool"
> guestfs_get_program: libguestfs-test-tool
> libguestfs: trace: get_recovery_proc
> libguestfs: trace: get_recovery_proc = 1
> guestfs_get_recovery_proc: 1
> libguestfs: trace: get_smp
> libguestfs: trace: get_smp = 1
> guestfs_get_smp: 1
> libguestfs: trace: get_sockdir
> libguestfs: trace: get_sockdir = "/run/user/1000"
> guestfs_get_sockdir: /run/user/1000
> libguestfs: trace: get_tmpdir
> libguestfs: trace: get_tmpdir = "/tmp"
> guestfs_get_tmpdir: /tmp
> libguestfs: trace: get_trace
> libguestfs: trace: get_trace = 1
> guestfs_get_trace: 1
> libguestfs: trace: get_verbose
> libguestfs: trace: get_verbose = 1
> guestfs_get_verbose: 1
> host_cpu: x86_64
> Launching appliance, timeout set to 600 seconds.
> libguestfs: trace: launch
> libguestfs: trace: max_disks
> libguestfs: trace: max_disks = 255
> libguestfs: trace: version
> libguestfs: trace: version =  release: 2, extra: , >
> libguestfs: trace: get_backend
> libguestfs: trace: get_backend = "direct"
> libguestfs: launch: program=libguestfs-test-tool
> libguestfs: launch: version=1.46.2
> libguestfs: launch: backend registered: unix
> libguestfs: launch: backend registered: uml
> libguestfs: launch: backend registered: libvirt
> libguestfs: launch: backend registered: direct
> libguestfs: launch: backend=direct
> libguestfs: launch: tmpdir=/tmp/libguestfsX16XPh
> libguestfs: launch: umask=0002
> libguestfs: launch: euid=1000
> libguestfs: trace: get_cachedir
> libguestfs: trace: get_cachedir = "/var/tmp"
> libguestfs: begin building supermin appliance
> libguestfs: run supermin
> libguestfs: command: run: /usr/bin/supermin
> libguestfs: command: run: \ --build
> libguestfs: command: run: \ --verbose
> libguestfs: command: run: \ --if-newer
> libguestfs: command: run: \ --lock /var/tmp/.guestfs-1000/lock
> libguestfs: command: run: \ --copy-kernel
> libguestfs: command: run: \ -f ext2
> libguestfs: command: run: \ --host-cpu x86_64
> libguestfs: command: run: \ /usr/lib/x86_64-linux-gnu/guestfs/supermin.d
> libguestfs: command: run: \ -o /var/tmp/.guestfs-1000/appliance.d
> supermin: version: 5.2.1
> supermin: package handler: debian/dpkg
> supermin: acquiring 

Re: [Libguestfs] [PATCH libnbd 0/4] Miscellaneous Rust cleanups

2023-10-18 Thread Richard W.M. Jones
On Wed, Oct 18, 2023 at 09:22:55AM -0500, Eric Blake wrote:
> On Tue, Oct 10, 2023 at 03:06:06PM +0100, Richard W.M. Jones wrote:
> > Add an overview libnbd-rust(3) man page pointing to the real
> > documentation.  This is like OCaml & Golang.
> > 
> > When reviewing the real rustdocs I noticed they basically converted
> > the man pages into plain text, resulting in lots of problems such as
> > internal links not working, no `code` annotations, etc.  So I wrote a
> > simple POD to rustdoc translator.  It is by no means perfect, but it
> > fixes many of the issues.
> > 
> > Also build the examples, to make sure we don't get any regressions.
> 
> For the series:
> Reviewed-by: Eric Blake 
> 
> There is an XXX about displying links rather than just italicized
> texts for pod L<> constructs linking to non-libnbd pages, which can be
> solved later, but your approach for now is definitely better than
> plain text.

Upstream in d4e8bb2d7d..8aa2f75cdf

The only change I made was to write better / longer commit messages.

Thanks,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH libnbd 0/4] Miscellaneous Rust cleanups

2023-10-18 Thread Richard W.M. Jones
On Wed, Oct 18, 2023 at 09:22:55AM -0500, Eric Blake wrote:
> On Tue, Oct 10, 2023 at 03:06:06PM +0100, Richard W.M. Jones wrote:
> > Add an overview libnbd-rust(3) man page pointing to the real
> > documentation.  This is like OCaml & Golang.
> > 
> > When reviewing the real rustdocs I noticed they basically converted
> > the man pages into plain text, resulting in lots of problems such as
> > internal links not working, no `code` annotations, etc.  So I wrote a
> > simple POD to rustdoc translator.  It is by no means perfect, but it
> > fixes many of the issues.
> > 
> > Also build the examples, to make sure we don't get any regressions.
> 
> For the series:
> Reviewed-by: Eric Blake 
> 
> There is an XXX about displying links rather than just italicized
> texts for pod L<> constructs linking to non-libnbd pages, which can be
> solved later, but your approach for now is definitely better than
> plain text.

Yeah I couldn't work out how that can be done, except to add an
explicit link to one of those man page sites (and then, which one?!)
I don't see existing Rust pages linking to man pages, unfortunately.

Thanks for the review.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] A note on migration of this list to lists.libguestfs.org

2023-10-18 Thread Richard W.M. Jones


We will soon be migrating this list over to a new hosting service at
lists.libguestfs.org.  All subscriptions will be moved automatically.
The main visible change will be that the email address of the list
will be different (since we will not longer be allowed to use the
@redhat.com address).

I'll have more details later.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Reported issues in nbdkit_parse_probability (was: Fwd: AUR Comment for nbdkit)

2023-10-14 Thread Richard W.M. Jones
On Sat, Oct 14, 2023 at 01:49:34PM +0100, Richard W.M. Jones wrote:
> https://aur.archlinux.org/packages/nbdkit#comment-938282

> FAIL: test-public
> =
> 
> Wrong return value for inf, got 0, expected -1
> Wrong result for inf, got inf, expected 0
> Wrong return value for nan, got 0, expected -1
> Wrong return value for nan/inf, got 0, expected -1
> Wrong return value for inf/inf, got 0, expected -1
> Wrong return value for nan/nan, got 0, expected -1
> Wrong return value for -0, got 0, expected -1
> Wrong return value for 1e200/1e-200, got 0, expected -1
> Wrong result for 1e200/1e-200, got inf, expected 0
> FAIL test-public (exit status: 1)

This turned out to be caused by using -Ofast or -ffast-math, and more
specifically the -ffinite-math-only option.  This option allows the
compiler to assume that all floating point operations have finite,
non-NaN parameters, thus functions like isfinite(3) in the standard
library are simply expected not to work.

As I said on the bug report the workaround is to use
-fno-finite-math-only, but better would be not to use the options
-Ofast/-ffast-math in the first place.

I don't think this is something we need to fix upstream, since by
using those options you're breaking the compiler, so in my view as the
user you get to keep the pieces.

Also there's not a reliable way to even detect this.  Using
-ffast-math defines "__FAST_MATH__", but apparently nothing is defined
for the specific option -ffinite-math-only.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] Reported issues in nbdkit_parse_probability (was: Fwd: AUR Comment for nbdkit)

2023-10-14 Thread Richard W.M. Jones
https://aur.archlinux.org/packages/nbdkit#comment-938282

- Forwarded message from not...@aur.archlinux.org -

Date: Sat, 14 Oct 2023 12:40:02 +
Subject: AUR Comment for nbdkit

dreieck [1] added the following comment to nbdkit [2]:

`check()` fails for me, too:

```
[...]
make  check-TESTS
make[3]: Entering directory
'/tmp/makepkg/build/nbdkit/src/nbdkit-1.36.0/server'
make[4]: Entering directory
'/tmp/makepkg/build/nbdkit/src/nbdkit-1.36.0/server'
FAIL: test-public

Testsuite summary for nbdkit 1.36.0

# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

See server/test-suite.log

```

`server/test-suite.log`:
```
==
   nbdkit 1.36.0: server/test-suite.log
==

# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: test-public
=

Wrong return value for inf, got 0, expected -1
Wrong result for inf, got inf, expected 0
Wrong return value for nan, got 0, expected -1
Wrong return value for nan/inf, got 0, expected -1
Wrong return value for inf/inf, got 0, expected -1
Wrong return value for nan/nan, got 0, expected -1
Wrong return value for -0, got 0, expected -1
Wrong return value for 1e200/1e-200, got 0, expected -1
Wrong result for 1e200/1e-200, got inf, expected 0
FAIL test-public (exit status: 1)
```

Regards!

--
If you no longer wish to receive notifications about this package,
please go to the package page [2] and select "Disable notifications".

[1] https://aur.archlinux.org/account/dreieck/
[2] https://aur.archlinux.org/pkgbase/nbdkit/

- End forwarded message -

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH libnbd 4/4] rust: Build the examples

2023-10-10 Thread Richard W.M. Jones
It's worth checking that the examples actually build, so we don't get
any regressions.
---
 rust/Makefile.am | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/rust/Makefile.am b/rust/Makefile.am
index 5c73512c87..a7700d69bd 100644
--- a/rust/Makefile.am
+++ b/rust/Makefile.am
@@ -90,7 +90,8 @@ EXTRA_DIST = \
 if HAVE_RUST
 
 all-local: libnbd-sys/libnbd_version target/debug/liblibnbd.rlib \
-   target/doc/libnbd/index.html
+   target/doc/libnbd/index.html \
+   target/debug/examples/get-size
 
 libnbd-sys/libnbd_version: Makefile
rm -f libnbd-sys/libnbd_version.t
@@ -103,6 +104,10 @@ target/debug/liblibnbd.rlib: $(source_files)
 target/doc/libnbd/index.html: $(source_files)
$(abs_top_builddir)/run $(CARGO) doc
 
+# This will actually build all the examples:
+target/debug/examples/get-size: $(source_files)
+   $(abs_top_builddir)/run $(CARGO) build --examples
+
 if HAVE_POD
 
 man_MANS = libnbd-rust.3
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH libnbd 3/4] rust: Write a custom translator from POD to rustdoc

2023-10-10 Thread Richard W.M. Jones
For more faithful translation of API documentation, write a custom
translator from POD to rustdoc.
---
 generator/Rust.ml | 99 ++-
 1 file changed, 80 insertions(+), 19 deletions(-)

diff --git a/generator/Rust.ml b/generator/Rust.ml
index 302bd4c48b..fd703f0493 100644
--- a/generator/Rust.ml
+++ b/generator/Rust.ml
@@ -476,27 +476,88 @@ let
   pr "\n"
 
 (* Print the comment for a rust function for a handle call. *)
-let print_rust_handle_call_comment call =
+let rec print_rust_handle_call_comment name call =
   (* Print comments. *)
   if call.shortdesc <> "" then
 pr "/// %s\n"
   (String.concat "\n/// " (String.split_on_char '\n' call.shortdesc));
   if call.longdesc <> "" then (
-(* If a short comment was printed, print a blank comment line befor the
-   long description. *)
+(* If a short comment was printed, print a blank comment line before
+   the long description. *)
 if call.shortdesc <> "" then pr "/// \n";
-(* Print all lines of the long description. Since Rust comments are
-   supposed to be Markdown, all indented lines will be treated as code
-   blocks. Hence we trim all lines. Also brackets ("[" and "]") must be
-   escaped. *)
-List.iter
-  (fun line ->
-let unindented = String.trim line in
-let escaped =
-  Str.global_replace (Str.regexp {|\(\[\|\]\)|}) {|\\\1|} unindented
-in
-pr "/// %s\n" escaped)
-  (pod2text call.longdesc))
+let md = longdesc_to_markdown name call.longdesc in
+List.iter (pr "/// %s\n") md
+  )
+
+(* Convert POD to rustdoc markdown. *)
+and longdesc_to_markdown name longdesc =
+  (* Replace any POD <> expression *)
+  let content =
+Str.global_substitute (Str.regexp {|[A-Z]<[^>]+?>|})
+  (fun s ->
+let expr = Str.matched_string s in
+let len = String.length expr in
+let c = expr.[0] and content = String.sub expr 2 (len-3) in
+match c with
+| 'C' -> sprintf "`%s`" content (* C<...> becomes `...` *)
+| 'B' -> sprintf "%s" content
+| 'I' | 'F' -> sprintf "%s" content
+| 'E' -> sprintf "&%s;" content
+| 'L' ->
+   let len = String.length content in
+   if String.starts_with ~prefix:"nbd_" content then (
+ let n = String.sub content 4 (len - 7) in
+ if n <> "get_error" && n <> "get_errno" && n <> "close" then
+   sprintf "[%s](Handle::%s)" n n
+ else
+   sprintf "`%s`" n
+   )
+   else (* external manual page - how to link XXX *)
+ sprintf "%s" content
+| _ ->
+   failwithf "rust: API documentation for %s contains '%s' which
+  cannot be converted to Rust markdown" name expr
+  )
+  longdesc in
+
+  (* Split input into lines for rest of the processing. *)
+  let lines = nsplit "\n" content in
+
+  (* Surround any group of lines starting with whitespace with ```text *)
+  let lines =
+List.map (fun line -> String.starts_with ~prefix:" " line, line) lines in
+  let (lines : (bool * string list) list) = group_by lines in
+  let lines =
+List.map (function
+  | true (* verbatim *), lines -> [ "```text" ] @ lines @ [ "```" ]
+  | false, lines -> lines
+) lines in
+  let lines = List.flatten lines in
+
+  (* Replace any = directives *)
+  filter_map (
+fun s ->
+  (* This is a very approximate way to translate bullet lists. *)
+  if String.starts_with ~prefix:"=over" s ||
+ String.starts_with ~prefix:"=back" s then
+None
+  else if String.starts_with ~prefix:"=item" s then (
+let len = String.length s in
+let s' = String.sub s 5 (len-5) in
+Some ("-" ^ s')
+  )
+  else if String.starts_with ~prefix:"=head" s then (
+let i = int_of_string (String.make 1 s.[5]) in
+let len = String.length s in
+let s' = String.sub s 6 (len-6) in
+Some (String.make i '#' ^ s')
+  )
+  else if String.starts_with ~prefix:"=" s then
+failwithf "rust: API documentation for %s contains '%s' which
+   cannot be converted to Rust markdown" name s
+  else
+Some s
+  ) lines
 
 (* Print a Rust expression which converts Rust like arguments to FFI like
arguments, makes a call on the raw FFI handle, and converts the return
@@ -533,7 +594,7 @@ let
 String.concat ", "
   (List.map2 (sprintf "%s: %s") rust_args_names rust_args_types)
   in
-  print_rust_handle_call_comment call;
+  print_rust_handle_call_comment name call;
   (* Print visibility modifier. *)
   if NameSet.mem name hidden_handle_calls then (
 (* If this is hidden to the public API, it might be used only if some 
feature
@@ -653,7 +714,7 @@ let
 
 (* Print the Rust function for a synchronous handle call. *)
 let print_rust_sync_handle_call name call =
-  print_rust_handle_call_comment call;
+  

[Libguestfs] [PATCH libnbd 2/4] rust: Add overview documentation

2023-10-10 Thread Richard W.M. Jones
---
 .gitignore   |  1 +
 docs/libnbd.pod  |  4 
 rust/Makefile.am | 13 +++
 rust/libnbd-rust.pod | 53 
 4 files changed, 71 insertions(+)

diff --git a/.gitignore b/.gitignore
index 36bf8b60f8..0b1cf7646a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -180,6 +180,7 @@ Makefile.in
 /python/run-python-tests
 /run
 /rust/Cargo.lock
+/rust/libnbd-rust.3
 /rust/libnbd-sys/Cargo.lock
 /rust/libnbd-sys/libnbd_version
 /rust/libnbd-sys/src/generated.rs
diff --git a/docs/libnbd.pod b/docs/libnbd.pod
index f15ea6403d..2fc78212a5 100644
--- a/docs/libnbd.pod
+++ b/docs/libnbd.pod
@@ -51,6 +51,10 @@ Using the API from OCaml.
 
 Using the API from Go.
 
+=item L
+
+Using the API from Rust.
+
 =item L
 
 Using the NBD shell (nbdsh) for command line and Python scripting.
diff --git a/rust/Makefile.am b/rust/Makefile.am
index 75738a0c30..5c73512c87 100644
--- a/rust/Makefile.am
+++ b/rust/Makefile.am
@@ -84,6 +84,7 @@ source_files = \
 
 EXTRA_DIST = \
$(source_files) \
+   libnbd-rust.pod \
$(NULL)
 
 if HAVE_RUST
@@ -102,6 +103,18 @@ target/debug/liblibnbd.rlib: $(source_files)
 target/doc/libnbd/index.html: $(source_files)
$(abs_top_builddir)/run $(CARGO) doc
 
+if HAVE_POD
+
+man_MANS = libnbd-rust.3
+CLEANFILES += $(man_MANS)
+
+libnbd-rust.3: libnbd-rust.pod $(top_builddir)/podwrapper.pl
+   $(PODWRAPPER) --section=3 --man $@ \
+   --html $(top_builddir)/html/$@.html \
+   $<
+
+endif HAVE_POD
+
 TESTS_ENVIRONMENT = \
LIBNBD_DEBUG=1 \
$(MALLOC_CHECKS) \
diff --git a/rust/libnbd-rust.pod b/rust/libnbd-rust.pod
new file mode 100644
index 00..622f0b4ac5
--- /dev/null
+++ b/rust/libnbd-rust.pod
@@ -0,0 +1,53 @@
+=head1 NAME
+
+libnbd-rust - how to use libnbd from Rust
+
+=head1 SYNOPSIS
+
+ let nbd = libnbd::Handle::new().unwrap();
+ nbd.connect_uri("nbd://localhost").unwrap();
+ let size = nbd.get_size().unwrap();
+ println!("{size} bytes");
+
+In C add:
+
+ [dependencies]
+ libnbd = VERSION | { path = "libnbd/rust" }
+
+=head1 DESCRIPTION
+
+This manual page documents how to use libnbd to access Network Block
+Device (NBD) servers from the Rust programming language.
+
+The Rust bindings work very similarly to the C bindings so you should
+start by reading L.
+
+There is also a higher level asynchronous API using Tokio.
+
+If you build libnbd from source, the main documentation can be found
+in F
+
+For the ordinary interface, start by reading the documentation for
+C.  For the higher level asynchronous API, start by reading
+C.
+
+C is a very low level wrapper around the libnbd API which
+should not be used directly.
+
+=head1 EXAMPLES
+
+This directory contains examples written in Rust:
+
+L
+
+=head1 SEE ALSO
+
+L.
+
+=head1 AUTHORS
+
+Tage Johansson
+
+=head1 COPYRIGHT
+
+Copyright Tage Johansson
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH libnbd 1/4] rust: Annotate 'endif' with corresponding label

2023-10-10 Thread Richard W.M. Jones
Although not required, it makes automake 'Makefile.am's easier to read.
---
 rust/Makefile.am | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/rust/Makefile.am b/rust/Makefile.am
index 027097af9b..75738a0c30 100644
--- a/rust/Makefile.am
+++ b/rust/Makefile.am
@@ -117,6 +117,6 @@ clean-local:
$(CARGO) clean
$(CARGO) clean --manifest-path cargo_test/Cargo.toml
 
-endif
+endif HAVE_RUST
 
 CLEANFILES += libnbd-sys/libnbd_version
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH libnbd 0/4] Miscellaneous Rust cleanups

2023-10-10 Thread Richard W.M. Jones
Add an overview libnbd-rust(3) man page pointing to the real
documentation.  This is like OCaml & Golang.

When reviewing the real rustdocs I noticed they basically converted
the man pages into plain text, resulting in lots of problems such as
internal links not working, no `code` annotations, etc.  So I wrote a
simple POD to rustdoc translator.  It is by no means perfect, but it
fixes many of the issues.

Also build the examples, to make sure we don't get any regressions.

Rich.


___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [nbdkit] Two POSIX questions for you ...

2023-10-10 Thread Richard W.M. Jones
On Tue, Oct 10, 2023 at 09:05:43AM +0200, Laszlo Ersek wrote:
> On 10/9/23 16:51, Richard W.M. Jones wrote:
> > 
> > So one thing we could do for this test is to require (for the test)
> > that /bin/sh is bash, something like the patch below.  I was only able
> > to test this in the positive case.
> > 
> > diff --git a/tests/test-log-script-info.sh b/tests/test-log-script-info.sh
> > index fa9b2ed32..d65f6415d 100755
> > --- a/tests/test-log-script-info.sh
> > +++ b/tests/test-log-script-info.sh
> > @@ -38,6 +38,10 @@ requires_run
> >  requires_nbdinfo
> >  requires_filter log
> >  
> > +# This test implicitly assumes /bin/sh is bash, see:
> > +# https://listman.redhat.com/archives/libguestfs/2023-October/032767.html
> > +BASH_VERSION=no requires /bin/sh -c 'test "x$BASH_VERSION" != "xno"'
> > +
> >  if ! nbdinfo --help | grep -- --map ; then
> >  echo "$0: nbdinfo --map option required to run this test"
> >  exit 77
> > 
> > 
> 
> Clever!
> 
> Reviewed-by: Laszlo Ersek 
> 
> You can test it with "dash" BTW (it must be available in Fedora, because
> it is available in EPEL-9):
> 
> BASH_VERSION=no /bin/bash -c 'test "x$BASH_VERSION" != "xno"; echo $?'
> BASH_VERSION=no /bin/dash -c 'test "x$BASH_VERSION" != "xno"; echo $?'

Thanks - after testing with dash as you suggested, I pushed this
change which is similar in spirit:

https://gitlab.com/nbdkit/nbdkit/-/commit/99712c09835ae430912aed4848579fdfab01a576

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [nbdkit] Two POSIX questions for you ...

2023-10-09 Thread Richard W.M. Jones


So one thing we could do for this test is to require (for the test)
that /bin/sh is bash, something like the patch below.  I was only able
to test this in the positive case.

diff --git a/tests/test-log-script-info.sh b/tests/test-log-script-info.sh
index fa9b2ed32..d65f6415d 100755
--- a/tests/test-log-script-info.sh
+++ b/tests/test-log-script-info.sh
@@ -38,6 +38,10 @@ requires_run
 requires_nbdinfo
 requires_filter log
 
+# This test implicitly assumes /bin/sh is bash, see:
+# https://listman.redhat.com/archives/libguestfs/2023-October/032767.html
+BASH_VERSION=no requires /bin/sh -c 'test "x$BASH_VERSION" != "xno"'
+
 if ! nbdinfo --help | grep -- --map ; then
 echo "$0: nbdinfo --map option required to run this test"
 exit 77


-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [nbdkit] Two POSIX questions for you ...

2023-10-09 Thread Richard W.M. Jones
On Mon, Oct 09, 2023 at 01:32:07PM +0200, Laszlo Ersek wrote:
> On 10/9/23 13:12, Richard W.M. Jones wrote:
> > On Mon, Oct 09, 2023 at 12:57:14PM +0200, Laszlo Ersek wrote:
> >> On 10/9/23 09:46, Richard W.M. Jones wrote:
> >>> Hi Eric, a couple of POSIX questions for you from nbdkit.
> >>>
> >>> The first question is from an AUR comment on nbdkit:
> >>>
> >>>   https://aur.archlinux.org/packages/nbdkit#comment-937381
> >>>
> >>> I think there's a bash-ism in the logscript parameter in this test:
> >>>
> >>>   
> >>> https://gitlab.com/nbdkit/nbdkit/-/blame/master/tests/test-log-script-info.sh#L51
> >>>
> >>> I believe it is happening in the $(( .. )) expression.  How do we
> >>> write that so it'll work in a posix shell?
> >>
> >> (I'm not Eric, but curious! :) )
> >>
> >> Here I think we should just explicitly insist on bash.
> > 
> > Unfortunately we can't do that, not easily anyway.
> > 
> > We use bash for writing the test scripts because it's the sane choice
> > for shell programming, and we declare /bin/bash (or something with
> > 'env') at the top of each of those scripts.  This introduces a *build*
> > dependency from nbdkit to /bin/bash.  That's all fine.
> > 
> > However when nbdkit is installed in production & runs an external
> > script, it uses system(3) which uses /bin/sh.  That might not be bash,
> > and indeed bash might not even be installed on the same machine as
> > nbdkit.
> > 
> > [Note: I think the whole Debian dash-for-/bin/sh is the stupidest idea
> > I ever heard, but (a) it's a thing and (b) there's BSD and Macs where
> > they don't want to use bash for licensing reasons.]
> > 
> > Arguably for the tests we could have some way to cause nbdkit to use
> > /bin/bash instead of /bin/sh when running external scripts, but it's
> > major complexity to implement.
> > 
> > So we gotta use a lowest common denominator for these external
> > scripts, even in our tests.  Note only this one test is affected.
> > 
> > [Aside: Windows is a whole separate kettle of fish.  Currently the
> > Windows port of nbdkit doesn't support --run for other reasons, but if
> > it did it would probably run some Windows-ish thing (COMMAND.COM?
> > Power Shell?).  I'm not sure how Windows behaves for the other
> > external commands we try to run like logscript.]
> > 
> >> Shell arrays are
> >> a bash-specific feature, and the extent array is deeply ingrained. See
> >> especially commit df63b23b6280 ("log: Use strict shell quoting for every
> >> parameter displayed in the log file.", 2021-01-04). In particular the
> >> assignment
> >>
> >> extents=(0x0 0x8000 "hole,zero" 0x8000 0x8000 "")
> >>
> >> turns "extents" into an array variable.
> >>
> >> In the bug tracker, comment
> >> <https://aur.archlinux.org/packages/nbdkit#comment-937375> says, "Thus
> >> this is an upstream issue; their scripts are calling sh when they should
> >> be calling bash". I think that's correct; for logscript=..., we should
> >> require /bin/bash in the manual, and execute the script with /bin/bash
> >> explicitly, not just system().
> > 
> > This would create a runtime dependency from nbdkit to /bin/bash which
> > I'd like to avoid.
> 
> The runtime dependency is already there in our logscript interface; the
> 
>   name=(a b c ... z)
> 
> syntax is already bash-only, for defining a shell array.

Hmm, the extents are indeed passed as a flattened array, ie.

  extents=(  "" [repeats ...])

I don't think I was aware this wasn't in POSIX.

However that's a dependency from _nbdkit-log-filter_ to bash, which is
slightly different.  (Nevertheless it'd be nice if we could remove it ...)

> So the question is basically how to best emulate an array in the POSIX
> shell. Some (rough) options that occur to me:
> 
> - Use named variables such as name_0, name_1, name_2, ... and so on.
> Requires eval tricks, and if the array is large, it creates many
> variables, which some shells (?) may have issues with.
> 
> - Generate a shell function with a huge case statement; like "get_name
> 0" should print "a", "get_name 1" should print "b", etc. The caller
> would then do
> 
>   element=$(get_name $idx)
> 
> - write the elements of the array to a text file (one, quoted, element
> per line), and then use a combination of "tail" and &q

Re: [Libguestfs] [nbdkit] Two POSIX questions for you ...

2023-10-09 Thread Richard W.M. Jones
On Mon, Oct 09, 2023 at 12:57:14PM +0200, Laszlo Ersek wrote:
> On 10/9/23 09:46, Richard W.M. Jones wrote:
> > Hi Eric, a couple of POSIX questions for you from nbdkit.
> > 
> > The first question is from an AUR comment on nbdkit:
> > 
> >   https://aur.archlinux.org/packages/nbdkit#comment-937381
> > 
> > I think there's a bash-ism in the logscript parameter in this test:
> > 
> >   
> > https://gitlab.com/nbdkit/nbdkit/-/blame/master/tests/test-log-script-info.sh#L51
> > 
> > I believe it is happening in the $(( .. )) expression.  How do we
> > write that so it'll work in a posix shell?
> 
> (I'm not Eric, but curious! :) )
> 
> Here I think we should just explicitly insist on bash.

Unfortunately we can't do that, not easily anyway.

We use bash for writing the test scripts because it's the sane choice
for shell programming, and we declare /bin/bash (or something with
'env') at the top of each of those scripts.  This introduces a *build*
dependency from nbdkit to /bin/bash.  That's all fine.

However when nbdkit is installed in production & runs an external
script, it uses system(3) which uses /bin/sh.  That might not be bash,
and indeed bash might not even be installed on the same machine as
nbdkit.

[Note: I think the whole Debian dash-for-/bin/sh is the stupidest idea
I ever heard, but (a) it's a thing and (b) there's BSD and Macs where
they don't want to use bash for licensing reasons.]

Arguably for the tests we could have some way to cause nbdkit to use
/bin/bash instead of /bin/sh when running external scripts, but it's
major complexity to implement.

So we gotta use a lowest common denominator for these external
scripts, even in our tests.  Note only this one test is affected.

[Aside: Windows is a whole separate kettle of fish.  Currently the
Windows port of nbdkit doesn't support --run for other reasons, but if
it did it would probably run some Windows-ish thing (COMMAND.COM?
Power Shell?).  I'm not sure how Windows behaves for the other
external commands we try to run like logscript.]

> Shell arrays are
> a bash-specific feature, and the extent array is deeply ingrained. See
> especially commit df63b23b6280 ("log: Use strict shell quoting for every
> parameter displayed in the log file.", 2021-01-04). In particular the
> assignment
> 
> extents=(0x0 0x8000 "hole,zero" 0x8000 0x8000 "")
> 
> turns "extents" into an array variable.
> 
> In the bug tracker, comment
> <https://aur.archlinux.org/packages/nbdkit#comment-937375> says, "Thus
> this is an upstream issue; their scripts are calling sh when they should
> be calling bash". I think that's correct; for logscript=..., we should
> require /bin/bash in the manual, and execute the script with /bin/bash
> explicitly, not just system().

This would create a runtime dependency from nbdkit to /bin/bash which
I'd like to avoid.

> > Secondly while looking into this I was trying variations on:
> > 
> >   $ POSIXLY_CORRECT=1 ./nbdkit -fv data '1 2 3' --run 'nbdinfo $uri'
> > 
> > This doesn't actually cause bash to emulate a posix shell, but it does
> > uncover a different bug:
> > 
> >   nbdkit: error: raw|base64|data parameter must be specified exactly once
> > 
> > This seems to be happening because getopt_long in wrapper.c behaves
> > somehow differently parsing when POSIXLY_CORRECT is set.  However I
> > couldn't work out exactly why.
> > 
> >   
> > https://gitlab.com/nbdkit/nbdkit/-/blob/master/wrapper.c?ref_type=heads#L278
> > 
> > I guess the wrapper ought to work if POSIXLY_CORRECT is set (?)
> 
> IIRC, POSIXLY_CORRECT makes getopt() stop parsing options when the first
> non-option argument (i.e., first operand) is reached. So "-f" and "-v"
> are taken as options, then the two arguments "data" and '1 2 3' are
> taken as operands,  and then "--run" and the rest are taken as operands
> as well.
> 
> I think the following loop is relevant:
> 
>   /* Are there any non-option arguments? */
>   if (optind < argc) {
> /* Ensure any further parameters can never be parsed as options by
>  * real nbdkit.
>  */
> passthru ("--");
> 
> /* The first non-option argument is the plugin name.  If it is a
>  * short name then rewrite it.
>  */
> if (is_short_name (argv[optind])) {
>   const char *language;
> 
>   /* Plugins written in scripting languages. */
>   if (is_script_plugin (argv[optind], )) {
> passthru_format ("%s/plugins/%s/.libs/nbdkit-%s-plugin." SOEXT,
>  builddir, language, language);
> passthru_format ("%s/

[Libguestfs] [nbdkit] Two POSIX questions for you ...

2023-10-09 Thread Richard W.M. Jones
Hi Eric, a couple of POSIX questions for you from nbdkit.

The first question is from an AUR comment on nbdkit:

  https://aur.archlinux.org/packages/nbdkit#comment-937381

I think there's a bash-ism in the logscript parameter in this test:

  
https://gitlab.com/nbdkit/nbdkit/-/blame/master/tests/test-log-script-info.sh#L51

I believe it is happening in the $(( .. )) expression.  How do we
write that so it'll work in a posix shell?

 - - -

Secondly while looking into this I was trying variations on:

  $ POSIXLY_CORRECT=1 ./nbdkit -fv data '1 2 3' --run 'nbdinfo $uri'

This doesn't actually cause bash to emulate a posix shell, but it does
uncover a different bug:

  nbdkit: error: raw|base64|data parameter must be specified exactly once

This seems to be happening because getopt_long in wrapper.c behaves
somehow differently parsing when POSIXLY_CORRECT is set.  However I
couldn't work out exactly why.

  https://gitlab.com/nbdkit/nbdkit/-/blob/master/wrapper.c?ref_type=heads#L278

I guess the wrapper ought to work if POSIXLY_CORRECT is set (?)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [libnbd PATCH 1/2] utils: Slightly simplify human_size()

2023-10-07 Thread Richard W.M. Jones
On Fri, Oct 06, 2023 at 10:18:08AM -0500, Eric Blake wrote:
> Use an array of characters instead of strings for less .data storage.
> Merge the loop conditional for fewer lines of code.
> 
> Signed-off-by: Eric Blake 
> ---
>  common/include/human-size.h | 14 ++
>  1 file changed, 6 insertions(+), 8 deletions(-)
> 
> diff --git a/common/include/human-size.h b/common/include/human-size.h
> index 47729c3c..8b1e0132 100644
> --- a/common/include/human-size.h
> +++ b/common/include/human-size.h
> @@ -159,7 +159,7 @@ human_size_parse (const char *str,
>  static inline char *
>  human_size (char *buf, uint64_t bytes, bool *human)
>  {
> -  static const char ext[][2] = { "E", "P", "T", "G", "M", "K", "" };
> +  static const char ext[] = "EPTGMK";
>size_t i;
> 
>if (buf == NULL) {
> @@ -170,18 +170,16 @@ human_size (char *buf, uint64_t bytes, bool *human)
> 
>/* Work out which extension to use, if any. */
>i = 6;
> -  if (bytes != 0) {
> -while ((bytes & 1023) == 0) {
> -  bytes >>= 10;
> -  i--;
> -}
> +  while (bytes && (bytes & 1023) == 0) {
> +bytes >>= 10;
> +i--;
>}
> 
>/* Set the flag to true if we're going to add a human-readable extension. 
> */
>if (human)
> -*human = ext[i][0] != '\0';
> +    *human = ext[i] != '\0';
> 
> -  snprintf (buf, HUMAN_SIZE_LONGEST, "%" PRIu64 "%s", bytes, ext[i]);
> +  snprintf (buf, HUMAN_SIZE_LONGEST, "%" PRIu64 "%.1s", bytes, [i]);
>return buf;
>  }

Reviewed-by: Richard W.M. Jones 

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [libnbd PATCH 2/2] info: Show human sizes for block_size values

2023-10-07 Thread Richard W.M. Jones
On Fri, Oct 06, 2023 at 10:18:09AM -0500, Eric Blake wrote:
> Adding a human-readable size for block constraints is useful.  For:
> 
> $ ./run nbdinfo -- [ nbdkit memory \
>--filter=blocksize-policy blocksize-preferred=32k 1M ] | grep pref
> 
> this changes pre-patch:
>   block_size_preferred: 32768
> to post-patch:
>   block_size_preferred: 32768 (32K)

I think info/nbdinfo.pod needs to be updated.

Rich.

> Signed-off-by: Eric Blake 
> ---
>  info/show.c | 26 +++---
>  1 file changed, 19 insertions(+), 7 deletions(-)
> 
> diff --git a/info/show.c b/info/show.c
> index 6aeffb54..ac483f34 100644
> --- a/info/show.c
> +++ b/info/show.c
> @@ -35,6 +35,7 @@
>  #include "nbdinfo.h"
> 
>  static void show_boolean (const char *name, bool cond);
> +static void show_size (const char *name, int64_t size);
>  static int collect_context (void *opaque, const char *name);
>  static char *get_content (struct nbd_handle *, int64_t size);
> 
> @@ -181,13 +182,9 @@ show_one_export (struct nbd_handle *nbd, const char 
> *desc,
>show_boolean ("can_trim", can_trim);
>  if (can_zero >= 0)
>show_boolean ("can_zero", can_zero);
> -if (block_minimum > 0)
> -  fprintf (fp, "\t%s: %" PRId64 "\n", "block_size_minimum", 
> block_minimum);
> -if (block_preferred > 0)
> -  fprintf (fp, "\t%s: %" PRId64 "\n", "block_size_preferred",
> -   block_preferred);
> -if (block_maximum > 0)
> -  fprintf (fp, "\t%s: %" PRId64 "\n", "block_size_maximum", 
> block_maximum);
> +show_size ("block_size_minimum", block_minimum);
> +show_size ("block_size_preferred", block_preferred);
> +show_size ("block_size_maximum", block_maximum);
>}
>else {
>  if (first)
> @@ -304,6 +301,21 @@ show_boolean (const char *name, bool cond)
>ansi_restore (fp);
>  }
> 
> +/* Used for displaying sizes in non-JSON output. */
> +void show_size (const char *name, int64_t size)
> +{
> +  char size_str[HUMAN_SIZE_LONGEST];
> +  bool human_size_flag = false;
> +
> +  if (size > 0) {
> +human_size (size_str, size, _size_flag);
> +if (human_size_flag)
> +  fprintf (fp, "\t%s: %" PRId64 " (%s)\n", name, size, size_str);
> +else
> +  fprintf (fp, "\t%s: %" PRId64 "\n", name, size);
> +  }
> +}
> +
>  static int
>  collect_context (void *opaque, const char *name)
>  {
> -- 
> 2.41.0
> 
> ___
> Libguestfs mailing list
> Libguestfs@redhat.com
> https://listman.redhat.com/mailman/listinfo/libguestfs

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Some comments on the libnbd apk in Alpine

2023-10-06 Thread Richard W.M. Jones
On Fri, Oct 06, 2023 at 03:54:46PM -0400, Sean McAvoy wrote:
> Hi Richard,
> the package in Alpine (edge) has recently been updated to 1.18.0. The
> mirrors should reflect that soon

Thank you

Rich.

> 
> Thanks,
> 
> Sean
> 
> On Sat, Sep 23, 2023 at 6:11 AM Richard W.M. Jones  wrote:
> >
> > On Sat, Sep 23, 2023 at 11:39:12AM +0200, Sean McAvoy wrote:
> > > Hello Richard,
> > > I have updated the version to 1.16.0 and integrated your suggestions in 
> > > Merge
> > > Request [1].
> > > I have stared the project and will get updates when new releases are made 
> > > and
> > > will update the Alpine package soon after.
> >
> > Great thanks.  Just a note that the stable branch (1.16.4)
> > has a few fixes:
> >
> > https://download.libguestfs.org/libnbd/1.16-stable/
> >
> > Rich.
> >
> > > Thanks for making great software.
> > >
> > > Sean
> > >
> > >
> > > [1] https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/52132
> > >
> > > On Thu, Sep 21, 2023 at 6:02 PM Richard W.M. Jones  
> > > wrote:
> > >
> > > Hi Sean, the maintainers of libnbd here.  Thanks for packaging libnbd
> > > in Alpine:
> > >
> > > https://git.alpinelinux.org/aports/tree/testing/libnbd/APKBUILD
> > >
> > > I have a few comments:
> > >
> > > (1) The url field should be "https://gitlab.com/nbdkit/libnbd;
> > > (The one you are using is an old, stale mirror)
> > >
> > > (2) libxml2-dev is not listed as a dependency.
> > >
> > > We had a request from an Alpine user to add this dependency because it
> > > enables several important features in libnbd, namely NBD URI support
> > > and some utilities which require URI support.
> > >
> > > It shouldn't be too much trouble to add since it appears that libxml2
> > > is already built in Alpine.
> > >
> > > (3) libnbd 1.18 (next stable) will be released in a week or two:
> > > 
> > > https://listman.redhat.com/archives/libguestfs/2023-September/032590.html
> > >
> > > If you have any questions or comments about the package you can send
> > > them to me and/or the mailing list (without subscribing if you don't
> > > want to).
> > >
> > > Rich.
> > >
> > > --
> > > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/
> > > ~rjones
> > > Read my programming and virtualization blog: http://rwmj.wordpress.com
> > > virt-p2v converts physical machines to virtual machines.  Boot with a
> > > live CD or over the network (PXE) and turn machines into KVM guests.
> > > http://libguestfs.org/virt-v2v
> > >
> > >
> >
> > --
> > Richard Jones, Virtualization Group, Red Hat 
> > http://people.redhat.com/~rjones
> > Read my programming and virtualization blog: http://rwmj.wordpress.com
> > virt-builder quickly builds VMs from scratch
> > http://libguestfs.org/virt-builder.1.html
> >

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


Re: [Libguestfs] [PATCH libnbd 3/5] common: Combine human-size.h headers into one

2023-10-06 Thread Richard W.M. Jones
On Fri, Oct 06, 2023 at 09:18:31AM -0500, Eric Blake wrote:
> On Sun, Sep 03, 2023 at 04:23:23PM +0100, Richard W.M. Jones wrote:
> > Copy the human_size() function from common/utils/ into the new
> > human-size.h header in common/include/.  Remove human-size.c and
> > combine the tests into one.
> > ---
> >  common/include/human-size.h  | 51 ++
> 
> This file was created by inheriting nbdkit's weaker BSD licensing...
> 
> >  common/include/test-human-size.c | 79 +---
> >  common/utils/Makefile.am | 10 +---
> >  common/utils/human-size.c| 56 
> >  common/utils/human-size.h| 49 --
> 
> ...while this file was originally created with libnbd's LGPLv2+
> licensing.  By merging LGPLv2+ code into a file containing only a BSD
> license header, you have created an ambiguity on what license should
> be assumed when using human_size().  Could you explicitly clarify that
> the relaxing of the license was intentional, so that it is safe to
> then merge libnbd's code into nbdkit without dragging in LGPLv2+ code?

Yes, let's use the weaker (BSD) license for this new code.  From the
git history I authored both originally.

> > +static inline char *
> > +human_size (char *buf, uint64_t bytes, bool *human)
> > +{
> > +  static const char ext[][2] = { "E", "P", "T", "G", "M", "K", "" };
> > +  size_t i;
> 
> Code motion, so this is pre-existing, but this seems rather lengthy,
> compared to a more compact:
> 
> static const char ext[] = "EPTGMK";
> 
> > +
> > +  if (buf == NULL) {
> > +buf = malloc (HUMAN_SIZE_LONGEST);
> > +if (buf == NULL)
> > +  return NULL;
> > +  }
> > +
> > +  /* Work out which extension to use, if any. */
> > +  i = 6;
> > +  if (bytes != 0) {
> > +while ((bytes & 1023) == 0) {
> > +  bytes >>= 10;
> > +  i--;
> > +}
> > +  }
> > +
> > +  /* Set the flag to true if we're going to add a human-readable 
> > extension. */
> > +  if (human)
> > +*human = ext[i][0] != '\0';
> 
> *human = ext[i] != '\0';
> 
> > +
> > +  snprintf (buf, HUMAN_SIZE_LONGEST, "%" PRIu64 "%s", bytes, ext[i]);
> 
> snprintf (buf, HUMAN_SIZE_LONGEST, "%" PRIu64 ".1s", bytes, [i]);
> 
> > +  return buf;
> > +}
> > +

Do you want to suggest / push a patch for these bits?

Thanks,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH virt-v2v] -it ssh: Double quote ssh command which tests remote file exists

2023-10-02 Thread Richard W.M. Jones
On Mon, Oct 02, 2023 at 04:09:50PM +0200, Laszlo Ersek wrote:
> On 10/2/23 11:03, Richard W.M. Jones wrote:
> > Double quoting was removed in
> > commit e2af12ba69c4463bb73d30db63290a887cdd41eb ("input: -i vmx:
> > Remove support for openssh scp < 8.8", Nov 2021).  However it should
> > only have been removed from scp commands, not for this ssh command
> > where it is still required.
> > 
> > See: https://github.com/libguestfs/virt-v2v/issues/35
> > Thanks: Laszlo Ersek for diagnosis and suggesting the fix
> > Reported-by: Bill Sanders
> > ---
> >  input/parse_domain_from_vmx.ml | 6 +-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > diff --git a/input/parse_domain_from_vmx.ml b/input/parse_domain_from_vmx.ml
> > index 15ee093f59..2e75e78506 100644
> > --- a/input/parse_domain_from_vmx.ml
> > +++ b/input/parse_domain_from_vmx.ml
> > @@ -97,7 +97,11 @@ let remote_file_exists uri path =
> >   | None -> ""
> >   | Some user -> quote user ^ "@")
> >  (quote (server_of_uri uri))
> > -(quote path) in
> > +(* Double quoting is necessary for 'ssh', first to protect
> > + * from the local shell, second to protect from the remote
> > + * shell.  
> > https://github.com/libguestfs/virt-v2v/issues/35#issuecomment-1741730963
> > + *)
> > +(quote (quote path)) in
> >if verbose () then
> >  eprintf "%s\n%!" cmd;
> >Sys.command cmd = 0
> 
> Reviewed-by: Laszlo Ersek 

I pushed this patch, and the timezone patches (previously reviewed) as
c5c9aa2f77..22c5b98ab7

Even though the timezone patches didn't apparently fix the other bug,
they didn't make it worse and they do seem to be worthwhile changes.

Thanks,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH virt-v2v] -it ssh: Double quote ssh command which tests remote file exists

2023-10-02 Thread Richard W.M. Jones
Double quoting was removed in
commit e2af12ba69c4463bb73d30db63290a887cdd41eb ("input: -i vmx:
Remove support for openssh scp < 8.8", Nov 2021).  However it should
only have been removed from scp commands, not for this ssh command
where it is still required.

See: https://github.com/libguestfs/virt-v2v/issues/35
Thanks: Laszlo Ersek for diagnosis and suggesting the fix
Reported-by: Bill Sanders
---
 input/parse_domain_from_vmx.ml | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/input/parse_domain_from_vmx.ml b/input/parse_domain_from_vmx.ml
index 15ee093f59..2e75e78506 100644
--- a/input/parse_domain_from_vmx.ml
+++ b/input/parse_domain_from_vmx.ml
@@ -97,7 +97,11 @@ let remote_file_exists uri path =
  | None -> ""
  | Some user -> quote user ^ "@")
 (quote (server_of_uri uri))
-(quote path) in
+(* Double quoting is necessary for 'ssh', first to protect
+ * from the local shell, second to protect from the remote
+ * shell.  
https://github.com/libguestfs/virt-v2v/issues/35#issuecomment-1741730963
+ *)
+(quote (quote path)) in
   if verbose () then
 eprintf "%s\n%!" cmd;
   Sys.command cmd = 0
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-10-02 Thread Richard W.M. Jones
On Mon, Oct 02, 2023 at 01:00:00PM +0800, Ming Xie wrote:
> Hi Richard,
> 
> On Fri, Sep 29, 2023 at 5:45 AM Richard W.M. Jones  wrote:
> 
> On Thu, Sep 28, 2023 at 07:41:55PM +0800, Ming Xie wrote:
> > Hi Richard,
> >
> > On Thu, Sep 28, 2023 at 6:14 PM Richard W.M. Jones 
> wrote:
> >
> >     Ming,
> >
> >     For some reason the build seems "stuck" at the moment, but
> >     if it finishes could you see if this fixes the RHEL 9 case?
> >
> >     https://kojihub.stream.centos.org/koji/taskinfo?taskID=2879547
> 
> It finished building now.
> 
> 
> Use virt-v2v-2.3.4-6.qemugatzbug1.el9.x86_64 to test and get the following
> results, the problem has not been fixed yet.
> 
>        qemu-ga
>   win10-non-UTC-8 PASS
>   Win10-efi-UTC-8  FAIL
>   Win11-non-efi UTC-8   PASS
>   Win11-efi-UTC-8  FAIL(can't find qemu-ga log in c:\)
>   Win11-efi-UTC+8 FAIL(can't find qemu-ga log in c:\)
>   Win2019-efi-UTC-8  PASS
>   Win2022-efi-UTC-8  PASS

OK thanks for testing it.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


Re: [Libguestfs] libguestfs kernel panic

2023-09-30 Thread Richard W.M. Jones
On Sat, Sep 30, 2023 at 10:42:55AM +, Clark, Dennis wrote:
> Ok, Looking at my original installation procedure, including the slight tweaks
> to get libvirt-7 loaded, and checking the current server … it appears the
> libvirt-7 packages mention the O/S version (eg el8.7), and there are packages
> for several versions, but not for el8.8 … and yet my server (after patching) 
> is
> now running OL 8.8 … is that the issue ?

No I don't think so.

The main thing is to try to find out which part of init.c
is crashing (and later, why).

> > [0.845792] traps: init[1] general protection fault ip:43faa8
> 
> > sp:7ffe9b9904b0 error:0 in init[40+a7000]

As I don't have OEL, could you try extracting the init binary.  It's
buried in the supermin executable, so it's best to get it from the
debug sources:

(1) debuginfo-install supermin

(2) rpm -ql supermin-debugsource | grep src/format-ext2-init-bin.h

(eg. it will print something like /src/format-ext2-init-bin.h )

(3) Attach the copy of /src/format-ext2-init-bin.h or send
it to me by email if it's too big for the list.  You might have
to compress it.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


Re: [Libguestfs] libguestfs kernel panic

2023-09-30 Thread Richard W.M. Jones
On Thu, Sep 28, 2023 at 12:30:28PM +, Clark, Dennis wrote:
> [root@ol-kvm-h01 work]# LIBGUESTFS_BACKEND=direct libguestfs-test-tool
...
> libguestfs: launch: version=1.44.0rhel=8,release=
> 5.0.1.module+el8.7.0+21035+a8208c98,libvirt
> 
> \x1b[2J[0.00] Linux version 5.15.0-105.125.6.2.2.el8uek.x86_64 
> (mockbuild at host-100-100-224-56) (gcc (GCC) 11.2.1 20220127 (Red Hat 
> 11.2.1-9.1.0.6), GNU ld version 2.36.1-2.0.1.el8) #2 SMP Tue Sep 19 21:02:08 
> PDT 2023

^ Looks like OEL.

> [0.843934] Run /init as init process

^ This messsage come from the kernel.

> [0.845792] traps: init[1] general protection fault ip:43faa8
> sp:7ffe9b9904b0 error:0 in init[40+a7000]
> 
> [0.849636] Kernel panic - not syncing: Attempted to kill init! exitcode=
> 0x000b

Then the supermin embedded init process appears to crash.  That's
somewhere probably very early on in this code:

  https://github.com/libguestfs/supermin/blob/master/init/init.c

Before I look further at this, is this unmodified OEL 8.7.0?
And you didn't recompile supermin or libguestfs?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-28 Thread Richard W.M. Jones
On Thu, Sep 28, 2023 at 07:41:55PM +0800, Ming Xie wrote:
> Hi Richard,
> 
> On Thu, Sep 28, 2023 at 6:14 PM Richard W.M. Jones  wrote:
> 
> Ming,
> 
> For some reason the build seems "stuck" at the moment, but
> if it finishes could you see if this fixes the RHEL 9 case?
> 
> https://kojihub.stream.centos.org/koji/taskinfo?taskID=2879547

It finished building now.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-28 Thread Richard W.M. Jones
On Thu, Sep 28, 2023 at 07:41:55PM +0800, Ming Xie wrote:
> Hi Richard,
> 
> On Thu, Sep 28, 2023 at 6:14 PM Richard W.M. Jones  wrote:
> 
> Ming,
> 
> For some reason the build seems "stuck" at the moment, but
> if it finishes could you see if this fixes the RHEL 9 case?
> 
> https://kojihub.stream.centos.org/koji/taskinfo?taskID=2879547
> 
> 
> Sure, no problem, but I don't see the package available now, I will check 
> later.

The build seems stuck.  I asked on the RHEL mailing list
if someone can look at it.

> BTW, is there a bug for this issue?

Not at the moment.

Rich.


> Regards
> Ming Xie
> 

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-28 Thread Richard W.M. Jones
On Thu, Sep 28, 2023 at 04:03:33PM +0800, Ming Xie wrote:
>    Yes, I reproduced the problem using rhel9 virt-v2v, because I saw that the
> customer was also using rhel9 virt-v2v (version: 2.2), I saw that you wanted 
> to
> solve the problem by fixing the  date of the schtask command, but I see that
> the schtask command used to install qemu-ga has been changed to the one shown
> in "qemu-ga-command-rhel9.png" on rhel9

The \windows\temp\install-qemu-ga-x86_64.msi.ps1 script will
contain the actual install commands.

I'll send you an updated v2v package anyway.

Rich.

>   Besides,  qemu-ga can be installed successfully after converting win11-efi
> -UTC-8 guest on rhel8.9, which means I cannot reproduce the issue on rhel8 and
> the schtask command used for installing qemu-ga is old one as ''
> qemu-ga-command-rhel8.png' ' shown on rhel8.9, so I'm not sure if the fix can
> be work with rhel9 virt-v2v  
> 
> 
> Regards
> Ming Xie




-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-28 Thread Richard W.M. Jones
Ming,

For some reason the build seems "stuck" at the moment, but
if it finishes could you see if this fixes the RHEL 9 case?

https://kojihub.stream.centos.org/koji/taskinfo?taskID=2879547

Thanks,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [libnbd PATCH] info: Try harder for graceful disconnect from server

2023-09-28 Thread Richard W.M. Jones
On Wed, Sep 27, 2023 at 04:22:48PM -0500, Eric Blake wrote:
> On Fri, Sep 22, 2023 at 04:03:19PM -0500, Eric Blake wrote:
> > On Fri, Sep 22, 2023 at 09:47:55PM +0100, Richard W.M. Jones wrote:
> > > On Fri, Sep 22, 2023 at 12:33:36PM -0500, Eric Blake wrote:
> > > > There are a number of ways in which gathering information can fail.
> > > > But when possible, it makes sense to let the server know that we are
> > > > done talking, as it minimizes the likelihood that nbdinfo's error
> > > > message will be obscured by an accompanying error message by the
> > > > server complaining about an unclean disconnect.
> > > > 
> > > > For example, with a one-off qemu temporarily patched as mentioned in
> > > > commit 0f8ee8c6 to advertise sizes larger than 2^63, kicking off
> > > > 'qemu-nbd -f raw -r file &' produces:
> > > > 
> > > > pre-patch:
> > > > 
> > > > $ ./run nbdinfo --size nbd://localhost
> > > > /home/eblake/libnbd/info/.libs/nbdinfo: nbd_get_size: server claims 
> > > > size 9223372036854781440 which does not fit in signed result: Value too 
> > > > large for defined data type
> > > > qemu-nbd: option negotiation failed: Failed to read opts magic: 
> > > > Unexpected end-of-file before all data were read
> > > 
> > > This doesn't necessarily seem like a bug?
> > 
> > It's a quality of service thing - qemu is just being noisy that the
> > client closed the socket abruptly without giving any reason why.  It
> > doesn't change libnbd's behavior (nbdinfo has already reported its
> > error message), but may confuse people reading qemu-nbd logs.
> > 
> > It may also be a case where we could patch qemu-nbd to not output a
> > complaint if the client exited at a clean point in the back-and-forth
> > transactions.  We still want to be noisy if the socket closes after
> > the first byte has been read, but if there are zero bytes available,
> > announcing that the client no longer cares does not add much value.
> > 
> > > 
> > > This feels like quite a significant change in behaviour to me.  In
> > > particular I'm worried if it interacts in some subtle way with the
> > > forking done by the "[ ... ]" syntax for running servers on the
> > > command line (or any of the other ways that libnbd might fork/exec).
> > 
> > Interesting observation.  atexit() handlers are not preserved across
> > exec, and I think all our fork() paths end in either exec or _exit
> > (also where atexit handlers are ignored), so I don't think we are
> > risking calling the handler twice.

So there's a couple of other things I don't like about this patch
after further reflection:

(1) It causes nbdinfo to do extra I/O on the error path.  While I
think we're only writing (NBD_CMD_DISC?) and not expecting a reply,
there might be some situation where a server (especially a local one)
is able to block progress by refusing to read, causing nbdinfo to
hang on exit.  I'm not sure how realistic that is.

(2) It hints that the best way to use the API is to always issue a
shutdown before close, but in reality that's not required.

I think it's also trying to solve something which isn't a problem.

Rich.

> > > 
> > > Can we hold this patch until after 1.18 is released and then put it
> > > in?  Should only be a week or two.
> > 
> > Sure, being conservative on this one is fine by me.  I'll delay it
> > until after 1.18.
> > 
> > > 
> > > Provisionally ACKed for post-1.18
> 
> Now in as commit fd4f3fea, so we can start getting some soak time under CI.
> 
> -- 
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.
> Virtualization:  qemu.org | libguestfs.org

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-27 Thread Richard W.M. Jones
On Wed, Sep 27, 2023 at 08:46:58PM +0800, Ming Xie wrote:
> Hi Richard,
> 
>    Sorry, I missed the email, I found out that the ova OS used by the
> customer is win11 uefi, so I tried to reproduce the error today using
> win11-uefi guest
> 
> Steps:
> 1. Convert win10-efi with UTC-8 timezone, win11-non-efi with UTC-8 timezone,
> win11-efi with UTC-8 timezone, win11-efi with UTC+8 timezone, win2022-efi with
> UTC-8 timezone from VMware by v2v, then check the installation status after
> finishing conversion
> 
> 
> Summary the test result as below:
> 
>     qemu-ga
> 
>   Win10-efi-UTC-8  PASS
>   Win11-non-efi UTC-8   PASS
>   Win11-efi-UTC-8  FAIL(can't find qemu-ga log in c:\)
>   Win11-efi-UTC+8 FAIL(can't find qemu-ga log in c:\)
>   Win2022-efi-UTC-8  PASS

Thanks for testing.  The symptoms look the same as Lee reported.

You're running virt-v2v from RHEL 9?  If so I will prepare a scratch
RHEL 9 package with the patches which we think might fix this, for
testing.

Rich.

>   Based on the above results, I think this problem is only caused by win11-efi
> and has nothing to do with the time zone "UTC-8"
> 
> The following are existing bugs about qemu-ga:
> 
> Bug 1820144 - cannot install qemu-ga to some guests even if scheduled qemu-ga
> installation task exist
> Bug 2114809 - Can't install qemu-ga because of network address error after
> converting MD-RAID1 win11 host by virt-p2v
> Bug 1820152 - Fail to execute installation-qemu-ga.msi for win2012r2 and
> win8.1-i386 guests
> 
> 
> Thanks & Regards
> Ming Xie
> 
> On Fri, Sep 22, 2023 at 6:40 PM Richard W.M. Jones  wrote:
> 
> On Thu, Sep 21, 2023 at 07:47:52PM +0200, Lee Garrett wrote:
> > On 21.09.23 19:43, Richard W.M. Jones wrote:
> > >So this is probably another instance or variation of the timezone
> > >formatting problem (of schtasks).  Which version of virt-v2v is this?
> > >I want to check that you have a version with all the latest patches in
> > >this area.
> >
> > It's 2.2.0-1 from Debian (12) bookworm. I've verified that it
> > doesn't have any distro-specific patches.
> >
> > (https://salsa.debian.org/libvirt-team/virt-v2v/-/tree/debian/master/
> debian
> > would have a patches/series file in this case)
> 
> The timezone fixes are:
> 
> commit 597d177567234c3a539098c423649781424eeb6f
> Author: Laszlo Ersek 
> Date:   Tue Mar 8 15:30:51 2022 +0100
> 
>     convert_windows: rewrite "configure_qemu_ga" script purely in
> PowerShell
> 
> commit d9dc6c42ae64ba92993dbd9477f003ba73fcfa2f
> Author: Richard W.M. Jones 
> Date:   Fri Nov 12 08:47:55 2021 +
> 
>     convert/convert_windows.ml: Handle date formats with dots instead of /
> 
> They are all included in >= 2.0
> 
> I wonder if 597d177567 has a subtle flaw, or if we introduced a bug
> somewhere when refactoring this code later.
> 
> Lee: Do you have a theory about exactly what is wrong with the
> schtasks date?  Like what was it supposed to be, assuming it was 120
> seconds in the future from boot time, versus what it was set to:
> 
> > Firstboot-qemu-ga                        9/21/2023 4:04:00 PM   Ready
> 
> Could a date or time field have not been swapped or been corrupted
> in some predictable way?
> 
> The code we run is here:
> 
> https://github.com/libguestfs/libguestfs-common/blob/
> e70d89a58dae068be2e19c7c21558707261af96a/mlcustomize/inject_virtio_win.ml#
> L571
> 
> Ming: this could be a bug affecting PST (UTC-8) guests, perhaps
> somehow related to having a single digit month field?
> 
> Rich.
> 
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/
> ~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines.  Supports shell scripting,
> bindings from many languages.  http://libguestfs.org
> 
> 







-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


[Libguestfs] ANNOUNCE: nbdkit 1.36 and libnbd 1.18 released

2023-09-27 Thread Richard W.M. Jones
 formats, like "10%" or "1:10".

   New "nbdkit_peer_security_context" function which returns the security
   context (usually SELinux label) of the client.

   Bug fixes
   Fix long-standing double-free in nbdkit-file-plugin(1) when the client
   closes the connection during negotiation (thanks Eric Blake).

   nbdkit-file-plugin now has improved error messages when you try to open
   a file with an empty name ("").

   Documentation
   Miscellaneous documentation improvements for Windows (Brian Carnes).

   Fuzzing instructions updated for Clang 17, AFL++ 4.08c.

   Tests
   Ongoing CI maintenance and fixes (Laszlo Ersek, Eric Blake).

   Many fixes in the test suite for MinGW and Wine.

   Several internal consistency checks were added to the test suite, to
   ensure, for example, that we always use "requires_run" when the test
   uses the nbdkit --run option.

   Build
   Optionally "zlib-ng" can be used (instead of "zlib") for faster
   decompression in various places.

   No longer link with msvcrt on Windows.

   "awk" is a build requirement.  It was implicitly required before
   (because it is needed by the "./configure" script generated by
   autoconf), but now the dependency is explicit (Eric Blake).

   Internals
   "podwrapper" has a new --replace parameter which makes it easier to use
   "AC_SUBST"-substitutions in documentation.

SEE ALSO
   nbdkit(1).

AUTHORS
   Authors of nbdkit 1.36:

   Alan Somers
   Brian Carnes
   Eric Blake
   Laszlo Ersek
   Richard W.M. Jones


--

   libnbd-release-notes-1.18 - release notes for libnbd 1.18

   These are the release notes for libnbd stable release 1.18.  This
   describes the major changes since 1.16.

   libnbd 1.18.0 was released on 27 September 2023.

   Security
   Eric Blake found a case where a server could cause libnbd to crash,
   although not in the normal libnbd configuration.  For more information
   see the announcement here:
   https://listman.redhat.com/archives/libguestfs/2023-July/032035.html

   Eric Blake found an issue with nbd_get_size(3) where a server that
   returns a size > (uint64_t)INT64_MAX and < (uint64_t)-1 would cause
   nbd_get_size to return a negative number (which is not -1 and so
   callers may or may not treat it as an error).  While no client code in
   libnbd itself is affected by this, it could affect external clients.
   libnbd ≥ 1.16.5 now returns an error (-1) and sets nbd_get_errno to
   "EOVERFLOW" in this case.  We have applied for a decision on a CVE.
   See the announcement here:
   https://listman.redhat.com/archives/libguestfs/2023-September/032711.html

   During routine fuzzing we found several security problems which had
   been introduced during this development cycle and have subsequently
   been fixed.  Stable (even numbered) releases of libnbd should not be
   vulnerable; do not use the development (odd numbered) releases in
   production.

   If you find a security issue, please read SECURITY in the source
   (online here: https://gitlab.com/nbdkit/libnbd/blob/master/SECURITY).
   To find out about previous security issues in libnbd, see
   libnbd-security(3).

   New APIs
   nbd_block_status_64(3)
   nbd_aio_block_status_64(3)
   Make a 64 bit block status request, see "Protocol" below (Eric
   Blake).

   nbd_block_status_filter(3)
   nbd_aio_block_status_filter(3)
   Send filtered block status command, see "Protocol" below (Eric
   Blake).

   nbd_can_block_status_payload(3)
   Find out if the server supports filtered block status command (Eric
   Blake).

   nbd_set_request_extended_headers(3)
   nbd_get_request_extended_headers(3)
   nbd_get_extended_headers_negotiated(3)
   nbd_opt_extended_headers(3)
   nbd_aio_opt_extended_headers(3)
   Set/get whether we request extended headers from the server, and
   find out if we negotiated extended headers, see "Protocol" below
   (Eric Blake).

   Enhancements to existing APIs
   "qemu:" meta-context constants (eg. "qemu:dirty-bitmap" as
   "LIBNBD_CONTEXT_QEMU_DIRTY_BITMAP") are now available through the C,
   Golang, OCaml and Python language bindings (Eric Blake).

   nbd_shutdown(3) now works correctly when in opt mode (Eric Blake).

   nbd_set_string(3) adds "LIBNBD_STRICT_AUTO_FLAG" which allows the
   client to test how servers behave when the payload length flag is
   adjusted (Eric Blake).

   Protocol
   libnbd now supports N

[Libguestfs] [PATCH v2v v2 6/6] convert: Find out if Windows guest is expecting RTC set to UTC

2023-09-25 Thread Richard W.M. Jones
Read HKLM\SYSTEM\CurrentControlSet\Control\TimeZoneInformation key
"RealTimeIsUniversal" to see if the Windows guest is expecting RTC
set to localtime (not present) or UTC (present and set to 1).

See: https://wiki.archlinux.org/title/System_time#UTC_in_Microsoft_Windows
See: 
https://listman.redhat.com/archives/libguestfs/2023-September/thread.html#32556
Reported-by: Lee Garrett
Reviewed-by: Laszlo Ersek 
---
 convert/convert_windows.ml | 38 +-
 tests/test-v2v-i-ova.xml   |  2 +-
 2 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/convert/convert_windows.ml b/convert/convert_windows.ml
index 7e3aa8d789..34cf341b8e 100644
--- a/convert/convert_windows.ml
+++ b/convert/convert_windows.ml
@@ -103,6 +103,42 @@ let convert (g : G.guestfs) _ inspect i_firmware 
block_driver _ static_ips =
   (* If the Windows guest has AV installed. *)
   let has_antivirus = Windows.detect_antivirus inspect in
 
+  (* Does the guest expect the RTC to be set to UTC or localtime?
+   * See https://wiki.archlinux.org/title/System_time#UTC_in_Microsoft_Windows
+   * Note this might be a QWORD on 64 bit Windows instances.
+   *)
+  let rtc_utc =
+Registry.with_hive_readonly g inspect.i_windows_system_hive
+  (fun reg ->
+   try
+ let key_path = [ "Control"; "TimeZoneInformation" ] in
+ let path = inspect.i_windows_current_control_set :: key_path in
+ let node =
+   match Registry.get_node reg path with
+   | None -> raise Not_found
+   | Some node -> node in
+ let valueh = g#hivex_node_get_value node "RealTimeIsUniversal" in
+ if valueh = 0L then raise Not_found;
+ let t = g#hivex_value_type valueh in
+ let data = g#hivex_value_value valueh in
+ let is_utc =
+   match t with
+   | 0_L (* REG_NONE *) -> false (* localtime *)
+   | 4_L (* REG_DWORD *) -> data = "\001\000\000\000"
+   | 11_L (* REG_QWORD *) -> data = "\001\000\000\000\000\000\000\000"
+   | _ (* who knows ... *) ->
+ warning (f_"unknown CurrentControlSet\\Control\\\
+ TimeZoneInformation key RealTimeIsUniversal \
+ type 0x%Lx, assuming RTC set to UTC") t;
+ true in
+ is_utc
+   with Not_found ->
+ (* If the key is not found then by default we assume
+  * that Windows is expecting the RTC to be set to localtime.
+  *)
+ false
+  ) in
+
   (* Open the software hive (readonly) and find the Xen PV uninstaller,
* if it exists.
*)
@@ -275,7 +311,7 @@ let convert (g : G.guestfs) _ inspect i_firmware 
block_driver _ static_ips =
   gcaps_arch = Utils.kvm_arch inspect.i_arch;
   gcaps_arch_min_version = 0;
   gcaps_virtio_1_0 = virtio_win_installed.Inject_virtio_win.virtio_1_0;
-  gcaps_rtc_utc = true;
+  gcaps_rtc_utc = rtc_utc;
 } in
 
 guestcaps
diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
index a41827bfd5..fa7b4dbfc5 100644
--- a/tests/test-v2v-i-ova.xml
+++ b/tests/test-v2v-i-ova.xml
@@ -18,7 +18,7 @@
   
 hvm
   
-  
+  
   destroy
   restart
   restart
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v v2 3/6] output/create_libvirt_xml.ml: Refactor os_section

2023-09-25 Thread Richard W.M. Jones
Minor refactoring of how  section is generated in XML output.
There is no change in the output.

Reviewed-by: Laszlo Ersek 
---
 output/create_libvirt_xml.ml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/output/create_libvirt_xml.ml b/output/create_libvirt_xml.ml
index 964acd25fd..12586fde17 100644
--- a/output/create_libvirt_xml.ml
+++ b/output/create_libvirt_xml.ml
@@ -292,10 +292,10 @@ let create_libvirt_xml ?pool source inspect
e "nvram" ["template", vars_template] [] ] in
 
 List.push_back_list os loader;
-!os in
+e "os" [] !os in
 
   List.push_back_list body [
-e "os" [] os_section;
+os_section;
 
 e "on_poweroff" [] [PCData "destroy"];
 e "on_reboot" [] [PCData "restart"];
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v v2 4/6] -o libvirt: Add to libvirt XML

2023-09-25 Thread Richard W.M. Jones
Reviewed-by: Laszlo Ersek 
---
 output/create_libvirt_xml.ml | 6 ++
 tests/test-v2v-i-ova.xml | 1 +
 2 files changed, 7 insertions(+)

diff --git a/output/create_libvirt_xml.ml b/output/create_libvirt_xml.ml
index 12586fde17..30119d1357 100644
--- a/output/create_libvirt_xml.ml
+++ b/output/create_libvirt_xml.ml
@@ -294,8 +294,14 @@ let create_libvirt_xml ?pool source inspect
 List.push_back_list os loader;
 e "os" [] !os in
 
+  (* The  section. *)
+  let clock_section =
+let offset = if guestcaps.gcaps_rtc_utc then "utc" else "localtime" in
+e "clock" [ "offset", offset ] [] in
+
   List.push_back_list body [
 os_section;
+clock_section;
 
 e "on_poweroff" [] [PCData "destroy"];
 e "on_reboot" [] [PCData "restart"];
diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
index e5907ea1cc..a41827bfd5 100644
--- a/tests/test-v2v-i-ova.xml
+++ b/tests/test-v2v-i-ova.xml
@@ -18,6 +18,7 @@
   
 hvm
   
+  
   destroy
   restart
   restart
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v v2 5/6] -o qemu: Set -rtc base=localtime when guest expects RTC set to localtime

2023-09-25 Thread Richard W.M. Jones
I didn't set the -rtc flag in the normal (UTC) case as that is the
default.

Reviewed-by: Laszlo Ersek 
---
 output/output_qemu.ml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/output/output_qemu.ml b/output/output_qemu.ml
index 57b1e58da2..1a5f7f7147 100644
--- a/output/output_qemu.ml
+++ b/output/output_qemu.ml
@@ -158,6 +158,8 @@ module QEMU = struct
 arg_list "-device" ["vmgenid"; sprintf "guid=%s" genid; "id=vmgenid0"]
 );
 
+if not guestcaps.gcaps_rtc_utc then arg "-rtc" "base=localtime";
+
 arg_list "-machine" (machine_str ::
(if smm then ["smm=on"] else []) @
["accel=kvm:tcg"]);
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [[PATCH v2v v2 0/6] convert: Find out if Windows guest is expecting RTC set to UTC

2023-09-25 Thread Richard W.M. Jones
The main changes are:

 - BIOS -> RTC passim

 - Split out the minor refactoring patch

 - Enhanced commit messages

 - Add Laszlo's R-b tag

Rich.


___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v v2 1/6] types: Add gcaps_rtc_utc to record if the RTC is set to UTC or localtime

2023-09-25 Thread Richard W.M. Jones
Almost every Linux guest expects the motherboard Real Time Clock (RTC)
to be set to UTC and they adjust the time displayed based on their
timezone (which may be different for each user).

Most Windows guests expect the RTC to be set to the local time.
Windows can be configured to use a UTC clock.  We can detect this by
looking at the Windows registry.

To cope with this difference we need to add a guestcaps flag based on
what we think the guest is expecting.  (We might also use the source
hypervisor RTC setting, but it is not thought to be as reliable as
inspecting the guest.)

This change simply adds the flag to guestcaps, and sets it to always
true, so there is no change to the output.

Reviewed-by: Laszlo Ersek 
---
 lib/types.mli  | 5 +
 convert/convert_linux.ml   | 1 +
 convert/convert_windows.ml | 1 +
 lib/types.ml   | 3 +++
 4 files changed, 10 insertions(+)

diff --git a/lib/types.mli b/lib/types.mli
index 65ef2e35cf..3446bb64bd 100644
--- a/lib/types.mli
+++ b/lib/types.mli
@@ -277,6 +277,11 @@ type guestcaps = {
   gcaps_virtio_1_0 : bool;
   (** The guest supports the virtio devices that it does at the virtio-1.0
   protocol level. *)
+
+  gcaps_rtc_utc : bool;
+  (** Is the RTC set to UTC ([true]) or localtime ([false])?  For
+  Linux guests this is always true.  For Windows we find out
+  what the guest is expecting by looking at the registry. *)
 }
 (** Guest capabilities after conversion.  eg. Was virtio found or installed? *)
 
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
index 8d7020848f..bd54568b10 100644
--- a/convert/convert_linux.ml
+++ b/convert/convert_linux.ml
@@ -220,6 +220,7 @@ let convert (g : G.guestfs) source inspect i_firmware _ 
keep_serial_console _ =
   gcaps_arch = Utils.kvm_arch inspect.i_arch;
   gcaps_arch_min_version = arch_min_version;
   gcaps_virtio_1_0 = virtio_1_0;
+  gcaps_rtc_utc = true; (* almost all Linux expect RTC to be UTC *)
 } in
 
 guestcaps
diff --git a/convert/convert_windows.ml b/convert/convert_windows.ml
index 122d95469d..7e3aa8d789 100644
--- a/convert/convert_windows.ml
+++ b/convert/convert_windows.ml
@@ -275,6 +275,7 @@ let convert (g : G.guestfs) _ inspect i_firmware 
block_driver _ static_ips =
   gcaps_arch = Utils.kvm_arch inspect.i_arch;
   gcaps_arch_min_version = 0;
   gcaps_virtio_1_0 = virtio_win_installed.Inject_virtio_win.virtio_1_0;
+  gcaps_rtc_utc = true;
 } in
 
 guestcaps
diff --git a/lib/types.ml b/lib/types.ml
index 75c14fd4f6..d6f9a266c5 100644
--- a/lib/types.ml
+++ b/lib/types.ml
@@ -399,6 +399,7 @@ type guestcaps = {
   gcaps_arch : string;
   gcaps_arch_min_version : int;
   gcaps_virtio_1_0 : bool;
+  gcaps_rtc_utc : bool;
 }
 and guestcaps_block_type = Virtio_blk | Virtio_SCSI | IDE
 and guestcaps_net_type = Virtio_net | E1000 | RTL8139
@@ -429,6 +430,7 @@ let string_of_guestcaps gcaps =
gcaps_arch = %s\n\
gcaps_arch_min_version = %d\n\
gcaps_virtio_1_0 = %b\n\
+   gcaps_rtc_utc = %b\n\
   "
   (string_of_block_type gcaps.gcaps_block_bus)
   (string_of_net_type gcaps.gcaps_net_bus)
@@ -440,6 +442,7 @@ let string_of_guestcaps gcaps =
   gcaps.gcaps_arch
   gcaps.gcaps_arch_min_version
   gcaps.gcaps_virtio_1_0
+  gcaps.gcaps_rtc_utc
 
 type target_buses = {
   target_virtio_blk_bus : target_bus_slot array;
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v v2 2/6] -o kubevirt: Add comment about future support for clock = localtime

2023-09-25 Thread Richard W.M. Jones
Reviewed-by: Laszlo Ersek 
---
 output/create_kubevirt_yaml.ml | 5 +
 1 file changed, 5 insertions(+)

diff --git a/output/create_kubevirt_yaml.ml b/output/create_kubevirt_yaml.ml
index 689555e4dc..e3a3f422df 100644
--- a/output/create_kubevirt_yaml.ml
+++ b/output/create_kubevirt_yaml.ml
@@ -54,6 +54,11 @@ let create_kubevirt_yaml source inspect
   "pit", Assoc [ "tickPolicy", String "delay" ];
   "rtc", Assoc [ "tickPolicy", String "catchup" ];
 ];
+(* XXX Note that we may need to set "localtime" here
+ * depending on guestcaps.gcaps_rtc_utc.  However that
+ * requires the following PR to be merged in Kubevirt:
+ * https://github.com/kubevirt/kubevirt/pull/9587
+ *)
 "utc", List []
   ]
 )
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH v2v 5/5] convert: Find out if Windows guest is expected BIOS set to localtime

2023-09-25 Thread Richard W.M. Jones
On Mon, Sep 25, 2023 at 05:58:00PM +0200, Laszlo Ersek wrote:
> On 9/25/23 16:04, Richard W.M. Jones wrote:
> > Read HKLM\SYSTEM\CurrentControlSet\Control\TimeZoneInformation key
> > "RealTimeIsUniversal" to see if the Windows guest is expecting BIOS
> > set to localtime (not present) or UTC (present and set to 1).
> > 
> > See: https://wiki.archlinux.org/title/System_time#UTC_in_Microsoft_Windows
> > See: 
> > https://listman.redhat.com/archives/libguestfs/2023-September/thread.html#32556
> > Reported-by: Lee Garrett
> > ---
> >  convert/convert_windows.ml | 38 +-
> >  tests/test-v2v-i-ova.xml   |  2 +-
> >  2 files changed, 38 insertions(+), 2 deletions(-)
> > 
> > diff --git a/convert/convert_windows.ml b/convert/convert_windows.ml
> > index f6e039be7e..84e8f7b7d3 100644
> > --- a/convert/convert_windows.ml
> > +++ b/convert/convert_windows.ml
> > @@ -103,6 +103,42 @@ let convert (g : G.guestfs) _ inspect i_firmware 
> > block_driver _ static_ips =
> >(* If the Windows guest has AV installed. *)
> >let has_antivirus = Windows.detect_antivirus inspect in
> >  
> > +  (* Does the guest expect the BIOS to be set to UTC or localtime?
> > +   * See 
> > https://wiki.archlinux.org/title/System_time#UTC_in_Microsoft_Windows
> > +   * Note this might be a QWORD on 64 bit Windows instances.
> > +   *)
> > +  let bios_utc =
> > +Registry.with_hive_readonly g inspect.i_windows_system_hive
> > +  (fun reg ->
> > +   try
> > + let key_path = [ "Control"; "TimeZoneInformation" ] in
> > + let path = inspect.i_windows_current_control_set :: key_path in
> > + let node =
> > +   match Registry.get_node reg path with
> > +   | None -> raise Not_found
> > +   | Some node -> node in
> > + let valueh = g#hivex_node_get_value node "RealTimeIsUniversal" in
> > + if valueh = 0L then raise Not_found;
> > + let t = g#hivex_value_type valueh in
> > + let data = g#hivex_value_value valueh in
> > + let is_utc =
> > +   match t with
> > +   | 0_L (* REG_NONE *) -> false (* localtime *)
> > +   | 4_L (* REG_DWORD *) -> data = "\001\000\000\000"
> > +   | 11_L (* REG_QWORD *) -> data = 
> > "\001\000\000\000\000\000\000\000"
> > +   | _ (* who knows ... *) ->
> > + warning (f_"unknown CurrentControlSet\\Control\\\
> > + TimeZoneInformation key RealTimeIsUniversal \
> > + type 0x%Lx, assuming BIOS set to UTC") t;
> > + true in
> 
> [*]
> 
> > + is_utc
> > +   with Not_found ->
> > + (* If the key is not found then by default we assume
> > +  * that Windows is expecting the BIOS to be set to localtime.
> > +  *)
> > + false
> > +  ) in
> > +
> >(* Open the software hive (readonly) and find the Xen PV uninstaller,
> > * if it exists.
> > *)
> 
> [*] Just guessing, but I'm not sure if assuming UTC is right when the
> key is found but its type is unrecognized, in light of determining
> localtime when the key is absent or has type REG_NONE.
> 
> Either way, the warning is honest about it!

I don't know - I would need to examine lots of Windows machines to
find out what they really do.  But I expect that this field only ever
contains dword:1 or hex(b):1 [qword:1], or is absent.

> > @@ -275,7 +311,7 @@ let convert (g : G.guestfs) _ inspect i_firmware 
> > block_driver _ static_ips =
> >gcaps_arch = Utils.kvm_arch inspect.i_arch;
> >gcaps_arch_min_version = 0;
> >gcaps_virtio_1_0 = virtio_win_installed.Inject_virtio_win.virtio_1_0;
> > -  gcaps_bios_utc = true;
> > +  gcaps_bios_utc = bios_utc;
> >  } in
> >  
> >  guestcaps
> > diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
> > index a41827bfd5..fa7b4dbfc5 100644
> > --- a/tests/test-v2v-i-ova.xml
> > +++ b/tests/test-v2v-i-ova.xml
> > @@ -18,7 +18,7 @@
> >
> >  hvm
> >
> > -  
> > +  
> >destroy
> >restart
> >restart
> 
> With the BIOS -> RTC typo (?) fixed everywhere:
> 
> series
> Reviewed-by: Laszlo Ersek 

Thanks, I'll post v2 anyway since I've written it now.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH v2v 0/5] convert: Find out if Windows guest is expecting BIOS localtime or UTC

2023-09-25 Thread Richard W.M. Jones
On Mon, Sep 25, 2023 at 05:49:18PM +0200, Laszlo Ersek wrote:
> On 9/25/23 17:48, Laszlo Ersek wrote:
> > On 9/25/23 16:04, Richard W.M. Jones wrote:
> >> [Alice: See patch 2]
> >>
> >> [This patch is a bit rough, it could do with better commit messages
> >> and some tests.  Please test it to see if it solves the Windows
> >> conversion issue described in the thread below.]
> >>
> >> We currently do not set any  field in guest output.  Most
> >> Windows guests expect the BIOS to be set to localtime, whereas almost
> >> all Linux guests would expect it to be set to UTC.  It is also
> >> possible to configure a Windows guest to expect BIOS set to UTC.
> >>
> >> The default is usually BIOS set to UTC, so for many Windows guests
> >> this would be wrong.  This specifically may cause problems when
> >> scheduling qemu-ga installation, see the thread here:
> >>
> >> https://listman.redhat.com/archives/libguestfs/2023-September/thread.html#32556
> >>
> >> but could cause other general issues with time in the guest.
> >>
> >> One way to implement this would be to copy the source hypervisor
> >> information across; however I'm not confident this information is read
> >> correctly.  A better way is to read out what the guest is expecting
> >> from the Windows registry.  (For Linux we just assume BIOS is always
> >> UTC, since that's the default for almost any Linux guest which hasn't
> >> been dual-booted with Windows, which for VMs would be incredibly
> >> rare.)
> > 
> > I think the word "BIOS" is incorrectly used all over the series; I'd
> > rather say "RTC" / "real time clock".
> 
> ... meaning patch subject lines, commit message bodies, and code.

OK :-)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH v2v 3/5] -o libvirt: Add to libvirt XML

2023-09-25 Thread Richard W.M. Jones
On Mon, Sep 25, 2023 at 05:47:26PM +0200, Laszlo Ersek wrote:
> On 9/25/23 16:04, Richard W.M. Jones wrote:
> > ---
> >  output/create_libvirt_xml.ml | 10 --
> >  tests/test-v2v-i-ova.xml |  1 +
> >  2 files changed, 9 insertions(+), 2 deletions(-)
> > 
> > diff --git a/output/create_libvirt_xml.ml b/output/create_libvirt_xml.ml
> > index 964acd25fd..f97272ca31 100644
> > --- a/output/create_libvirt_xml.ml
> > +++ b/output/create_libvirt_xml.ml
> > @@ -292,10 +292,16 @@ let create_libvirt_xml ?pool source inspect
> > e "nvram" ["template", vars_template] [] ] in
> >  
> >  List.push_back_list os loader;
> > -!os in
> > +e "os" [] !os in
> > +
> > +  (* The  section. *)
> > +  let clock_section =
> > +let offset = if guestcaps.gcaps_bios_utc then "utc" else "localtime" in
> > +e "clock" [ "offset", offset ] [] in
> >  
> >List.push_back_list body [
> > -e "os" [] os_section;
> > +os_section;
> > +clock_section;
> >  
> >  e "on_poweroff" [] [PCData "destroy"];
> >  e "on_reboot" [] [PCData "restart"];
> 
> I don't think we need to change what "os_section" is bound to, here; I
> understand the idea is to increase consistency, but for me it makes the
> patch harder to read -- ultimately it is a refactoring, and then adding
> in clock_section is the new thing.
> 
> If you can split these apart, that's optimal; if not, this is viable too
> IMO.

Yup, that is a separate clean-up, will split.

Rich.

> Thanks
> Laszlo
> 
> > diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
> > index e5907ea1cc..a41827bfd5 100644
> > --- a/tests/test-v2v-i-ova.xml
> > +++ b/tests/test-v2v-i-ova.xml
> > @@ -18,6 +18,7 @@
> >
> >  hvm
> >
> > +  
> >destroy
> >restart
> >restart

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH v2v 0/5] convert: Find out if Windows guest is expecting BIOS localtime or UTC

2023-09-25 Thread Richard W.M. Jones
On Mon, Sep 25, 2023 at 04:26:59PM +0200, Alice Frosi wrote:
> Hi Rich,
> 
> On Mon, Sep 25, 2023 at 4:10 PM Richard W.M. Jones  wrote:
> 
> [Alice: See patch 2]
> 
>
> I'm not 100% sure about the source of this work. However, we had in
> KubeVirt people interested in using localtime with Windows [1]. Yes,
> I see that you pointed to that PR in patch 2. The problem with that
> PR is the migration. What happens if we migrate the VM to a host
> that is in another timezone? Is it going to break the application
> running inside the VM?

Does every Windows VM running in Kubevirt have its registry adjusted
so it expects the BIOS to be set to UTC?  (ie:
https://wiki.archlinux.org/title/System_time#UTC_in_Microsoft_Windows).
This is a non-standard setting, but I can't understand how Windows VMs
would have correct clocks if this were not the case.

BTW you could find out by running:

  $ virt-win-reg window.img 'HKLM\SYSTEM'

on some Kubevirt Windows VMs, and looking for:

  [\ControlSet001\Control\TimeZoneInformation]

in the output.  See if there is a key called "RealTimeIsUniversal" and
what it is set to.

If it is the case that this is always set for existing Kubevirt
guests, then virt-v2v would have to be changed so it also sets this
registry entry when the output is Kubevirt.

I agree that migration across timezones is highly problematic for the
BIOS = localtime situation.  I don't see any solution for that except
disallowing it.

> IMO, that was the major blocker of that PR. I initially suggested that we
> either avoid migration as a first step or try to migrate only on nodes at the
> same timezone,
> Do you know if anyone has experience on this particular problem?

I guess RHV must have encountered this before, but I don't know any
specifics.

Thanks,

Rich.

> [1] https://github.com/kubevirt/kubevirt/pull/9587
> 
> Many thanks,
> Alice

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


[Libguestfs] [PATCH v2v 0/5] convert: Find out if Windows guest is expecting BIOS localtime or UTC

2023-09-25 Thread Richard W.M. Jones
[Alice: See patch 2]

[This patch is a bit rough, it could do with better commit messages
and some tests.  Please test it to see if it solves the Windows
conversion issue described in the thread below.]

We currently do not set any  field in guest output.  Most
Windows guests expect the BIOS to be set to localtime, whereas almost
all Linux guests would expect it to be set to UTC.  It is also
possible to configure a Windows guest to expect BIOS set to UTC.

The default is usually BIOS set to UTC, so for many Windows guests
this would be wrong.  This specifically may cause problems when
scheduling qemu-ga installation, see the thread here:

https://listman.redhat.com/archives/libguestfs/2023-September/thread.html#32556

but could cause other general issues with time in the guest.

One way to implement this would be to copy the source hypervisor
information across; however I'm not confident this information is read
correctly.  A better way is to read out what the guest is expecting
from the Windows registry.  (For Linux we just assume BIOS is always
UTC, since that's the default for almost any Linux guest which hasn't
been dual-booted with Windows, which for VMs would be incredibly
rare.)

Rich.


___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v 5/5] convert: Find out if Windows guest is expected BIOS set to localtime

2023-09-25 Thread Richard W.M. Jones
Read HKLM\SYSTEM\CurrentControlSet\Control\TimeZoneInformation key
"RealTimeIsUniversal" to see if the Windows guest is expecting BIOS
set to localtime (not present) or UTC (present and set to 1).

See: https://wiki.archlinux.org/title/System_time#UTC_in_Microsoft_Windows
See: 
https://listman.redhat.com/archives/libguestfs/2023-September/thread.html#32556
Reported-by: Lee Garrett
---
 convert/convert_windows.ml | 38 +-
 tests/test-v2v-i-ova.xml   |  2 +-
 2 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/convert/convert_windows.ml b/convert/convert_windows.ml
index f6e039be7e..84e8f7b7d3 100644
--- a/convert/convert_windows.ml
+++ b/convert/convert_windows.ml
@@ -103,6 +103,42 @@ let convert (g : G.guestfs) _ inspect i_firmware 
block_driver _ static_ips =
   (* If the Windows guest has AV installed. *)
   let has_antivirus = Windows.detect_antivirus inspect in
 
+  (* Does the guest expect the BIOS to be set to UTC or localtime?
+   * See https://wiki.archlinux.org/title/System_time#UTC_in_Microsoft_Windows
+   * Note this might be a QWORD on 64 bit Windows instances.
+   *)
+  let bios_utc =
+Registry.with_hive_readonly g inspect.i_windows_system_hive
+  (fun reg ->
+   try
+ let key_path = [ "Control"; "TimeZoneInformation" ] in
+ let path = inspect.i_windows_current_control_set :: key_path in
+ let node =
+   match Registry.get_node reg path with
+   | None -> raise Not_found
+   | Some node -> node in
+ let valueh = g#hivex_node_get_value node "RealTimeIsUniversal" in
+ if valueh = 0L then raise Not_found;
+ let t = g#hivex_value_type valueh in
+ let data = g#hivex_value_value valueh in
+ let is_utc =
+   match t with
+   | 0_L (* REG_NONE *) -> false (* localtime *)
+   | 4_L (* REG_DWORD *) -> data = "\001\000\000\000"
+   | 11_L (* REG_QWORD *) -> data = "\001\000\000\000\000\000\000\000"
+   | _ (* who knows ... *) ->
+ warning (f_"unknown CurrentControlSet\\Control\\\
+ TimeZoneInformation key RealTimeIsUniversal \
+ type 0x%Lx, assuming BIOS set to UTC") t;
+ true in
+ is_utc
+   with Not_found ->
+ (* If the key is not found then by default we assume
+  * that Windows is expecting the BIOS to be set to localtime.
+  *)
+ false
+  ) in
+
   (* Open the software hive (readonly) and find the Xen PV uninstaller,
* if it exists.
*)
@@ -275,7 +311,7 @@ let convert (g : G.guestfs) _ inspect i_firmware 
block_driver _ static_ips =
   gcaps_arch = Utils.kvm_arch inspect.i_arch;
   gcaps_arch_min_version = 0;
   gcaps_virtio_1_0 = virtio_win_installed.Inject_virtio_win.virtio_1_0;
-  gcaps_bios_utc = true;
+  gcaps_bios_utc = bios_utc;
 } in
 
 guestcaps
diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
index a41827bfd5..fa7b4dbfc5 100644
--- a/tests/test-v2v-i-ova.xml
+++ b/tests/test-v2v-i-ova.xml
@@ -18,7 +18,7 @@
   
 hvm
   
-  
+  
   destroy
   restart
   restart
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v 2/5] -o kubevirt: Add comment about future support for clock = localtime

2023-09-25 Thread Richard W.M. Jones
---
 output/create_kubevirt_yaml.ml | 5 +
 1 file changed, 5 insertions(+)

diff --git a/output/create_kubevirt_yaml.ml b/output/create_kubevirt_yaml.ml
index 689555e4dc..b96c742832 100644
--- a/output/create_kubevirt_yaml.ml
+++ b/output/create_kubevirt_yaml.ml
@@ -54,6 +54,11 @@ let create_kubevirt_yaml source inspect
   "pit", Assoc [ "tickPolicy", String "delay" ];
   "rtc", Assoc [ "tickPolicy", String "catchup" ];
 ];
+(* XXX Note that we may need to set "localtime" here
+ * depending on guestcaps.gcaps_bios_utc.  However that
+ * requires the following PR to be merged in Kubevirt:
+ * https://github.com/kubevirt/kubevirt/pull/9587
+ *)
 "utc", List []
   ]
 )
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v 4/5] -o qemu: Set -rtc base=localtime when guest expects BIOS set to localtime

2023-09-25 Thread Richard W.M. Jones
I didn't set the -rtc flag in the normal (UTC) case as that is the
default.
---
 output/output_qemu.ml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/output/output_qemu.ml b/output/output_qemu.ml
index 57b1e58da2..a9d7b37071 100644
--- a/output/output_qemu.ml
+++ b/output/output_qemu.ml
@@ -158,6 +158,8 @@ module QEMU = struct
 arg_list "-device" ["vmgenid"; sprintf "guid=%s" genid; "id=vmgenid0"]
 );
 
+if not guestcaps.gcaps_bios_utc then arg "-rtc" "base=localtime";
+
 arg_list "-machine" (machine_str ::
(if smm then ["smm=on"] else []) @
["accel=kvm:tcg"]);
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v 3/5] -o libvirt: Add to libvirt XML

2023-09-25 Thread Richard W.M. Jones
---
 output/create_libvirt_xml.ml | 10 --
 tests/test-v2v-i-ova.xml |  1 +
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/output/create_libvirt_xml.ml b/output/create_libvirt_xml.ml
index 964acd25fd..f97272ca31 100644
--- a/output/create_libvirt_xml.ml
+++ b/output/create_libvirt_xml.ml
@@ -292,10 +292,16 @@ let create_libvirt_xml ?pool source inspect
e "nvram" ["template", vars_template] [] ] in
 
 List.push_back_list os loader;
-!os in
+e "os" [] !os in
+
+  (* The  section. *)
+  let clock_section =
+let offset = if guestcaps.gcaps_bios_utc then "utc" else "localtime" in
+e "clock" [ "offset", offset ] [] in
 
   List.push_back_list body [
-e "os" [] os_section;
+os_section;
+clock_section;
 
 e "on_poweroff" [] [PCData "destroy"];
 e "on_reboot" [] [PCData "restart"];
diff --git a/tests/test-v2v-i-ova.xml b/tests/test-v2v-i-ova.xml
index e5907ea1cc..a41827bfd5 100644
--- a/tests/test-v2v-i-ova.xml
+++ b/tests/test-v2v-i-ova.xml
@@ -18,6 +18,7 @@
   
 hvm
   
+  
   destroy
   restart
   restart
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH v2v 1/5] types: Add gcaps_bios_utc to record if the BIOS is set to UTC or localtime

2023-09-25 Thread Richard W.M. Jones
This change simply adds the flag to guestcaps, and sets it to always
true, so there is no change to the output.
---
 lib/types.mli  | 5 +
 convert/convert_linux.ml   | 1 +
 convert/convert_windows.ml | 1 +
 lib/types.ml   | 3 +++
 4 files changed, 10 insertions(+)

diff --git a/lib/types.mli b/lib/types.mli
index 65ef2e35cf..44b35faeb0 100644
--- a/lib/types.mli
+++ b/lib/types.mli
@@ -277,6 +277,11 @@ type guestcaps = {
   gcaps_virtio_1_0 : bool;
   (** The guest supports the virtio devices that it does at the virtio-1.0
   protocol level. *)
+
+  gcaps_bios_utc : bool;
+  (** Is the BIOS set to UTC ([true]) or localtime ([false])?  For
+  Linux guests this is always true.  For Windows we find out
+  what the guest is expecting by looking at the registry. *)
 }
 (** Guest capabilities after conversion.  eg. Was virtio found or installed? *)
 
diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml
index 8d7020848f..a70d43224c 100644
--- a/convert/convert_linux.ml
+++ b/convert/convert_linux.ml
@@ -220,6 +220,7 @@ let convert (g : G.guestfs) source inspect i_firmware _ 
keep_serial_console _ =
   gcaps_arch = Utils.kvm_arch inspect.i_arch;
   gcaps_arch_min_version = arch_min_version;
   gcaps_virtio_1_0 = virtio_1_0;
+  gcaps_bios_utc = true; (* almost all Linux expect BIOS to be UTC *)
 } in
 
 guestcaps
diff --git a/convert/convert_windows.ml b/convert/convert_windows.ml
index 122d95469d..f6e039be7e 100644
--- a/convert/convert_windows.ml
+++ b/convert/convert_windows.ml
@@ -275,6 +275,7 @@ let convert (g : G.guestfs) _ inspect i_firmware 
block_driver _ static_ips =
   gcaps_arch = Utils.kvm_arch inspect.i_arch;
   gcaps_arch_min_version = 0;
   gcaps_virtio_1_0 = virtio_win_installed.Inject_virtio_win.virtio_1_0;
+  gcaps_bios_utc = true;
 } in
 
 guestcaps
diff --git a/lib/types.ml b/lib/types.ml
index 75c14fd4f6..0f659ebd9a 100644
--- a/lib/types.ml
+++ b/lib/types.ml
@@ -399,6 +399,7 @@ type guestcaps = {
   gcaps_arch : string;
   gcaps_arch_min_version : int;
   gcaps_virtio_1_0 : bool;
+  gcaps_bios_utc : bool;
 }
 and guestcaps_block_type = Virtio_blk | Virtio_SCSI | IDE
 and guestcaps_net_type = Virtio_net | E1000 | RTL8139
@@ -429,6 +430,7 @@ let string_of_guestcaps gcaps =
gcaps_arch = %s\n\
gcaps_arch_min_version = %d\n\
gcaps_virtio_1_0 = %b\n\
+   gcaps_bios_utc = %b\n\
   "
   (string_of_block_type gcaps.gcaps_block_bus)
   (string_of_net_type gcaps.gcaps_net_bus)
@@ -440,6 +442,7 @@ let string_of_guestcaps gcaps =
   gcaps.gcaps_arch
   gcaps.gcaps_arch_min_version
   gcaps.gcaps_virtio_1_0
+  gcaps.gcaps_bios_utc
 
 type target_buses = {
   target_virtio_blk_bus : target_bus_slot array;
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-25 Thread Richard W.M. Jones
On Mon, Sep 25, 2023 at 01:25:06PM +0100, Daniel P. Berrangé wrote:
> On Mon, Sep 25, 2023 at 01:14:40PM +0100, Richard W.M. Jones wrote:
> > On Mon, Sep 25, 2023 at 01:09:15PM +0200, Lee Garrett wrote:
> > > On 23.09.23 19:37, Laszlo Ersek wrote:
> > > >On 9/22/23 16:47, Lee Garrett wrote:
> > > >>On 22.09.23 14:54, Richard W.M. Jones wrote:
> > > >>>On Fri, Sep 22, 2023 at 11:40:03AM +0100, Richard W.M. Jones wrote:
> > > >>>>On Thu, Sep 21, 2023 at 07:47:52PM +0200, Lee Garrett wrote:
> > > >>>>>On 21.09.23 19:43, Richard W.M. Jones wrote:
> > > >>>>>>So this is probably another instance or variation of the timezone
> > > >>>>>>formatting problem (of schtasks).  Which version of virt-v2v is 
> > > >>>>>>this?
> > > >>>>>>I want to check that you have a version with all the latest patches 
> > > >>>>>>in
> > > >>>>>>this area.
> > > >>>>>
> > > >>>>>It's 2.2.0-1 from Debian (12) bookworm. I've verified that it
> > > >>>>>doesn't have any distro-specific patches.
> > > >>>>>
> > > >>>>>(https://salsa.debian.org/libvirt-team/virt-v2v/-/tree/debian/master/debian
> > > >>>>>would have a patches/series file in this case)
> > > >>>>
> > > >>>>The timezone fixes are:
> > > >>>>
> > > >>>>commit 597d177567234c3a539098c423649781424eeb6f
> > > >>>>Author: Laszlo Ersek 
> > > >>>>Date:   Tue Mar 8 15:30:51 2022 +0100
> > > >>>>
> > > >>>>  convert_windows: rewrite "configure_qemu_ga" script purely in
> > > >>>>PowerShell
> > > >>>>
> > > >>>>commit d9dc6c42ae64ba92993dbd9477f003ba73fcfa2f
> > > >>>>Author: Richard W.M. Jones 
> > > >>>>Date:   Fri Nov 12 08:47:55 2021 +
> > > >>>>
> > > >>>>  convert/convert_windows.ml: Handle date formats with dots
> > > >>>>instead of /
> > > >>>>
> > > >>>>They are all included in >= 2.0
> > > >>>>
> > > >>>>I wonder if 597d177567 has a subtle flaw, or if we introduced a bug
> > > >>>>somewhere when refactoring this code later.
> > > >>>>
> > > >>>>Lee: Do you have a theory about exactly what is wrong with the
> > > >>>>schtasks date?  Like what was it supposed to be, assuming it was 120
> > > >>>>seconds in the future from boot time, versus what it was set to:
> > > >>>>
> > > >>>>>Firstboot-qemu-ga    9/21/2023 4:04:00 PM   Ready
> > > >>>>
> > > >>>>Could a date or time field have not been swapped or been corrupted
> > > >>>>in some predictable way?
> > > >>>
> > > >>>Or in even simpler terms, what is the time (and timezone) that
> > > >>>this ^^^ machine was booted?
> > > >>
> > > >>I believe I have it figured out.
> > > >>The guest local time is currently 7:08 AM (a few minutes after
> > > >>firstboot/provisioning), pacific daylight time (UTC-7, though Windows
> > > >>displays it as "UTC-08:00"). This is the timezone that the guest comes
> > > >>configured with at first boot. The task is scheduled for 2:01 PM,
> > > >>meaning it's scheduled to run ~7 hours in the future.
> > > >>
> > > >>So it seems like the task was meant to be scheduled for 2:01 PM UTC (=
> > > >>7:01 AM PDT), but for some reason was scheduled for 2:01 PM *local 
> > > >>time*.
> > > >>
> > > >> From what I can see, the host machine time zone is irrelevant (UTC+2).
> > > >>
> > > >>I don't know where the timezone mixup comes from, though. Running
> > > >>`(get-date)` in the powershell at this point correctly returns the local
> > > >>time (7:08 AM). I guess during injection the time is in UTC, and
> > > >>schtasks.exe has no awareness of timezones?
> > > >
> > > >Right, I think there is a timezone disagreement between how we format 
> > > >the timestamp and how schtas

Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-25 Thread Richard W.M. Jones
On Mon, Sep 25, 2023 at 01:09:15PM +0200, Lee Garrett wrote:
> On 23.09.23 19:37, Laszlo Ersek wrote:
> >On 9/22/23 16:47, Lee Garrett wrote:
> >>On 22.09.23 14:54, Richard W.M. Jones wrote:
> >>>On Fri, Sep 22, 2023 at 11:40:03AM +0100, Richard W.M. Jones wrote:
> >>>>On Thu, Sep 21, 2023 at 07:47:52PM +0200, Lee Garrett wrote:
> >>>>>On 21.09.23 19:43, Richard W.M. Jones wrote:
> >>>>>>So this is probably another instance or variation of the timezone
> >>>>>>formatting problem (of schtasks).  Which version of virt-v2v is this?
> >>>>>>I want to check that you have a version with all the latest patches in
> >>>>>>this area.
> >>>>>
> >>>>>It's 2.2.0-1 from Debian (12) bookworm. I've verified that it
> >>>>>doesn't have any distro-specific patches.
> >>>>>
> >>>>>(https://salsa.debian.org/libvirt-team/virt-v2v/-/tree/debian/master/debian
> >>>>>would have a patches/series file in this case)
> >>>>
> >>>>The timezone fixes are:
> >>>>
> >>>>commit 597d177567234c3a539098c423649781424eeb6f
> >>>>Author: Laszlo Ersek 
> >>>>Date:   Tue Mar 8 15:30:51 2022 +0100
> >>>>
> >>>>  convert_windows: rewrite "configure_qemu_ga" script purely in
> >>>>PowerShell
> >>>>
> >>>>commit d9dc6c42ae64ba92993dbd9477f003ba73fcfa2f
> >>>>Author: Richard W.M. Jones 
> >>>>Date:   Fri Nov 12 08:47:55 2021 +
> >>>>
> >>>>  convert/convert_windows.ml: Handle date formats with dots
> >>>>instead of /
> >>>>
> >>>>They are all included in >= 2.0
> >>>>
> >>>>I wonder if 597d177567 has a subtle flaw, or if we introduced a bug
> >>>>somewhere when refactoring this code later.
> >>>>
> >>>>Lee: Do you have a theory about exactly what is wrong with the
> >>>>schtasks date?  Like what was it supposed to be, assuming it was 120
> >>>>seconds in the future from boot time, versus what it was set to:
> >>>>
> >>>>>Firstboot-qemu-ga    9/21/2023 4:04:00 PM   Ready
> >>>>
> >>>>Could a date or time field have not been swapped or been corrupted
> >>>>in some predictable way?
> >>>
> >>>Or in even simpler terms, what is the time (and timezone) that
> >>>this ^^^ machine was booted?
> >>
> >>I believe I have it figured out.
> >>The guest local time is currently 7:08 AM (a few minutes after
> >>firstboot/provisioning), pacific daylight time (UTC-7, though Windows
> >>displays it as "UTC-08:00"). This is the timezone that the guest comes
> >>configured with at first boot. The task is scheduled for 2:01 PM,
> >>meaning it's scheduled to run ~7 hours in the future.
> >>
> >>So it seems like the task was meant to be scheduled for 2:01 PM UTC (=
> >>7:01 AM PDT), but for some reason was scheduled for 2:01 PM *local time*.
> >>
> >> From what I can see, the host machine time zone is irrelevant (UTC+2).
> >>
> >>I don't know where the timezone mixup comes from, though. Running
> >>`(get-date)` in the powershell at this point correctly returns the local
> >>time (7:08 AM). I guess during injection the time is in UTC, and
> >>schtasks.exe has no awareness of timezones?
> >
> >Right, I think there is a timezone disagreement between how we format the 
> >timestamp and how schtasks.exe takes it.
> >
> >What matters here is the /ST (start time) flag.
> >
> >Today we have (in the common submodule):
> >
> >   add "$d = (get-date).AddSeconds(120)";
> >   add "$dtfinfo = 
> > [System.Globalization.DateTimeFormatInfo]::CurrentInfo";
> >   add "$sdp = $dtfinfo.ShortDatePattern";
> >   add "$sdp = $sdp -replace 'y+', ''";
> >   add "$sdp = $sdp -replace 'M+', 'MM'";
> >   add "$sdp = $sdp -replace 'd+', 'dd'";
> >   add "schtasks.exe /Create /SC ONCE `";
> >   add "  /ST $d.ToString('HH:mm') /SD $d.ToString($sdp) `";
> >   add "  /RU SYSTEM /TN Firstboot-qemu-ga `";
> >   add (sprintf "  /TR \"C:\\%s /forcerestart /qn /l+*vx C:\\%s.log\""
> >  ms

Re: [Libguestfs] [EXTERNAL] - Re: Libguestfs desired version installation

2023-09-25 Thread Richard W.M. Jones
On Mon, Sep 25, 2023 at 09:09:32AM +, Teja Konapalli wrote:
> Hi Richard,
>
> Yes, you are correct my ask is without upgrading OS version(from 8.2
> to 9.0) is it possible to upgrade libguestfs alone to different
> versions.

No, that is not supported.

Rich.

> Teja
> 
> -Original Message-----
> From: Richard W.M. Jones  
> Sent: Monday, September 25, 2023 2:36 PM
> To: Teja Konapalli 
> Cc: libguestfs@redhat.com; Raja Ram Sharma ; Divyanshu 
> Kumar 
> Subject: [EXTERNAL] - Re: Libguestfs desired version installation
> 
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open attachments unless you recognize the sender and know the 
> content is safe. If you feel that the email is suspicious, please report it 
> using PhishAlarm.
> 
> 
> I read the first email, you don't need to keep sending it.
> 
> If you upgrade RHEL from 8.2 to the latest (8.8 or 8.9) you will get 
> libguestfs 1.44.0.
> 
> If you upgrade to RHEL 9 you will get libguestfs 1.50.1.
> 
> Since you have (still) not described the exact environment you're using, or 
> even accurately described the problem you're having, I don't know if this 
> will make any difference.
> 
> Rich.
> 
> --
> Richard Jones, Virtualization Group, Red Hat 
> https://urldefense.com/v3/__http://people.redhat.com/*rjones__;fg!!Obbck6kTJA!amwb8TmXfcQ2vZeYTfXu4kiNMITCUSbhMBLKj2ZMXMJhxiXtMaNfgWaYXNR9-hQoAQHPNKXHHdDJYB-efg$
> Read my programming and virtualization blog: 
> https://urldefense.com/v3/__http://rwmj.wordpress.com__;!!Obbck6kTJA!amwb8TmXfcQ2vZeYTfXu4kiNMITCUSbhMBLKj2ZMXMJhxiXtMaNfgWaYXNR9-hQoAQHPNKXHHdBA-vNKXQ$
> nbdkit - Flexible, fast NBD server with plugins 
> https://urldefense.com/v3/__https://gitlab.com/nbdkit/nbdkit__;!!Obbck6kTJA!amwb8TmXfcQ2vZeYTfXu4kiNMITCUSbhMBLKj2ZMXMJhxiXtMaNfgWaYXNR9-hQoAQHPNKXHHdBuQVbubQ$

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Libguestfs desired version installation

2023-09-25 Thread Richard W.M. Jones


I read the first email, you don't need to keep sending it.

If you upgrade RHEL from 8.2 to the latest (8.8 or 8.9) you
will get libguestfs 1.44.0.

If you upgrade to RHEL 9 you will get libguestfs 1.50.1.

Since you have (still) not described the exact environment you're
using, or even accurately described the problem you're having, I don't
know if this will make any difference.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Some comments on the libnbd apk in Alpine

2023-09-23 Thread Richard W.M. Jones
On Sat, Sep 23, 2023 at 11:39:12AM +0200, Sean McAvoy wrote:
> Hello Richard,
> I have updated the version to 1.16.0 and integrated your suggestions in Merge
> Request [1].
> I have stared the project and will get updates when new releases are made and
> will update the Alpine package soon after.

Great thanks.  Just a note that the stable branch (1.16.4)
has a few fixes:

https://download.libguestfs.org/libnbd/1.16-stable/

Rich.

> Thanks for making great software.
> 
> Sean
> 
> 
> [1] https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/52132
> 
> On Thu, Sep 21, 2023 at 6:02 PM Richard W.M. Jones  wrote:
> 
> Hi Sean, the maintainers of libnbd here.  Thanks for packaging libnbd
> in Alpine:
> 
> https://git.alpinelinux.org/aports/tree/testing/libnbd/APKBUILD
> 
> I have a few comments:
> 
> (1) The url field should be "https://gitlab.com/nbdkit/libnbd;
> (The one you are using is an old, stale mirror)
> 
> (2) libxml2-dev is not listed as a dependency.
> 
> We had a request from an Alpine user to add this dependency because it
> enables several important features in libnbd, namely NBD URI support
> and some utilities which require URI support.
> 
> It shouldn't be too much trouble to add since it appears that libxml2
> is already built in Alpine.
> 
> (3) libnbd 1.18 (next stable) will be released in a week or two:
> https://listman.redhat.com/archives/libguestfs/2023-September/032590.html
> 
> If you have any questions or comments about the package you can send
> them to me and/or the mailing list (without subscribing if you don't
> want to).
> 
> Rich.
> 
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/
> ~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-p2v converts physical machines to virtual machines.  Boot with a
> live CD or over the network (PXE) and turn machines into KVM guests.
> http://libguestfs.org/virt-v2v
> 
> 

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


Re: [Libguestfs] [libnbd PATCH] info: Try harder for graceful disconnect from server

2023-09-22 Thread Richard W.M. Jones
On Fri, Sep 22, 2023 at 12:33:36PM -0500, Eric Blake wrote:
> There are a number of ways in which gathering information can fail.
> But when possible, it makes sense to let the server know that we are
> done talking, as it minimizes the likelihood that nbdinfo's error
> message will be obscured by an accompanying error message by the
> server complaining about an unclean disconnect.
> 
> For example, with a one-off qemu temporarily patched as mentioned in
> commit 0f8ee8c6 to advertise sizes larger than 2^63, kicking off
> 'qemu-nbd -f raw -r file &' produces:
> 
> pre-patch:
> 
> $ ./run nbdinfo --size nbd://localhost
> /home/eblake/libnbd/info/.libs/nbdinfo: nbd_get_size: server claims size 
> 9223372036854781440 which does not fit in signed result: Value too large for 
> defined data type
> qemu-nbd: option negotiation failed: Failed to read opts magic: Unexpected 
> end-of-file before all data were read

This doesn't necessarily seem like a bug?

This feels like quite a significant change in behaviour to me.  In
particular I'm worried if it interacts in some subtle way with the
forking done by the "[ ... ]" syntax for running servers on the
command line (or any of the other ways that libnbd might fork/exec).

Can we hold this patch until after 1.18 is released and then put it
in?  Should only be a week or two.

Provisionally ACKed for post-1.18

Rich.

> post-patch:
> $ ./run nbdinfo --size nbd://localhost
> /home/eblake/libnbd/info/.libs/nbdinfo: nbd_get_size: server claims size 
> 9223372036854781440 which does not fit in signed result: Value too large for 
> defined data type
> 
> if nbdinfo is run in the same terminal as qemu-nbd.
> 
> Signed-off-by: Eric Blake 
> ---
>  info/main.c | 13 +
>  1 file changed, 13 insertions(+)
> 
> diff --git a/info/main.c b/info/main.c
> index c8248c61..05f19f43 100644
> --- a/info/main.c
> +++ b/info/main.c
> @@ -88,6 +88,17 @@ usage (FILE *fp, int exitcode)
>exit (exitcode);
>  }
> 
> +void
> +clean_shutdown (void)
> +{
> +  /* If we are connected but detect an error, try to give the server
> +   * notice that we are done talking.  Ignore failures, as this is
> +   * only a courtesy measure.
> +   */
> +  if (nbd)
> +nbd_shutdown (nbd, 0);
> +}
> +
>  int
>  main (int argc, char *argv[])
>  {
> @@ -277,6 +288,7 @@ main (int argc, char *argv[])
>  fprintf (stderr, "%s: %s\n", progname, nbd_get_error ());
>  exit (EXIT_FAILURE);
>}
> +  atexit (clean_shutdown);
>nbd_set_uri_allow_local_file (nbd, true); /* Allow ?tls-psk-file. */
> 
>/* Set optional modes in the handle. */
> @@ -353,6 +365,7 @@ main (int argc, char *argv[])
>free_exports ();
>nbd_shutdown (nbd, 0);
>nbd_close (nbd);
> +  nbd = NULL;
> 
>/* Close the output stream and copy it to the real stdout. */
>if (fclose (fp) == EOF) {
> -- 
> 2.41.0
> 



-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-22 Thread Richard W.M. Jones
On Fri, Sep 22, 2023 at 05:27:25PM +0200, Lee Garrett wrote:
> On 22.09.23 16:47, Lee Garrett wrote:
> >On 22.09.23 14:54, Richard W.M. Jones wrote:
> >>On Fri, Sep 22, 2023 at 11:40:03AM +0100, Richard W.M. Jones wrote:
> >>>On Thu, Sep 21, 2023 at 07:47:52PM +0200, Lee Garrett wrote:
> >>>>On 21.09.23 19:43, Richard W.M. Jones wrote:
> >>>>>So this is probably another instance or variation of the timezone
> >>>>>formatting problem (of schtasks).  Which version of virt-v2v is this?
> >>>>>I want to check that you have a version with all the latest patches in
> >>>>>this area.
> >>>>
> >>>>It's 2.2.0-1 from Debian (12) bookworm. I've verified that it
> >>>>doesn't have any distro-specific patches.
> >>>>
> >>>>(https://salsa.debian.org/libvirt-team/virt-v2v/-/tree/debian/master/debian
> >>>>would have a patches/series file in this case)
> >>>
> >>>The timezone fixes are:
> >>>
> >>>commit 597d177567234c3a539098c423649781424eeb6f
> >>>Author: Laszlo Ersek 
> >>>Date:   Tue Mar 8 15:30:51 2022 +0100
> >>>
> >>> convert_windows: rewrite "configure_qemu_ga" script purely in 
> >>>PowerShell
> >>>
> >>>commit d9dc6c42ae64ba92993dbd9477f003ba73fcfa2f
> >>>Author: Richard W.M. Jones 
> >>>Date:   Fri Nov 12 08:47:55 2021 +
> >>>
> >>> convert/convert_windows.ml: Handle date formats with dots instead of /
> >>>
> >>>They are all included in >= 2.0
> >>>
> >>>I wonder if 597d177567 has a subtle flaw, or if we introduced a bug
> >>>somewhere when refactoring this code later.
> >>>
> >>>Lee: Do you have a theory about exactly what is wrong with the
> >>>schtasks date?  Like what was it supposed to be, assuming it was 120
> >>>seconds in the future from boot time, versus what it was set to:
> >>>
> >>>>Firstboot-qemu-ga    9/21/2023 4:04:00 PM   Ready
> >>>
> >>>Could a date or time field have not been swapped or been corrupted
> >>>in some predictable way?
> >>
> >>Or in even simpler terms, what is the time (and timezone) that
> >>this ^^^ machine was booted?
> >
> >I believe I have it figured out.
> >The guest local time is currently 7:08 AM (a few minutes after
> >firstboot/provisioning), pacific daylight time (UTC-7, though
> >Windows displays it as "UTC-08:00"). This is the timezone that the
> >guest comes configured with at first boot. The task is scheduled
> >for 2:01 PM, meaning it's scheduled to run ~7 hours in the future.
> >
> >So it seems like the task was meant to be scheduled for 2:01 PM
> >UTC (= 7:01 AM PDT), but for some reason was scheduled for 2:01 PM
> >*local time*.
> >
> > From what I can see, the host machine time zone is irrelevant (UTC+2).
> >
> >I don't know where the timezone mixup comes from, though. Running
> >`(get-date)` in the powershell at this point correctly returns the
> >local time (7:08 AM). I guess during injection the time is in UTC,
> >and schtasks.exe has no awareness of timezones?
> 
> I digged a bit into the XML description and noticed this:
> 
> 
> 
> To my knowledge Windows runs the BIOS clock in local time. So This
> should probably be
> 
> 
>   
>   
>   
>   
> 
> 
> as this is what libvirt uses in the preset for Windows 11 hosts. I
> manually changed this before first boot, however it didn't solve the
> issue. It only increased the offset to +9 hours in the future (7
> hours difference to UTC, +2 hours of local timezone). How does the
> firstboot injection exactly happen? Is the guest actually booted for
> it?

No, the guest is not running.

Rich.

> >
> >>
> >>Rich.
> >>
> >>>The code we run is here:
> >>>
> >>>https://github.com/libguestfs/libguestfs-common/blob/e70d89a58dae068be2e19c7c21558707261af96a/mlcustomize/inject_virtio_win.ml#L571
> >>>
> >>>Ming: this could be a bug affecting PST (UTC-8) guests, perhaps
> >>>somehow related to having a single digit month field?
> >>>
> >>>Rich.
> >>>
> >>>-- 
> >>>Richard Jones, Virtualization Group, Red Hat 
> >>>http://people.redhat.com/~rjones
> >>>Read my programming and virtualization blog: http://rwmj.wordpress.com
> >>>libguestfs lets you edit virtual machines.  Supports shell scripting,
> >>>bindings from many languages.  http://libguestfs.org
> >>
> >

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-22 Thread Richard W.M. Jones
On Fri, Sep 22, 2023 at 11:40:03AM +0100, Richard W.M. Jones wrote:
> On Thu, Sep 21, 2023 at 07:47:52PM +0200, Lee Garrett wrote:
> > On 21.09.23 19:43, Richard W.M. Jones wrote:
> > >So this is probably another instance or variation of the timezone
> > >formatting problem (of schtasks).  Which version of virt-v2v is this?
> > >I want to check that you have a version with all the latest patches in
> > >this area.
> > 
> > It's 2.2.0-1 from Debian (12) bookworm. I've verified that it
> > doesn't have any distro-specific patches.
> > 
> > (https://salsa.debian.org/libvirt-team/virt-v2v/-/tree/debian/master/debian
> > would have a patches/series file in this case)
> 
> The timezone fixes are:
> 
> commit 597d177567234c3a539098c423649781424eeb6f
> Author: Laszlo Ersek 
> Date:   Tue Mar 8 15:30:51 2022 +0100
> 
> convert_windows: rewrite "configure_qemu_ga" script purely in PowerShell
> 
> commit d9dc6c42ae64ba92993dbd9477f003ba73fcfa2f
> Author: Richard W.M. Jones 
> Date:   Fri Nov 12 08:47:55 2021 +
> 
> convert/convert_windows.ml: Handle date formats with dots instead of /
> 
> They are all included in >= 2.0
> 
> I wonder if 597d177567 has a subtle flaw, or if we introduced a bug
> somewhere when refactoring this code later.
> 
> Lee: Do you have a theory about exactly what is wrong with the
> schtasks date?  Like what was it supposed to be, assuming it was 120
> seconds in the future from boot time, versus what it was set to:
> 
> > Firstboot-qemu-ga9/21/2023 4:04:00 PM   Ready
> 
> Could a date or time field have not been swapped or been corrupted
> in some predictable way?

Or in even simpler terms, what is the time (and timezone) that
this ^^^ machine was booted?

Rich.

> The code we run is here:
> 
> https://github.com/libguestfs/libguestfs-common/blob/e70d89a58dae068be2e19c7c21558707261af96a/mlcustomize/inject_virtio_win.ml#L571
> 
> Ming: this could be a bug affecting PST (UTC-8) guests, perhaps
> somehow related to having a single digit month field?
> 
> Rich.
> 
> -- 
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines.  Supports shell scripting,
> bindings from many languages.  http://libguestfs.org

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-22 Thread Richard W.M. Jones
On Thu, Sep 21, 2023 at 07:47:52PM +0200, Lee Garrett wrote:
> On 21.09.23 19:43, Richard W.M. Jones wrote:
> >So this is probably another instance or variation of the timezone
> >formatting problem (of schtasks).  Which version of virt-v2v is this?
> >I want to check that you have a version with all the latest patches in
> >this area.
> 
> It's 2.2.0-1 from Debian (12) bookworm. I've verified that it
> doesn't have any distro-specific patches.
> 
> (https://salsa.debian.org/libvirt-team/virt-v2v/-/tree/debian/master/debian
> would have a patches/series file in this case)

The timezone fixes are:

commit 597d177567234c3a539098c423649781424eeb6f
Author: Laszlo Ersek 
Date:   Tue Mar 8 15:30:51 2022 +0100

convert_windows: rewrite "configure_qemu_ga" script purely in PowerShell

commit d9dc6c42ae64ba92993dbd9477f003ba73fcfa2f
Author: Richard W.M. Jones 
Date:   Fri Nov 12 08:47:55 2021 +

convert/convert_windows.ml: Handle date formats with dots instead of /

They are all included in >= 2.0

I wonder if 597d177567 has a subtle flaw, or if we introduced a bug
somewhere when refactoring this code later.

Lee: Do you have a theory about exactly what is wrong with the
schtasks date?  Like what was it supposed to be, assuming it was 120
seconds in the future from boot time, versus what it was set to:

> Firstboot-qemu-ga9/21/2023 4:04:00 PM   Ready

Could a date or time field have not been swapped or been corrupted
in some predictable way?

The code we run is here:

https://github.com/libguestfs/libguestfs-common/blob/e70d89a58dae068be2e19c7c21558707261af96a/mlcustomize/inject_virtio_win.ml#L571

Ming: this could be a bug affecting PST (UTC-8) guests, perhaps
somehow related to having a single digit month field?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [libnbd PATCH v2 2/6] fuzzing: Disable client-side strictness checks

2023-09-22 Thread Richard W.M. Jones
On Thu, Sep 21, 2023 at 10:22:08PM +0100, Richard W.M. Jones wrote:
> On Thu, Sep 21, 2023 at 03:58:01PM -0500, Eric Blake wrote:
> > When fuzzing, it is more desirable to always provoke the server into
> > sending a response, rather than sometimes accidentally skipping a wire
> > call because a client-side strictness test failed.
> > 
> > [Our fuzzer could probably be made even more powerful by changing the
> > fuzzer input file to be a series of records, where each record is the
> > API to call and then the server's response; right now, the sequence of
> > APIs called is hard-coded, which is not as powerful at testing
> > potential cross-command coupling. But that's a project for another
> > day.]
> 
> Intrigued by how this would work exactly.  The current file is just a
> stream of bytes from the "server" (the output from libnbd is
> discarded).  Would the input file be of the form:
> 
>   ( [ request + args ] [ stream of bytes from server ] )*
> 
> where [ request + args ] would be a simple serialized representation
> of a synchronous nbd_* API call?  I think that would work ...

Actually I don't think this would work.  The stream of bytes is
variable length so it would need a sentinel to indicate the end of one
stream (hard to do in-stream) or a length field up front.  However
fuzzing doesn't work well with length + data since the fuzzer doesn't
understand the relationship.

This could work (I've abbreviated "request + args" to just "request"
and "stream ..." to "stream"):

  ( [ request ] )* [ sentinel ] [ stream ](* case B *)

where [ sentinel ] is some sentinel that can be distinguished from a
request.

The idea here is the fuzzing wrapper would read the set of requests
into an internal structure, then process the stream as it does now
while issuing the requests.

However I still don't think the fuzzer would work well with this.  The
problem is that the stream of bytes is still related to the set of
requests.  For example if a new request was inserted then some later
point in the stream would need to change, but the fuzzer wouldn't
understand this relationship.  [Now I'm thinking if only we could use
an AI transformer it would solve this problem too!  Transformers can
find these kind of long-range moving dependencies.]

This seems better still:

  [ request ] [ stream ] (* case C *)

There's a single request (which can represent exactly one nbd_*
request), and a single associated stream.  The fuzzer would generate
alternate nbd_* APIs as it mutated the request field.

However the problem here is we no longer have any way to describe
multiple dependent requests.

So ...

I think my answer is to go back to case B:

  ( [ request ] )* [ sentinel ] [ stream ]

but make the starting test cases all look (nearly) like case C:

  [ request ] [ sentinel ] [ stream ]

The fuzzer would start by finding the simple test cases looking like
C, and then might create longer test cases looking like B later in the
run (or might not, we'd need to examine the output after a long run).

Anyway, something for a rainy day.

Rich.

> Rich.
> 
> > Signed-off-by: Eric Blake 
> > Reviewed-by: Richard W.M. Jones 
> > Reviewed-by: Laszlo Ersek 
> > ---
> >  fuzzing/libnbd-fuzz-wrapper.c | 5 -
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> > 
> > diff --git a/fuzzing/libnbd-fuzz-wrapper.c b/fuzzing/libnbd-fuzz-wrapper.c
> > index cbd55380..fcd1d04c 100644
> > --- a/fuzzing/libnbd-fuzz-wrapper.c
> > +++ b/fuzzing/libnbd-fuzz-wrapper.c
> > @@ -193,8 +193,11 @@ client (int sock)
> >}
> > 
> >/* Note we ignore errors in these calls because we are only
> > -   * interested in whether the process crashes.
> > +   * interested in whether the process crashes.  Likewise, we don't
> > +   * want to accidentally avoid sending traffic to the server merely
> > +   * because client side strictness sees a problem.
> > */
> > +  nbd_set_strict_mode (nbd, 0);
> > 
> >/* Enable a metadata context, for block status below. */
> >nbd_add_meta_context (nbd, LIBNBD_CONTEXT_BASE_ALLOCATION);
> > -- 
> > 2.41.0
> 
> -- 
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-top is 'top' for virtual machines.  Tiny program with many
> powerful monitoring features, net stats, disk stats, logging, etc.
> http://people.redhat.com/~rjones/virt-top

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [libnbd PATCH v2 6/6] info: Tolerate missing size

2023-09-21 Thread Richard W.M. Jones
On Thu, Sep 21, 2023 at 03:58:05PM -0500, Eric Blake wrote:
> As previous patches showed, the NBD spec does not yet forbid a server
> sending us a size that does not fit in int64_t.  We should gracefully
> handle this during nbdinfo, rather than giving up early.
> 
> With the same one-line hack to qemu to set the most significant bit of
> the export size, output changes from pre-patch:
> 
> $ ./run nbdinfo nbd://localhost
> /home/eblake/libnbd/info/.libs/nbdinfo: nbd_get_size: server claims size 
> 9223372036854781440 which does not fit in signed result: Value too large for 
> defined data type
> qemu-nbd: option negotiation failed: Failed to read opts magic: Unexpected 
> end-of-file before all data were read
> 
> to post-patch:
> 
> $ ./run nbdinfo nbd://localhost
> protocol: newstyle-fixed without TLS, using extended packets
> ...
>   block_size_maximum: 33554432
> 
> or
> 
> $ ./run nbdinfo nbd://localhost --json
> {
> "protocol": "newstyle-fixed",
> ...
>   "block_size_maximum": 33554432,
>   "export-size-str": "unavailable"
>   } ]
> }
> 
> Sadly, since writing a server with such large export sizes requires a
> one-off hack, I don't see the point in adding a unit test.
> 
> Signed-off-by: Eric Blake 
> ---
>  info/show.c | 25 +
>  1 file changed, 13 insertions(+), 12 deletions(-)
> 
> diff --git a/info/show.c b/info/show.c
> index a71d837e..3d80545e 100644
> --- a/info/show.c
> +++ b/info/show.c
> @@ -46,7 +46,7 @@ show_one_export (struct nbd_handle *nbd, const char *desc,
>   bool first, bool last)
>  {
>int64_t i, size;
> -  char size_str[HUMAN_SIZE_LONGEST];
> +  char size_str[HUMAN_SIZE_LONGEST] = "unavailable";
>bool human_size_flag;
>char *export_name = NULL;
>char *export_desc = NULL;
> @@ -89,13 +89,10 @@ show_one_export (struct nbd_handle *nbd, const char *desc,
>  return false;
>}
>size = nbd_get_size (nbd);
> -  if (size == -1) {
> -fprintf (stderr, "%s: %s\n", progname, nbd_get_error ());
> -exit (EXIT_FAILURE);
> +  if (size >= 0) {
> +human_size (size_str, size, _size_flag);
>}
> 
> -  human_size (size_str, size, _size_flag);
> -
>if (uri_is_meaningful ())
>  uri = nbd_get_uri (nbd);
> 
> @@ -130,7 +127,8 @@ show_one_export (struct nbd_handle *nbd, const char *desc,
>  show_context = true;
> 
>/* Get content last, as it moves the connection out of negotiating */
> -  content = get_content (nbd, size);
> +  if (size >= 0)
> +content = get_content (nbd, size);
> 
>if (!json_output) {
>  ansi_colour (ANSI_FG_BOLD_BLACK, fp);
> @@ -140,10 +138,12 @@ show_one_export (struct nbd_handle *nbd, const char 
> *desc,
>  fprintf (fp, ":\n");
>  if (desc && *desc)
>fprintf (fp, "\tdescription: %s\n", desc);
> -if (human_size_flag)
> -  fprintf (fp, "\texport-size: %" PRIi64 " (%s)\n", size, size_str);
> -else
> -  fprintf (fp, "\texport-size: %" PRIi64 "\n", size);
> +if (size >= 0) {
> +  if (human_size_flag)
> +fprintf (fp, "\texport-size: %" PRIi64 " (%s)\n", size, size_str);
> +  else
> +fprintf (fp, "\texport-size: %" PRIi64 "\n", size);
> +}
>  if (content)
>fprintf (fp, "\tcontent: %s\n", content);
>  if (uri)
> @@ -273,7 +273,8 @@ show_one_export (struct nbd_handle *nbd, const char *desc,
> block_maximum);
> 
>  /* Put this one at the end because of the stupid comma thing in JSON. */
> -fprintf (fp, "\t\"export-size\": %" PRIi64 ",\n", size);
> +if (size >= 0)
> +  fprintf (fp, "\t\"export-size\": %" PRIi64 ",\n", size);
>  fprintf (fp, "\t\"export-size-str\": \"%s\"\n", size_str);
> 
>  if (last)

Reviewed-by: Richard W.M. Jones 


-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [libnbd PATCH v2 4/6] block_status: Fix assertion with large server size

2023-09-21 Thread Richard W.M. Jones
On Thu, Sep 21, 2023 at 03:58:03PM -0500, Eric Blake wrote:
> As mentioned in the previous commit ("api: Sanitize sizes larger than
> INT64_MAX"), the NBD spec does not (yet) prohibit a server from
> advertising a size larger than INT64_MAX.  While we can't report such
> size to the user, v1.16 was at least internally consistent with the
> server's size everywhere else.
> 
> But when adding code to implement 64-bit block status, I intentionally
> wanted to guarantee that the callback sees a positive int64_t length
> even when the server's wire value can be 64 bits, for ease in writing
> language bindings (OCaml in particular benefitted from that
> guarantee), even though I didn't document that in the API until now.
> That was because I had blindly assumed that the server's exportsize
> fit in 63 bits, and therefore I didn't have to worry about arithmetic
> overflow once I capped the extent length to exportsize.  But the
> fuzzer quickly proved me wrong.  What's more, with the same one-line
> hack to qemu as shown in the previous commit to advertise a size with
> the high-order bit set,
> 
> $ ./run nbdsh --base -u nbd://localhost -c - <<\EOF
> > def f(*k):
> >  pass
> > h.block_status(1,0,f)
> > EOF
> nbdsh: generator/states-reply-chunk.c:554: 
> enter_STATE_REPLY_CHUNK_REPLY_RECV_BS_ENTRIES: Assertion `h->exportsize <= 
> INT64_MAX' failed.
> Aborted (core dumped)
> 
> even though it did not dump core in v1.16.
> 
> Since my assumption was bad, rewrite the logic to increment total
> after bounds-checking rather than before, and to bounds-check based on
> INT64_MAX+1-64M rather than on the export size.  As before, we never
> report a zero-length extent to the callback.  Whether or not secalert
> advises us to create a CVE for the previous patch, this bug does not
> deserve its own CVE as it was only introduced in recent unstable
> releases.
> 
> Fixes: e8d837d306 ("block_status: Add some sanity checking of server 
> lengths", v1.17.4)
> Thanks: Richard W.M. Jones 
> Signed-off-by: Eric Blake 
> ---
>  generator/states-reply-chunk.c | 49 ++
>  generator/C.ml |  2 +-
>  2 files changed, 27 insertions(+), 24 deletions(-)
> 
> diff --git a/generator/states-reply-chunk.c b/generator/states-reply-chunk.c
> index 2cebe456..20407d91 100644
> --- a/generator/states-reply-chunk.c
> +++ b/generator/states-reply-chunk.c
> @@ -547,11 +547,16 @@  REPLY.CHUNK_REPLY.RECV_BS_ENTRIES:
>break;
>  }
> 
> +/* Be careful to avoid arithmetic overflow, even when the user
> + * disabled LIBNBD_STRICT_BOUNDS to pass a suspect offset, or the
> + * server returns suspect lengths or advertised exportsize larger
> + * than 63 bits.  We guarantee that callbacks will not see a
> + * length exceeding INT64_MAX or the advertised h->exportsize.
> + */
>  name = h->meta_contexts.ptr[i].name;
> -total = 0;
> -cap = h->exportsize - cmd->offset;
> -assert (cap <= h->exportsize);
> -assert (h->exportsize <= INT64_MAX);
> +total = cap = 0;
> +if (cmd->offset <= h->exportsize)
> +  cap = h->exportsize - cmd->offset;
> 
>  /* Need to byte-swap the entries returned into the callback size
>   * requested by the caller.  The NBD protocol allows truncation as
> @@ -560,10 +565,11 @@  REPLY.CHUNK_REPLY.RECV_BS_ENTRIES:
>   * don't like.  We stop iterating on a zero-length extent (error
>   * only if it is the first extent), on an extent beyond the
>   * exportsize (unconditional error after truncating to
> - * exportsize), and on an extent exceeding a 32-bit callback (no
> - * error, and to simplify alignment, we truncate to 4G-64M); but
> - * do not diagnose issues with the server's length alignments,
> - * flag values, nor compliance with the REQ_ONE command flag.
> + * exportsize), and on an extent exceeding a callback length limit
> + * (no error, and to simplify alignment, we truncate to 64M before
> + * the limit); but we do not diagnose issues with the server's
> + * length alignments, flag values, nor compliance with the REQ_ONE
> + * command flag.
>   */
>  for (i = 0, stop = false; i < h->bs_count && !stop; ++i) {
>if (type == NBD_REPLY_TYPE_BLOCK_STATUS) {
> @@ -572,16 +578,12 @@  REPLY.CHUNK_REPLY.RECV_BS_ENTRIES:
>}
>else {
>  orig_len = len = be64toh (h->bs_raw.wide[i].length);
> -if (len > h->exportsize) {
> -  /* Since we already asserted exportsize is at most 63 bits,
> -   * this ensures the extent length will appear

Re: [Libguestfs] [libnbd PATCH v2 5/6] block_status: Fix assertion on bad 64-bit block status reply

2023-09-21 Thread Richard W.M. Jones
On Thu, Sep 21, 2023 at 03:58:04PM -0500, Eric Blake wrote:
> If a server replies to a block status command with an invalid count in
> NBD_REPLY_TYPE_BLOCK_STATUS_EXT, we correctly detect the server's
> error, but fail to mark that we've consumed enough data off the wire
> to resync back to the server's next reply.  Rich's fuzzing run
> initially found this, but I was able to quickly write a one-byte patch
> on top of my pending qemu patches [1] to reproduce it:
> 
> [1] https://lists.gnu.org/archive/html/qemu-devel/2023-08/msg05231.html
> 
> | diff --git i/nbd/server.c w/nbd/server.c
> | index 898580a9b0b..bd8d46ba3c4 100644
> | --- i/nbd/server.c
> | +++ w/nbd/server.c
> | @@ -2326,7 +2326,7 @@ nbd_co_send_extents(NBDClient *client, NBDRequest 
> *request, NBDExtentArray *ea,
> |  iov[1].iov_base = _ext;
> |  iov[1].iov_len = sizeof(meta_ext);
> |  stl_be_p(_ext.context_id, context_id);
> | -stl_be_p(_ext.count, ea->count);
> | +stl_be_p(_ext.count, !ea->count);
> |
> |  nbd_extent_array_convert_to_be(ea);
> |  iov[2].iov_base = ea->extents;
> 
> then with a just-built 'qemu-nbd -f raw -r file -t &' running, we have
> pre-patch:
> 
> $ ./run nbdsh --base -u nbd://localhost -c - <<\EOF
> > def f(*k):
> >  pass
> > try:
> >  h.block_status(1,0,f)
> > except nbd.Error as ex:
> >  print(ex.string)
> > h.shutdown()
> > EOF
> nbdsh: generator/states-reply-chunk.c:701: 
> enter_STATE_REPLY_CHUNK_REPLY_FINISH: Assertion `h->payload_left == 0' failed.
> Aborted (core dumped)
> 
> vs. post-patch:
> 
> $ ./run nbdsh --base -u nbd://localhost -c - <<\EOF
> > def f(*k):
> >  pass
> > try:
> >  h.block_status(1,0,f)
> > except nbd.Error as ex:
> >  print(ex.string)
> > h.shutdown()
> > EOF
> nbd_block_status: block-status: command failed: Protocol error
> 
> Appears to be a casualty of rebasing: I added h->payload_left
> verification fairly late in the game, then floated it earlier in the
> series, and missed a spot where I added a state machine jump to RESYNC
> without having updated h->payload_left.  An audit of h->hlen
> modification sites show that all other chunked reads updated
> h->payload_left appropriately (often in the next statement, but
> sometimes in a later state when that made logic easier).
> 
> Requires a non-compliant server, and only possible when extended
> headers are negotiated, which does not affect any stable released
> libnbd.  Thus, there is no reason to create a CVE, although since I
> will already be doing a security info email about previous patches
> also addressing fuzzer findings, I can mention this at the same time.
> 
> Fixes: ab992766cd ("block_status: Accept 64-bit extents during block status")
> Thanks: Richard W.M. Jones 
> Signed-off-by: Eric Blake 
> ---
>  generator/states-reply-chunk.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/generator/states-reply-chunk.c b/generator/states-reply-chunk.c
> index 20407d91..5a31c192 100644
> --- a/generator/states-reply-chunk.c
> +++ b/generator/states-reply-chunk.c
> @@ -476,6 +476,7 @@  REPLY.CHUNK_REPLY.RECV_BS_HEADER:
>if (h->bs_count != be32toh (h->sbuf.reply.payload.bs_hdr_64.count)) {
>  h->rbuf = NULL;
>  h->rlen = h->payload_left;
> +h->payload_left = 0;
>  SET_NEXT_STATE (%RESYNC);
>  return 0;
>}

Reviewed-by: Richard W.M. Jones 


-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [libnbd PATCH v2 2/6] fuzzing: Disable client-side strictness checks

2023-09-21 Thread Richard W.M. Jones
On Thu, Sep 21, 2023 at 03:58:01PM -0500, Eric Blake wrote:
> When fuzzing, it is more desirable to always provoke the server into
> sending a response, rather than sometimes accidentally skipping a wire
> call because a client-side strictness test failed.
> 
> [Our fuzzer could probably be made even more powerful by changing the
> fuzzer input file to be a series of records, where each record is the
> API to call and then the server's response; right now, the sequence of
> APIs called is hard-coded, which is not as powerful at testing
> potential cross-command coupling. But that's a project for another
> day.]

Intrigued by how this would work exactly.  The current file is just a
stream of bytes from the "server" (the output from libnbd is
discarded).  Would the input file be of the form:

  ( [ request + args ] [ stream of bytes from server ] )*

where [ request + args ] would be a simple serialized representation
of a synchronous nbd_* API call?  I think that would work ...

Rich.

> Signed-off-by: Eric Blake 
> Reviewed-by: Richard W.M. Jones 
> Reviewed-by: Laszlo Ersek 
> ---
>  fuzzing/libnbd-fuzz-wrapper.c | 5 -
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/fuzzing/libnbd-fuzz-wrapper.c b/fuzzing/libnbd-fuzz-wrapper.c
> index cbd55380..fcd1d04c 100644
> --- a/fuzzing/libnbd-fuzz-wrapper.c
> +++ b/fuzzing/libnbd-fuzz-wrapper.c
> @@ -193,8 +193,11 @@ client (int sock)
>}
> 
>/* Note we ignore errors in these calls because we are only
> -   * interested in whether the process crashes.
> +   * interested in whether the process crashes.  Likewise, we don't
> +   * want to accidentally avoid sending traffic to the server merely
> +   * because client side strictness sees a problem.
> */
> +  nbd_set_strict_mode (nbd, 0);
> 
>/* Enable a metadata context, for block status below. */
>nbd_add_meta_context (nbd, LIBNBD_CONTEXT_BASE_ALLOCATION);
> -- 
> 2.41.0

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [libnbd PATCH v2 3/6] api: Sanitize sizes larger than INT64_MAX

2023-09-21 Thread Richard W.M. Jones
ric Blake 
> ---
>  generator/API.ml | 6 +-
>  lib/flags.c  | 6 ++
>  2 files changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/generator/API.ml b/generator/API.ml
> index 14988403..c4547615 100644
> --- a/generator/API.ml
> +++ b/generator/API.ml
> @@ -2492,7 +2492,11 @@   "get_size", {
>  permitted_states = [ Negotiating; Connected; Closed ];
>  shortdesc = "return the export size";
>  longdesc = "\
> -Returns the size in bytes of the NBD export."
> +Returns the size in bytes of the NBD export.
> +
> +Note that this call fails with C for an unlikely
> +server that advertises a size which cannot fit in a 64-bit
> +signed integer."
>  ^ non_blocking_test_call_description;
>  see_also = [SectionLink "Size of the export"; Link "opt_info"];
>  example = Some "examples/get-size.c";
> diff --git a/lib/flags.c b/lib/flags.c
> index 7e6ddedd..394abe87 100644
> --- a/lib/flags.c
> +++ b/lib/flags.c
> @@ -253,6 +253,12 @@ nbd_unlocked_get_size (struct nbd_handle *h)
>  return -1;
>}
> 
> +  if (h->exportsize > INT64_MAX) {
> +set_error (EOVERFLOW, "server claims size %" PRIu64
> +   " which does not fit in signed result", h->exportsize);
> +return -1;
> +  }
> +
>return h->exportsize;
>  }

Reviewed-by: Richard W.M. Jones 


-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-21 Thread Richard W.M. Jones
On Thu, Sep 21, 2023 at 06:24:40PM +0200, Lee Garrett wrote:
> On 21.09.23 18:08, Richard W.M. Jones wrote:
> >On Thu, Sep 21, 2023 at 04:50:26PM +0200, Lee Garrett wrote:
> >>After testing it again today, the guest agent is still not
> >>installed, however this time the C:\Program
> >>Files\Guestfs\Firstboot\log.txt exists:
> >>
> >>starting firstboot service
> >>running "C:\Program Files\Guestfs\Firstboot\scripts\5000-0001-wait-pnp.bat"
> >> 1 file(s) moved.
> >>Wait for PnP to complete
> >> exit code 0
> >>running "C:\Program 
> >>Files\Guestfs\Firstboot\scripts\5000-0002-install-qemu-ga-x86_64-msi-ps1.bat"
> >> 1 file(s) moved.
> >>Removing any previously scheduled qemu-ga installation
> >>ERROR: The system cannot find the file specified.
> >>Scheduling delayed installation of qemu-ga from qemu-ga-x86_64.msi
> >>SUCCESS: The scheduled task "Firstboot-qemu-ga" has successfully been 
> >>created.
> >> exit code 0
> >>running "C:\Program
> >>Files\Guestfs\Firstboot\scripts\5000-0003-uninstall-VMware-Tools.bat"
> >> 1 file(s) moved.
> >>uninstalling VMware Tools
> >> exit code 0
> >>uninstalling firstboot service
> >>Service uninstalled successfully
> >>
> >>
> >>The folder scripts is empty, all scripts are in scripts-done. The
> >>contents of the folder are:
> >>5000-0001-wait-pnp.bat
> >>123 5000-0001-wait-pnp.log
> >>5000-0002-install-qemu-ga-x86_64-msi-ps1.bat
> >>5000-0003-uninstall-VMware-Tools.bat
> >>5000-0003-uninstall-VMware-Tools.log
> >>
> >>Note that the .log is missing for qemu-ga.
> >
> >This is what we'd expect, except maybe the missing log for qemu-ga.
> >It's a little odd that it is missing, but it's also possible that the
> >log file doesn't get written if there's no output.
> >
> >>At least today I could twice provision the windows 11 VM, and in
> >>both cases the main log.txt was generated by the time I checked. How
> >>are the services scheduled? Could it be that they were delayed by >
> >>1h?
> >
> >Yes, we do actually try to delay qemu-ga installation.  It should only
> >be delayed for about 120 seconds (not 1 hour).
> >
> >https://github.com/libguestfs/libguestfs-common/blob/e70d89a58dae068be2e19c7c21558707261af96a/mlcustomize/inject_virtio_win.ml#L580
> >
> >However there have been timezone bugs with this in the past (and maybe
> >there still are), see the sordid history in these two links:
> >
> >https://github.com/libguestfs/libguestfs-common/blob/e70d89a58dae068be2e19c7c21558707261af96a/mlcustomize/inject_virtio_win.ml#L558
> 
> I see. That's not reassuring. :(
> 
> >
> >Which TZ are you in?
> 
> The host machine is UTC+2 (Berlin summer time), however the Windows
> 11 trial vmware image itself defaults to UTC-8 (Pacific Time US &
> Canada).
> 
> After a little bit of digging this indeed seems to be the issue.
> 
> `schtasks.exe /Query` gives:
> 
> PS C:\Users\User> schtasks.exe /Query
> 
> Folder: \
> TaskName Next Run Time  Status
>  == 
> ===========
> Firstboot-qemu-ga9/21/2023 4:04:00 PM   Ready
> [...]
> 
> which is ~7 hours in the future. Btw, running
> `.\5000-0002-install-qemu-ga-x86_64-msi-ps1.bat` by hand works just
> fine. So it definitely is just the scheduling issue that's
> preventing it from running.

So this is probably another instance or variation of the timezone
formatting problem (of schtasks).  Which version of virt-v2v is this?
I want to check that you have a version with all the latest patches in
this area.

Rich.

> >
> >Rich.
> >
> >>On 11.09.23 21:29, Richard W.M. Jones wrote:
> >>>
> >>>If we suspect that the whole firstboot mechanism might not be working
> >>>with the new version of Windows, one way to test it (on this one, or a
> >>>freshly installed Windows VM) would be:
> >>>
> >>>   $ virt-customize -a windows.img --firstboot-command 'echo hello'
> >>>
> >>>and see if "hello" is written in some form to the log.txt file inside
> >>>the guest after it boots.
> >>>
> >>>If that doesn't work then it's likely some change in Windows which is
> >>>breaking firstboot support.
> >>>
> >>>Rich.
> >>>
> >>
> >>Regards, Lee
> >
> 
> Regards, Lee

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-21 Thread Richard W.M. Jones
On Thu, Sep 21, 2023 at 04:50:26PM +0200, Lee Garrett wrote:
> After testing it again today, the guest agent is still not
> installed, however this time the C:\Program
> Files\Guestfs\Firstboot\log.txt exists:
> 
> starting firstboot service
> running "C:\Program Files\Guestfs\Firstboot\scripts\5000-0001-wait-pnp.bat"
> 1 file(s) moved.
> Wait for PnP to complete
>  exit code 0
> running "C:\Program 
> Files\Guestfs\Firstboot\scripts\5000-0002-install-qemu-ga-x86_64-msi-ps1.bat"
> 1 file(s) moved.
> Removing any previously scheduled qemu-ga installation
> ERROR: The system cannot find the file specified.
> Scheduling delayed installation of qemu-ga from qemu-ga-x86_64.msi
> SUCCESS: The scheduled task "Firstboot-qemu-ga" has successfully been created.
>  exit code 0
> running "C:\Program
> Files\Guestfs\Firstboot\scripts\5000-0003-uninstall-VMware-Tools.bat"
> 1 file(s) moved.
> uninstalling VMware Tools
>  exit code 0
> uninstalling firstboot service
> Service uninstalled successfully
> 
> 
> The folder scripts is empty, all scripts are in scripts-done. The
> contents of the folder are:
> 5000-0001-wait-pnp.bat
> 123 5000-0001-wait-pnp.log
> 5000-0002-install-qemu-ga-x86_64-msi-ps1.bat
> 5000-0003-uninstall-VMware-Tools.bat
> 5000-0003-uninstall-VMware-Tools.log
> 
> Note that the .log is missing for qemu-ga.

This is what we'd expect, except maybe the missing log for qemu-ga.
It's a little odd that it is missing, but it's also possible that the
log file doesn't get written if there's no output.

> At least today I could twice provision the windows 11 VM, and in
> both cases the main log.txt was generated by the time I checked. How
> are the services scheduled? Could it be that they were delayed by >
> 1h?

Yes, we do actually try to delay qemu-ga installation.  It should only
be delayed for about 120 seconds (not 1 hour).

https://github.com/libguestfs/libguestfs-common/blob/e70d89a58dae068be2e19c7c21558707261af96a/mlcustomize/inject_virtio_win.ml#L580

However there have been timezone bugs with this in the past (and maybe
there still are), see the sordid history in these two links:

https://github.com/libguestfs/libguestfs-common/blob/e70d89a58dae068be2e19c7c21558707261af96a/mlcustomize/inject_virtio_win.ml#L558

Which TZ are you in?

Rich.

> On 11.09.23 21:29, Richard W.M. Jones wrote:
> >
> >If we suspect that the whole firstboot mechanism might not be working
> >with the new version of Windows, one way to test it (on this one, or a
> >freshly installed Windows VM) would be:
> >
> >   $ virt-customize -a windows.img --firstboot-command 'echo hello'
> >
> >and see if "hello" is written in some form to the log.txt file inside
> >the guest after it boots.
> >
> >If that doesn't work then it's likely some change in Windows which is
> >breaking firstboot support.
> >
> >Rich.
> >
> 
> Regards, Lee

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] Some comments on the libnbd apk in Alpine

2023-09-21 Thread Richard W.M. Jones
Hi Sean, the maintainers of libnbd here.  Thanks for packaging libnbd
in Alpine:

https://git.alpinelinux.org/aports/tree/testing/libnbd/APKBUILD

I have a few comments:

(1) The url field should be "https://gitlab.com/nbdkit/libnbd;
(The one you are using is an old, stale mirror)

(2) libxml2-dev is not listed as a dependency.

We had a request from an Alpine user to add this dependency because it
enables several important features in libnbd, namely NBD URI support
and some utilities which require URI support.

It shouldn't be too much trouble to add since it appears that libxml2
is already built in Alpine.

(3) libnbd 1.18 (next stable) will be released in a week or two:
https://listman.redhat.com/archives/libguestfs/2023-September/032590.html

If you have any questions or comments about the package you can send
them to me and/or the mailing list (without subscribing if you don't
want to).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH libguestfs] daemon: Omit 'file -S' option on older distros that lack support

2023-09-21 Thread Richard W.M. Jones
This is upstream in c95d8c4cf64142.

Also added a few commits to fix the Ruby tests which had broken
because of a recent change to Ruby.  I'll cut a development release
soon.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH libguestfs] daemon: Omit 'file -S' option on older distros that lack support

2023-09-21 Thread Richard W.M. Jones
On Thu, Sep 21, 2023 at 03:56:05PM +0200, Laszlo Ersek wrote:
...
> > +open Std_utils
> > +
> > +(* Does [file] support the [-S] / [--no-sandbox] option
> > + * (not on OpenSUSE LEAP 15).
> > + *)
> > +let file_has_S_option = lazy (
> > +  let out = Utils.command "file" ["file"; "--help"] in
> 
> I think this executes "file file --help", which happens to work, but the
> intent is probably just "file --help". (IOW, IMO this should be
> 
> Utils.command "file" ["--help"]
> 
> .)

Indeed it does.

BTW I did check this patch by running:

  $ make && LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 make -C tests/ check 
TESTS=c-api/tests

and examining tests/c-api/tests.log, and that shows that file --help
(or rather, "file file --help") is being run exactly once, and the -S
option was still being included in the parameters.

> Reviewed-by: Laszlo Ersek 

Thanks, I'll tidy up and fix the things you mentioned.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH libguestfs] daemon: Omit 'file -S' option on older distros that lack support

2023-09-21 Thread Richard W.M. Jones
OpenSUSE LEAP 15 lacks support for this option, so test for it before
using it.

See-also: 
https://listman.redhat.com/archives/libguestfs/2023-September/032613.html
Report-by: Olaf Hering
Fixes: commit 23986d3c4f4d1f9cbac44cc743d3e6af721e4237
---
 daemon/Makefile.am |  2 ++
 daemon/file.ml | 10 --
 daemon/file_helper.ml  | 29 +
 daemon/file_helper.mli | 19 +++
 daemon/filearch.ml |  5 -
 5 files changed, 62 insertions(+), 3 deletions(-)

diff --git a/daemon/Makefile.am b/daemon/Makefile.am
index bb2e58d014..01c0f6416c 100644
--- a/daemon/Makefile.am
+++ b/daemon/Makefile.am
@@ -280,6 +280,7 @@ SOURCES_MLI = \
devsparts.mli \
file.mli \
filearch.mli \
+   file_helper.mli \
findfs.mli \
inspect.mli \
inspect_fs.mli \
@@ -321,6 +322,7 @@ SOURCES_ML = \
btrfs.ml \
cryptsetup.ml \
devsparts.ml \
+   file_helper.ml \
file.ml \
filearch.ml \
isoinfo.ml \
diff --git a/daemon/file.ml b/daemon/file.ml
index 1f87b190c1..f0ef181938 100644
--- a/daemon/file.ml
+++ b/daemon/file.ml
@@ -43,7 +43,10 @@ let file path =
 | S_SOCK -> "socket"
 | S_REG ->
(* Regular file, so now run [file] on it. *)
-   let out = command "file" ["-zSb"; Sysroot.sysroot_path path] in
+   let file_options =
+ sprintf "-z%sb"
+   (if File_helper.file_has_S_option () then "S" else "") in
+   let out = command "file" [file_options; Sysroot.sysroot_path path] in
 
(*  We need to remove the trailing \n from output of file(1).
 *
@@ -54,6 +57,9 @@ let file path =
String.trimr out
   )
   else (* it's a device *) (
-let out = command "file" ["-zSbsL"; path] in
+let file_options =
+  sprintf "-z%sbsL"
+(if File_helper.file_has_S_option () then "S" else "") in
+let out = command "file" [file_options; path] in
 String.trimr out
   )
diff --git a/daemon/file_helper.ml b/daemon/file_helper.ml
new file mode 100644
index 00..f8c4bcbe56
--- /dev/null
+++ b/daemon/file_helper.ml
@@ -0,0 +1,29 @@
+(* guestfs-inspection
+ * Copyright (C) 2009-2023 Red Hat Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *)
+
+open Std_utils
+
+(* Does [file] support the [-S] / [--no-sandbox] option
+ * (not on OpenSUSE LEAP 15).
+ *)
+let file_has_S_option = lazy (
+  let out = Utils.command "file" ["file"; "--help"] in
+  String.find out "--no-sandbox" >= 0
+
+)
+let file_has_S_option () = Lazy.force file_has_S_option
diff --git a/daemon/file_helper.mli b/daemon/file_helper.mli
new file mode 100644
index 00..a644cf6de2
--- /dev/null
+++ b/daemon/file_helper.mli
@@ -0,0 +1,19 @@
+(* guestfs-inspection
+ * Copyright (C) 2009-2023 Red Hat Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *)
+
+val file_has_S_option : unit -> bool
diff --git a/daemon/filearch.ml b/daemon/filearch.ml
index 7c858129db..cf784f18a2 100644
--- a/daemon/filearch.ml
+++ b/daemon/filearch.ml
@@ -128,7 +128,10 @@ and cpio_arch magic orig_path path =
 | bin :: bins ->
let bin_path = tmpdir // bin in
if is_regular_file bin_path then (
- let out = command "file" ["-zSb"; bin_path] in
+ let file_options =
+   sprintf "-z%sb"
+ (if File_helper.file_has_S_option () then "S" else "") in
+ let out = command "file" [file_options; bin_path] in
  file_architecture_of_magic out orig_path bin_path
)
else
-- 
2.41.0


Re: [Libguestfs] regression: file does not understand the -S option

2023-09-21 Thread Richard W.M. Jones
On Wed, Sep 20, 2023 at 11:42:55PM +0200, Olaf Hering wrote:
> Recently a commit was added to call 'file -zSb' instead of 'file -zb'.
> 
> This causes a regression on Leap 15 (but not on Tumbleweed), because
> file 5.32 does not understand the -S option.
> 
> How can this be fixed properly, to handle both cases either at runtime
> or at buildtime?

The background to this was:

  https://github.com/libguestfs/libguestfs/issues/100

It took a while to work out what was going on in the original bug
report, but it turned out that Arch (IIRC) enabled the seccomp feature
in the 'file' command.  This filters what system calls 'file' is
allowed to make, which strengthens security as 'file' is often run on
untrusted inputs.

Unfortunately the seccomp rules for 'file' don't cope with running
external programs (ie. 'file -z' which runs zcat).  We filed a bug to
try to get that fixed:

  https://bugzilla.redhat.com/show_bug.cgi?id=2148753
  https://bugs.astron.com/view.php?id=406

but the fix to seccomp policy was rejected recently in both Fedora &
upstream.

The patch we added to libguestfs turns off seccomp sandboxing, both
because it's broken (see above) and because we don't really need it as
we run stuff in a virtual machine already:

  
https://github.com/libguestfs/libguestfs/commit/23986d3c4f4d1f9cbac44cc743d3e6af721e4237

I didn't realise there were distros that lack support for the
'file -S' option.

So I guess the fix is to detect if 'file' has the -S option ...
I think we can just grep 'file --help' for the -S / --no-sandbox
option.  Let me try for a patch now.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [EXTERNAL] - Re: LIBGUESTFS mount disk failure

2023-09-20 Thread Richard W.M. Jones
On Wed, Sep 20, 2023 at 09:53:19AM +, Divyanshu Kumar wrote:
> Hi @Richard W.M. Jones,
>
> I am attaching a sample program of how we are using guestfs apis
> where we are getting this error guestfs_mount_local fail.

It works fine, I see no errors when I run it.

This is the third and last time I'll repeat this.  If you ignore it
again then I won't reply to any more emails from you.  So I advise you
to read it very carefully:

  I need to see the program that you're trying to run, or a
  self-contained reproducer, and I also need to see the _complete_
  output from the program from start to finish, not just small parts
  of it.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [EXTERNAL] - Re: LIBGUESTFS mount disk failure

2023-09-19 Thread Richard W.M. Jones
On Tue, Sep 19, 2023 at 11:59:43AM +, Raja Ram Sharma wrote:
> Hi Richard, 
> 
> Thank you for the response, will set both handler and let you know.
> 
> when we try to execute guestfs_mount_local. 
> Getting below error :
> 
> [ 2023-09-19 13:34:05 ]  errno: ( 0 ) : mount_local: unknown option 554101136 
> (this can happen if a program is compiled against a newer version of 
> libguestfs, then dynamically linked to an older version)

I attach what I said previously.

Rich.

> Setup details :
> OS : (PRETTY_NAME="Red Hat Enterprise Linux 8.2 (Ootpa)")
> Guesfs.h version : 1.50
> Guestfish version : guestfish 
> 1.38.4rhel=8,release=15.module+el8.2.0+5297+222a20af,libvirt
> 
> In Below scenario:
> 1. guestfs_mount_local_run
> 2. umount /mountpoint
> 3. Again, while trying to mount, after sleep of 20sec.
> 
> 
> Thanks
> RR
> 
> 
> 
>  
> 
> -Original Message-
> From: Richard W.M. Jones  
> Sent: Tuesday, September 19, 2023 1:15 PM
> To: Raja Ram Sharma 
> Cc: Teja Konapalli ; libguestfs@redhat.com; 
> Divyanshu Kumar 
> Subject: Re: [EXTERNAL] - Re: LIBGUESTFS mount disk failure
> 
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open attachments unless you recognize the sender and know the 
> content is safe. If you feel that the email is suspicious, please report it 
> using PhishAlarm.
> 
> 
> On Sun, Sep 17, 2023 at 04:11:25PM +, Raja Ram Sharma wrote:
> > One more help is needed : 
> > https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/i
> > ssues/124__;!!Obbck6kTJA!blek-uFsj3xPSaSiNQrS-1lk59I6-udl4zqOJCxZ6LLeu
> > pvAFl6tPfod_GU2BLm_19gJUs3OXp6_IUEM$
> > /***/
> > I can get guestfs_last_error using function_callback and write to log file.
> > But "event Handling" is not working Also, std::cout and std::cerr still 
> > printing on terminal.
> > Is there any workaround for Event Handling?
> >  How to completely disable/enable standard output?
> > /***/
> >
> > In short, Any Debugging sample program, 
> > https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/i
> > ssues/124__;!!Obbck6kTJA!blek-uFsj3xPSaSiNQrS-1lk59I6-udl4zqOJCxZ6LLeu
> > pvAFl6tPfod_GU2BLm_19gJUs3OXp6_IUEM$
> 
> In the two programs in that issue, the first program sets an event callback 
> and the second program sets an error handler.  To capture all messages you 
> must set both together.
> 
> If you can produce a single, short reproducer which uses both types of 
> handlers and still prints to stderr, then I can look at that.
> 
> Rich.
> 
> --
> Richard Jones, Virtualization Group, Red Hat 
> https://urldefense.com/v3/__http://people.redhat.com/*rjones__;fg!!Obbck6kTJA!blek-uFsj3xPSaSiNQrS-1lk59I6-udl4zqOJCxZ6LLeupvAFl6tPfod_GU2BLm_19gJUs3OXuHRnZTv$
> Read my programming and virtualization blog: 
> https://urldefense.com/v3/__http://rwmj.wordpress.com__;!!Obbck6kTJA!blek-uFsj3xPSaSiNQrS-1lk59I6-udl4zqOJCxZ6LLeupvAFl6tPfod_GU2BLm_19gJUs3OXrVS1VaN$
> Fedora Windows cross-compiler. Compile Windows programs, test, and build 
> Windows installers. Over 100 libraries supported.
> https://urldefense.com/v3/__http://fedoraproject.org/wiki/MinGW__;!!Obbck6kTJA!blek-uFsj3xPSaSiNQrS-1lk59I6-udl4zqOJCxZ6LLeupvAFl6tPfod_GU2BLm_19gJUs3OXjCvHvVp$

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
--- Begin Message ---
On Sun, Sep 17, 2023 at 03:55:24AM +, Raja Ram Sharma wrote:
> Hi Richard, 
> 
> Wish you a happy weekend! 
> 
> Guestfs_launch fails after below error :
>   
>   libguestfs: is_openable: /dev/kvm: No such file or directory
>   libguestfs: trace: get_backend_setting "force_tcg"
>   libguestfs: trace: get_backend_setting = NULL (error)
>   ^^^
> Same line has also printed in " LIBGUESTFS_BACKEND=direct 
> libguestfs-test-tool " at line $96-$98
>   ^^^
>   Could not access KVM kernel module: No such file or directory
>   qemu-kvm: failed to initialize KVM: No such file or directory
>   ^^^

Thanks for providing the second output.

The above messages aren't errors.

I really need much more context.  I need to see the program that
you're trying to run, or a self-contained reproducer, and I also need
to see the _complete_ output from the program from start to finish,
not just small parts of it.

Rich.

-- 
Richard 

Re: [Libguestfs] [External] Re: [PATCH] VxFS Filesystem support to libguestfs

2023-09-19 Thread Richard W.M. Jones
A new patch is attached.  This is only build tested so far.

To build with this patch I had to do:

  ./configure --enable-werror \
  --with-supermin-packager-config=$PWD/localyum.conf 

where localyum.conf contains:

--
[local]
name=local
baseurl=file:///root/dvd1-redhatlinux/rhel9_x86_64/rpms/
enabled=1
gpgcheck=0
--

(adjust the path to point to the Veritas DVD mount point)


I made various changes in this patch:

- Add copyright and license notice to list_vxvm.ml & vxfs.c.  Please
  ensure all files have copyright and license notices, and if necessary
  get them checked by your legal.

- Change vm.ml{,i} -> list_vxvm.ml{,i}

- In packagelist.in, move package names to their own section.

- Remove perl/python packages as perl/python is not present in
  the appliance.

- Fix commit message.

- Line length under 80 columns in a few places.

- Function names at start of line.

- List SOURCES_MLI alphabetically.

- daemon/listfs.ml: Remove comment about has_vxvm.

- do_vxfs_start -> vxfs_start (since do_* is reserved prefix in daemon).

- Skip vxfs_start if !vxdisk available, to save start up time.

- Add a prototype for vxfs_start.

- Don't use reply_with_error in vxfs_start, as we're not called
  during protocol context.

- Use only one *err.

- Add daemon/list_vxvm.mli & daemon/vxvm_type.mli to .gitignore


vxfs_start has several issues which need to be addressed:

 - Why does it return if a service (eg. veki) is not available?  That
   would prevent all further initialization.

 - I think this would be better done as part of appliance/init, and
   delete this function completely.



Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
>From 997cd736acf59432804aebb469fc1c6dbe56a9fd Mon Sep 17 00:00:00 2001
From: "ravi.singh" 
Date: Fri, 15 Sep 2023 15:40:50 +0530
Subject: [PATCH] Add VxFS Filesystem support

---
 .gitignore|   2 +
 appliance/init|   2 +
 appliance/packagelist.in  |   8 +++
 daemon/Makefile.am|   9 ++-
 daemon/daemon.h   |   3 +
 daemon/guestfsd.c |   3 +
 daemon/list_vxvm.ml   |  78 
 daemon/listfs.ml  |  13 
 daemon/vxfs.c | 122 ++
 daemon/vxvm_type.ml   |  17 ++
 generator/actions_core.ml |  24 
 generator/proc_nr.ml  |   2 +
 lib/MAX_PROC_NR   |   2 +-
 po/POTFILES   |   1 +
 14 files changed, 284 insertions(+), 2 deletions(-)
 create mode 100644 daemon/list_vxvm.ml
 create mode 100644 daemon/vxfs.c
 create mode 100644 daemon/vxvm_type.ml

diff --git a/.gitignore b/.gitignore
index ee5ea74dd..28d327ea0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -96,6 +96,7 @@ Makefile.in
 /daemon/ldm.mli
 /daemon/link.mli
 /daemon/listfs.mli
+/daemon/list_vxvm.mli
 /daemon/lvm.mli
 /daemon/lvm_dm.mli
 /daemon/lvm-tokenization.c
@@ -118,6 +119,7 @@ Makefile.in
 /daemon/stubs-?.c
 /daemon/stubs.h
 /daemon/types.ml
+/daemon/vxvm_type.mli
 /depcomp
 /docs/guestfs-building.1
 /docs/guestfs-faq.1
diff --git a/appliance/init b/appliance/init
index d41056659..535a3d6b1 100755
--- a/appliance/init
+++ b/appliance/init
@@ -22,6 +22,8 @@ mount -t proc /proc /proc
 mount -t sysfs /sys /sys
 # devtmpfs is required since udev 176
 mount -t devtmpfs /dev /dev
+# Create dev directory for all VxVM device files.
+mkdir -p /dev/vx
 ln -s /proc/self/fd /dev/fd
 
 # Parse the kernel command line early (must be after /proc is mounted).
diff --git a/appliance/packagelist.in b/appliance/packagelist.in
index 20b08c470..06d4aaf6d 100644
--- a/appliance/packagelist.in
+++ b/appliance/packagelist.in
@@ -271,5 +271,13 @@ exfat-utils
 exfatprogs
 fuse-exfat
 
+dnl Proprietary Veritas packages for VxFS support
+dnl VRTSperl
+dnl VRTSpython
+VRTSveki
+VRTSvlic
+VRTSvxfs
+VRTSvxvm
+
 dnl Define this by doing: ./configure --with-extra-packages="..."
 EXTRA_PACKAGES
diff --git a/daemon/Makefile.am b/daemon/Makefile.am
index bd1920c61..b251bebf4 100644
--- a/daemon/Makefile.am
+++ b/daemon/Makefile.am
@@ -51,6 +51,8 @@ generator_built = \
link.mli \
listfs.mli \
lvm.mli \
+   list_vxvm.mli \
+   vxvm_type.mli \
lvm_dm.mli \
md.mli \
mount.mli \
@@ -208,6 +210,7 @@ guestfsd_SOURCES = \
wc.c \
xattr.c \
xfs.c \
+   vxfs.c \
yara.c \
zero.c \
zerofree.c
@@ -292,6 +295,7 @@ SOURCES_MLI = \
isoinfo.mli \
ldm.mli \
link.mli \
+   list_vxvm.mli \
listfs.mli \
lvm.mli \
lvm_dm.mli \
@@ -307,7 +311,8 @@ SOURCES_MLI = \
statvfs.mli \

Re: [Libguestfs] [External] Re: [PATCH] VxFS Filesystem support to libguestfs

2023-09-19 Thread Richard W.M. Jones
On Fri, Sep 15, 2023 at 11:37:32AM +, Ravi Singh wrote:
> >>> Fixed. Added VRTS* packages to packagelist.in and removed the
> above scripts from hostfiles.

OK, that should work.  I will test it out later.

One issue we'll probably have is how to package this up for Fedora or
RHEL as those require that the RPMs are available at build time (which
is not possible as they are not open source).

It may be that you will need to build a "libguestfs-vxfs" package
yourself, built against your RPMs.  It's fairly simple to do this but
let's cross that bridge later.

> >>> The approach here we took to pass the package name as an extra parameter
> during libguestfs initial configure stage to eliminate our package
> dependencies. But as you suggested added to packagelist.in.
> 
> e.g:
> ./configure --with-extra-packages="VRTSperl VRTSpython VRTSveki VRTSvlic
> VRTSvxfs VRTSvxvm"

I see.  That would work, but including the package names in
appliance/packagelist.in makes everything easier.

> > +  /* Start vxfs services */
> > +  do_vxfs_start();
> > +
> 
> >>> Fixed. That was not intentional. Simple function works fine.

Good.

> >> Please find the attached zip file, contains latest patch and commands to
> create VxVM volumes and VxFS for testing this patch.

Alright, I'll try this out soon.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [EXTERNAL] - Re: LIBGUESTFS mount disk failure

2023-09-19 Thread Richard W.M. Jones
On Sun, Sep 17, 2023 at 04:11:25PM +, Raja Ram Sharma wrote:
> One more help is needed : https://github.com/libguestfs/libguestfs/issues/124
> /***/
> I can get guestfs_last_error using function_callback and write to log file.
> But "event Handling" is not working Also, std::cout and std::cerr still 
> printing on terminal.
> Is there any workaround for Event Handling?
>  How to completely disable/enable standard output?
> /***/
> 
> In short, Any Debugging sample program, 
> https://github.com/libguestfs/libguestfs/issues/124 

In the two programs in that issue, the first program sets an event
callback and the second program sets an error handler.  To capture all
messages you must set both together.

If you can produce a single, short reproducer which uses both types of
handlers and still prints to stderr, then I can look at that.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [libnbd PATCH] generator: Fix assertion with ill-formed 64-bit block status reply

2023-09-19 Thread Richard W.M. Jones
On Mon, Sep 18, 2023 at 02:31:28PM -0500, Eric Blake wrote:
> If a server replies to a block status command with an invalid length
> in NBD_REPLY_TYPE_BLOCK_STATUS_EXT, we correctly detect the server's
> error, but fail to mark that we've consumed enough data off the wire
> to resync back to the server's next reply.  Symptoms seen during a
> fuzzing run:
> 
> │  $ ./fuzzing/libnbd-fuzz-wrapper
> │+id\:01\,sig\:06\,src\:000396\,time\:16888628\,execs\:54159419\,op\:havoc\,rep\:4
> │  libnbd-fuzz-wrapper: generator/states-reply-chunk.c:698: 
> enter_STATE_REPLY_CHUNK_REPLY_FINISH:
> │+Assertion `h->payload_left == 0' failed.
> │  Aborted (core dumped)
> │  read: Connection reset by peer
> 
> Appears to be a casualty of rebasing: I added h->payload_left
> verification fairly late in the game, then floated it earlier in the
> series, and missed a spot where I added a state machine jump to RESYNC
> without having updated h->payload_left.  An audit of all other
> assignments to h->rlen in that file was able to find corresponding
> assignments to h->payload_left (often the next statement, but
> sometimes split across states based on what made the next state easier
> to code).
> 
> Requires a non-compliant server, but I was able to come up with a
> one-line tweak to pending qemu patches that could trigger it.  Not
> creating a CVE as it only appears in unstable releases.
> 
> Fixes: ab992766cd ("block_status: Accept 64-bit extents during block status")
> Thanks: Richard W.M. Jones 
> Signed-off-by: Eric Blake 
> ---
> 
> I'm investigating another crash that Rich sent me off-list, but I
> suspect it will be a similar non-CVE situation caused by my recent
> 64-bit extension patches.
> 
> I'll wait to apply this one for just a bit more, in case I can get the
> corpus file or two from Rich's fuzzing run to add to
> fuzzing/testcase_dir.
> 
>  generator/states-reply-chunk.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/generator/states-reply-chunk.c b/generator/states-reply-chunk.c
> index 2cebe456..a5d3aefe 100644
> --- a/generator/states-reply-chunk.c
> +++ b/generator/states-reply-chunk.c
> @@ -476,6 +476,7 @@  REPLY.CHUNK_REPLY.RECV_BS_HEADER:
>if (h->bs_count != be32toh (h->sbuf.reply.payload.bs_hdr_64.count)) {
>  h->rbuf = NULL;
>  h->rlen = h->payload_left;
> +h->payload_left = 0;
>  SET_NEXT_STATE (%RESYNC);
>  return 0;
>}

ACK, thanks.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


Re: [Libguestfs] [EXTERNAL] - Re: LIBGUESTFS mount disk failure

2023-09-17 Thread Richard W.M. Jones
On Sun, Sep 17, 2023 at 03:55:24AM +, Raja Ram Sharma wrote:
> Hi Richard, 
> 
> Wish you a happy weekend! 
> 
> Guestfs_launch fails after below error :
>   
>   libguestfs: is_openable: /dev/kvm: No such file or directory
>   libguestfs: trace: get_backend_setting "force_tcg"
>   libguestfs: trace: get_backend_setting = NULL (error)
>   ^^^
> Same line has also printed in " LIBGUESTFS_BACKEND=direct 
> libguestfs-test-tool " at line $96-$98
>   ^^^
>   Could not access KVM kernel module: No such file or directory
>   qemu-kvm: failed to initialize KVM: No such file or directory
>   ^^^

Thanks for providing the second output.

The above messages aren't errors.

I really need much more context.  I need to see the program that
you're trying to run, or a self-contained reproducer, and I also need
to see the _complete_ output from the program from start to finish,
not just small parts of it.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [EXTERNAL] - Re: LIBGUESTFS mount disk failure

2023-09-16 Thread Richard W.M. Jones
Thanks for providing the libguestfs-test-tool output in a previous
message.

> While: guestfs_lanch error:
>   libguestfs: trace: set_verbose true
>   libguestfs: trace: set_verbose = 0
>   libguestfs: trace: set_backend "direct"
>   libguestfs: trace: set_backend = 0

Here you're setting backend = "direct", and while that isn't
necessarily a problem in itself, the libguestfs-test-tool output that
you provided was for the (normal, for RHEL) libvirt backend.

This could make a difference, so what's the output of:

  $ LIBGUESTFS_BACKEND=direct libguestfs-test-tool

?

By the way, unless you have a specific reason to set
'LIBGUESTFS_BACKEND=direct' that you can articulate, it is best not to
set this as it is less secure and less well tested.  If you just
copied that environment variable from somewhere without understanding
why, then don't do that.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [EXTERNAL] - Re: LIBGUESTFS mount disk failure

2023-09-16 Thread Richard W.M. Jones
Hi, can you run:

  libguestfs-test-tool

and send us the complete output.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] LIBGUESTFS mount disk failure

2023-09-16 Thread Richard W.M. Jones
On Sat, Sep 16, 2023 at 03:56:07PM +, Teja Konapalli wrote:
> Hi Richard & Team,
> 
>  
> 
> Could you please help here on the below failure of mounting disk using
> libguestfs.
> 
>  
> 
> OS version: RHEL 8.2
> 
> Libguestfs version: 1.38.0 stable
> 
>  
> 
>  
> 
> Error logs:
> 
>  
> 
> libguestfs: trace: add_drive_ro "/var/opt/tmp/GRE//RAJA/RAJAT-flat.vmdk"
> libguestfs: trace: add_drive "/var/opt/tmp/GRE//RAJA/RAJAT-flat.vmdk"
> "readonly:true"
> libguestfs: creating COW overlay to protect original drive content
> libguestfs: trace: disk_create "/tmp/libguestfsCpl8My/overlay3.qcow2" "qcow2"
> -1 "backingfile:/var/opt/tmp/GRE//RAJA/RAJAT-flat.vmdk"
> libguestfs: command: run: qemu-img
> libguestfs: command: run: \ create
> libguestfs: command: run: \ -f qcow2
> libguestfs: command: run: \ -o backing_file=/var/opt/tmp/GRE//RAJA/
> RAJAT-flat.vmdk
> libguestfs: command: run: \ /tmp/libguestfsCpl8My/overlay3.qcow2
> Formatting '/tmp/libguestfsCpl8My/overlay3.qcow2', fmt=qcow2 size=21474836480
> backing_file=/var/opt/tmp/GRE//RAJA/RAJAT-flat.vmdk cluster_size=65536
> lazy_refcounts=off refcount_bits=16
> libguestfs: trace: disk_create = -1 (error)

^^ There should be an error message between this two lines.  Without
seeing what the error is, it's hard to know what the problem is.

Is this happening in a program you wrote?  If so you probably want to
read the bit about how to handle error messages:

https://libguestfs.org/guestfs.3.html#error-handling

Rich.

> libguestfs: trace: add_drive = -1 (error)
> libguestfs: trace: add_drive_ro = -1 (error)
> libguestfs: trace: close
> libguestfs: closing guestfs handle 0x1e89340 (state 0)
> libguestfs: command: run: rm
> libguestfs: command: run: \ -rf /tmp/libguestfsCpl8My
> 

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH] VxFS Filesystem support to libguestfs

2023-09-14 Thread Richard W.M. Jones
On Thu, Jun 08, 2023 at 11:38:53AM +0100, Richard W.M. Jones wrote:
> From: Ravi Singh 

First, thanks to various people who helped me to get Infoscale Storage
installed.

I have questions ...

>  appliance/hostfiles.in|   4 ++
>  appliance/init|   2 +
>  daemon/Makefile.am|   7 +++
>  daemon/guestfsd.c |   3 +
>  daemon/listfs.ml  |  17 ++
>  daemon/vm.ml  |  68 ++
>  daemon/vm.mli |  24 
>  daemon/vxfs.c | 119 ++
>  daemon/vxvm_type.ml   |  17 ++
>  daemon/vxvm_type.mli  |  24 
>  generator/actions_core.ml |  32 ++
>  generator/proc_nr.ml  |   3 +
>  lib/MAX_PROC_NR   |   2 +-
>  po/POTFILES   |   1 +
>  14 files changed, 322 insertions(+), 1 deletion(-)
> 
> diff --git a/appliance/hostfiles.in b/appliance/hostfiles.in
> index e78c79bd34..e519ad1787 100644
> --- a/appliance/hostfiles.in
> +++ b/appliance/hostfiles.in
> @@ -17,3 +17,7 @@ dnl   OPENMANDRIVA=1 For OpenMandriva.
>  /etc/ld.so.cache
>  /lib/lsb/*
>  /usr/share/augeas/lenses/*.aug
> +/etc/vx/vxfs
> +/etc/vx/veki
> +/etc/vx/vxvm-startup

These are executable configuration files which start various services.
Later on, do_vxfs_start will test if these files exist and then run
them to bring up the vxfs services inside the appliance.  However I'm
not exactly sure how that could possibly work as these scripts refer
to programs that are not copied into the appliance.

> +/opt/VRTS/bin/fstyp

This is a symlink to /usr/lib/fs/vxfs/fstyp which is a binary.  While
it's not impossible for hostfiles.in to be used for scripts (like the
ones above) and binaries, it's better to use packagelist.in to list
the packages (ie. RPMs) you actually want to copy in.  For example:

# rpm -qf /usr/lib/fs/vxfs/fstyp
VRTSvxfs-8.0.2.-RHEL9.x86_64
]# rpm -qf /etc/vx/vxfs
VRTSvxfs-8.0.2.-RHEL9.x86_64
# rpm -qf /etc/vx/veki 
VRTSveki-8.0.2.-RHEL9.x86_64
# rpm -qf /etc/vx/vxvm-startup 
VRTSvxvm-8.0.2.-RHEL9.x86_64

Simply listing VRTSvxvm in packagelist.in would copy the whole package
into the appliance, including all those files, and (if the RPMs are
set up correctly) all dependencies that they need to run.

I actually can't understand how this can work as written.

> diff --git a/appliance/init b/appliance/init
> index d410566597..535a3d6b19 100755
> --- a/appliance/init
> +++ b/appliance/init
> @@ -22,6 +22,8 @@ mount -t proc /proc /proc
>  mount -t sysfs /sys /sys
>  # devtmpfs is required since udev 176
>  mount -t devtmpfs /dev /dev
> +# Create dev directory for all VxVM device files.
> +mkdir -p /dev/vx
>  ln -s /proc/self/fd /dev/fd
>  
>  # Parse the kernel command line early (must be after /proc is mounted).
> diff --git a/daemon/Makefile.am b/daemon/Makefile.am
> index bb2e58d014..8d3d9c8255 100644
> --- a/daemon/Makefile.am
> +++ b/daemon/Makefile.am
> @@ -51,6 +51,8 @@ generator_built = \
>   link.mli \
>   listfs.mli \
>   lvm.mli \
> + vm.mli \
> + vxvm_type.mli \
>   lvm_dm.mli \
>   md.mli \
>   mount.mli \
> @@ -208,6 +210,7 @@ guestfsd_SOURCES = \
>   wc.c \
>   xattr.c \
>   xfs.c \
> + vxfs.c \
>   yara.c \
>   zero.c \
>   zerofree.c
> @@ -294,6 +297,8 @@ SOURCES_MLI = \
>   link.mli \
>   listfs.mli \
>   lvm.mli \
> + vm.mli \
> + vxvm_type.mli \
>   lvm_dm.mli \
>   lvm_utils.mli \
>   md.mli \
> @@ -328,6 +333,8 @@ SOURCES_ML = \
>   ldm.ml \
>   link.ml \
>   lvm.ml \
> + vm.ml \
> + vxvm_type.ml \
>   lvm_utils.ml \
>   lvm_dm.ml \
>   findfs.ml \
> diff --git a/daemon/guestfsd.c b/daemon/guestfsd.c
> index 33c297de33..7b202ea53c 100644
> --- a/daemon/guestfsd.c
> +++ b/daemon/guestfsd.c
> @@ -273,6 +273,9 @@ main (int argc, char *argv[])
> */
>udev_settle ();
>  
> +  /* Start vxfs services */
> +  do_vxfs_start();
> +

This is added as a public API (see generator/actions_core.ml change
later).  But it seems like that wasn't intentional.  Do you intend
that programs using libguestfs would need to run 'guestfs_vxfs_start'
themselves?  And if so what would be the situation when that is
necessary?

If it's not necessary for this to be a public API then just make this
into a normal function.

>/* Send the magic length message which indicates that
> * userspace is up inside the guest.
> */
> diff --git a/daemon/listfs.ml b/daemon/listfs.ml
> index 2376b61dbc..10e32d8c04 100644
> --- a/daemon/listfs.ml
> +++ b/daemon/listfs.ml
> @@ -25,6 +25,11 @@ open Std_utils
>   * contain filesystems, so we filter them

[Libguestfs] Plans for nbdkit 1.36 & libnbd 1.18

2023-09-12 Thread Richard W.M. Jones
The current stable versions of nbdkit & libnbd were first released on:

nbdkit 1.34.0 => 14 April 2023
libnbd 1.16.0 => 18 April 2023

which is about 5 months ago.  Since we tend to release these projects
approximately every 6 months, I'd like to aim for a release at the end
of September or beginning of October, about 3 weeks from now.

If there are large features, then let's discuss whether to add them
now or else hold off.  (IIUC all 64 bit NBD patches are now upstream?
Are there more?)

I'll prepare draft release notes in the coming weeks.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [EXTERNAL] - Re: LIBGUESTFS supported version

2023-09-12 Thread Richard W.M. Jones
On Tue, Sep 12, 2023 at 07:55:23AM +, Teja Konapalli wrote:
> Hi Richard,
> 
> There's no information here.  What program are you using?  --> we are using 
> sample c++ program

Did you compile the C++ program yourself?  Did you compile it against
the same version of libguestfs-devel?

Rich.

>  What exact version of the program and libguestfs?  -->version 1.38.0
> What operating system are you using? -->RHEL 8.2
>   How did you obtain and install libguestfs?  Installed through this command 
> sudo yum install libguestfs-tools .
> 
> Regards,
> Teja K
> 
> -Original Message-
> From: Richard W.M. Jones  
> Sent: Tuesday, September 12, 2023 1:14 PM
> To: Teja Konapalli 
> Cc: libguestfs@redhat.com
> Subject: Re: [EXTERNAL] - Re: [Libguestfs] LIBGUESTFS supported version
> 
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open attachments unless you recognize the sender and know the 
> content is safe. If you feel that the email is suspicious, please report it 
> using PhishAlarm.
> 
> 
> On Tue, Sep 12, 2023 at 06:44:54AM +, Teja Konapalli wrote:
> > Hi Richard & Team,
> >
> > Am getting below issue frequently when I try to mount using libguestfs.
> >
> > [ 2023-09-12 08:17:43 ]  errno: ( 0 ) : mount_local: unknown option 
> > 1804310928 (this can happen if a program is compiled against a newer 
> > version of libguestfs, then dynamically linked to an older version) [ 
> > 2023-09-12 08:17:45 ]  errno: ( 0 ) : mount_local: unknown option 
> > 1804310928 (this can happen if a program is compiled against a newer 
> > version of libguestfs, then dynamically linked to an older version) [ 
> > 2023-09-12 08:17:47 ]  errno: ( 0 ) : mount_local: unknown option 
> > 1804310928 (this can happen if a program is compiled against a newer 
> > version of libguestfs, then dynamically linked to an older version)
> 
> There's no information here.  What program are you using?  What exact version 
> of the program and libguestfs?  What operating system are you using?  How did 
> you obtain and install libguestfs?  etc.
> 
> Rich.
> 
> --
> Richard Jones, Virtualization Group, Red Hat 
> https://urldefense.com/v3/__http://people.redhat.com/*rjones__;fg!!Obbck6kTJA!YruXPeOhVGJTOULQGG0dNevCF1Ltq_Nd2Xkegc0y3D_mKq0C475vj-qMH9c_4xHbKhFvICspUcSQPHr5kQ$
> Read my programming and virtualization blog: 
> https://urldefense.com/v3/__http://rwmj.wordpress.com__;!!Obbck6kTJA!YruXPeOhVGJTOULQGG0dNevCF1Ltq_Nd2Xkegc0y3D_mKq0C475vj-qMH9c_4xHbKhFvICspUcR-Y2VZqA$
> libguestfs lets you edit virtual machines.  Supports shell scripting, 
> bindings from many languages.  
> https://urldefense.com/v3/__http://libguestfs.org__;!!Obbck6kTJA!YruXPeOhVGJTOULQGG0dNevCF1Ltq_Nd2Xkegc0y3D_mKq0C475vj-qMH9c_4xHbKhFvICspUcQFlpFVjg$

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [EXTERNAL] - Re: LIBGUESTFS supported version

2023-09-12 Thread Richard W.M. Jones
On Tue, Sep 12, 2023 at 06:44:54AM +, Teja Konapalli wrote:
> Hi Richard & Team,
> 
> Am getting below issue frequently when I try to mount using libguestfs.
> 
> [ 2023-09-12 08:17:43 ]  errno: ( 0 ) : mount_local: unknown option 
> 1804310928 (this can happen if a program is compiled against a newer version 
> of libguestfs, then dynamically linked to an older version)
> [ 2023-09-12 08:17:45 ]  errno: ( 0 ) : mount_local: unknown option 
> 1804310928 (this can happen if a program is compiled against a newer version 
> of libguestfs, then dynamically linked to an older version)
> [ 2023-09-12 08:17:47 ]  errno: ( 0 ) : mount_local: unknown option 
> 1804310928 (this can happen if a program is compiled against a newer version 
> of libguestfs, then dynamically linked to an older version)

There's no information here.  What program are you using?  What exact
version of the program and libguestfs?  What operating system are you
using?  How did you obtain and install libguestfs?  etc.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-11 Thread Richard W.M. Jones


If we suspect that the whole firstboot mechanism might not be working
with the new version of Windows, one way to test it (on this one, or a
freshly installed Windows VM) would be:

  $ virt-customize -a windows.img --firstboot-command 'echo hello'

and see if "hello" is written in some form to the log.txt file inside
the guest after it boots.

If that doesn't work then it's likely some change in Windows which is
breaking firstboot support.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH nbdkit 10/10] XXX docs: Remove references to -U - when it is implicit

2023-09-11 Thread Richard W.M. Jones


This series, minus the delayed documentation patch, plus the
--dump-config patch, was pushed as:

ff3c9eb0e..868954b23

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH nbdkit 10/10] XXX docs: Remove references to -U - when it is implicit

2023-09-11 Thread Richard W.M. Jones
On Mon, Sep 11, 2023 at 09:34:50AM -0500, Eric Blake wrote:
> On Sat, Sep 09, 2023 at 02:57:58PM +0100, Richard W.M. Jones wrote:
> > XXX NOTE XXX
> > 
> > I would not apply this patch immediately, since online documentation
> > will get updated as soon as I do that.  Best to wait until after 1.36
> > is released at least.
> 
> At the earliest, on the day that we are ready to cut 1.36 (I don't see
> a problem with the docs being updated once the tarball will be ready,
> rather than having to wait yet longer to 1.36.1).  But I agree that we
> aren't quite ready for the stable release yet.
> 
> > 
> > XXX END NOTE XXX
> > ---
> >  docs/nbdkit-captive.pod   | 6 +++---
> >  filters/cacheextents/nbdkit-cacheextents-filter.pod   | 2 +-
> >  filters/checkwrite/nbdkit-checkwrite-filter.pod   | 6 +++---
> >  filters/pause/nbdkit-pause-filter.pod | 2 +-
> >  filters/retry/nbdkit-retry-filter.pod | 2 +-
> >  plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod | 4 ++--
> >  plugins/nbd/nbdkit-nbd-plugin.pod | 2 +-
> >  plugins/random/nbdkit-random-plugin.pod   | 2 +-
> >  plugins/sparse-random/nbdkit-sparse-random-plugin.pod | 2 +-
> >  plugins/torrent/nbdkit-torrent-plugin.pod | 6 +++---
> >  plugins/vddk/nbdkit-vddk-plugin.pod   | 4 ++--
> >  BENCHMARKING  | 4 ++--
> 
> Unless I'm mistaken, BENCHMARKING is the only file in this list that
> is not live on the website, and therefore which could be hoisted into
> 9/10 if desired.  But I see no harm in leaving it here.
> 
> Obviously, we'd better not forget to apply this at the correct later
> date, and you'll have to touch up the commit message at that point.
> But once we are ready, feel free to add
> 
> Reviewed-by: Eric Blake 

I now have a special local branch called
'2023-run-U-push-after-1.36-is-released' :-)

Thanks for the reviews.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH nbdkit 04/10] server: Calculate $uri in one place

2023-09-11 Thread Richard W.M. Jones
On Mon, Sep 11, 2023 at 09:19:12AM -0500, Eric Blake wrote:
> On Mon, Sep 11, 2023 at 03:09:21PM +0100, Richard W.M. Jones wrote:
> > > > -  case SERVICE_MODE_UNIXSOCKET:
> > > > -fprintf (fp, "nbd%s+unix://", tls == 2 ? "s" : "");
> > > > -if (export_name && strcmp (export_name, "") != 0) {
> > > > -  putc ('/', fp);
> > > > -  uri_quote (export_name, fp);
> > > > -}
> > > > -fprintf (fp, "\\?socket=");
> > > > -uri_quote (unixsocket, fp);
> > > 
> > > Beforehand, we were manually shell-quoting the ? in the Unix URI...
> > 
> > The shell quoting here was only marginally useful before this change.
> > In theory there might be a file in a subdirectory called
> > 'nbd+unix:/Xsocket=' which would match :-)
> 
> > > > +  switch (service_mode) {
> > > > +  case SERVICE_MODE_UNIXSOCKET:
> > > > +fprintf (fp, "nbd%s+unix://", tls == 2 ? "s" : "");
> > > > +if (export_name && strcmp (export_name, "") != 0) {
> > > > +  putc ('/', fp);
> > > > +  uri_quote (export_name, fp);
> > > > +}
> > > > +fprintf (fp, "?socket=");
> > > > +uri_quote (unixsocket, fp);
> > > 
> > > ...where the manual shell-quoting is no longer injected.  Yes, this
> > > looks correct (the appearance of the quoting, using '' instead of \,
> > > may be different, but the resulting string as parsed by the shell is
> > > the same).
> 
> My point was that pre-patch, we had:
> 
> nbd+unix://\?socket=...
> 
> and post-patch, we have:
> 
> 'nbd+unix://?socket=...'
> 
> but both forms are equally quoted; after the shell removes \ or ''
> quoting, we are left with:
> 
> nbd+unix://?socket=...
> 
> and neither form allowed the glob expansion of the (unlikely) file
> nbd+unix:/Xsocket=...
> 
> My comment was more about why changing the fprintf("\\?socket=")
> pre-patch to fprintf("?socket=") post-patch was correct.

Agreed, thanks!

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH nbdkit 05/10] server: Add the NBD URI to debug output

2023-09-11 Thread Richard W.M. Jones
On Mon, Sep 11, 2023 at 09:04:13AM -0500, Eric Blake wrote:
> Do we also want to output a debug statement when a URI is not
> possible, such as under -s?

Yes, I'll add something for those two cases.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH nbdkit 04/10] server: Calculate $uri in one place

2023-09-11 Thread Richard W.M. Jones
On Mon, Sep 11, 2023 at 09:01:47AM -0500, Eric Blake wrote:
> On Sat, Sep 09, 2023 at 02:57:52PM +0100, Richard W.M. Jones wrote:
> > Move the calculation of $uri to the main function (well, inlined
> > there), and out of --run code.
> > 
> > This is largely code motion.  In theory it changes the content of $uri
> > since we now shell quote it after generating it, but this ought not to
> > have any practical effect.
> > ---
> >  server/internal.h |  1 +
> >  server/captive.c  | 41 ++--
> >  server/main.c | 79 +++
> >  3 files changed, 82 insertions(+), 39 deletions(-)
> > 
> 
> > +++ b/server/captive.c
> > @@ -75,45 +75,8 @@ run_command (void)
> >  
> >/* Construct $uri. */
> >fprintf (fp, "uri=");
> > -  switch (service_mode) {
> > -  case SERVICE_MODE_SOCKET_ACTIVATION:
> > -  case SERVICE_MODE_LISTEN_STDIN:
> > -break;  /* can't form a URI, leave it blank */
> > -  case SERVICE_MODE_UNIXSOCKET:
> > -fprintf (fp, "nbd%s+unix://", tls == 2 ? "s" : "");
> > -if (export_name && strcmp (export_name, "") != 0) {
> > -  putc ('/', fp);
> > -  uri_quote (export_name, fp);
> > -}
> > -fprintf (fp, "\\?socket=");
> > -uri_quote (unixsocket, fp);
> 
> Beforehand, we were manually shell-quoting the ? in the Unix URI...

The shell quoting here was only marginally useful before this change.
In theory there might be a file in a subdirectory called
'nbd+unix:/Xsocket=' which would match :-)


> > -break;
> > -  case SERVICE_MODE_VSOCK:
> > -/* 1 = VMADDR_CID_LOCAL */
> > -fprintf (fp, "nbd%s+vsock://1", tls == 2 ? "s" : "");
> > -if (port) {
> > -  putc (':', fp);
> > -  shell_quote (port, fp);
> > -}
> > -if (export_name && strcmp (export_name, "") != 0) {
> > -  putc ('/', fp);
> > -  uri_quote (export_name, fp);
> > -}
> > -break;
> > -  case SERVICE_MODE_TCPIP:
> > -fprintf (fp, "nbd%s://localhost", tls == 2 ? "s" : "");
> > -if (port) {
> > -  putc (':', fp);
> > -  shell_quote (port, fp);
> > -}
> > -if (export_name && strcmp (export_name, "") != 0) {
> > -  putc ('/', fp);
> > -  uri_quote (export_name, fp);
> > -}
> > -break;
> > -  default:
> > -abort ();
> > -  }
> > +  if (uri)
> > +shell_quote (uri, fp);
> 
> ...while here, we shell-quote the entire string...

Right, and '?' is not listed as a "safe char" so it should be quoted:

https://gitlab.com/nbdkit/nbdkit/-/blob/ff3c9eb0e2afb24def80950cbfc963c14b037ba5/common/utils/quote.c#L54

> >putc ('\n', fp);
> >  
> >/* Since nbdkit 1.24, $nbd is a synonym for $uri. */
> > diff --git a/server/main.c b/server/main.c
> > index 2a332bfdd..54eb348ba 100644
> > --- a/server/main.c
> 
> > +static char *
> > +make_uri (void)
> 
> > +
> > +  switch (service_mode) {
> > +  case SERVICE_MODE_UNIXSOCKET:
> > +fprintf (fp, "nbd%s+unix://", tls == 2 ? "s" : "");
> > +if (export_name && strcmp (export_name, "") != 0) {
> > +  putc ('/', fp);
> > +  uri_quote (export_name, fp);
> > +}
> > +fprintf (fp, "?socket=");
> > +uri_quote (unixsocket, fp);
> 
> ...where the manual shell-quoting is no longer injected.  Yes, this
> looks correct (the appearance of the quoting, using '' instead of \,
> may be different, but the resulting string as parsed by the shell is
> the same).
> 
> > +break;
> > +  case SERVICE_MODE_VSOCK:
> > +/* 1 = VMADDR_CID_LOCAL */
> > +fprintf (fp, "nbd%s+vsock://1", tls == 2 ? "s" : "");
> > +if (port) {
> > +  putc (':', fp);
> > +  fputs (port, fp);
> > +}
> > +if (export_name && strcmp (export_name, "") != 0) {
> > +  putc ('/', fp);
> > +  uri_quote (export_name, fp);
> > +}
> > +break;
> > +  case SERVICE_MODE_TCPIP:
> > +fprintf (fp, "nbd%s://localhost", tls == 2 ? "s" : "");
> > +if (port) {
> > +  putc (':', fp);
> > +  fputs (port, fp);
> > +}
> > +if (export_name && strcmp (export_name, "&q

Re: [Libguestfs] [PATCH nbdkit 00/10] Make --run imply -U -

2023-09-11 Thread Richard W.M. Jones
On Mon, Sep 11, 2023 at 08:18:04AM -0500, Eric Blake wrote:
> On Mon, Sep 11, 2023 at 01:11:46PM +0100, Richard W.M. Jones wrote:
> > On Mon, Sep 11, 2023 at 07:08:48AM -0500, Eric Blake wrote:
> > > On Sat, Sep 09, 2023 at 02:57:48PM +0100, Richard W.M. Jones wrote:
> > > > Should have done this a long time ago.  I feel it is about time we
> > > > change the default of nbdkit --run to imply -U -, rather than opening
> > > > a public port.
> > > > 
> > > > Patch series turned out to be a little bit more complicated than I
> > > > anticipated, but it contains some nice clean ups.
> > > 
> > > Indeed, just from the summary it sounds like a good idea.  nbdkit can
> > > take advantage of it immediately, but libnbd will need to continue to
> > > explicitly use -U - for a while longer (as long as older nbdkit tends
> > > to be the version still installed on various CI platforms).
> > 
> > Although it might not be useful for libnbd, we could also had a
> > --dump-config sentinel, something like:
> > 
> >   $ nbdkit --dump-config
> >   ...
> >   run_default_socket=Unix
> > 
> > (and add run_default_socket=TCP to older branches).  If you can think
> > of a better name for this ... :-)
> 
> That actually sounds good as-is.  I don't know if backporting a patch
> to expose run_default_socket=TCP makes a difference (after all, the
> absence of run_default_socket= in the config output serves the same
> purpose), but I'll leave that up to your call.

Something like the attached patch.  It leaves it open whether
we add the sentinel to older versions or not.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
>From 75dad8e87c50c6c4cb3f2f882ed9a6fe25177bb1 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" 
Date: Mon, 11 Sep 2023 14:06:03 +0100
Subject: [PATCH] server: Add --dump-config run_default_socket=Unix sentinel

In builds where --run implies -U - you will now see:

$ nbdkit --dump-config
...
run_default_socket=Unix

This is absent or has a different value in older versions of nbdkit.
---
 server/main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/server/main.c b/server/main.c
index 978a720cf..3a429fa46 100644
--- a/server/main.c
+++ b/server/main.c
@@ -190,6 +190,7 @@ dump_config (void)
   printf ("%s=%s\n", "name", PACKAGE_NAME);
   printf ("%s=%s\n", "plugindir", plugindir);
   printf ("%s=%s\n", "root_tls_certificates_dir", root_tls_certificates_dir);
+  printf ("%s=%s\n", "run_default_socket", "Unix");
   printf ("%s=%s\n", "sbindir", sbindir);
 #ifdef HAVE_LIBSELINUX
   printf ("selinux=yes\n");
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs


Re: [Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-11 Thread Richard W.M. Jones
On Sun, Sep 10, 2023 at 11:34:22AM +0100, Richard W.M. Jones wrote:
> This message got caught in moderation because the attachment
> was large.  I put the attachment here:
> 
> http://oirase.annexia.org/tmp/2023-lgarrett-virt-v2v-debug-output.txt
> 
> - Forwarded message from Lee Garrett  -
> 
> Date: Sun, 10 Sep 2023 00:09:17 +0200
> From: Lee Garrett 
> To: libguestfs@redhat.com
> Subject: virt-v2v creating image that does not install guest agent on first
>  boot
> 
> Hi,
> 
> rwmjones from #guestfs on libera IRC pointed me to this mail address.
> 
> I've noticed that converting the latest Windows 11 trial image via
> virt-v2v does not install the guest agent any more on first boot. I
> have let the machine settle for ~10 minutes to ensure it's not just
> because I'm too impatient. I can see C:\qemu-ga-x86-64.msi, and also a
> bunch of files in  "C:\Program Files\Guestfs\Firstboot\" and
> .\Scripts, so the files are definitely copied onto the VM image, just
> not executed for some reason.

I can see from the log that these are installed in the image.

> The vmware image was downloaded from 
> https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/.
> The issue should be 100% reproducible (at least for me). I recall that
> an older image from two, three weeks ago downloaded from the same
> location did not have this issue. There guest agent was installed and
> working on the first boot. Unfortunately I overwrote the image so I
> cannot verify this.
> 
> I have attached the debug output of the virt-v2v run of the latest
> image where this issue is apparent (guest agent not being installed).
> 
> Please let me know if there's anything else I can check, or if you
> need any further info.

The main thing to look for is any debugging output which should appear
in c:\program files\guestfs\firstboot\log.txt

It would be interesting to know if this file is present at all, and
what exactly it contains.

I've CC'd my colleague Ming on this to see if she has noticed anything
with recent Windows 11.

Rich.

> Cheers,
> Lee
> 
> - End forwarded message -
> 
> -- 
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines.  Supports shell scripting,
> bindings from many languages.  http://libguestfs.org
> ___
> Libguestfs mailing list
> Libguestfs@redhat.com
> https://listman.redhat.com/mailman/listinfo/libguestfs

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [EXTERNAL] - Re: LIBGUESTFS supported version

2023-09-11 Thread Richard W.M. Jones
On Mon, Sep 11, 2023 at 12:18:21PM +, Teja Konapalli wrote:
> Hi ,
>
> Yes, was mounting one disk using libguestfs version 1.38 but the
> disk filesystem type is unknown, can't it be possible to mount
> unknown filesystem type disk?

This might not be related to the version of libguestfs, since
libguestfs uses your host kernel and host utilities to access the disk
(running them in a VM).  What exact "unknown filesystem" type was it?

Rich.

> Thanks
> Teja K
> 
> -Original Message-
> From: Richard W.M. Jones  
> Sent: Wednesday, September 6, 2023 4:50 PM
> To: Teja Konapalli 
> Cc: libguestfs@redhat.com
> Subject: [EXTERNAL] - Re: [Libguestfs] LIBGUESTFS supported version
> 
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open attachments unless you recognize the sender and know the 
> content is safe. If you feel that the email is suspicious, please report it 
> using PhishAlarm.
> 
> 
> On Wed, Sep 06, 2023 at 09:04:17AM +, Teja Konapalli wrote:
> > Am trying to install libguestfs in my redhat 8.2 version default its 
> > installing 1.38.4. Could you please help us with the installation of 
> > version 1.50 libguestfs (through command line in RHEL 8.2) and 
> > supported RHEL versions of 1.50.
> 
> The latest version of libguestfs for RHEL 8 is 1.44.0, but you would have to 
> update to at least RHEL 8.4 (preferably later) to get that.
> 
> libguestfs 1.50 is only available in RHEL 9.
> 
> Is there a specific task you are trying to accomplish?
> 
> Rich.
> 
> --
> Richard Jones, Virtualization Group, Red Hat 
> https://urldefense.com/v3/__http://people.redhat.com/*rjones__;fg!!Obbck6kTJA!Y-zNijcNl5D5cSaJLK5etpMyCrqIEDFmMdfXxFfSw_hfENnkcM7aYnaNOaWTdxPuaUzHUJi7luHZwudwvQ$
> Read my programming and virtualization blog: 
> https://urldefense.com/v3/__http://rwmj.wordpress.com__;!!Obbck6kTJA!Y-zNijcNl5D5cSaJLK5etpMyCrqIEDFmMdfXxFfSw_hfENnkcM7aYnaNOaWTdxPuaUzHUJi7luF4dn9wCQ$
> virt-p2v converts physical machines to virtual machines.  Boot with a live CD 
> or over the network (PXE) and turn machines into KVM guests.
> https://urldefense.com/v3/__http://libguestfs.org/virt-v2v__;!!Obbck6kTJA!Y-zNijcNl5D5cSaJLK5etpMyCrqIEDFmMdfXxFfSw_hfENnkcM7aYnaNOaWTdxPuaUzHUJi7luEIQaD0aw$

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



Re: [Libguestfs] [PATCH nbdkit 00/10] Make --run imply -U -

2023-09-11 Thread Richard W.M. Jones
On Mon, Sep 11, 2023 at 07:08:48AM -0500, Eric Blake wrote:
> On Sat, Sep 09, 2023 at 02:57:48PM +0100, Richard W.M. Jones wrote:
> > Should have done this a long time ago.  I feel it is about time we
> > change the default of nbdkit --run to imply -U -, rather than opening
> > a public port.
> > 
> > Patch series turned out to be a little bit more complicated than I
> > anticipated, but it contains some nice clean ups.
> 
> Indeed, just from the summary it sounds like a good idea.  nbdkit can
> take advantage of it immediately, but libnbd will need to continue to
> explicitly use -U - for a while longer (as long as older nbdkit tends
> to be the version still installed on various CI platforms).

Although it might not be useful for libnbd, we could also had a
--dump-config sentinel, something like:

  $ nbdkit --dump-config
  ...
  run_default_socket=Unix

(and add run_default_socket=TCP to older branches).  If you can think
of a better name for this ... :-)

> > Last patch updating the documentation wouldn't be applied any time
> > soon, so that the old docs stay around on the website.
> 
> Also a good idea.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] Fwd: virt-v2v creating image that does not install guest agent on first boot

2023-09-10 Thread Richard W.M. Jones
This message got caught in moderation because the attachment
was large.  I put the attachment here:

http://oirase.annexia.org/tmp/2023-lgarrett-virt-v2v-debug-output.txt

- Forwarded message from Lee Garrett  -

Date: Sun, 10 Sep 2023 00:09:17 +0200
From: Lee Garrett 
To: libguestfs@redhat.com
Subject: virt-v2v creating image that does not install guest agent on first
 boot

Hi,

rwmjones from #guestfs on libera IRC pointed me to this mail address.

I've noticed that converting the latest Windows 11 trial image via
virt-v2v does not install the guest agent any more on first boot. I
have let the machine settle for ~10 minutes to ensure it's not just
because I'm too impatient. I can see C:\qemu-ga-x86-64.msi, and also a
bunch of files in  "C:\Program Files\Guestfs\Firstboot\" and
.\Scripts, so the files are definitely copied onto the VM image, just
not executed for some reason.

The vmware image was downloaded from 
https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/.
The issue should be 100% reproducible (at least for me). I recall that
an older image from two, three weeks ago downloaded from the same
location did not have this issue. There guest agent was installed and
working on the first boot. Unfortunately I overwrote the image so I
cannot verify this.

I have attached the debug output of the virt-v2v run of the latest
image where this issue is apparent (guest agent not being installed).

Please let me know if there's anything else I can check, or if you
need any further info.

Cheers,
Lee

- End forwarded message -

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs



[Libguestfs] [PATCH nbdkit 04/10] server: Calculate $uri in one place

2023-09-09 Thread Richard W.M. Jones
Move the calculation of $uri to the main function (well, inlined
there), and out of --run code.

This is largely code motion.  In theory it changes the content of $uri
since we now shell quote it after generating it, but this ought not to
have any practical effect.
---
 server/internal.h |  1 +
 server/captive.c  | 41 ++--
 server/main.c | 79 +++
 3 files changed, 82 insertions(+), 39 deletions(-)

diff --git a/server/internal.h b/server/internal.h
index 0652f8a86..656b6d8ce 100644
--- a/server/internal.h
+++ b/server/internal.h
@@ -142,6 +142,7 @@ extern const char *user, *group;
 extern bool verbose;
 extern bool vsock;
 extern enum service_mode service_mode;
+extern char *uri;
 extern bool configured;
 extern int saved_stdin;
 extern int saved_stdout;
diff --git a/server/captive.c b/server/captive.c
index 40c4bb2ca..51dafca34 100644
--- a/server/captive.c
+++ b/server/captive.c
@@ -75,45 +75,8 @@ run_command (void)
 
   /* Construct $uri. */
   fprintf (fp, "uri=");
-  switch (service_mode) {
-  case SERVICE_MODE_SOCKET_ACTIVATION:
-  case SERVICE_MODE_LISTEN_STDIN:
-break;  /* can't form a URI, leave it blank */
-  case SERVICE_MODE_UNIXSOCKET:
-fprintf (fp, "nbd%s+unix://", tls == 2 ? "s" : "");
-if (export_name && strcmp (export_name, "") != 0) {
-  putc ('/', fp);
-  uri_quote (export_name, fp);
-}
-fprintf (fp, "\\?socket=");
-uri_quote (unixsocket, fp);
-break;
-  case SERVICE_MODE_VSOCK:
-/* 1 = VMADDR_CID_LOCAL */
-fprintf (fp, "nbd%s+vsock://1", tls == 2 ? "s" : "");
-if (port) {
-  putc (':', fp);
-  shell_quote (port, fp);
-}
-if (export_name && strcmp (export_name, "") != 0) {
-  putc ('/', fp);
-  uri_quote (export_name, fp);
-}
-break;
-  case SERVICE_MODE_TCPIP:
-fprintf (fp, "nbd%s://localhost", tls == 2 ? "s" : "");
-if (port) {
-  putc (':', fp);
-  shell_quote (port, fp);
-}
-if (export_name && strcmp (export_name, "") != 0) {
-  putc ('/', fp);
-  uri_quote (export_name, fp);
-}
-break;
-  default:
-abort ();
-  }
+  if (uri)
+shell_quote (uri, fp);
   putc ('\n', fp);
 
   /* Since nbdkit 1.24, $nbd is a synonym for $uri. */
diff --git a/server/main.c b/server/main.c
index 2a332bfdd..54eb348ba 100644
--- a/server/main.c
+++ b/server/main.c
@@ -66,6 +66,7 @@
 #include "ascii-string.h"
 #include "exit-with-parent.h"
 #include "nbd-protocol.h"
+#include "open_memstream.h"
 #include "realpath.h"
 #include "strndup.h"
 #include "syslog.h"
@@ -79,6 +80,7 @@
 #endif
 
 static char *make_random_fifo (void);
+static char *make_uri (void);
 static struct backend *open_plugin_so (size_t i, const char *filename,
int short_name);
 static struct backend *open_filter_so (struct backend *next, size_t i,
@@ -118,6 +120,7 @@ bool verbose;   /* -v */
 bool vsock; /* --vsock */
 unsigned int socket_activation; /* $LISTEN_FDS and $LISTEN_PID set */
 enum service_mode service_mode; /* serving over TCP, Unix, etc */
+char *uri;  /* NBD URI */
 bool configured;/* .config_complete done */
 int saved_stdin = -1;   /* dup'd stdin during -s/--run */
 int saved_stdout = -1;  /* dup'd stdout during -s/--run */
@@ -630,6 +633,11 @@ main (int argc, char *argv[])
   else
 service_mode = SERVICE_MODE_TCPIP;
 
+  /* By the point we have enough information to calculate the NBD URI.
+   * Note this may be NULL.
+   */
+  uri = make_uri ();
+
   /* The remaining command line arguments are the plugin name and
* parameters.  If --help, --version or --dump-plugin were specified
* then we open the plugin so that we can display the per-plugin
@@ -799,6 +807,7 @@ main (int argc, char *argv[])
 
   free (unixsocket);
   free (pidfile);
+  free (uri);
 
   if (random_fifo) {
 unlink (random_fifo);
@@ -856,6 +865,76 @@ make_random_fifo (void)
   return NULL;
 }
 
+static char *
+make_uri (void)
+{
+  FILE *fp;
+  size_t len = 0;
+  char *r = NULL;
+
+  switch (service_mode) {
+  case SERVICE_MODE_SOCKET_ACTIVATION:
+  case SERVICE_MODE_LISTEN_STDIN:
+/* can't form a URI, uri will be NULL */
+return NULL;
+  default: ;
+  }
+
+  fp = open_memstream (, );
+  if (fp == NULL) {
+perror ("open_memstream");
+exit (EXIT_FAILURE);
+  }
+
+  switch (service_mode) {
+  case SERVICE_MODE_UNIXSOCKET:
+fprintf (fp, "nbd%s+unix://", tls == 2 ? "s" : "");
+if (export_name && strcmp (export_name, "") != 0) {
+  putc ('/', fp);
+  uri_quote (export_name, fp);
+}
+fprintf (fp, "?socket=");
+uri_quote (unixsocket, fp);
+break;
+  case SERVICE_MODE_VSOCK:
+/* 1 = VMADDR_CID_LOCAL */
+fprintf (fp, "nbd%s+vsock://1", tls == 2 ? "s" : "");
+if (port) {
+  putc (':', fp);
+  fputs (port, fp);
+}
+if (export_name && strcmp 

[Libguestfs] [PATCH nbdkit 01/10] server: Introduce service_mode concept

2023-09-09 Thread Richard W.M. Jones
Previously there were two places where similiar-ish logic was used to
decide if we are going to serve over a socket activation, -s, Unix
socket, AF_VSOCK or TCP/IP.  Let's abstract that into a service_mode.

One place where we did this was when calculating the $uri variable for
--run.  This change adjusts and fixes this calculation (revealed as I
made the above change).  In particular if --port was not set then the
$uri would contain fairly bogus values in some cases:

  $ nbdkit --vsock null 1M --run 'echo $uri ; nbdinfo $uri'
  nbd
  nbdinfo: nbd_connect_uri: NBD URI does not have a scheme: valid NBD URIs 
should start with a scheme like nbd://, nbds:// or nbd+unix://: Invalid argument

(note uri='nbd')

After this commit:

  $ nbdkit --vsock null 1M --run 'echo $uri ; nbdinfo $uri'
  nbd+vsock://1
  protocol: newstyle-fixed without TLS, using structured packets
  export="":
export-size: 1048576 (1M)
content: data
uri: nbd+vsock://1:10809/
  ...
---
 server/internal.h | 11 +++
 server/captive.c  | 56 ---
 server/main.c | 75 ++-
 3 files changed, 91 insertions(+), 51 deletions(-)

diff --git a/server/internal.h b/server/internal.h
index fcfbf0573..0652f8a86 100644
--- a/server/internal.h
+++ b/server/internal.h
@@ -108,6 +108,16 @@ enum log_to {
   LOG_TO_NULL,   /* --log=null forced on the command line */
 };
 
+enum service_mode {
+  /* These two modes cannot form an NBD URI: */
+  SERVICE_MODE_SOCKET_ACTIVATION, /* socket activation. */
+  SERVICE_MODE_LISTEN_STDIN,  /* -s */
+
+  SERVICE_MODE_UNIXSOCKET,/* -U */
+  SERVICE_MODE_VSOCK, /* --vsock */
+  SERVICE_MODE_TCPIP, /* --port */
+};
+
 extern int tcpip_sock_af;
 extern struct debug_flag *debug_flags;
 extern const char *export_name;
@@ -131,6 +141,7 @@ extern char *unixsocket;
 extern const char *user, *group;
 extern bool verbose;
 extern bool vsock;
+extern enum service_mode service_mode;
 extern bool configured;
 extern int saved_stdin;
 extern int saved_stdout;
diff --git a/server/captive.c b/server/captive.c
index 2361bb60b..31fd949e5 100644
--- a/server/captive.c
+++ b/server/captive.c
@@ -78,36 +78,44 @@ run_command (void)
 
   /* Construct $uri. */
   fprintf (fp, "uri=");
-  if (tls == 2) /* --tls=require */
-fprintf (fp, "nbds");
-  else
-fprintf (fp, "nbd");
-  if (port) {
-if (!vsock) {
-  fprintf (fp, "://localhost:");
-  shell_quote (port, fp);
-  if (strcmp (export_name, "") != 0) {
-putc ('/', fp);
-uri_quote (export_name, fp);
-  }
-}
-else {
-  fprintf (fp, "+vsock://1:"); /* 1 = VMADDR_CID_LOCAL */
-  shell_quote (port, fp);
-  if (strcmp (export_name, "") != 0) {
-putc ('/', fp);
-uri_quote (export_name, fp);
-  }
-}
-  }
-  else if (unixsocket) {
-fprintf (fp, "+unix://");
+  switch (service_mode) {
+  case SERVICE_MODE_SOCKET_ACTIVATION:
+  case SERVICE_MODE_LISTEN_STDIN:
+break;  /* can't form a URI, leave it blank */
+  case SERVICE_MODE_UNIXSOCKET:
+fprintf (fp, "nbd%s+unix://", tls == 2 ? "s" : "");
 if (strcmp (export_name, "") != 0) {
   putc ('/', fp);
   uri_quote (export_name, fp);
 }
 fprintf (fp, "\\?socket=");
 uri_quote (unixsocket, fp);
+break;
+  case SERVICE_MODE_VSOCK:
+/* 1 = VMADDR_CID_LOCAL */
+fprintf (fp, "nbd%s+vsock://1", tls == 2 ? "s" : "");
+if (port) {
+  putc (':', fp);
+  shell_quote (port, fp);
+}
+if (strcmp (export_name, "") != 0) {
+  putc ('/', fp);
+  uri_quote (export_name, fp);
+}
+break;
+  case SERVICE_MODE_TCPIP:
+fprintf (fp, "nbd%s://localhost", tls == 2 ? "s" : "");
+if (port) {
+  putc (':', fp);
+  shell_quote (port, fp);
+}
+if (strcmp (export_name, "") != 0) {
+  putc ('/', fp);
+  uri_quote (export_name, fp);
+}
+break;
+  default:
+abort ();
   }
   putc ('\n', fp);
 
diff --git a/server/main.c b/server/main.c
index 528a2dfea..2a332bfdd 100644
--- a/server/main.c
+++ b/server/main.c
@@ -117,6 +117,7 @@ const char *user, *group;   /* -u & -g */
 bool verbose;   /* -v */
 bool vsock; /* --vsock */
 unsigned int socket_activation; /* $LISTEN_FDS and $LISTEN_PID set */
+enum service_mode service_mode; /* serving over TCP, Unix, etc */
 bool configured;/* .config_complete done */
 int saved_stdin = -1;   /* dup'd stdin during -s/--run */
 int saved_stdout = -1;  /* dup'd stdout during -s/--run */
@@ -617,6 +618,18 @@ main (int argc, char *argv[])
 exit (EXIT_FAILURE);
   }
 
+  /* By the point we have enough information to calculate the service mode. */
+  if (socket_activation)
+service_mode = SERVICE_MODE_SOCKET_ACTIVATION;
+  else if (listen_stdin)
+service_mode = SERVICE_MODE_LISTEN_STDIN;
+  else 

[Libguestfs] [PATCH nbdkit 09/10] tests: Remove references to -U - when it is implicit

2023-09-09 Thread Richard W.M. Jones
In tests where we used 'nbdkit -U - ... --run', remove -U - as that is
now implicit.
---
 tests/Makefile.am|  2 +-
 plugins/rust/test-ramdisk.sh |  4 +--
 tests/test-S3.sh |  2 +-
 tests/test-blkio.sh  |  2 +-
 tests/test-block-size-constraints.sh |  2 +-
 tests/test-blocksize-default.sh  |  2 +-
 tests/test-blocksize-error-policy.sh |  2 +-
 tests/test-blocksize-extents.sh  |  2 +-
 tests/test-blocksize-policy.sh   | 12 +++
 tests/test-blocksize-sharding.sh |  2 +-
 tests/test-blocksize-write-disconnect.sh |  2 +-
 tests/test-captive-tls.sh|  2 +-
 tests/test-captive.sh|  4 +--
 tests/test-cc-cpp.sh |  2 +-
 tests/test-cc-ocaml.sh   |  2 +-
 tests/test-cc.sh |  2 +-
 tests/test-checkwrite-bounds.sh  |  2 +-
 tests/test-checkwrite.sh |  2 +-
 tests/test-cow-block-size.sh |  2 +-
 tests/test-cow-extents-large.sh  |  2 +-
 tests/test-cow-null.sh   |  2 +-
 tests/test-cow.sh|  2 +-
 tests/test-curl-file.sh  |  2 +-
 tests/test-curl-header-script-fail.sh|  2 +-
 tests/test-data-bad.sh   |  2 +-
 tests/test-data-extents.sh   |  2 +-
 tests/test-data-format.sh|  8 ++---
 tests/test-data-optimum.sh   |  2 +-
 tests/test-data-partition.sh |  2 +-
 tests/test-data-raw-copy.sh  |  2 +-
 tests/test-data-reloffset.sh |  4 +--
 tests/test-data-size.sh  |  2 +-
 tests/test-debug-flags.sh| 22 ++---
 tests/test-delay-close.sh|  2 +-
 tests/test-delay-open.sh |  2 +-
 tests/test-disk2data.sh  |  2 +-
 tests/test-eflags.sh |  2 +-
 tests/test-eval-cache.sh |  4 +--
 tests/test-eval-exports.sh   |  8 ++---
 tests/test-eval-file.sh  |  2 +-
 tests/test-eval.sh   |  2 +-
 tests/test-evil-cosmic.sh|  2 +-
 tests/test-evil-large-p.sh   |  6 ++--
 tests/test-evil-small-p.sh   |  6 ++--
 tests/test-exportname.sh | 40 
 tests/test-extentlist.sh |  2 +-
 tests/test-file-dir.sh   |  6 ++--
 tests/test-file-extents.sh   |  2 +-
 tests/test-ip-filter-anyunix.sh  |  4 +--
 tests/test-ip-filter-gid.sh  |  4 +--
 tests/test-ip-filter-pid.sh  |  4 +--
 tests/test-ip-filter-security.sh |  4 +--
 tests/test-ip-filter-uid.sh  |  4 +--
 tests/test-linuxdisk-copy-out.sh |  2 +-
 tests/test-log-script-info.sh|  2 +-
 tests/test-long-name.sh  | 14 -
 tests/test-luks-copy-zero.sh |  2 +-
 tests/test-luks-info.sh  |  2 +-
 tests/test-multi-conn-name.sh|  4 +--
 tests/test-multi-conn.sh | 18 +--
 tests/test-nbd-block-size.sh |  4 +--
 tests/test-nbd-extents.sh|  2 +-
 tests/test-nbd-vsock.sh  |  2 +-
 tests/test-nbdkit-backend-debug.sh   |  6 ++--
 tests/test-nofilter.sh   |  2 +-
 tests/test-nozero.sh |  2 +-
 tests/test-null-extents.sh   |  2 +-
 tests/test-ocaml-fork.sh |  2 +-
 tests/test-ocaml-list-exports.sh |  2 +-
 tests/test-offset-extents.sh |  2 +-
 tests/test-offset-truncate.sh|  6 ++--
 tests/test-old-plugins.sh|  2 +-
 tests/test-ondemand-list.sh  |  2 +-
 tests/test-ones.sh   |  6 ++--
 tests/test-parallel-file.sh  |  6 ++--
 tests/test-parallel-nbd.sh   |  4 +--
 tests/test-parallel-sh.sh|  6 ++--
 tests/test-partition-4k-gpt.sh   |  2 +-
 tests/test-partition-4k-mbr.sh   |  2 +-
 tests/test-partition1.sh |  2 +-
 tests/test-partition2.sh |  8 ++---
 tests/test-partitioning1.sh  |  6 ++--
 tests/test-partitioning4.sh  |  2 +-
 tests/test-partitioning6.sh  |  2 +-
 tests/test-protect-ranges.sh |  2 +-
 tests/test-qcow2dec-map.sh   |  2 +-
 tests/test-qcow2dec.sh   |  2 +-
 tests/test-random-copy.sh|  2 +-
 tests/test-random-sock.sh|  4 +--
 tests/test-rate-dynamic.sh   |  2 +-
 tests/test-rate.sh   |  2 +-
 tests/test-readahead-copy.sh |  4 +--
 tests/test-readahead.sh  |  2 +-
 tests/test-retry-extents.sh  |  2 +-
 tests/test-retry-open.sh |  2 +-
 tests/test-retry-readonly.sh |  2 +-
 

  1   2   3   4   5   6   7   8   9   10   >