On Thu, Oct 23, 2025 at 03:15:09PM +0200, Arsen Arsenović wrote: > "Richard W.M. Jones" <[email protected]> writes: > > > I wonder if you can grab > > 'copy/copy-allocated-destination-zero-asynch.sh.log' > > and/or 'copy/copy-allocated-destination-zero-synch.sh.log' from the > > failing tests? > > Sure thing. The contents of the logs of the synchronous version are: > > + requires cmp --version > + requires truncate --version > + requires stat --version > + requires test -r /dev/zero > + inp=copy-allocated-destination-zero-synch.in > + out=copy-allocated-destination-zero-synch.out > + cleanup_fn rm -f copy-allocated-destination-zero-synch.in > copy-allocated-destination-zero-synch.out > + local _hook=_cleanup_hook0 > + declare -ag _cleanup_hook0 > + eval '_cleanup_hook0=("$@")' > ++ _cleanup_hook0=("$@") > + rm -f copy-allocated-destination-zero-synch.in > copy-allocated-destination-zero-synch.out > + truncate -s 33554432 copy-allocated-destination-zero-synch.out > + truncate -r copy-allocated-destination-zero-synch.out > copy-allocated-destination-zero-synch.in > + nbdcopy --allocated --destination-is-zero --synchronous > --request-size=32K copy-allocated-destination-zero-synch.in > copy-allocated-destination-zero-synch.out > libnbd: debug: nbd1: nbd_create: allocated the handle (9134 bytes) > libnbd: debug: nbd1: nbd_is_uri: enter: > uri="copy-allocated-destination-zero-synch.in" > libnbd: debug: nbd1: nbd_is_uri: leave: ret=0 > libnbd: debug: nbd1: nbd_close: closing handle > libnbd: debug: nbd2: nbd_create: allocated the handle (9134 bytes) > libnbd: debug: nbd2: nbd_is_uri: enter: > uri="copy-allocated-destination-zero-synch.out" > libnbd: debug: nbd2: nbd_is_uri: leave: ret=0 > libnbd: debug: nbd2: nbd_close: closing handle > + echo Output: > Output: > + ls -lsh copy-allocated-destination-zero-synch.out > 512 -rw-r--r-- 1 portage portage 32M Oct 23 13:49 > copy-allocated-destination-zero-synch.out > ++ stat -c %s copy-allocated-destination-zero-synch.out > + size=33554432 > ++ stat -c %b copy-allocated-destination-zero-synch.out > + balloc=1 > ++ stat -c %B copy-allocated-destination-zero-synch.out > + bsize=512 > + alloc=512 > + '[' 33554432 -ne 33554432 ']' > + '[' 512 -ne 33554432 ']' > + echo > '/var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/copy-allocated-destination-zero-synch.sh: > allocated size is not 32M' > > /var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/copy-allocated-destination-zero-synch.sh: > allocated size is not 32M > + exit 1 > + _run_cleanup_hooks > + local _status=1 _i > + set +e > + trap '' INT QUIT TERM EXIT ERR > + echo > /var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/copy-allocated-destination-zero-synch.sh: > run cleanup hooks: exit code 1 > > /var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/copy-allocated-destination-zero-synch.sh: > run cleanup hooks: exit code 1 > + (( _i = 0 )) > + (( _i < 1 )) > + eval '"${_cleanup_hook0[@]}"' > ++ rm -f copy-allocated-destination-zero-synch.in > copy-allocated-destination-zero-synch.out > + (( ++_i )) > + (( _i < 1 )) > + exit 1 > FAIL copy-allocated-destination-zero-synch.sh (exit status: 1) > > I suspect you might find the following interesting: > > $ truncate -s 33554432 copy-allocated-destination-zero-synch.out > $ truncate -r copy-allocated-destination-zero-synch.out > copy-allocated-destination-zero-synch.in > $ ls -lsh copy* > 512 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:10 > copy-allocated-destination-zero-synch.in > 512 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:10 > copy-allocated-destination-zero-synch.out > $ sudo > /var/tmp/portage/sys-libs/libnbd-1.23.9/work/libnbd-1.23.9/copy/nbdcopy > --allocated --destination-is-zero --synchronous --request-size=32K > copy-allocated-destination-zero-synch.in > copy-allocated-destination-zero-synch.out > $ ls -lsh copy* > 512 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:13 > copy-allocated-destination-zero-synch.in > 512 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:13 > copy-allocated-destination-zero-synch.out > > (forgive the 'sudo', portage has a strict umask so I can't access that > path from my user) > > On tmpfs, I get the following instead: > > /tmp/foo$ ls -lsh * > 0 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:12 > copy-allocated-destination-zero-synch.in > 0 -rw-r--r-- 1 arsen arsen 32M Oct 23 15:12 > copy-allocated-destination-zero-synch.out > > ... and, after running nbdcopy: > > /tmp/foo$ ls -lsh * > 0 -rw-r--r-- 1 arsen arsen 32M Oct 23 13:58 > copy-allocated-destination-zero-synch.in > 32M -rw-r--r-- 1 arsen arsen 32M Oct 23 13:59 > copy-allocated-destination-zero-synch.out
I don't have OpenZFS to test, but what seems to be happening on regular filesystems is we call truncate on the destination before doing the copy: https://gitlab.com/nbdkit/libnbd/-/blob/6182a590ced685694574b74884c9cec53668f879/copy/main.c#L500 which eventually calls fallocate with mode == 0 here: https://gitlab.com/nbdkit/libnbd/-/blob/6182a590ced685694574b74884c9cec53668f879/copy/file-ops.c#L587 Confirmed in strace: 2745865 ftruncate(4, 0) = 0 2745865 ftruncate(4, 33554432) = 0 2745865 fallocate(4, 0, 0, 33554432) = 0 In Linux this is documented as: Allocating disk space The default operation (i.e., mode is zero) of fallocate() allocates the disk space within the range specified by offset and len. [...] which is the behaviour that we want here, allocating the space. We make no further writes to the output disk in this test, but end up with a 32M fully allocated, zeroed disk (what we want). ZoL is probably ignoring fallocate and not allocating the space. It may be returning ENOTSUP / EOPNOTSUPP, which we deliberately ignore. I'm fairly sure this is a libnbd bug however, since we shouldn't ignore unsupported fallocate (if ZoL returns that), if the --allocated flag is set. Eric, any thoughts? The code path that this pair of flags takes is very convoluted ... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit _______________________________________________ Libguestfs mailing list -- [email protected] To unsubscribe send an email to [email protected]
