On 2023-11-05 16:34, Martin Matuska wrote:
OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can work with pools that have feature@block_cloning enabled. The sysctl variable vfs.zfs.bclone_enabled affects the behavior of zfs_clone_range() which is called by copy_file_range(). When it is set to 0, zfs_clone_range() does not do block cloning. If it is set to anything else than 0, zfs_clone_range() does block cloning (if all conditions are met - same ZFS pool, correct data alignment, etc.).In FreeBSD-main, this tunable is enabled and I plan to enable it in stable/14 somewhere around December 11, 2023.As of today I personally use block cloning on all my systems.
I'd like to share a different data point. It still panics on my storage (running -CURRENT about a week ago) when enabled and can be triggered by "make buildworld buildkernel". I wasn't able to capture earlier coredump until the most recent one, which panicked with:
cpuid = 2 time = 1699593456 KDB: stack backtrace:db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe022f2bd7e0
vpanic() at vpanic+0x132/frame 0xfffffe022f2bd910 spl_panic() at spl_panic+0x3a/frame 0xfffffe022f2bd970 dmu_brt_clone() at dmu_brt_clone+0x555/frame 0xfffffe022f2bd9e0 zfs_clone_range() at zfs_clone_range+0xa4c/frame 0xfffffe022f2bdbb0zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe022f2bdc30
vn_copy_file_range() at vn_copy_file_range+0x163/frame 0xfffffe022f2bdce0kern_copy_file_range() at kern_copy_file_range+0x380/frame 0xfffffe022f2bddb0
sys_copy_file_range() at sys_copy_file_range+0x78/frame 0xfffffe022f2bde00 amd64_syscall() at amd64_syscall+0x153/frame 0xfffffe022f2bdf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe022f2bdf30--- syscall (569, FreeBSD ELF64, copy_file_range), rip = 0x7fbb2da4ada, rsp = 0x7fbb02c5d48, rbp = 0x7fbb02c61e0 ---
Uptime: 2h32m27sDumping 7800 out of 32696 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
#0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57#1 doadump (textdump=textdump@entry=1) at /usr/src/sys/kern/kern_shutdown.c:405 #2 0xffffffff80694480 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:526 #3 0xffffffff8069497f in vpanic (fmt=0xffffffff82603415 "VERIFY3(nbps == numbufs) failed (%llu == %llu)\n", ap=ap@entry=0xfffffe022f2bd950) at /usr/src/sys/kern/kern_shutdown.c:970 #4 0xffffffff8232999a in spl_panic (file=<optimized out>, func=<optimized out>, line=<unavailable>, fmt=<unavailable>) at /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_misc.c:103 #5 0xffffffff823a6605 in dmu_brt_clone (os=os@entry=0xfffff800c5ce4000, object=<optimized out>, offset=offset@entry=0, length=length@entry=207477, tx=tx@entry=0xfffff8071a108d00, bps=bps@entry=0xfffffe01e218c000, nbps=2, replay=0)
at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:2303
#6 0xffffffff8250f67c in zfs_clone_range (inzp=0xfffff804416ac000,
inoffp=0xfffff800b81cb048, outzp=0xfffff806f58f03a0,
outoffp=0xfffff800b8063048, lenp=lenp@entry=0xfffffe022f2bdbf0,
cr=0xfffff8000a6fe600)
at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1326
#7 0xffffffff8234b3ba in zfs_freebsd_copy_file_range
(ap=0xfffffe022f2bdc48) at
/usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294
#8 0xffffffff8079f443 in VOP_COPY_FILE_RANGE (invp=0xfffff804416cb1c0,
inoffp=0xfffff800b81cb048, outvp=0xfffff806f51d3380,
outoffp=0xfffff800b8063048, lenp=0xfffffe022f2bdd48,
incred=0xfffff8000a6fe600, flags=<optimized out>,
outcred=<optimized out>, fsizetd=<optimized out>) at ./vnode_if.h:2385
#9 vn_copy_file_range (invp=invp@entry=0xfffff804416cb1c0,
inoffp=inoffp@entry=0xfffff800b81cb048,
outvp=outvp@entry=0xfffff806f51d3380,
outoffp=outoffp@entry=0xfffff800b8063048,
lenp=lenp@entry=0xfffffe022f2bdd48, flags=flags@entry=0,
incred=0xfffff8000a6fe600, outcred=0xfffff8000a6fe600,
fsize_td=0xfffffe022925b3a0) at /usr/src/sys/kern/vfs_vnops.c:3087
#10 0xffffffff8079a070 in kern_copy_file_range
(td=td@entry=0xfffffe022925b3a0, infd=<optimized out>,
inoffp=0xfffff800b81cb048, inoffp@entry=0x0, outfd=<optimized out>,
outoffp=0xfffff800b8063048, outoffp@entry=0x0, len=9223372036854775807,
flags=0) at /usr/src/sys/kern/vfs_syscalls.c:4973
#11 0xffffffff8079a178 in sys_copy_file_range (td=0xfffffe022925b3a0,
uap=0xfffffe022925b7a0) at /usr/src/sys/kern/vfs_syscalls.c:5011
#12 0xffffffff80a97aa3 in syscallenter (td=0xfffffe022925b3a0) at
/usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188
#13 amd64_syscall (td=0xfffffe022925b3a0, traced=0) at
/usr/src/sys/amd64/amd64/trap.c:1194
#14 <signal handler called> #15 0x000007fbb2da4ada in ?? ()and disabling bclone does appear to allow me to finish buildworld / buildkernel.
The pool didn't have redaction_list_spill enabled.The ASSERT3U(nbps, ==, numbufs); in dmu_brt_clone was added when block clone is first implemented.
It seems that I am the only person who is seeing this as of today. It seems that block clone was indeed being used for some data:
saturn bcloneused 1.18M - saturn bclonesaved 1.21M - saturn bcloneratio 2.02x - The pool have dedup enabled for some datasets.Any suggestions? (In extreme cases I can recreate the storage pool from backup or copy the data somewhere else, then recreate the pool, then copy data back, but I'd like to avoid that if possible)
Cheers,
mm On 04/11/2023 13:35, Mark Millard wrote:On Nov 4, 2023, at 04:38, Mike Karels <[email protected]> wrote:On 4 Nov 2023, at 4:01, Ronald Klop wrote:On 11/4/23 02:39, Mark Millard wrote:It looks to me like releng/14.0 (as of 14.0-RC4) still has: int zfs_bclone_enabled; SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTLFLAG_RWTUN, &zfs_bclone_enabled, 0, "Enable block cloning"); leaving block cloning effectively disabled by default, no matter what the pool has enabled. https://www.freebsd.org/releases/14.0R/relnotes/ also reports: QUOTE OpenZFS has been upgraded to version 2.2. New features include: •block cloning, which allows shallow copies of blocks in file copies. This is optional, and disabled by default; it can be enabled with sysctl vfs.zfs.bclone_enabled=1.END QUOTEI think this answers your question in the subject.I think so too (and I wrote that text).Thanks for the confirmation of the final intent. I believe this makes: QUOTE author Brian Behlendorf <[email protected]> 2023-05-25 20:53:08 +0000 committer GitHub <[email protected]> 2023-05-25 20:53:08 +0000 commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch) tree dd01dfce6aeef357ade1775acf18aade535c6271 . . . Update compatibility.d filesAdd an openzfs-2.2 compatibility file for the next release. Edon-R support has been enabled for FreeBSD removing the need for different FreeBSD and Linux files. Symlinks for the -linux and -freebsd names are created for any scripts expecting that convention. Additionally, a symlink for ubunutu-22.04 was added. Signed-off-by: Brian Behlendorf <[email protected]> Closes #14833END QUOTE technically incorrect in that compatibility.d/openzfs-2.2-freebsd should be distinct in content from compatibility.d/openzfs-2.2 so that block cloning would not be enabled.Just curiousity on my part about the default completeness of openzfs-2.2 support, not an objection either way.I haven't seen new issues with block cloning in the last few weeks mentioned on the mailing lists. All known issues are fixed AFAIK. But I can imagine that the risk+effect ratio of data corruption is seen as a bit too high for a 14.0 release for this particular feature. That does not diminish the rest of the completeness of openzfs-2.2.NB: I'm not involved in developing openzfs or the decision making in the release. Just repeating what I read on the lists.There was another block cloning fix in 14.0-RC4; see the commit log. Maybe there will be no more issues, but it seems that corner cases were still being found recently.Looks like I'll stay at openzfs-2.1 pool features until there is a release that no longer has the default status: 0 for sysctl vfs.zfs.bclone_enabled I use main [so: 15 now] but only enable openzfs-2.* pool features supported by default on some FreeBSD release, that has an accurate compatibility.d/openzfs-2.*-freebsd file. === Mark Millard marklmi at yahoo.com
OpenPGP_signature.asc
Description: OpenPGP digital signature
