On Jun 05 12:37:10, s...@spacehopper.org wrote:
> reminded by the dwz mail, rsync would also like to use xxhash if
> available:
>
> 'The xxHash library (https://cyan4973.github.io/xxHash/) provides
> extremely fast checksum functions that can make the "rsync algorithm"
> run much more quickly, especially when matching blocks in large files.
> Installing this development library adds xxhash checksums as the default
> checksum algorithm. You'll need at least v0.8.0 if you want rsync to
> include the full range of its checksum algorithms.'
Thank you for enabling this. I am testing an current/amd64,
rsyncing a 4G dir of video files, about 150-250 MB each.
I am touching the files before every run,
otherwise rsync just finishes almost instantly,
based on the mtime (right?).
$ touch /dload/Catastrophe/S*/*
$ time rsync -Hai4 --del /path/dir/ remote:/path/dir/
Is that a scenario where faster checksums are supposed
to make things faster, matching blocks in large files?
Before:
3m07.07s real 0m20.55s user 0m08.15s system
3m11.96s real 0m19.84s user 0m08.07s system
3m06.73s real 0m19.96s user 0m07.89s system
After:
3m06.68s real 0m19.88s user 0m07.99s system
3m13.86s real 0m19.83s user 0m08.38s system
3m06.63s real 0m20.67s user 0m08.02s system
Jan
> while xxHash does provide standard shared+static libraries, it is more
> commonly used as a "header-only library" (done here and also in dwz)
> so there's no additional run dependency in rsync for this.
>
> ok?
>
> Index: Makefile
> ===================================================================
> RCS file: /cvs/ports/net/rsync/Makefile,v
> retrieving revision 1.97
> diff -u -p -r1.97 Makefile
> --- Makefile 5 Jan 2023 21:59:21 -0000 1.97
> +++ Makefile 5 Jun 2023 11:36:48 -0000
> @@ -1,6 +1,7 @@
> COMMENT = mirroring/synchronization over low bandwidth links
>
> DISTNAME = rsync-3.2.7
> +REVISION = 0
> CATEGORIES = net
> HOMEPAGE = https://rsync.samba.org/
>
> @@ -19,12 +20,12 @@ MODULES = lang/python
>
> MODPY_RUNDEP = No
>
> -BUILD_DEPENDS = textproc/py-commonmark${MODPY_FLAVOR}
> +BUILD_DEPENDS = textproc/py-commonmark${MODPY_FLAVOR} \
> + sysutils/xxhash
>
> SEPARATE_BUILD =Yes
> CONFIGURE_STYLE =gnu
> CONFIGURE_ARGS =--disable-lz4 \
> - --disable-xxhash \
> --disable-zstd \
> --with-included-popt \
> --with-included-zlib \
> @@ -33,6 +34,8 @@ CONFIGURE_ARGS =--disable-lz4 \
> --with-rsh=/usr/bin/ssh \
> --with-nobody-user=_rsync \
> --with-nobody-group=_rsync
> +CONFIGURE_ENV +=CPPFLAGS="-I${LOCALBASE}/include -DXXH_INLINE_ALL=1" \
> + ac_cv_search_XXH64_createState=""
>
> .include <bsd.port.arch.mk>
>
> @@ -41,8 +44,7 @@ CONFIGURE_ARGS +=--enable-md5-asm
> .endif
>
> .if ${FLAVOR:Miconv}
> -CONFIGURE_ENV +=CPPFLAGS='-I${LOCALBASE}/include' \
> - LDFLAGS='-L${LOCALBASE}/lib'
> +CONFIGURE_ENV +=LDFLAGS='-L${LOCALBASE}/lib'
> LIB_DEPENDS += converters/libiconv
> WANTLIB += iconv
> .endif
>
>
