On 2020/08/26 14:03, Marc Espie wrote: > On Wed, Aug 26, 2020 at 12:48:27PM +0100, Stuart Henderson wrote: > > This updates rsync to 3.2.3, which amongst other things includes > > https://github.com/WayneD/rsync/commit/9375a8c4c2b79a9d44ceae00de7284b77d79033c > > which greatly reduces maxrss (https://github.com/WayneD/rsync/issues/81). > > > > Also adds a patch for the --rsh problem I ran into (issue open upstream). > > > > OK? > > Okay. > > > > Not included in the diff, but also I would like to reopen the discussion > > about dependencies. In order to lower resource use for CVS repository > > mirrors (7.3G of data across 450K files) I would like to have xxhash > > and lz4 enabled in the default ports build. > > > > xxhash and lz4 are nearly self-contained ports, they use the base > > compiler, they use gmake but otherwise add no build dependencies above > > what's already needed for rsync, and are available on all archs we > > build packages for. > > > > xxhash is a single C file, takes ~10 seconds to build, the package > > weighs in at 200K. > > > > lz4 is a little larger, 4 C files in the library, and there are tools > > too so it takes more like 45 seconds to build, the package is 300K. > > > > These are not onerous requirements. > > > > zstd is a bit different, it uses "COMPILER=base-clang ports-gcc". > > Seems this is needed for -Wvla which is used in the Makefile (easily > > patched), and maybe for something related to atomics on certain archs > > (hppa and powerpc are the usual suspects for this). It would be nice > > to have zstd as well (it's an excellent compression tool and nicely > > licensed) but that wants more work first so I am not proposing that at > > this point. > > I don't think we'll be able to make it default, Theo will object. > If you want it, you have to convince *him*, not me. > > The problem is not onerousness, but rather Theo being scared of more > compression algorithms, which tend to be rife with overflows...
Oh, I guess the commit message was wrong then "Disable lz4, zstd and xxhash again The dependencies are too big" lz4 has a reasonably good track record (not spotless but not rife with overflows) and (unlike rsync's internal zlib, which is 1.2.8 plus some rsync patch) is now extensively fuzzed https://github.com/lz4/lz4/releases I am far more concerned about rsync itself which is already a labyrinth of so many options that it's nigh on impossible to get wide test coverage, but *shrug* there's no usable alternative for what it does, I jail it, what more can I do? > rsync is cheap enough that more flavors is not really an issue. > It is an issue, after being presented with an overwhelming list of various flavours people will tend to pick the default option, so mirror operators won't benefit from clients having access to cheaper algorithms. FWIW this is the diff I have locally, Index: Makefile =================================================================== RCS file: /cvs/ports/net/rsync/Makefile,v retrieving revision 1.89 diff -u -p -r1.89 Makefile --- Makefile 26 Aug 2020 12:40:39 -0000 1.89 +++ Makefile 26 Aug 2020 12:42:56 -0000 @@ -5,6 +5,7 @@ COMMENT = mirroring/synchronization over DISTNAME = rsync-3.2.3 CATEGORIES = net HOMEPAGE = https://rsync.samba.org/ +REVISION = 0 MAINTAINER = Marc Espie <[email protected]> @@ -14,27 +15,32 @@ FLAVOR ?= # GPLv3 PERMIT_PACKAGE = Yes -WANTLIB = c crypto +WANTLIB = c crypto lz4 xxhash MASTER_SITES = https://rsync.samba.org/ftp/rsync/src/ \ https://ftp.funet.fi/pub/mirrors/samba.org/pub/rsync/src/ SEPARATE_BUILD =Yes CONFIGURE_STYLE =gnu -CONFIGURE_ARGS =--disable-lz4 \ - --disable-xxhash \ - --disable-zstd \ +CONFIGURE_ARGS = --disable-zstd \ --with-included-popt \ --with-included-zlib \ --with-rsyncd-conf="${SYSCONFDIR}/rsyncd.conf" \ --with-rsh=/usr/bin/ssh \ --with-nobody-group=_rsync -.if ${FLAVOR:Miconv} -CONFIGURE_ENV +=CPPFLAGS='-I${LOCALBASE}/include' \ +CONFIGURE_ENV += CPPFLAGS='-I${LOCALBASE}/include' \ LDFLAGS='-L${LOCALBASE}/lib' + +LIB_DEPENDS += archivers/lz4 \ + sysutils/xxhash + +.if ${FLAVOR:Miconv} LIB_DEPENDS += converters/libiconv WANTLIB += iconv +.else +# --disable-iconv-open only covers iconv_open not libiconv_open +CONFIGURE_ENV += ac_cv_func_libiconv_open=false .endif DOCDIR = ${PREFIX}/share/doc/rsync
