Re: Rsync 3.2.3pre1 released

[Wayne Davison via rsync]

On Tue, Jul 28, 2020 at 4:43 PM Nelson H. F. Beebe via rsync <
rsync@lists.samba.org> wrote:

> xxhash in particular is not a package that we have needed before, so none
> of the clients has it.
>

All new, wonderful libraries have a point in time where they begin to get
used.  If you transfer large files then you probably want it.  If you use
compression then you also probably want at least zstd.  The INSTALL file

includes full details on the things you might want to install before
running configure and the suggested package-install commands that would
have probably helped you out.

I also noted that the configure script asks for installation of cmarkgfm or
> commonmark
>

No, it does not for a release. It sounds like you grabbed a github repo
snapshot tar file instead of a release tar file. The release tars include
various generated files to make a normal install easier, and do not require
any commonmark stuff nor autoconf stuff.  One other thing that is not well
documented is that you can grab the latest generated files for the latest
git commit using "./prepare-source fetchgen".  Currently that requires an
existing rsync and an openssl binary (since samba's rsync daemon requires
an ssl connection these days).

The configure script bails out when a library is not found
>

Yeah, that is on the annoying side.  It would be nice if it created a full
list of the missing items and then failed with a full summary of what
should be either added or disabled.

Thanks for the package lists.  Many of those were already in the INSTALL
docs, but I haven't had access to a Fedora host, for instance, and getting
some info there is helpful.  You did err on the side of installing too much
stuff, though.  You installed some cmark tools which aren't related to the
python3 library, for instance.  And you installed some other python
libraries, such as a python version of xxhash, which is also not needed.
The extra installs don't harm anything, though (other than your disk space).

..wayne..
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: rsync-3.2.3pre1 released

[Randall S. Becker via rsync]

So far on my NonStop platform boxes, the rsync 3.2.3pre1 looks stable
building off our git clone.

Regards,
Randall

-- Brief whoami:
 NonStop developer since approximately 2112884442
 UNIX developer since approximately 421664400
-- In my real life, I talk too much.



-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Rsync 3.2.3pre1 released

[Ben RUBSON via rsync]

> On 28 Jul 2020, at 20:53, Nelson H. F. Beebe via rsync 
>  wrote:
> 
> To my surprise, ALL of the builds failed, and examination of the build
> logs showed they were all due to missing libraries or header files,
> notably for one or more of lz4, openssl, xxhash, and zstd.  Once I
> installed those packages, I got successful builds.

This is mainly to "force you" using these librairies, which bring very nice 
speedup / CPU saving.
This is also to be sure packages maintainers from the various systems will use 
them, so that these improvements are quickly adopted.

> I believe that it would be much better to simply disable the code that
> needs the missing library or its header files, and keep on running the
> configure script, with a prominent final report, something like:
> 
>   WARNING: configuration is complete, but some features are
>   missing because libraries and/or header files were not
>   found, or were too old, for these packages:
> 
>   lz4 xxhash

The risk is that some (many ?) would no care at all and continue their builds, 
without then the benefit from these libraries.
The worst risk being to have packages distributed without these features 
enabled.

> FreeBSD 11.4 and 12:
>pkg install cmark liblz4 openssl py37-cmarkgfm py37-xxhash zstd
> FreeBSD 13:
>pkg install cmark cmarkgfm liblz4 openssl py37-xxhash zstd

Not sure you need py37-xxhash, xxhash shold be enough.

Ben


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Rsync 3.2.3pre1 released

[Nelson H. F. Beebe via rsync]

I downloaded rsync-3.2.3pre1.tar.gz this morning and started builds on
a modest set of hosts; due to a machine-room A/C failure, many of my
normal build hosts have been shutdown.

To my surprise, ALL of the builds failed, and examination of the build
logs showed they were all due to missing libraries or header files,
notably for one or more of lz4, openssl, xxhash, and zstd.  Once I
installed those packages, I got successful builds.

Our site has a small number of servers for each O/S + CPU combination,
and hundreds of clients.  Thus, it is important that any new libraries
added to a server to get a build to complete must also be installed on
ALL of the clients, which is a big job.  xxhash in particular is not a
package that we have needed before, so none of the clients has it.
Thus, to get a distributable rsync executable, I need to add various
--disable-XXX options to ensure that server builds do not require
additional software installation on clients.

I also noted that the configure script asks for installation of
cmarkgfm or commonmark, because they are needed for documentation
builds.  In my view, that is wrong: docs in software distributions
should not require any building, and should be ready to install by
simple copy commands as standard manual pages in groff format,
possibly supplemented by .html and .pdf variants.

The configure script bails out when a library is not found, which is
akin to a compiler that quits on the first syntax error.  In both
cases, repeated attempts are necessary, fixing one problem at a time.

I believe that it would be much better to simply disable the code that
needs the missing library or its header files, and keep on running the
configure script, with a prominent final report, something like:

WARNING: configuration is complete, but some features are
missing because libraries and/or header files were not
found, or were too old, for these packages:

lz4 xxhash

We have a large software installation base at my site, with more than
25,000 installed executables in our search paths, and I've never
before had to install cmarkgfm or commonmark, so until my work this
morning, none of our servers had those packages.

With these package installations, I got successful builds and "make
check" runs on almost all of the systems I worked on today, except as
noted below.

After this posting, I continue to do builds on other systems, so I
might have further reports.  Of the systems below, the O/S versions
are mostly recent ones on x86_64, although in many cases, I have
several older O/S versions, and machines with numerous other CPU
types; they all remain to be tested as our test farm is gradually
brought back up, A/C conditions permitting.


CentOS 7:

yum install \
cmark-lib cmark-devel cmark \
lz4 lz4-devel lz4-static\
xxhash xxhash-devel xxhash-libs \
zstd libzstd libzstd-devel  \
openssl-devel openssl-static


CentOS 8 and Oracle 8:

yum install \
cmark-lib cmark-devel cmark \
lz4 lz4-devel   \
xxhash xxhash-devel xxhash-libs \
zstd libzstd libzstd-devel  \
openssl-devel



Fedora 33:

dnf install \
cmark-lib cmark-devel cmark \
lz4-devel lz4-libs lz4-static   \
xxhash xxhash-devel xxhash-libs \
zstd libzstd libzstd-devel  \
openssl-devel openssl-static



FreeBSD 10.4:

pkg install \
cmark liblz4 openssl py36-gfm py36-xxhash xxhash zstd

configure.sh: error: Failed to find ZSTD_minCLevel function in zstd 
lib.
Use --disable-zstd to continue without zstd compression.

./configure --disable-zstd --prefix=$prefix && gmake all check



FreeBSD 11.4 and 12:

pkg install \
cmark liblz4 openssl py37-cmarkgfm py37-xxhash zstd



FreeBSD 13:
pkg install \
cmark cmarkgfm liblz4 openssl py37-xxhash zstd



ClonOS 19.09:
pkg install \
cmark liblz4 openssl py37-cmarkgfm py37-xxhash zstd



NetBSD 9.0:
pkg_add