The branch, master has been updated via 9a06b2ed Preparing for release of 3.3.0pre1 [buildall] via 273dced2 Update the NEWS. via b6e23219 Mention that `--crtimes` support is spotty. via fe95a936 Fix issue with trailing --sparse --inplace blocks. Fixes #450. via 6ae7f408 Add --force-link-text to md-convert. via 0f599d36 Fix overflow of sum2 buffer for sha1 rolling checksums. Fixed #353. via c3d3b49d Make use of .UR & .UE for links. via c69dc7a5 Tweak shell protection news to mention a few more characters. via 2c82006b add rrsync option to enforce `--ignore-existing` (#461) via 0698ea9a Fix flist string comparison issue in tr_TR.utf-8 locale. via 90df93e4 Don't call memcmp() on an empty lastdir. from 5c93dedf Add backtick to SHELL_CHARS.
https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 9a06b2edb0ea1a226bcc642682c07bacd2ea47d3 Author: Wayne Davison <wa...@opencoder.net> Date: Sat Apr 29 09:01:43 2023 -0700 Preparing for release of 3.3.0pre1 [buildall] commit 273dced284e2bef03ccdf626e44aec4ce6bab6fe Author: Wayne Davison <wa...@opencoder.net> Date: Sat Apr 29 08:52:42 2023 -0700 Update the NEWS. commit b6e2321973e163f7c7f22fad4db5fd1bc3638710 Author: Wayne Davison <wa...@opencoder.net> Date: Sat Apr 29 08:20:30 2023 -0700 Mention that `--crtimes` support is spotty. commit fe95a9369ac7a1d8d47b036094f91946cacccf53 Author: Wayne Davison <wa...@opencoder.net> Date: Sat Apr 29 07:45:52 2023 -0700 Fix issue with trailing --sparse --inplace blocks. Fixes #450. commit 6ae7f4085a5012038c29ae1c70f626cae7714b69 Author: Wayne Davison <wa...@opencoder.net> Date: Sun Apr 23 08:26:32 2023 -0700 Add --force-link-text to md-convert. commit 0f599d3641bd197ff01f5182fad33169aa46a055 Author: Wayne Davison <wa...@opencoder.net> Date: Sat Apr 22 08:49:48 2023 -0700 Fix overflow of sum2 buffer for sha1 rolling checksums. Fixed #353. commit c3d3b49d722439f18be24a0414aa49b4b035841f Author: Wayne Davison <wa...@opencoder.net> Date: Sat Apr 22 08:37:01 2023 -0700 Make use of .UR & .UE for links. commit c69dc7a5ab473bb52a575b5803026c2694761084 Author: Wayne Davison <wa...@opencoder.net> Date: Thu Mar 30 12:56:33 2023 -0700 Tweak shell protection news to mention a few more characters. commit 2c82006b1fcddcdfb12aceb838f4dff7531b6a7c Author: dogvisor <13217529+dogvi...@users.noreply.github.com> Date: Thu Mar 30 21:55:56 2023 +0200 add rrsync option to enforce `--ignore-existing` (#461) The `-no-overwrite` rrsync option disallows the updating of existing files for incoming rrsync copies. commit 0698ea9aeb4044d615dc0ff2c6da24cea1bcc98e Author: Wayne Davison <wa...@opencoder.net> Date: Sun Feb 5 19:46:45 2023 -0800 Fix flist string comparison issue in tr_TR.utf-8 locale. commit 90df93e446f9ebbfd4ce97d6755c5fe1f45f9fd0 Author: Wayne Davison <wa...@opencoder.net> Date: Sun Jan 8 21:35:39 2023 -0800 Don't call memcmp() on an empty lastdir. ----------------------------------------------------------------------- Summary of changes: NEWS.md | 58 +++++++++++++++++++++++++++++++++++++++++++++--- checksum.c | 3 ++- fileio.c | 42 +++++++++++++++++++---------------- flist.c | 4 ++-- generator.c | 4 ++-- latest-year.h | 2 +- match.c | 4 ++-- maybe-make-man | 9 +++++++- md-convert | 32 ++++++++++++++++++++++++-- options.c | 4 ++-- packaging/lsb/rsync.spec | 12 +++++----- receiver.c | 4 ++-- rsync.1.md | 4 +++- support/rrsync | 4 ++++ support/rrsync.1.md | 7 +++++- version.h | 2 +- 16 files changed, 149 insertions(+), 46 deletions(-) Changeset truncated at 500 lines: diff --git a/NEWS.md b/NEWS.md index 555d16f5..1d3a158b 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,53 @@ +# NEWS for rsync 3.3.0 (UNRELEASED) + +## Changes in this version: + +### BUG FIXES: + +- Fixed a bug with `--sparse --inplace` where a trailing gap in the source + file would not clear out the trailing data in the destination file. + +- Fixed an buffer overflow in the checksum2 code if SHA1 is being used for + the checksum2 algorithm. + +- Fixed a string-comparison issue in the internal file-list code that affected + tr_TR.utf-8. + +- Add a backtick to the list of characters that the filename quoting needs to + escape using backslashes. + +- Make sure that a local transfer marks the sender side as trusted. + +- Change the argv handling to work with a newer popt library -- one that likes + to free more data than it used to. + +- Rsync now calls `OpenSSL_add_all_algorithms()` when compiled against an older + openssl library. + +- Fixed a problem in the daemon auth for older protocols (29 and before) if the + openssl library is being used to compute md4 checksums. + +- Fixed `rsync -VV` on Cygwin -- it needed a flush of stdout. + +### ENHANCEMENTS: + +- Enhanced rrsync with the `-no-overwrite` option that allows you to ensure + that existing files on your restricted but writable directory can't be + modified. + +- Enhanced the manpages to mark links with .UR & .UE. If your nroff doesn't + support these idioms, touch the file `.md2man-force` in the source directory + so that `md-convert` gets called with the `--force-link-text` option, and + that should ensure that your manpages are still readable even with the + ignored markup. + +- Some manpage improvements on the handling of [global] modules. + +- Changed the mapfrom & mapto perl scripts (in the support dir) into a single + python script named idmap. Converted a couple more perl scripts into python. + +------------------------------------------------------------------------------ + # NEWS for rsync 3.2.7 (20 Oct 2022) ## Changes in this version: @@ -217,9 +267,10 @@ - A new form of arg protection was added that works similarly to the older `--protect-args` ([`-s`](rsync.1#opt)) option but in a way that avoids breaking things like rrsync (the restricted rsync script): rsync now uses - backslash escaping for sending "shell-active" characters to the remote - shell. This includes spaces, so fetching a remote file via a simple quoted - filename value now works by default without any extra quoting: + backslash escaping for sending "shell-active" characters to the remote shell + (such as `$(){}<>#&` and others). This includes spaces, so fetching a remote + file via a quoted filename value now works by default without any extra + quoting: ```shell rsync -aiv host:'a simple file.pdf' . @@ -4696,6 +4747,7 @@ | RELEASE DATE | VER. | DATE OF COMMIT\* | PROTOCOL | |--------------|--------|------------------|-------------| +| ?? May 2023 | 3.3.0 | | 31 | | 20 Oct 2022 | 3.2.7 | | 31 | | 09 Sep 2022 | 3.2.6 | | 31 | | 14 Aug 2022 | 3.2.5 | | 31 | diff --git a/checksum.c b/checksum.c index c6007f36..cb21882c 100644 --- a/checksum.c +++ b/checksum.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2004-2022 Wayne Davison + * Copyright (C) 2004-2023 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -300,6 +300,7 @@ uint32 get_checksum1(char *buf1, int32 len) } #endif +/* The "sum" buffer must be at least MAX_DIGEST_LEN bytes! */ void get_checksum2(char *buf, int32 len, char *sum) { #ifdef USE_OPENSSL diff --git a/fileio.c b/fileio.c index f80af19e..69c9a7b4 100644 --- a/fileio.c +++ b/fileio.c @@ -3,7 +3,7 @@ * * Copyright (C) 1998 Andrew Tridgell * Copyright (C) 2002 Martin Pool - * Copyright (C) 2004-2020 Wayne Davison + * Copyright (C) 2004-2023 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -40,30 +40,34 @@ OFF_T preallocated_len = 0; static OFF_T sparse_seek = 0; static OFF_T sparse_past_write = 0; -int sparse_end(int f, OFF_T size) +int sparse_end(int f, OFF_T size, int updating_basis_or_equiv) { - int ret; - - sparse_past_write = 0; - - if (!sparse_seek) - return 0; + int ret = 0; + if (updating_basis_or_equiv) { + if (sparse_seek && do_punch_hole(f, sparse_past_write, sparse_seek) < 0) + ret = -1; +#ifdef HAVE_FTRUNCATE /* A compilation formality -- in-place requires ftruncate() */ + else /* Just in case the original file was longer */ + ret = do_ftruncate(f, size); +#endif + } else if (sparse_seek) { #ifdef HAVE_FTRUNCATE - ret = do_ftruncate(f, size); + ret = do_ftruncate(f, size); #else - if (do_lseek(f, sparse_seek-1, SEEK_CUR) != size-1) - ret = -1; - else { - do { - ret = write(f, "", 1); - } while (ret < 0 && errno == EINTR); - - ret = ret <= 0 ? -1 : 0; - } + if (do_lseek(f, sparse_seek-1, SEEK_CUR) != size-1) + ret = -1; + else { + do { + ret = write(f, "", 1); + } while (ret < 0 && errno == EINTR); + + ret = ret <= 0 ? -1 : 0; + } #endif + } - sparse_seek = 0; + sparse_past_write = sparse_seek = 0; return ret; } diff --git a/flist.c b/flist.c index 65b459b1..311bbcf1 100644 --- a/flist.c +++ b/flist.c @@ -4,7 +4,7 @@ * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2001, 2002 Martin Pool <m...@samba.org> - * Copyright (C) 2002-2022 Wayne Davison + * Copyright (C) 2002-2023 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -2367,7 +2367,7 @@ struct file_list *send_file_list(int f, int argc, char *argv[]) } dirlen = dir ? strlen(dir) : 0; - if (dirlen != lastdir_len || memcmp(lastdir, dir, dirlen) != 0) { + if (dirlen != lastdir_len || (dirlen && memcmp(lastdir, dir, dirlen) != 0)) { if (!change_pathname(NULL, dir, -dirlen)) goto bad_path; lastdir = pathname; diff --git a/generator.c b/generator.c index 21c4a595..110db28f 100644 --- a/generator.c +++ b/generator.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2002 Martin Pool <m...@samba.org> - * Copyright (C) 2003-2022 Wayne Davison + * Copyright (C) 2003-2023 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -783,7 +783,7 @@ static int generate_and_send_sums(int fd, OFF_T len, int f_out, int f_copy) for (i = 0; i < sum.count; i++) { int32 n1 = (int32)MIN(len, (OFF_T)sum.blength); char *map = map_ptr(mapbuf, offset, n1); - char sum2[SUM_LENGTH]; + char sum2[MAX_DIGEST_LEN]; uint32 sum1; len -= n1; diff --git a/latest-year.h b/latest-year.h index 37e8efbb..0dcf3464 100644 --- a/latest-year.h +++ b/latest-year.h @@ -1 +1 @@ -#define LATEST_YEAR "2022" +#define LATEST_YEAR "2023" diff --git a/match.c b/match.c index 6243994c..cdb30a15 100644 --- a/match.c +++ b/match.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2003-2022 Wayne Davison + * Copyright (C) 2003-2023 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -142,7 +142,7 @@ static void hash_search(int f,struct sum_struct *s, { OFF_T offset, aligned_offset, end; int32 k, want_i, aligned_i, backup; - char sum2[SUM_LENGTH]; + char sum2[MAX_DIGEST_LEN]; uint32 s1, s2, sum; int more; schar *map; diff --git a/maybe-make-man b/maybe-make-man index 0dc17305..c7af7393 100755 --- a/maybe-make-man +++ b/maybe-make-man @@ -8,6 +8,7 @@ fi inname="$1" srcdir=`dirname "$0"` flagfile="$srcdir/.md2man-works" +force_flagfile="$srcdir/.md2man-force" if [ ! -f "$flagfile" ]; then # We test our smallest manpage just to see if the python setup works. @@ -32,4 +33,10 @@ if [ ! -f "$flagfile" ]; then fi fi -"$srcdir/md-convert" "$srcdir/$inname" +if [ -f "$force_flagfile" ]; then + opt='--force-link-text' +else + opt='' +fi + +"$srcdir/md-convert" $opt "$srcdir/$inname" diff --git a/md-convert b/md-convert index a48689a7..5fd63a76 100755 --- a/md-convert +++ b/md-convert @@ -276,7 +276,10 @@ class TransformHtml(HTMLParser): bad_hashtags = set(), latest_targets = [ ], opt_prefix = 'opt', + a_href = None, + a_href_external = False, a_txt_start = None, + after_a_tag = False, target_suf = '', ) @@ -315,6 +318,13 @@ class TransformHtml(HTMLParser): for bad in st.referenced_hashtags - st.created_hashtags: warn('Unknown hashtag link in', self.fn + ':', '#' + bad) + def handle_UE(self): + st = self.state + if st.txt.startswith(('.', ',', '!', '?', ';', ':')): + st.man_out[-1] = ".UE " + st.txt[0] + "\n" + st.txt = st.txt[1:] + st.after_a_tag = False + def handle_starttag(self, tag, attrs_list): st = self.state if args.debug: @@ -387,13 +397,20 @@ class TransformHtml(HTMLParser): for var, val in attrs_list: if var == 'href': if val.startswith(('https://', 'http://', 'mailto:', 'ftp:')): - pass # nothing to check + if st.after_a_tag: + self.handle_UE() + st.man_out.append(manify(st.txt.strip()) + "\n") + st.man_out.append(".UR " + val + "\n") + st.txt = '' + st.a_href = val + st.a_href_external = True elif '#' in val: pg, tgt = val.split('#', 1) if pg and pg not in VALID_PAGES or '#' in tgt: st.bad_hashtags.add(val) elif tgt in ('', 'opt', 'dopt'): st.a_href = val + st.a_href_external = False elif pg == '': st.referenced_hashtags.add(tgt) if tgt in st.latest_targets: @@ -409,6 +426,8 @@ class TransformHtml(HTMLParser): st = self.state if args.debug: self.output_debug('END', (tag,)) + if st.after_a_tag: + self.handle_UE() if tag in CONSUMES_TXT or st.dt_from == tag: txt = st.txt.strip() st.txt = '' @@ -473,7 +492,15 @@ class TransformHtml(HTMLParser): elif tag == 'hr': return elif tag == 'a': - if st.a_href: + if st.a_href_external: + st.txt = st.txt.strip() + if args.force_link_text or st.a_href != st.txt: + st.man_out.append(manify(st.txt) + "\n") + st.man_out.append(".UE\n") # This might get replaced with a punctuation version in handle_UE() + st.after_a_tag = True + st.a_href_external = False + st.txt = '' + elif st.a_href: atxt = st.txt[st.a_txt_start:] find = 'href="' + st.a_href + '"' for j in range(len(st.html_out)-1, 0, -1): @@ -612,6 +639,7 @@ if __name__ == '__main__': parser = argparse.ArgumentParser(description="Convert markdown into html and (optionally) nroff. Each input filename must have a .md suffix, which is changed to .html for the output filename. If the input filename ends with .num.md (e.g. foo.1.md) then a nroff file is also output with the input filename's .md suffix removed (e.g. foo.1).", add_help=False) parser.add_argument('--test', action='store_true', help="Just test the parsing without outputting any files.") parser.add_argument('--dest', metavar='DIR', help="Create files in DIR instead of the current directory.") + parser.add_argument('--force-link-text', action='store_true', help="Don't remove the link text if it matches the link href. Useful when nroff doesn't understand .UR and .UE.") parser.add_argument('--debug', '-D', action='count', default=0, help='Output copious info on the html parsing. Repeat for even more.') parser.add_argument("--help", "-h", action="help", help="Output this help message and exit.") parser.add_argument("mdfiles", metavar='FILE.md', nargs='+', help="One or more .md files to convert.") diff --git a/options.c b/options.c index c9c82527..93bbe7b0 100644 --- a/options.c +++ b/options.c @@ -3,7 +3,7 @@ * * Copyright (C) 1998-2001 Andrew Tridgell <tri...@samba.org> * Copyright (C) 2000, 2001, 2002 Martin Pool <m...@samba.org> - * Copyright (C) 2002-2022 Wayne Davison + * Copyright (C) 2002-2023 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -2340,7 +2340,7 @@ int parse_arguments(int *argc_p, const char ***argv_p) if (do_progress && !am_server) { if (!log_before_transfer && INFO_EQ(NAME, 0)) parse_output_words(info_words, info_levels, "name", DEFAULT_PRIORITY); - parse_output_words(info_words, info_levels, "flist2,progress", DEFAULT_PRIORITY); + parse_output_words(info_words, info_levels, "FLIST2,PROGRESS", DEFAULT_PRIORITY); } if (dry_run) diff --git a/packaging/lsb/rsync.spec b/packaging/lsb/rsync.spec index f2d7aa44..5df58b96 100644 --- a/packaging/lsb/rsync.spec +++ b/packaging/lsb/rsync.spec @@ -1,9 +1,9 @@ Summary: A fast, versatile, remote (and local) file-copying tool Name: rsync -Version: 3.2.7 -%define fullversion %{version} -Release: 1 -%define srcdir src +Version: 3.3.0 +%define fullversion %{version}pre1 +Release: 0.1.pre1 +%define srcdir src-previews Group: Applications/Internet License: GPL Source0: https://rsync.samba.org/ftp/rsync/%{srcdir}/rsync-%{fullversion}.tar.gz @@ -79,8 +79,8 @@ rm -rf $RPM_BUILD_ROOT %dir /etc/rsync-ssl/certs %changelog -* Thu Oct 20 2022 Wayne Davison <wa...@opencoder.net> -Released 3.2.7. +* Sat Apr 29 2023 Wayne Davison <wa...@opencoder.net> +Released 3.3.0pre1. * Fri Mar 21 2008 Wayne Davison <wa...@opencoder.net> Added installation of /etc/xinetd.d/rsync file and some commented-out diff --git a/receiver.c b/receiver.c index c9d7e01d..6b4b369e 100644 --- a/receiver.c +++ b/receiver.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996-2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2003-2022 Wayne Davison + * Copyright (C) 2003-2023 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -372,7 +372,7 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r, if (fd != -1 && offset > 0) { if (sparse_files > 0) { - if (sparse_end(fd, offset) != 0) + if (sparse_end(fd, offset, updating_basis_or_equiv) != 0) goto report_write_error; } else if (flush_write_file(fd) < 0) { report_write_error: diff --git a/rsync.1.md b/rsync.1.md index ee0a4f39..16dbf20c 100644 --- a/rsync.1.md +++ b/rsync.1.md @@ -1636,7 +1636,9 @@ expand it. 0. `--crtimes`, `-N,` This tells rsync to set the create times (newness) of the destination - files to the same value as the source files. + files to the same value as the source files. Your OS & filesystem must + support the setting of arbitrary creation (birth) times for this option + to be supported. 0. `--omit-dir-times`, `-O` diff --git a/support/rrsync b/support/rrsync index 94c85f59..4b4b87c5 100755 --- a/support/rrsync +++ b/support/rrsync @@ -258,6 +258,9 @@ def main(): if args.munge: rsync_opts.append('--munge-links') + + if args.no_overwrite: + rsync_opts.append('--ignore-existing') if not rsync_args: rsync_args = [ '.' ] @@ -364,6 +367,7 @@ if __name__ == '__main__': arg_parser.add_argument('-munge', action='store_true', help="Enable rsync's --munge-links on the server side.") arg_parser.add_argument('-no-del', action='store_true', help="Disable rsync's --delete* and --remove* options.") arg_parser.add_argument('-no-lock', action='store_true', help="Avoid the single-run (per-user) lock check.") + arg_parser.add_argument('-no-overwrite', action='store_true', help="Prevent overwriting existing files by enforcing --ignore-existing") arg_parser.add_argument('-help', '-h', action='help', help="Output this help message and exit.") arg_parser.add_argument('dir', metavar='DIR', help="The restricted directory to use.") args = arg_parser.parse_args() diff --git a/support/rrsync.1.md b/support/rrsync.1.md index 98f2cab0..24892900 100644 --- a/support/rrsync.1.md +++ b/support/rrsync.1.md @@ -5,7 +5,7 @@ rrsync - a script to setup restricted rsync users via ssh logins ## SYNOPSIS ``` -rrsync [-ro|-rw] [-munge] [-no-del] [-no-lock] DIR +rrsync [-ro|-rw] [-munge] [-no-del] [-no-lock] [-no-overwrite] DIR ``` The single non-option argument specifies the restricted _DIR_ to use. It can be @@ -85,6 +85,11 @@ The remainder of this manpage is dedicated to using the rrsync script. Avoid the single-run (per-user) lock check. Useful with [`-munge`](#opt). +0. `-no-overwrite` + + Enforce `--ignore-existing` on the server. Prevents overwriting existing + files when the server is the receiver. + 0. `-help`, `-h` Output this help message and exit. diff --git a/version.h b/version.h index fdfce4c4..da4bb368 100644 --- a/version.h +++ b/version.h @@ -1,2 +1,2 @@ -#define RSYNC_VERSION "3.2.7" +#define RSYNC_VERSION "3.3.0pre1" #define MAINTAINER_TZ_OFFSET -7.0 -- The rsync repository. _______________________________________________ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs