Re: --copy-unsafe-links unexpected behavior with unsafe symlinks
Read the section on --copy-unsafe-symlinks and make the change I recommended. That's all there is to it. When a dir name is in the transfer, it is not the top of the transfer tree. It's parent is. This is pretty standard rsync stuff, where trailing slashes are very important to rsync. ..wayne.. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --copy-unsafe-links unexpected behavior with unsafe symlinks
Hello Wayne
Thank you very much for trying to help,
I have the following three observations:
1) As recommended, I have searched the manual of rsync in order to try
to find your interpretation about what is going on but the only relevant
fact that I was able to find was how rsync defines unsafe links:
Symbolic links are considered unsafe if they are absolute
symlinks (start with /), empty, or if they contain enough ".."
components to ascend from the directory being copied.
In my case (I copy and paste below my original example for commodity)
TST/
+-- outside1
| |
| +-- outside1-file.txt
|
+-- outside2-file.txt
|
+-- tree
+-- inside.txt
|
+-- lnk-ans.txt -> ../../ans.txt
|
+-- o1-file.txt -> ../outside1/outside1-file.txt
|
+-- o1-folder -> ../outside1
|
+-- o2-file.txt -> ../outside2-file.txt
when I run the commands
$ cd TST
$ rsync --copy-unsafe-links -avz --delete tree XXX
In my opinion
tree/o1-file.txt
tree/o1-folder
tree/o2-file.txt
contain enough .. to ascend from the directory being copied (which
is "tree")
2) I've also taken into account your observation that
In this case, the "tree" directory is a part of the transfer
inside the top-of-transfer dir (the current directory)
which seems unusual to me (because in my case the directory being
copied is "tree" and not the current directory (am I missing something?))
and started the command rsync command from a another folder:
$ cd ~/dev
$ rsync --copy-unsafe-links -avz --delete ~/tmp/TST/tree ~/tmp/XXX
The result is the same
~/tmp/TST/tree/lnk-ans.txt
is copied as a file
And
~/tmp/TST/tree/o1-file.txt
~/tmp/TST/tree/o1-folder
~/tmp/TST/tree/o2-file.txt
are copied as broken symblinks.
3) I don't think that the intended behavior of rsync is to consider safe
the symlinks that point to files and folders that are located just above
the directory being copied ("tree" in this case).
If this would be the intended behavior, then there will not be possibile
to specify that you want and entire directory to be copied (with copies
of the files pointed by unsafe links), because independently of what you
specify in the command line as the source tree to be copied there will
always be the posibility that the tree contains some symlink that points
to a file that is outside the subtree but located under the parent that
contains the tree and this file will not be copied.
Best regards
Cristian
On Sun, Nov 28, 2021 at 12:53 PM Cristian via rsync
wrote:
$ rsync --copy-unsafe-links -avz --delete tree XXX
In this case, the "tree" directory is a part of the transfer inside the
top-of-transfer dir (the current directory). Thus any symlinks that don't
try to escape the current directory are considered to be OK. You should
instead suffix "tree" with a slash and add "tree" to your destination path:
$ rsync --copy-unsafe-links -aivz --delete tree/ host:/old/path/tree/
This is discussed in the man page
(search for "unsafe").
..wayne..
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[SCM] The rsync repository. - branch master updated
The branch, master has been updated via c9832790 Improve rrsync usage and some more NEWS tweaks. from ee9199b5 More NEWS improvements. https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log - commit c98327902086cb6e7f525aa263169344c6306771 Author: Wayne Davison Date: Mon Jan 3 00:37:57 2022 -0800 Improve rrsync usage and some more NEWS tweaks. --- Summary of changes: NEWS.md | 49 + support/rrsync | 2 +- support/rrsync.1.md | 7 --- 3 files changed, 30 insertions(+), 28 deletions(-) Changeset truncated at 500 lines: diff --git a/NEWS.md b/NEWS.md index 339b31ee..dc523136 100644 --- a/NEWS.md +++ b/NEWS.md @@ -110,7 +110,7 @@ - More ASM optimizations from Shark64. - - Transformed rrsync (in support dir) into a python script with improvements: + - Transformed rrsync into a python script with improvements: - Security has been beefed up. - The known rsync options were updated to include recent additions. - Make rrsync reject `-L`, `-K`, & `-k` by default to make it harder to @@ -123,19 +123,19 @@ options on the server side. - The log format has been tweaked slightly to add seconds to the timestamp and to output the command executed as a tuple (making the args clearer). - - An rrsync.1 man page was added. + - An rrsync.1 man page was added (in the support dir with rrsync). - - Added options to the lsh script (in the support dir) to facilitate rrsync - testing. + - Added options to the lsh script to facilitate rrsync testing. (See the + support dir.) - - Transformed the atomic-rsync script (in the support dir) into a python - script and added the ability to ignore one or more non-zero exit codes. - By default, it now ignores code 24 (file vanished). + - Transformed the atomic-rsync script into a python script and added the + ability to ignore one or more non-zero exit codes. By default, it now + ignores code 24, the file-vanished exit code. (See the support dir.) - - Transformed the munge-symlinks script (in the support dir) into python. + - Transformed the munge-symlinks script into python. (See the support dir.) - - Improved the rsync-no-vanished script (in the support dir) to not join - stdout & stderr together. + - Improved the rsync-no-vanished script to not join stdout & stderr together. + (See the support dir.) - Work around a glibc bug where lchmod() breaks in a chroot w/o /proc mounted. @@ -877,7 +877,7 @@ non-bundled zlib. See the `--new-compress` and `--old-compress` options in the manpage. - - Added the rsync-no-vanished shell script (in the support dir). + - Added the rsync-no-vanished shell script. (See the support dir.) - Made configure more prominently mention when we failed to find yodl (in case the user wants to be able to generate manpages from `*.yo` files). @@ -1210,7 +1210,7 @@ - Fix some issues with the post-processing of the man pages. - - Fixed the user home-dir handling in the lsh script (in the support dir). + - Fixed the user home-dir handling in the lsh script. (See the support dir.) - Some minor manpage improvements. @@ -1328,10 +1328,10 @@ reject an attempt to supply one (can configure `--with-included-popt` if your system's popt library doesn't yet have this fix). - - A couple minor option tweaks to the rrsync script (in the support dir), and - also some regex changes that make vim highlighting happier. + - A couple minor option tweaks to the rrsync script, and also some regex + changes that make vim highlighting happier. (See the support dir.) - - Fixed some issues in the mnt-excl script (in the support dir). + - Fixed some issues in the mnt-excl script. (See the support dir.) - Various manpage improvements. @@ -1541,9 +1541,9 @@ ### ENHANCEMENTS: - - Made the atomic-rsync script (in the support dir) able to perform a fully - atomic update of the copied hierarchy when the destination is setup using a - particular symlink idiom. + - Made the atomic-rsync script able to perform a fully atomic update of the + copied hierarchy when the destination is setup using a particular symlink + idiom. (See the support dir.) -- @@ -1811,8 +1811,9 @@ - Fixed the inclusion of per-dir merge files from implied dirs. - - Fixed the rrsync script (in the support dir) to work with the latest options - that rsync sends (including its flag-specifying use of `-e` to the server). + - Fixed the rrsync script to work with the latest options that rsync sends, + including its flag-specifying use of `-e` to the server. (See the support + dir.) ### ENHANCEMENTS: @@ -2816,10 +2817,10 @@ - Made the `max verbosity` setting in the rsyncd.conf file
[SCM] The rsync repository. - annotated tag v3.2.4pre1 created
The annotated tag, v3.2.4pre1 has been created at ad07d8c1df543b9d84d8be36ec97e0f587a0ddaa (tag) tagging 3e44bbd3132c22f7ce1ae48c0ce262f237473038 (commit) replaces v3.2.3 tagged by Wayne Davison on Sun Jan 2 15:57:47 2022 -0800 - Log - Version 3.2.4pre1. -BEGIN PGP SIGNATURE- iG8EABECAC8WIQQASMiwJtTJbw5YnC9shZ+xS5aoxQUCYdI7+xEcd2F5bmVkQHNh bWJhLm9yZwAKCRBshZ+xS5aoxXA6AJ9nf0EtqLA8EJ1HPHxqrPBtJ+qg4QCg86um /Yo4eD8iSntEstiYL8hglxc= =3KpW -END PGP SIGNATURE- Achim Leitner (1): Linux: Handle protected_regular in inplace writes (#241) Andrew Aladjev (1): improved cross compilation detection (#252) Bart S (1): Updated GLIBC check in configure.ac (#175) Fabian H (2): add missing - in certopt (#210) add ssl/tls key option (#216) Issam Maghni (1): shell: test -a|o is not POSIX (#250) James Cook (1): Fix spelling error in man page. (#124) JindÅich MakoviÄka (1): Reduce memory usage (#235) Jonathan Davies (1): configure.ac: Test IPv6 compatibility instead of relying on library probes (#206) Marco Nenciarini (1): Fix rrsync directory normalization (#268) Matt McCutchen (1): rsync-ssl: Verify the hostname in the certificate when using openssl. Natanael Copa (1): Add test and fix regression for --delay-updates (#192) (#204) Rodrigo Osorio (1): Time-limit options are not being checked enough (#179) Shark64 (2): optimize avx2 code (#102) x86-64 AVX2 assemby implemenation of get_checksum1() (#174) TomasKorbar (1): Eventually add write permission when setting extended attributes (#212) Wayne Davison (129): Simplify where version.h is included. Add some comments about protocol versions. Simplify the compat logic for local_server Setup for 3.2.4dev. Create SECURITY.md Set CXX_OK=no when cross compiling. Allow git-version.h to be provided for the build Leave git-version.h out of GENFILES so it doesn't go in a release tar. Add git-version.h to "gen" target. Try cygwin build again. [buildall] Improve a sentence about --stderr=all. Another cygwin build attempt. [buildall] Try using the Windows version of curl. [buildall] Use new xxhash lib in cygwin build. [buildall] Handle a --mkpath failure Update the NEWS. Avoid spurious "is newer" messages with `--update`. Put CAN_HARDLINK_SYMLINK info into --version output. Check status of tests that pipe rsync's output & simplify output diffing. Avoid output variance in protocol 29. Improve the docs for `--archive`. Skip an append if sender's file gets shorter. A couple whitespace tweaks. More NEWS updates. Various file comparison improvements Make the extra info on the "exists" messages optional. Change from $build_cpu to $host_cpu as edo1 suggested. Change `--info=skip2` messages & add info on attr changes. Expand the max name_num_item list size. Make gcc die on init overflow of an array. Handle early gcc versions that don't understand -Wno-pedantic. Avoid some pedantic errors & old warnings. Use a better -Wno-pedantic heuristic. Silence a few more warnings. Update CI builds to new path-setting idiom. Use the right powershell env syntax. [buildall] Force HAVE_LCHMOD off for Linux (for now). Tweak cmd_txt routines in the packaging scripts. Include stdlib.h for exit() and consult HAVE_* macros more. Work around glibc's lchmod() issue a better way. Fix a typo. Cygwin python3 is now 3.8 w/o commonmark lib. [buildall] Add `--with-nobody-user=FOO` configure option. Put 0 in parens to silence an Xcode warning. A couple "make" tweaks. Check for netinet/ip.h after including netinet/in.h. Add a link to the man page to the README. Some pip-releated tweaking. Add optional netgroup.h include for NetBSD hosts. Stop checking for gmake in build scripts A few Cygwin build tweaks. [buildall] Rename util.c to util1.c Allow the generator's msg iobuf to get bigger too. See if explicitly installing m4 gets FreeBSD CI a newer gm4 version. Update FreeBSD CI to 12.2. Install bash on FreeBSD CI. Switch to using image_family for Cirrus CI. Fix git-set-file-times's handling of staged changed files. Fix plural of --group option. Unset DISPLAY in environment. Fix compiling without ftruncate. Need to also check stdout_format_has_i in some INFO-NAME checks. Set whole_file = 0 when whole_file < 0. Fixes issue 114. Make whole-line comments clearer. Support --crtimes on Cygwin. Update the options in rrsync. Add missing INET6 check. Make use of a git gist instead of the samba website. Add gist
[SCM] The rsync repository. - branch master updated
The branch, master has been updated via ee9199b5 More NEWS improvements. via f1a6998d Only send the `--no-W` kluge to a receiver. from 3e44bbd3 Preparing for release of 3.2.4pre1 https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log - commit ee9199b54274aed05776f55806dc18b3e2f20a32 Author: Wayne Davison Date: Mon Jan 3 00:02:28 2022 -0800 More NEWS improvements. commit f1a6998df29f0f41f882edf11cb957df78aeb267 Author: Wayne Davison Date: Sun Jan 2 23:51:04 2022 -0800 Only send the `--no-W` kluge to a receiver. --- Summary of changes: NEWS.md | 113 -- options.c | 2 +- 2 files changed, 66 insertions(+), 49 deletions(-) Changeset truncated at 500 lines: diff --git a/NEWS.md b/NEWS.md index 05215c08..339b31ee 100644 --- a/NEWS.md +++ b/NEWS.md @@ -30,11 +30,14 @@ ### BUG FIXES: - - Fixed a bug with `--inplace` + `--sparse` where the destination file could - get reconstructed with bogus data. This bug can be worked-around in older - rsync versions by also specifying `--no-W -M--no-W`. When running 3.2.4 or - newer for your copy, rsync now sends `--no-W` to the remote rsync in such a - scenario (just in case the remote rsync is a version with this bug). + - Fixed a bug with `--inplace` + `--sparse` (and a lack of `--whole-file`) + where the destination file could get reconstructed with bogus data. Since + the bug can also be avoided by using (the seemingly redundant) `--no-W` on + the receiving side, the latest rsync will now send `--no-W` to a remote + receiver when this option combination occurs. If your client rsync is not + new enough to do this for you (or if you're just paranoid), you can manually + specify `--no-W -M--no-W` (when not using `--whole-file`) to make sure the + bug is avoided. - Fixed a bug with `--mkpath` if a single-file copy specifies an existing destination dir with a non-existing destination filename. @@ -47,9 +50,9 @@ or it is skipped. Fixes a crash that could occur when the size changes to 0 in the middle of the send negotiations. - - When dealing with a special file in an alt-dest hierarchy, rsync now checks - the non-permissions mode bits to ensure that the 2 special files are really - the same. + - When dealing with special files (see `--specials`) in an alt-dest hierarchy, + rsync now checks the non-permission mode bits to ensure that the 2 special + files are really the same before hard-linking them together. - Fixed a bug where `--delay-updates` with stale partial data could cause a file to fail to update. @@ -58,12 +61,16 @@ should only have been output given `--verbose` or `--itemize-changes`. - Avoid a weird failure if you run a local copy with a (useless) `--rsh` - option that contains a `V`. + option that contains a `V` in the command. - Fixed a long-standing compression bug where the compression level of the first file transferred affected the level for all future files. Also, the - per-file compression skipping has apparently not worked in a very long time - (I checked back to 2.6.4), so it is now documented as being ineffective. + per-file compression skipping has apparently never worked, so it is now + documented as being ineffective. + + - Improved how the `--stop-at`, `--stop-after`, and `--time-limit` options + check to see if the allowed time is over, which should make rsync exit more + consistently. ### ENHANCEMENTS: @@ -83,7 +90,8 @@ - The rsync daemon can now handle a client address with an implied "%scope" suffix. - - Added support for `--atimes` on macOS and fixed using using it without `-t`. + - Added support for `--atimes` on macOS and fixed a bug where it wouldn't work + without `--times`. - Rsync can now update the xattrs on a read-only file when your user can temporarily add user-write permission to the file. (It always worked for a @@ -92,8 +100,8 @@ - Rsync can now work around an `--inplace` update of a file that is being refused due to the Linux fs.protected_regular sysctl setting. - - When `--chown`, `--usermap`, or `--groupmap` is used, rsync now implies - the appropriate `--owner` and/or `--group` option. + - When `--chown`, `--usermap`, or `--groupmap` is specified, rsync now makes + sure that the appropriate `--owner` and/or `--group` option is enabled. - Added the `--info=NONREG` setting to control if rsync should warn about non-regular files in the transfer. This is enabled by default (keeping the @@ -102,7 +110,7 @@ - More ASM optimizations from Shark64. - - Transformed support/rrsync into a python script with improvements: + - Transformed rrsync (in support dir) into a python script with improvements: - Security has been beefed up. - The known
