[Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12806

--- Comment #7 from Aryo Da  ---
I think this a severe bug for all backup use cases of rsync that take a full
snapshot with permissions (--perms) by creating hardlinks to unchanged files +
copies of changed files (--link-dest):

-> Whenever an old snapshot is deleted with `rsync --delete` the permissions of
the hardlinked files change and trigger another copy of the files in a new
snapshot (= waste of storage space and backup run-time).

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12806

--- Comment #6 from Aryo Da  ---
Created attachment 18117
  --> https://bugzilla.samba.org/attachment.cgi?id=18117=edit
MRE (minimal reproducible example) as bash script to reproduce the bug

Rename to "setup.sh" and make it executable...

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12806

--- Comment #5 from Heinz-Willi Wyes  ---
Lars Ellenberg provided a workaround for the behaviour. Using

rsync -d --delete --super `mktemp -d`/ snapshots/2017-05-26-15-00-53/

plays the trick.

Not sure whether the --super option has implications not suitable for other
scenarios. For me it works just fine. Nevertheless, there should be a patch for
rsync in order to overcome the questionable behaviour.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: [Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.

[Lars Ellenberg via rsync]

On Thu, Jun 08, 2017 at 07:09:24AM +, just subscribed for rsync-qa from 
bugzilla via rsync wrote:
> https://bugzilla.samba.org/show_bug.cgi?id=12806
> 
> --- Comment #4 from Heinz-Willi Wyes  ---
> I got a personal message from Lars Ellenberg that went also to
> samba-b...@samba.org and rsync...@samba.org. He wrote:
> 
> --
> Calling chmod only on (optimization: non-empty) directories would fix this.
> I don't need to chmod a *file* before unlinking it, I just need write
> permission on the directory it is located in.
> 
> (Now you have to convince the "appliance" to use a patched rsync ...)

Note that as long as you have only files (not directories) without write
permissions, you can potentially work around this using "--super" (or 
"--fake-super"),
because in that case rsync sets "am_root" to 2 (or -1),
"!am_root" becomes false in the various conditions,
and the do_chmod() will not even be called.

Not sure though what other implications --[fake-]super will have for your 
usecase.

Lars Ellenberg


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12806

--- Comment #4 from Heinz-Willi Wyes  ---
I got a personal message from Lars Ellenberg that went also to
samba-b...@samba.org and rsync...@samba.org. He wrote:

--
Calling chmod only on (optimization: non-empty) directories would fix this.
I don't need to chmod a *file* before unlinking it, I just need write
permission on the directory it is located in.

(Now you have to convince the "appliance" to use a patched rsync ...)

Cheers,

Lars Ellenberg


diff --git a/delete.c b/delete.c
index 88e4230..223b6d2 100644
--- a/delete.c
+++ b/delete.c
@@ -97,10 +97,10 @@ static enum delret delete_dir_contents(char *fname, uint16
flags)
 }

 strlcpy(p, fp->basename, remainder);
-if (!(fp->mode & S_IWUSR) && !am_root && fp->flags & FLAG_OWNED_BY_US)
-do_chmod(fname, fp->mode | S_IWUSR);
 /* Save stack by recursing to ourself directly. */
 if (S_ISDIR(fp->mode)) {
+if (!(fp->mode & S_IWUSR) && !am_root && fp->flags &
FLAG_OWNED_BY_US)
+do_chmod(fname, fp->mode | S_IWUSR);
 if (delete_dir_contents(fname, flags | DEL_RECURSE) != DR_SUCCESS)
 ret = DR_NOT_EMPTY;
 }
@@ -138,14 +138,13 @@ enum delret delete_item(char *fbuf, uint16 mode, uint16
flags)
 fbuf, (int)mode, (int)flags);
 }

-if (flags & DEL_NO_UID_WRITE)
-do_chmod(fbuf, mode | S_IWUSR);
-
 if (S_ISDIR(mode) && !(flags & DEL_DIR_IS_EMPTY)) {
 /* This only happens on the first call to delete_item() since
  * delete_dir_contents() always calls us w/DEL_DIR_IS_EMPTY. */
 ignore_perishable = 1;
 /* If DEL_RECURSE is not set, this just reports emptiness. */
+if (!(mode & S_IWUSR) && !am_root && flags & DEL_NO_UID_WRITE && flags
& DEL_RECURSE)
+do_chmod(fbuf, mode | S_IWUSR);
 ret = delete_dir_contents(fbuf, flags);
 ignore_perishable = 0;
 if (ret == DR_NOT_EMPTY || ret == DR_AT_LIMIT)
-

As far as I understand, there actually *is* a design flaw in the rsync
implementation that causes the behavior I described in my original post.

I suggested the patch to my provider. They replied that they rather wait for a
new release of rsync with that patch officially applied.

Now I wonder what the status is. Will there be a patched version of rsync?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: [Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.

[Lars Ellenberg via rsync]

Calling chmod only on (optimization: non-empty) directories would fix this.
I don't need to chmod a *file* before unlinking it,
I just need write permission on the directory it is located in.

(Now you have to convince the "appliance" to use a patched rsync ...)

Cheers,

Lars Ellenberg


diff --git a/delete.c b/delete.c
index 88e4230..223b6d2 100644
--- a/delete.c
+++ b/delete.c
@@ -97,10 +97,10 @@ static enum delret delete_dir_contents(char *fname, uint16 
flags)
}
 
strlcpy(p, fp->basename, remainder);
-   if (!(fp->mode & S_IWUSR) && !am_root && fp->flags & 
FLAG_OWNED_BY_US)
-   do_chmod(fname, fp->mode | S_IWUSR);
/* Save stack by recursing to ourself directly. */
if (S_ISDIR(fp->mode)) {
+   if (!(fp->mode & S_IWUSR) && !am_root && fp->flags & 
FLAG_OWNED_BY_US)
+   do_chmod(fname, fp->mode | S_IWUSR);
if (delete_dir_contents(fname, flags | DEL_RECURSE) != 
DR_SUCCESS)
ret = DR_NOT_EMPTY;
}
@@ -138,14 +138,13 @@ enum delret delete_item(char *fbuf, uint16 mode, uint16 
flags)
fbuf, (int)mode, (int)flags);
}
 
-   if (flags & DEL_NO_UID_WRITE)
-   do_chmod(fbuf, mode | S_IWUSR);
-
if (S_ISDIR(mode) && !(flags & DEL_DIR_IS_EMPTY)) {
/* This only happens on the first call to delete_item() since
 * delete_dir_contents() always calls us w/DEL_DIR_IS_EMPTY. */
ignore_perishable = 1;
/* If DEL_RECURSE is not set, this just reports emptiness. */
+   if (!(mode & S_IWUSR) && !am_root && flags & DEL_NO_UID_WRITE 
&& flags & DEL_RECURSE)
+   do_chmod(fbuf, mode | S_IWUSR);
ret = delete_dir_contents(fbuf, flags);
ignore_perishable = 0;
if (ret == DR_NOT_EMPTY || ret == DR_AT_LIMIT)

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12806

--- Comment #3 from Heinz-Willi Wyes  ---
(In reply to Kevin Korb from comment #2)
I'd like to, but I can't. I set up a local scenario in order to make the
problem most easily reproducible. But the real use case involves rsync in a
cloud backup application where I get no acces to the cloud provider's shell.
Using any other possible protocol so as e.g. ftp or cifs is practically not
possible. Each file operation with these protocols requires a roundtrip to my
local server. Such an attempt lasts for hours considering that there are some
100.000 files per snapshot. rsync does this in less than two minutes - but
resets the file permissions.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12806

--- Comment #2 from Kevin Korb  ---
Just use rm -rf to delete a backup.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12806

--- Comment #1 from Heinz-Willi Wyes  ---
Addendum.
Thought that the --inplace option could play the trick. I just employed the
following:

rsync -r --delete --inplace `mktemp -d`/ snapshots/2017-05-28-08-10-11/

But this does not help either. Permission bits get reset to 644 anyway.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html