Re: Rsync to Network Share works from cmd but fails from Scheduled Task
Leon Vanderploeg via rsync wrote: > Greetings, > > I have written a C program that runs on Windows servers and does overhead > and reporting for backup administration. The program calls rsync for the > actual data copy function. It copies data from a remote system to a local > drive and that works fine. However, I have recently made changes to copy > the data to a mapped drive on a NAS device instead of the local drive. > > The changes work fine when run from a command line, but fail when run from > a scheduled task or cron job: > rsync: mkdir "/cygdrive/q/restores" failed: No such file or directory > (2) > rsync error: error in file IO (code 11) at main.c(675) [Receiver=3.1.2] > Note: The folder already exists and permissions are open for Administrator. > > Command line that works is bash command to call the C program under > Administrator login. > Scheduled task built as Administrator with or without highest privileges > fails. > Tried Cron entry built under Administrator and it fails. > > Here is the Rsync command (abbreviated) called by the C program: > rsync -rltgoD --chmod=Du+w,u+r,u+x -e "ssh -p 33222 -i > \"/cygdrive/c/Users/administrator/.ssh/id_rsa\"" --stats --files-from=/... > -vv --exclude-from=/ @remote:/ /cygdrive/q/restores > > Note: q is the mapped drive to the NAS share. > > So, what can be done to track this down? Why would running as a scheduled > task or cron job fail when command line works? I am willing to do any kind > of tracing if someone can provide directions on how to do that. > > Thanks in advance, > Leon Hi, I think the most likely reason is that when you execute the command interactively, you are logged into Windows locally and your cached credentials are used to connect to the mapped drive. But that's not the case when the task is scheduled, and so the mapped drive isn't mapped when the scheduled task is attempted. Similarly, with Cygwin, you can only see /cygdrive/q/... while you are accessing it locally. If you ssh to a Windows host with Cygwin and sshd running, you can't see any mapped drives that are accessible to the local user, even if you ssh in as the same user. You will have to map the drive in the scheduled task before invoking rsync, and then unmap the drive afterwards. I don't know how to do that or handle the credentials that would be needed, but I think that's where to look. cheers, raf -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx permission issue
Hi,
I don't think it's about file flags. macOS has SIP
(System Integrity Protection) which, among other
things, restricts the ability of processes to access
certain parts of the file system.
Usually people ask about how to override this locally
which can be done by disabling SIP altogether, or for
file access only, adding the relevant app in:
System Preferences -> Secutiy & Privacy -> Full Disk Access
But It's not great. I think you can only add "apps" to
the list, not arbitrary executables. I've added
Terminal, iTerm.app and XQuartz.app but it doesn't work
for XQuartz/X11 for some reason, so I need to rsync
some parts from the file system from Terminal/iTerm
instead of from an xterm under XQuartz.
But the OP is asking about the remote end of the
connection. If you are allowed to manipulate the
security settings on the remote end, you could apply
the above ideas there but you'd need to know which app
needs to be added to the list so that rsync can
function. It's not obvious. It's probably the app that
starts sshd. It might be launchd. I can't imagine that
giving that full disk access is a great idea.
It might be possible to get the remote sshd to run the
remote side of rsync via an intermediary app that can
be added to the list of apps with full disk access,
such as Automator.
Maybe you could use (via rsync) ssh's -e / --rsh option
to specify what's needed. See this page for ideas:
How to Give Full Disk Access to a Binary in MacOS Mojave
https://n8henrie.com/2018/11/how-to-give-full-disk-access-to-a-binary-in-macos-mojave/
I haven't read it, just googled "full disk access launchd"
so not sure it'll help.
good luck,
raf
Perry Hutchison via rsync wrote:
> Roland via rsync wrote:
> > does somebody know how to circumvent that "extra file access restriction
> > feature" introduced in osx some time ago ?
>
> It may not be possible.
>
> Based on experience with FreeBSD, from which much of OSX is derived,
> I suspect you may be running into issues with "file flags"; check the
> OSX documentation for the chflags system call. In FreeBSD, I've seen
> that same error message when rsync attempts an operation that would
> violate chflags restrictions.
>
> > i already tried adding rsync binary to programms with "full disk access"
> > privilege ( system-prefs -> security & privacy -> privacy -> full disk
> > access)?? , since running as root is not sufficient - but it does not work.
>
> "full disk access" likely refers to the ability to read the disk
> directly, bypassing the filesystem. It's used by maintenance
> programs like fsck and fsdb, and by some full-disk backup programs,
> but it won't help rsync.
>
> > i want to make sure that every file on osx is getting backup
> >
> > i run rsync on linux to remotely backup osx system (via ssh).
> >
> > regards
> > roland
> >
> > rsync:
> > readlink_stat("/private/var/folders/_p/ky_w_lyj6ps7jcnkjkl5ss0mgn/0/com.apple.routined")
> > failed: Operation not permitted (1)
> > ...
>
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[SCM] The rsync repository. - branch master updated
The branch, master has been updated
via 08650cb1 Add a --copy-as=USER[:GROUP] option
via 24c28cd7 Match the latest git "clean" text.
from c0c6a97c Try to fix the iconv crash in bug 11338.
https://git.samba.org/?p=rsync.git;a=shortlog;h=master
- Log -
commit 08650cb14cd2b07fdebea8ec79cf78e7bfc473a6
Author: Wayne Davison
Date: Sun Mar 29 13:01:13 2020 -0700
Add a --copy-as=USER[:GROUP] option
This can be used by a root-run rsync to try to make reading or writing
files safer in a situation where you can't run the whole rsync command
as a non-root user.
commit 24c28cd715b30ac9e0b669ddadc895e340ed0e9c
Author: Wayne Davison
Date: Tue Mar 19 09:35:59 2019 -0700
Match the latest git "clean" text.
---
Summary of changes:
main.c | 78 +
options.c | 3 ++
packaging/nightly-rsync | 2 +-
rsync.yo| 77
4 files changed, 134 insertions(+), 26 deletions(-)
Changeset truncated at 500 lines:
diff --git a/main.c b/main.c
index 6a6ac559..f7905494 100644
--- a/main.c
+++ b/main.c
@@ -89,6 +89,7 @@ extern char *shell_cmd;
extern char *batch_name;
extern char *password_file;
extern char *backup_dir;
+extern char *copy_as;
extern char curr_dir[MAXPATHLEN];
extern char backup_dir_buf[MAXPATHLEN];
extern char *basis_dir[MAX_BASIS_DIRS+1];
@@ -231,6 +232,74 @@ void read_del_stats(int f)
stats.deleted_files += stats.deleted_specials = read_varint(f);
}
+static void become_copy_as_user()
+{
+ char *gname;
+ uid_t uid;
+ gid_t gid;
+
+ if (!copy_as)
+ return;
+
+ if (DEBUG_GTE(CMD, 2))
+ rprintf(FINFO, "[%s] copy_as=%s\n", who_am_i(), copy_as);
+
+ if ((gname = strchr(copy_as, ':')) != NULL)
+ *gname++ = '\0';
+
+ if (!user_to_uid(copy_as, , True)) {
+ rprintf(FERROR, "Invalid copy-as user: %s\n", copy_as);
+ exit_cleanup(RERR_SYNTAX);
+ }
+
+ if (gname) {
+ if (!group_to_gid(gname, , True)) {
+ rprintf(FERROR, "Invalid copy-as group: %s\n", gname);
+ exit_cleanup(RERR_SYNTAX);
+ }
+ } else {
+ struct passwd *pw;
+ if ((pw = getpwuid(uid)) == NULL) {
+ rsyserr(FERROR, errno, "getpwuid failed");
+ exit_cleanup(RERR_SYNTAX);
+ }
+ gid = pw->pw_gid;
+ }
+
+ if (setgid(gid) < 0) {
+ rsyserr(FERROR, errno, "setgid failed");
+ exit_cleanup(RERR_SYNTAX);
+ }
+#ifdef HAVE_SETGROUPS
+ if (setgroups(1, )) {
+ rsyserr(FERROR, errno, "setgroups failed");
+ exit_cleanup(RERR_SYNTAX);
+ }
+#endif
+#ifdef HAVE_INITGROUPS
+ if (!gname && initgroups(copy_as, gid) < 0) {
+ rsyserr(FERROR, errno, "initgroups failed");
+ exit_cleanup(RERR_SYNTAX);
+ }
+#endif
+
+ if (setuid(uid) < 0
+#ifdef HAVE_SETEUID
+|| seteuid(uid) < 0
+#endif
+ ) {
+ rsyserr(FERROR, errno, "setuid failed");
+ exit_cleanup(RERR_SYNTAX);
+ }
+
+ our_uid = MY_UID();
+ our_gid = MY_GID();
+ am_root = (our_uid == 0);
+
+ if (gname)
+ gname[-1] = ':';
+}
+
/* This function gets called from all 3 processes. We want the client side
* to actually output the text, but the sender is the only process that has
* all the stats we need. So, if we're a client sender, we do the report.
@@ -824,6 +893,8 @@ static void do_server_sender(int f_in, int f_out, int argc,
char *argv[])
exit_cleanup(RERR_SYNTAX);
}
+ become_copy_as_user();
+
dir = argv[0];
if (!relative_paths) {
if (!change_dir(dir, CD_NORMAL)) {
@@ -1027,6 +1098,8 @@ static void do_server_recv(int f_in, int f_out, int argc,
char *argv[])
return;
}
+ become_copy_as_user();
+
if (argc > 0) {
char *dir = argv[0];
argc--;
@@ -1186,6 +1259,9 @@ int client_run(int f_in, int f_out, pid_t pid, int argc,
char *argv[])
if (write_batch && !am_server)
start_write_batch(f_out);
+
+ become_copy_as_user();
+
flist = send_file_list(f_out, argc, argv);
if (DEBUG_GTE(FLIST, 3))
rprintf(FINFO,"file list sent\n");
@@ -1219,6 +1295,8 @@ int client_run(int f_in, int f_out, pid_t pid, int argc,
char *argv[])
io_start_buffering_out(f_out);
}
+ become_copy_as_user();
+
send_filter_list(read_batch ?
