Re: rsync support in authprogs - feedback requested

[Karl O. Pinc via rsync]

On Thu, 18 Feb 2021 12:22:33 -0500
Kevin Korb via rsync  wrote:

> You should both look into rrsync.  It comes with rsync and is designed
> to do exactly this.  

I'm not really interested in restricting rsync to particular
directories.  That seems to be what rrsync is for, although
it's a little hard to tell -- there's also a read-only option.

>  Unfortunately some Linux distros are maintained
> by insane people who install rrsync as if it was documentation
> (compressed and not executable) instead of a helper script which is
> what it is.  

FWIW, my uninformed guess is that Debian does not install
rrsync as an executable script because nobody has gotten around to
writing a it a man page.  A man page is required by Debian
policy for every executable.  

The good news is that rrsync is not compressed in Debian.  :)  The bad
news is I don't even see a bug report requesting rrsync, or
anything else in /usr/share/doc/rsync/scripts/, be executable.

Regards,

Karl 
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: rsync support in authprogs - feedback requested

[Karl O. Pinc via rsync]

On Wed, 17 Feb 2021 21:52:06 -0800
Bri Hatch via rsync  wrote:

> I recently added initial rsync support to authprogs.

> I'd be very interested in feedback 

For some 15 years+ (?) I've had a /root/.ssh/authorized keys line
that starts with:

"no-pty,no-agent-forwarding,no-port-forwarding,no-user-rc,no-X11-forwarding,command="rsync
 --server --daemon ."

Occasionally I frob the ssh restrictions as new ones are
introduced.

The remote end uses rsync to backup (with --link-dest) the
entire file system.  The idea (iirc) was to restrict
the given key so that it would only run rsync.
And I think this also forces the local end to use
/etc/rsyncd.conf, where there's an additional layer
of security via a secrets file and read-only can
be set to provide some control.

The remote end always runs rsync -- the direction of 
transfer is static, per-host-pair, but can be either
in or out. (Push or pull backups.) The above authorized_keys 
line does not enforce direction, which might be useful.

I only rarely think about tweaking the authorized_keys line, 
and the rsync options used haven't changed since I got them to work.
Without really thinking about it it seems that your
authprogs development might be useful.  

My purpose with this email is to let you do all the 
thinking and tell me of all the wonderful utility
your authprogs work can provides, either now or
in the future.  ;-)  Or at least give you some
background in case you want to develop in a direction
that you think would helpful to me.  If something comes
of this I might even turn my brain on again and
modify my systems.  :)

Regards,

Karl 
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Enabling easier contributions to rsync

[Karl O. Pinc via rsync]

On Tue, 26 May 2020 00:43:41 +0200
uxio prego via rsync  wrote:

> > On 25 May 2020, at 23:55, Wayne Davison via rsync
> >  wrote:
> > 
> > I've decided to give hosting it on github a try, especially since
> > there's been a lot of nice contributions lately.  Hopefully this
> > will make it easier for both the people sending patches as well as
> > for me to snag the changes.  I'll continue to push changes to the
> > samba git as well.

> Excellent call; for GitHub's temporary, but freedom's forever!

FWIW, I find sending an email with an attachment
a lot easier than having to login to github, clone, push,
and open a pull request.  I thought people went to github
because they like the workflow when _accepting_ patches.

Regards,

Karl 
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: [PATCH] Optimized assembler version of md5_process() for x86-64

[Karl O. Pinc via rsync]

On Sat, 23 May 2020 10:21:31 -0700
Wayne Davison via rsync  wrote:

> Adding optional support for openssl's crypto library is also a good
> idea.

There is also libressl to consider, if you're considering libraries.

Karl 
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Much improved speeds of rsync via SSH - something to consider

[Karl O. Pinc via rsync]

On Mon, 2 Apr 2018 15:05:14 +0200
Thanassis Tsiodras via rsync  wrote:

> I recently concluded a bug hunt to trace why my rsync-ing to an SBC
> was much slower than the corresponding iperf3-reported speeds. To
> give a concise summary of the situation, in slow wifi links using SSH
> with ProxyCommand tremendously speeds up things:

> $ rsync -avz --progress -e 'ssh -o "ProxyCommand nc %h %p"'
> ./sample.data root@192.168.1.150:/dev/shm/

> If it isn't clear - the speed of the upload went from 543 kbytes/sec
> to 2690 kbytes/sec.
> 
> If you want to see why - and how I traced this down - you can read my
> complete report in the UNIX StackExchange forum. To avoid being
> classified as a spammer, I won't include a link - search for question
> 434825 in the search bar. The executive summary is that SSH disables
> Nagle's algorithm by default - and in slow links this can cause
> tremendous impact (as you see above).
> 
> Hope this helps people that backup over slow links (wi-fi or
> otherwise).

In the interest of collecting ssh-related issues in a single
thread I add this note:

The HPN patches to the portable OpenSSH implementation can
improve speed over high speed links with long round trip times.

  https://www.psc.edu/hpn-ssh

It also helps in those cases where CPU is a bottleneck and
encryption of transmitted data unnecessary.

The FAQ is also interesting.

The executive summary is:

  SCP and the underlying SSH2 protocol implementation in OpenSSH is
  network performance limited by statically defined internal flow
  control buffers.

Regards,

Karl 
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Help - rsync runs from command line, fails from task scheduler, hangs at msg checking charset: UTF-8

[Karl O. Pinc via rsync]

On Sat, 29 Jul 2017 19:45:22 -0700 (PDT)
leonv12 via rsync  wrote:

> I don't get why it runs from the command line but not from a
> scheduled task. Any suggestions for a fix or a work-around?

Talk to someone who knows about task scheduling and its
failure modes?

Karl 
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Help - rsync runs from command line, fails from task scheduler, hangs at msg checking charset: UTF-8

[Karl O. Pinc via rsync]

On Fri, 28 Jul 2017 14:47:02 -0700 (PDT)
leonv12 via rsync  wrote:

>(18 args)*msg
> checking charset: UTF-8*[sender] io timeout after 3000 seconds --
> exiting[sender] _exit_cleanup(code=30, file=io.c, line=195):
> enteredrsync error: timeout in data send/receive (code 30) at
> io.c(195) [sender=3.1.2][sender] _exit_cleanup(code=30, file=io.c,
> line=195): about to call exit(30)Help will be much appreciated!

Reminds me of previous struggles I've had with MS Windows and
byte order marks.  (Something about the way it requires UTF-16
but breaks unless you use UTF-8, or something.)

https://en.wikipedia.org/wiki/Byte_order_mark


Karl 
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: [Bug 12819] [PATCH] sync() on receiving side for data consistency

[Karl O. Pinc via rsync]

On Fri, 16 Jun 2017 12:34:40 +0200
Ben RUBSON via rsync <rsync@lists.samba.org> wrote:

> > On 15 Jun 2017, at 19:29, Karl O. Pinc via rsync
> > <rsync@lists.samba.org> wrote:

> > The problem is that the --server (and, especially,
> > --daemon) documentation has gone away.  Or at least
> > left the man page. (v3.1.1, Debian 8, Jessie)  Except
> > for a hint that --server exists at the bottom.  
> 
> Are you looking for `man rsyncd.conf` ?

No, that tells me what --daemon does; how to run rsync
as a server.  It does not tell me how to invoke rsync at the 
remote end manually without doing server-side things
such as the reading of rsyncd.conf.

What I want documened is how to use a customized
transport that does not allow the client side to
send arbirtrary commands to the remote end.
The sort of thing done when using
ssh with keys and the command= option within an
authorized_keys file.

As mentioned, now I use command="rsync --server --daemon ."
in my authorized_keys file.
I once figured this out from old rsync man pages, but don't
see how to glean this command sequence from a more recent
man page.

Again, I might (eventually) get around to sending
in a man page patch if somebody explains how it's done.

Regards,

Karl <k...@meme.com>
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: [Bug 12819] [PATCH] sync() on receiving side for data consistency

[Karl O. Pinc via rsync]

On Thu, 15 Jun 2017 13:23:44 +
just subscribed for rsync-qa from bugzilla via rsync
 wrote:

> https://bugzilla.samba.org/show_bug.cgi?id=12819
> 
> --- Comment #7 from Ben RUBSON  ---

> Note that my patch simply adds a sync() just after recv_files(), so
> one sync() per connection, not per write operation.

> But we could make this a rsync option, so that one can enable /
> disable it on its own.

I think the "right" rsync option to add (because rsync does
not have enough options already ;-) is a --hook-post option.
It would run something (a `sync` in your case) on the
remote end after finishing.  There are clear security issues
here.

Rather than having --hook-post and having to do something
(a server side config option that says what --hook-post
can do?) to address the security concerns it seems much
simpler to improve the rsync documentation regarding running
the rsync server side.

I'm still using command="rsync --server --daemon ." in my
~/.ssh/authorized_keys file on the remote end.  It'd be simple 
enough to add, say, a "sync" to the end of this to force a sync
when rsync finishes.  The problem is that the --server (and, especially,
--daemon) documentation has gone away.  Or at least
left the man page. (v3.1.1, Debian 8, Jessie)  Except
for a hint that --server exists at the bottom.

If the server side of rsync was better documented then
perhaps a simple inetd rsync service (or --rsync-path
or -e value, etc.) would be easy for the end-user to 
cobble together to meet needs such as this.

Can somebody please explain --server?  (And --sender, I guess.)
I might (possibly) be motivated to send in a man page patch.

Regards,

Karl 
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: How to detect hanging rsync from bash-script ?

[Karl O. Pinc via rsync]

On Mon, 10 Apr 2017 06:50:24 + (UTC)
reiner otto via rsync  wrote:

> I am using rsync on an unreliable mobile WAN connection, which causes
> rsync receiver to hang for long time, when connection is broken
> during transfere. As the option "--timeout=" here does not hit for
> me, is there a recommended "best" method to detect such scenario ?

One good way would be to use rsync's --rsh option to
invoke ssh yourself with something like -o ServerAliveInterval.

It's not clear to me how you'd get the ssh error code back
out of rsync, in other words what error rsync returns in the
case of transport failure.  Maybe 10?  Perhaps someone else
can chime in here?


Karl 
Free Software:  "You don't pay back, you pay forward."
 -- Robert A. Heinlein

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html