Re: Aw: Re: [PATCH] Reduce memory usage

2021-10-03 Thread Rupert Gallagher via rsync
Original Message On Oct 2, 2021, 12:36, < devz...@web.de> wrote: >>In the exchange I argued that proper use of ram as a buffer would have cut >>down backup time to minutes instead of days. >could you give an example where rsync is slowing things down so much due to >ram

[SCM] The rsync repository. - branch master updated

2021-10-02 Thread Rsync CVS commit messages
The branch, master has been updated via 78b5bc66 Enable --atimes on macOS. via f41cdc75 Check ro in set_create_time() for Cygwin too. from c8e7c4b3 Avoid an issue where the size of st_dev != dev_t. https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log

Re: Re: [PATCH] Reduce memory usage

2021-10-02 Thread Jindřich Makovička via rsync
Just note this patch has nothing to do with memory consumption vs performance. It just avoids allocating memory that was left unused anyway. On Sat, Oct 2, 2021, 12:51 devzero--- via rsync wrote: > >In the exchange I argued that proper use of ram as a buffer would have > cut down backup time to

Aw: Re: [PATCH] Reduce memory usage

2021-10-02 Thread devzero--- via rsync
>In the exchange I argued that proper use of ram as a buffer would have cut down backup time to minutes instead of days. could you give an example where rsync is slowing things down so much due to ram constraints or inefficient ram use? please mind that disk bandwith and file copy bandwith is

Re: [PATCH] Reduce memory usage

2021-10-02 Thread Rupert Gallagher via rsync
If you look at my previous exchange in the list, I raised the need for more ram usage via a tool option. In the exchange I argued that proper use of ram as a buffer would have cut down backup time to minutes instead of days. At the time, my proposal was dismissed by someone saying that rsync

[SCM] The rsync repository. - branch master updated

2021-10-01 Thread Rsync CVS commit messages
The branch, master has been updated via c8e7c4b3 Avoid an issue where the size of st_dev != dev_t. via bff084c1 Always run `mkgitver` prior to a build from 16c8b05f Add more NEWS updates. https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log

[SCM] The rsync repository. - branch master updated

2021-10-01 Thread Rsync CVS commit messages
The branch, master has been updated via 16c8b05f Add more NEWS updates. via 15dd2058 Change do_chmod to always try lchmod() first (when possible). via c27180c0 Add a couple more options to rrsync. via 050fdd41 Allow the script to be run from inside the packaging

[SCM] The rsync repository. - branch master updated

2021-10-01 Thread Rsync CVS commit messages
The branch, master has been updated via ae1f0029 Reduce memory usage (#235) from 3911c238 Tweak SIMD & ASM option defaults. https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log - commit

[SCM] The rsync repository. - branch master updated

2021-10-01 Thread Rsync CVS commit messages
The branch, master has been updated via 3911c238 Tweak SIMD & ASM option defaults. via 3814dbb0 Make cygwin's curl grab the gist file. [buildall] via 82f023d7 Add --fsync option (promoted from patches). via ec57c57b Help avoid a --sparse --inplace bug in older

Re: [PATCH] Reduce memory usage

2021-09-28 Thread Johannes Altmanninger via rsync
Looks awesome, really nice catch! -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[PATCH] Reduce memory usage

2021-09-28 Thread Jindřich Makovička via rsync
In 2004, an allocation optimization has been added to the file list handling code, that preallocates 32k of file_struct pointers in a file_list. This optimization predates the incremental recursion feature, for which it is not appropriate anymore. When copying a tree containing a large number of

Re: [PATCH] Reduce memory usage

2021-09-28 Thread Jindřich Makovička via rsync
On Mon, 27 Sep 2021 22:03:01 +0200 Johannes Altmanninger wrote: > > On Sun, Sep 26, 2021 at 01:54:13PM +0200, Jindřich Makovička via > > rsync wrote: > > > > Applying the attached patch, which reduces the default allocation > > to 32 pointers, and preallocates 32K pointers only for the main > >

Re: [PATCH] Reduce memory usage

2021-09-27 Thread Johannes Altmanninger via rsync
> On Sun, Sep 26, 2021 at 01:54:13PM +0200, Jindřich Makovička via rsync wrote: > > Applying the attached patch, which reduces the default allocation to 32 > pointers, and preallocates 32K pointers only for the main file lists in > send_file_list and recv_file_list, reduces the peak memory usage

Re: [PATCH] Reduce memory usage

2021-09-27 Thread Jindřich Makovička via rsync
On Mon, 27 Sep 2021 13:38:22 +0200 Jindřich Makovička wrote: > On Sun, 26 Sep 2021 13:54:13 +0200 > Jindřich Makovička wrote: > > Hi, > > > > ... > > > > Applying the attached patch, which reduces the default allocation to > > 32 pointers, and preallocates 32K pointers only for the main file >

Re: [PATCH] Reduce memory usage

2021-09-27 Thread Jindřich Makovička via rsync
On Sun, 26 Sep 2021 13:54:13 +0200 Jindřich Makovička wrote: > Hi, > > ... > > Applying the attached patch, which reduces the default allocation to > 32 pointers, and preallocates 32K pointers only for the main file > lists in send_file_list and recv_file_list, reduces the peak memory > usage in

[SCM] The rsync repository. - branch master updated

2021-09-26 Thread Rsync CVS commit messages
The branch, master has been updated via 745ecf28 Fix a couple variable typos. from 265785b7 x86-64 AVX2 assemby implemenation of get_checksum1() (#174) https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log -

[SCM] The rsync repository. - branch master updated

2021-09-26 Thread Rsync CVS commit messages
The branch, master has been updated via 265785b7 x86-64 AVX2 assemby implemenation of get_checksum1() (#174) from 97f4d48a configure improvements https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log - commit

[PATCH] Reduce memory usage

2021-09-26 Thread Jindřich Makovička via rsync
Hi, When using rsync to back up the file system on my laptop, containing a pretty much default linux desktop, I was wondering how rsync uses over 100MB of RAM it allocates. It turned out that most of the memory is used for the arrays of file_struct pointers, most of which end up unused - much

Release schedule 3.2.4?

2021-09-23 Thread Rudi Heitbaum via rsync
Hi Wayne, I have been performing the package maintenance (bumps) for a LibreELEC over the past year in the lead up to the LE10 release. Just enquiring if you have a planned/proposed release coming up? Just enquiring given the PR requests into LibreELEC

high memory usage transferring about 27.000.000 files

2021-09-21 Thread Valentin Kleibel via rsync
We are trying to back up a large number of small files and have issues with high memory usage. On both ends debian 11.0 is running with rsync version 3.2.3 protocol version 31. Ideally we want to use the options `-a --delete --backup -e ssh` to run this transfer and pull a backup from the

rsync gets stuck on a file for a long time

2021-09-18 Thread jedenfalco via rsync
I am getting a number of errors from rsync ERROR: a3-swbase/05-nonfsw/scien/NatrResour/Seiscope/fullwav/fullwav--4.8/fwi/fwinv--3d/examples/ddm_nostk/ftopo0 failed verification -- update discarded. When this happens, rsync gets stuck on the file for a long time, slowing other rsync commands.

Adding timestamp before printing file name

2021-09-17 Thread hancooper via rsync
I am using rsync in the following manner rsync -av --inplace --update --log-file=/medhd/a3-swbase/05-nonfsw.log /medhc/a3-swbase/05-nonfsw /medhd/a3-swbase/ Normally the file name is outputted. Would it be possible to also add the time before the name?-- Please use reply-all for most replies

Passing result of shuf to rsync

2021-09-17 Thread hancooper via rsync
‐‐‐ Original Message ‐‐‐ On Saturday, September 18, 2021 12:58 AM, Kevin Korb wrote: > Quoting from the man page > > SORTED TRANSFER ORDER > Rsync always sorts the specified filenames into its internal > transfer list. This handles the merging together of the contents of identi‐ >

Re: Passing result of shuf to rsync

2021-09-17 Thread Kevin Korb via rsync
Quoting from the man page SORTED TRANSFER ORDER Rsync always sorts the specified filenames into its internal transfer list. This handles the merging together of the contents of identi‐ cally named directories, makes it easy to remove duplicate filenames, and may confuse someone

Passing result of shuf to rsync

2021-09-17 Thread hancooper via rsync
‐‐‐ Original Message ‐‐‐ On Saturday, September 18, 2021 12:29 AM, Kevin Korb wrote: > Well, what you have should function. But rsync is going to sort the list. I want rsync to use the list as given to it by shuf. > On 9/17/21 8:26 PM, hancooper wrote: > > > ‐‐‐ Original Message

Re: Passing result of shuf to rsync

2021-09-17 Thread Kevin Korb via rsync
Well, what you have should function. But rsync is going to sort the list. On 9/17/21 8:26 PM, hancooper wrote: > ‐‐‐ Original Message ‐‐‐ > On Saturday, September 18, 2021 12:06 AM, Kevin Korb via rsync > wrote: > >> Not really sure what you are trying to accomplish here. Seems like

Passing result of shuf to rsync

2021-09-17 Thread hancooper via rsync
‐‐‐ Original Message ‐‐‐ On Saturday, September 18, 2021 12:06 AM, Kevin Korb via rsync wrote: > Not really sure what you are trying to accomplish here. Seems like it > should work the way you have it. Note that many wonky rsync kludges are > due to people not realizing that rsync can

Re: Passing result of shuf to rsync

2021-09-17 Thread Kevin Korb via rsync
Not really sure what you are trying to accomplish here. Seems like it should work the way you have it. Note that many wonky rsync kludges are due to people not realizing that rsync can have multiple source arguments. Instead of the source simply being . it can be a list of stuff. Also, note

Passing result of shuf to rsync

2021-09-17 Thread hancooper via rsync
I am trying to pass filename results that have been shuffled, then pass the file to rsync. Not quite sure whether to use --files-from=- or some other way. shuf -n "$nf" -e ${dpath}/${incl} | rsync -av --update --files-from=- . "$dst"-- Please use reply-all for most replies to avoid omitting the

Skip file transfer on detecton of error code

2021-09-17 Thread hancooper via rsync
Occasionally rsync issues errors, which result in slowing a system during a transfer. Would it be possible to skip the transfer of the file rather than having rsync continue to insist, trying the transfer many times? rsync: read errors mapping

rsync dest high disk write request avg waiting time (w_await) and probable low speed

2021-09-13 Thread Charles via rsync
Is it likely that rsync is the cause of the symptoms? If so, how and can the rsync command be changed to prevent the problem? While running this command ... rsync --filter=protect _Changed and deleted files/ --exclude=/_Changed and deleted files/ --backup --backup-dir=_Changed and deleted

Rsync options –update and –append-verify

2021-09-11 Thread hancooper via rsync
‐‐‐ Original Message ‐‐‐ On Sunday, September 12, 2021 5:10 AM, Kevin Korb wrote: > --inplace can prevent rsync from running you out of disk space when > updating a large file. But you don't want to mix it with --update. > Judging by your other message you think that --update means

Re: Rsync options –update and –append-verify

2021-09-11 Thread Kevin Korb via rsync
--inplace can prevent rsync from running you out of disk space when updating a large file. But you don't want to mix it with --update. Judging by your other message you think that --update means modiify files that are already on both ends. It doesn't. It only means that rsync is forbidden to

Rsync options –update and –append-verify

2021-09-11 Thread hancooper via rsync
‐‐‐ Original Message ‐‐‐ On Sunday, September 12, 2021 2:34 AM, hancooper via rsync wrote: > ‐‐‐ Original Message ‐‐‐ > On Sunday, September 12, 2021 2:03 AM, Kevin Korb k...@sanitarium.net wrote: > > > I thought I did elaborate. If it is a problem for you then maybe you > >

Re: Rsync options –update and –append-verify

2021-09-11 Thread hancooper via rsync
‐‐‐ Original Message ‐‐‐ On Sunday, September 12, 2021 2:03 AM, Kevin Korb wrote: > I thought I did elaborate. If it is a problem for you then maybe you > shouldn't be using --update. Or you should let rsync delete incomplete > files upon abort as it does by default. I am using the

Re: Rsync options –update and –append-verify

2021-09-11 Thread Kevin Korb via rsync
I thought I did elaborate. If it is a problem for you then maybe you shouldn't be using --update. Or you should let rsync delete incomplete files upon abort as it does by default. On 9/11/21 9:29 PM, hancooper wrote: > ‐‐‐ Original Message ‐‐‐ > On Saturday, September 11, 2021 11:20 PM,

Rsync options –update and –append-verify

2021-09-11 Thread hancooper via rsync
‐‐‐ Original Message ‐‐‐ On Saturday, September 11, 2021 11:20 PM, Kevin Korb via rsync wrote: > --archive is all you really need. I actually wish --archive was the > default because it is all most people need and with the exception of > writing to a FAT filesystem it is almost always

Re: Rsync options –update and –append-verify

2021-09-11 Thread Kevin Korb via rsync
--archive is all you really need. I actually wish --archive was the default because it is all most people need and with the exception of writing to a FAT filesystem it is almost always needed. --append is for very special cases and should only be used if you really know you need it and why.

Rsync options –update and –append-verify

2021-09-11 Thread hancooper via rsync
I am struggling to understand exactly what the rsync options --update and --append-verify do. Doing info rsync gives -u, --update This forces rsync to skip any files which exist on the destina‐ tion and have a modified time that is newer than the source file. (If

Re: Question about rsync -uav dir1/. dir2/.: possib to link?

2021-09-04 Thread Kevin Korb via rsync
Yes, cpio -l can be useful since cpio can easily operate on the output from the very capable find command. On 9/4/21 8:34 PM, Dan Stromberg wrote: > > I was thinking --link-dest too. > > Sometimes this can be done with cpio too; check out the -pdlv options. > > On Sat, Sep 4, 2021 at 4:57 PM

Re: Question about rsync -uav dir1/. dir2/.: possib to link?

2021-09-04 Thread Dan Stromberg via rsync
I was thinking --link-dest too. Sometimes this can be done with cpio too; check out the -pdlv options. On Sat, Sep 4, 2021 at 4:57 PM Kevin Korb via rsync wrote: > Rsync does almost everything cp does but since it is designed to network > it never got that feature. I was thinking maybe

Re: Question about rsync -uav dir1/. dir2/.: possib to link?

2021-09-04 Thread Kevin Korb via rsync
Rsync does almost everything cp does but since it is designed to network it never got that feature. I was thinking maybe --link-dest could be tortured into doing it but if it can I can't figure out how. BTW, you have some pointless dots in there. On 9/4/21 6:41 PM, L A Walsh via rsync wrote: >

Question about rsync -uav dir1/. dir2/.: possib to link?

2021-09-04 Thread L A Walsh via rsync
I noticed in looking at download dirs for a project, that another mirror had "crept-in" for usage (where different mirrors are stored under mirror-URL names). To copy over the diffs, normally I'd do: rsync -uav dir1/. dir2/. (where dir1="the new mirror that I'd switched to by accident, and

variable compression feature request

2021-09-04 Thread griffin tucker via rsync
--compress or -z will compress during data transfer what about a variable compression rate? --optimal-compress or -za monitor the speed of the transfer and the cpu throughput and automatically find the optimum compression in almost-real-time with the goal of reduced transfer time i figure

Re: using "--checksum" for local sync

2021-08-27 Thread ff0 via rsync
> I could just have synced the file anyway? First important note to make known up front: When rsync qualifies a file for transfer, then by default it performs in-flight transfer checksums to verify the SRC and DST are binary same. The --checksum option is an extra step in determining "Does this

using "--checksum" for local sync

2021-08-27 Thread Fourhundred Thecat via rsync
Hello, I am using the "--checksum" option, when syncing files over the network. But when syncing large files locally, does it make sense to use --checksum, or would it slow things down? I mean, if it needs to calculate the checksum first, it could just have synced the file anyway. right?

Can rsync log files excluded from transfers?

2021-08-27 Thread ff0 via rsync
Hey folks, My installed distro rsync version is: 3.1.3 (3.2.3 available in backports). Per subject, it would be great for one of my use cases, to be able to see a list of files that were evaluated for transfer but excluded (not skipped). I've tried -vv and -ii and --info=skip and --info=all4

Speeding up rsync

2021-08-24 Thread hancooper via rsync
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Tuesday, August 24, 2021 9:40 PM, Kevin Korb wrote: > Most of the speedup is due to ssh being bound to a single CPU. You will > get a little more speed out of a single CPU system but the speedup is > unlikely to go past

Re: Speeding up rsync

2021-08-24 Thread Kevin Korb via rsync
Most of the speedup is due to ssh being bound to a single CPU. You will get a little more speed out of a single CPU system but the speedup is unlikely to go past 2 in parallel. Even with that you may end up adding more latency via disk seeks to accomplish much. On 8/24/21 5:12 PM, hancooper

Speeding up rsync

2021-08-24 Thread hancooper via rsync
‐‐‐ Original Message ‐‐‐ On Tuesday, August 24, 2021 9:12 PM, hancooper via rsync wrote: > ‐‐‐ Original Message ‐‐‐ > On Tuesday, August 24, 2021 8:25 PM, Kevin Korb via rsync > rsync@lists.samba.org wrote: > > > In my experience backing up multiple hosts to a single host can

Re: Speeding up rsync

2021-08-24 Thread hancooper via rsync
‐‐‐ Original Message ‐‐‐ On Tuesday, August 24, 2021 8:25 PM, Kevin Korb via rsync wrote: > In my experience backing up multiple hosts to a single host can speed > things up. Especially if using rsync over ssh with multiple CPUs on the > single host. You would need to do some

Re: Speeding up rsync

2021-08-24 Thread Kevin Korb via rsync
In my experience backing up multiple hosts to a single host can speed things up. Especially if using rsync over ssh with multiple CPUs on the single host. You would need to do some experimentation to determine the best number for your hardware and network. Also, if you exceed that number you

Speeding up rsync

2021-08-24 Thread hancooper via rsync
It makes sense to we to run multiple rsync commands to speed up a backup. At work, some have argued that if I sync the data all to the same host, there is no advantage in parallelization. Whether you sync 3x 1G in parallel or 1x 3G makes no noticeable difference. Speed in general does not

Teo En Ming's Guide to Configuring rsnapshot Backup for Linux Servers

2021-08-18 Thread Turritopsis Dohrnii Teo En Ming via rsync
Subject: Teo En Ming's Guide to Configuring rsnapshot Backup for Linux Servers rsnapshot backup for Linux servers is based on rsync. Author: Mr. Turritopsis Dohrnii Teo En Ming (TARGETED INDIVIDUAL) Country: Singapore Date: 18 August 2021 Wednesday Singapore Time Type of Publication: Plain Text

[Bug 14798] New: Metadata traffic --- uncompressed with -z, interaction with --bwlimit and ssh compression

2021-08-17 Thread just subscribed for rsync-qa from bugzilla via rsync
https://bugzilla.samba.org/show_bug.cgi?id=14798 Bug ID: 14798 Summary: Metadata traffic --- uncompressed with -z, interaction with --bwlimit and ssh compression Product: rsync Version: 3.1.3 Hardware: All

Re: suppress message: is uptodate

2021-08-16 Thread Fourhundred Thecat via rsync
> On 2021-08-15 07:09, Robin Lee Powell wrote: So, yeah, rsync is giving you that message because you added an extra flag to specifically tell it to do so. :) Try "name1" and see if that fixes it. thank you. I think that solved it. -- Please use reply-all for most replies to avoid

Re: suppress message: is uptodate

2021-08-14 Thread Robin Lee Powell via rsync
On Sun, Aug 15, 2021 at 07:03:18AM +0200, Fourhundred Thecat via rsync wrote: > Hello, > > is there a way to suppress this message when syncing files? > >is uptodate > > I would like to see only files that have been synced. > > The internet forums are full of people asking how to get

suppress message: is uptodate

2021-08-14 Thread Fourhundred Thecat via rsync
Hello, is there a way to suppress this message when syncing files? is uptodate I would like to see only files that have been synced. The internet forums are full of people asking how to get rid of this, even as far back as 10 years, and only hack I could find is to pipe it to grep.

Re: Which method (rsync-based) is better for backing up Linux servers to Synology NAS?

2021-08-09 Thread Dan Stromberg via rsync
I suppose I may as well mention: https://stromberg.dnsalias.org/~dstromberg/Backup.remote It just does rsync snapshotting with --link-dest, and keeps the last n snapshots. It's smart enough to resume a previously interrupted snapshot. It's pretty simple - both to set up and to use. I used to

Re: Which method (rsync-based) is better for backing up Linux servers to Synology NAS?

2021-08-09 Thread Bri Hatch via rsync
tl;dr: rsnapshot is usually the best plan. Details: rsnapshot copies files from remote machines and stores snapshotted directories locally. If you want to get the CentOS servers copied to the synology then you need to run rsnapshot from the synology. Synology doesn't have rsnapshot out of the box

Which method (rsync-based) is better for backing up Linux servers to Synology NAS?

2021-08-09 Thread Turritopsis Dohrnii Teo En Ming via rsync
Subject: Which method (rsync-based) is better for backing up Linux servers to Synology NAS? Good day from Singapore, Our customer has 2 CentOS 7.9 Linux servers with cPanel web hosting control panel installed. We would like to backup the 2 CentOS 7.9 Linux servers to Synology NAS. Which method

Re: How to manage root<-->root rsync keeping permissions?

2021-08-07 Thread Chris Green via rsync
Greg Minshall via rsync wrote: > >If you only do backups at 1am (or whenever), why would your > > backup machine enable ssh outside of the range 12:59 - 01:01? > > Greg's rule of windows: the narrower the window, the more likely it will > be hit. :) > But I use Linux, not windows..

Re: How to manage root<-->root rsync keeping permissions?

2021-08-07 Thread L A Walsh via rsync
On 2021/08/07 08:45, Chris Green via rsync wrote: Because cron/anacron isn't perfect and the machine being backed up nay not be turned on all the time so the time that it tries to backup is most definitely not fixed accurately! My *backups* of important data are incremental

Re: How to manage root<-->root rsync keeping permissions?

2021-08-07 Thread Greg Minshall via rsync
>If you only do backups at 1am (or whenever), why would your > backup machine enable ssh outside of the range 12:59 - 01:01? Greg's rule of windows: the narrower the window, the more likely it will be hit. :) -- Please use reply-all for most replies to avoid omitting the mailing list. To

Re: How to manage root<-->root rsync keeping permissions?

2021-08-07 Thread Bri Hatch via rsync
The original request was to be able to copy files from local /etc to a remote /etc while retaining file permissions (perms, owner, etc). This does require root on the receiving box in some way. Commenting on some previous suggestions: * connecting to the remote system as a non-root user and

Re: How to manage root<-->root rsync keeping permissions?

2021-08-07 Thread Chris Green via rsync
On Sat, Aug 07, 2021 at 08:10:47AM -0700, L A Walsh wrote: > On 2021/08/07 03:44, Chris Green via rsync wrote: > > L A Walsh via rsync wrote: > > > It seems to me, a safer bet would be to generate an ssh-cert > > > that allows a passwdless login from your sys to the remote. > > > > > The trouble

Re: How to manage root<-->root rsync keeping permissions?

2021-08-07 Thread raf via rsync
On Sat, Aug 07, 2021 at 11:44:34AM +0100, Chris Green via rsync wrote: > L A Walsh via rsync wrote: > > On 2021/08/03 07:09, Chris Green via rsync wrote: > > > I already have an rsync daemon server running elsewhere, I can add > > > this requirement to that I think. Thank you. > > > > >

Re: How to manage root<-->root rsync keeping permissions?

2021-08-07 Thread L A Walsh via rsync
On 2021/08/07 03:44, Chris Green via rsync wrote: L A Walsh via rsync wrote: It seems to me, a safer bet would be to generate an ssh-cert that allows a passwdless login from your sys to the remote. The trouble with that is that it leaves a big security hole. If you only

Re: How to manage root<-->root rsync keeping permissions?

2021-08-07 Thread Chris Green via rsync
L A Walsh via rsync wrote: > On 2021/08/03 07:09, Chris Green via rsync wrote: > > I already have an rsync daemon server running elsewhere, I can add > > this requirement to that I think. Thank you. > > > > > It seems to me, a safer bet would be to generate an ssh-cert > that allows a

Re: How to manage root<-->root rsync keeping permissions?

2021-08-07 Thread L A Walsh via rsync
On 2021/08/03 07:09, Chris Green via rsync wrote: I already have an rsync daemon server running elsewhere, I can add this requirement to that I think. Thank you. It seems to me, a safer bet would be to generate an ssh-cert that allows a passwdless login from your sys to the remote.

Re: How to manage root<-->root rsync keeping permissions?

2021-08-04 Thread Chris Green via rsync
Andy Smith via rsync wrote: > > > I've set it up so chris can run rsync with root permissions. > > However I'm not quite sure how to get it to work as one needs to say > > "sudo rsync" to get the root privilege. How do you do that? > > The first link I sent you had an example of that:

Re: How to manage root<-->root rsync keeping permissions?

2021-08-04 Thread Andy Smith via rsync
Hello, On Tue, Aug 03, 2021 at 03:05:27PM +0100, Chris Green via rsync wrote: > Remember, as I said, this is all Debianland with no real root login, > while I could add one I'd prefer not to. Your system already has a root user and if you added an SSH public key to its authorized_keys file

Re: How to manage root<-->root rsync keeping permissions?

2021-08-03 Thread raf via rsync
On Tue, Aug 03, 2021 at 10:51:34AM +, Andy Smith via rsync wrote: > Hi Chris, > > On Tue, Aug 03, 2021 at 09:48:37AM +0100, Chris Green via rsync wrote: > > But how do you handle the other end to restore the root ownership etc.? > > The script has to do something like:- > > > > rsync

Re: How to manage root<-->root rsync keeping permissions?

2021-08-03 Thread Chris Green via rsync
Andy Smith via rsync wrote: > Hi Chris, > > On Tue, Aug 03, 2021 at 11:48:31AM +0100, Chris Green via rsync wrote: > > If I used the --super option (in a command like the one above) and > > chris can run rsync as root on the remote end (via options in the > > sudoers file) will this do what I

Re: How to manage root<-->root rsync keeping permissions?

2021-08-03 Thread Chris Green via rsync
Paul Slootman via rsync wrote: > On Tue 03 Aug 2021, Chris Green via rsync wrote: > > > Is there a way to copy (for example) the /etc hierarchy from one > > system to another preserving root ownership of files and without > > revealing root passwords all over the place? > > Best way is to run

Re: How to manage root<-->root rsync keeping permissions?

2021-08-03 Thread Chris Green via rsync
Andy Smith via rsync wrote: > Hi Chris, > > On Tue, Aug 03, 2021 at 09:48:37AM +0100, Chris Green via rsync wrote: > > But how do you handle the other end to restore the root ownership etc.? > > The script has to do something like:- > > > > rsync -a /etc/ chris@remote:backups/etc/ > > > >

Re: How to manage root<-->root rsync keeping permissions?

2021-08-03 Thread Paul Slootman via rsync
On Tue 03 Aug 2021, Chris Green via rsync wrote: > Is there a way to copy (for example) the /etc hierarchy from one > system to another preserving root ownership of files and without > revealing root passwords all over the place? Best way is to run an rsync daemon on the source system, and be

Re: How to manage root<-->root rsync keeping permissions?

2021-08-03 Thread Andy Smith via rsync
Hi Chris, On Tue, Aug 03, 2021 at 11:48:31AM +0100, Chris Green via rsync wrote: > If I used the --super option (in a command like the one above) and > chris can run rsync as root on the remote end (via options in the > sudoers file) will this do what I want? I guess I can go away and try > it!

Re: How to manage root<-->root rsync keeping permissions?

2021-08-03 Thread Andy Smith via rsync
Hi Chris, On Tue, Aug 03, 2021 at 09:48:37AM +0100, Chris Green via rsync wrote: > But how do you handle the other end to restore the root ownership etc.? > The script has to do something like:- > > rsync -a /etc/ chris@remote:backups/etc/ > > So at the remote end it only has chris'

Re: How to manage root<-->root rsync keeping permissions?

2021-08-03 Thread Chris Green via rsync
Chris Green via rsync wrote: > Is there a way to copy (for example) the /etc hierarchy from one > system to another preserving root ownership of files and without > revealing root passwords all over the place? > > This is actually from and to Debian based systems (from Raspberry Pi > to Xubuntu)

How to manage root<-->root rsync keeping permissions?

2021-08-03 Thread Chris Green via rsync
Is there a way to copy (for example) the /etc hierarchy from one system to another preserving root ownership of files and without revealing root passwords all over the place? This is actually from and to Debian based systems (from Raspberry Pi to Xubuntu) so there's no actual root user login

Utility of --backup

2021-07-20 Thread Lisa via rsync
>From what I understand `--backup-dir` uses a hierarchical backup.  And >`--suffix` appends the value in `--suffix` to the end of each file. What happens when one specifies both `--backup-dir` and `dest`.  Is `--backup-dir` a replacement to `dest`, or not ? From: Charles To:

Utility of --backup

2021-07-20 Thread Lisa via rsync
From: Charles To: lisa-as...@perso.be;    rsync@lists.samba.org Subject: Re: Utility of --backup Date: 20/07/2021 02:37:50 Europe/Paris > And let me do what you suggest. What is the difference, and is --backup > better than the other ? Only in conjunction with -backup-dir, for example --backup

Re: Utility of --backup

2021-07-19 Thread Charles via rsync
> And let me do what you suggest. What is the difference, and is --backup > better than the other ? Only in conjunction with -backup-dir, for example --backup --backup-dir=_Changed and deleted files/2021/Jul/19@21:21 The man page's entry for --backup suggests using it with either

Re: Utility of --backup

2021-07-19 Thread Dan Stromberg via rsync
In Backup.rsync, which of course is a wrapper around rsync that can be used for backups, I do not use --backup, but I do use --link-dest: https://stromberg.dnsalias.org/~strombrg/Backup.remote.html On Sun, Jul 18, 2021 at 11:13 PM Lisa via rsync wrote: > I would like some feedback about the

Utility of --backup

2021-07-19 Thread Lisa via rsync
From: Charles via rsync To: rsync@lists.samba.org Subject: Re: Utility of --backup Date: 19/07/2021 14:26:59 Europe/Paris >IThe --backup option is great for creating "rolling full" backups which >look exactly like the backed up tree except for the existence of the >backup directory I am not

Re: Utility of --backup

2021-07-19 Thread Charles via rsync
IThe --backup option is great for creating "rolling full" backups which look exactly like the backed up tree except for the existence of the backup directory Here's how a Linux backup directory tree looks as created by backup utility bung's bu_rsync script +-- bin -> usr/bin +-- boot |  

Utility of --backup

2021-07-19 Thread Lisa via rsync
Correct, the --backup is almost never discussed.  Have never used it, but as I have been looking at the documentation, I have started to wonder about its usefulness. From: Robin Lee Powell To: lisa-as...@perso.be Subject: Re: Utility of --backup Date: 19/07/2021 08:33:45 Europe/Paris Cc:

Utility of --backup

2021-07-19 Thread Lisa via rsync
I would like some feedback about the --backup option in rsync. Is it worth using it for backups, or should I just use rsync commands that just transfer files without the use of --backup option? -b, --backup  make backups (see --suffix & --backup-dir) --backup-dir=DIR  make backups into

[SCM] The rsync repository. - branch master updated

2021-07-10 Thread Rsync CVS commit messages
The branch, master has been updated via 35d4f673 Update the options in rrsync. from 291a042b Support --crtimes on Cygwin. https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log - commit

[SCM] The rsync repository. - branch master updated

2021-07-08 Thread Rsync CVS commit messages
The branch, master has been updated via 291a042b Support --crtimes on Cygwin. via 9dad3721 Make whole-line comments clearer. from dbb1c2d1 Set whole_file = 0 when whole_file < 0. Fixes issue 114. https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log

[SCM] The rsync repository. - branch master updated

2021-07-04 Thread Rsync CVS commit messages
The branch, master has been updated via dbb1c2d1 Set whole_file = 0 when whole_file < 0. Fixes issue 114. via e8e34ed6 Need to also check stdout_format_has_i in some INFO-NAME checks. from c529782a Fix compiling without ftruncate.

[SCM] The rsync repository. - branch master updated

2021-06-27 Thread Rsync CVS commit messages
The branch, master has been updated via c529782a Fix compiling without ftruncate. via 2dfd4849 fix man page typo from a6bdf313 Unset DISPLAY in environment. https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log

Re: config

2021-06-25 Thread Joe via rsync
Part of the genius of Linux is the concept that each tool should do one thing well and with pipes it is extremely easy to chain these tools together in combinations unique to the current requirements of any task. I can't speak for the rsync developers, but I don't believe they would be open to

Re: config

2021-06-25 Thread Gerhard Obermayr via rsync
Am 25.06.2021 um 17:34 schrieb Knight, Dave: The rsync stdout typically lists directories "considered" with a "/" at the end and lists those files that actually get copied/sync'd by name with no "/" at the end.  If I understand your "problem" correctly, you want to see only the copied files.

Re: config

2021-06-25 Thread Joe via rsync
Once you have the closest superset of what you want to see in your rsync output, run the result through a filter to clean it up before sending it. sed and awk are great for this, but any language/script you are familiar with will do. Having regexes for the pattern matching will make the job

Re: config

2021-06-25 Thread Kevin Korb via rsync
Rsync by default displays nothing. There are more than one options that tell it to display the files it is touching. There are other (and duplicate) options that tell it to show everything. You didn't say what options you are using so we have no idea. Except that you aren't using

config

2021-06-25 Thread Gerhard Obermayr via rsync
Hello everyone, Briefly to my problem - or wish: By lucky coincidence I came into possession of an HP Proliant. My junior installed OpenMediaVault for me. I've been using it as a daily backup for my TeraStation ever since. Rsync is set up for this. Every day I get two emails from this backup,

Re: Permission denied errors with --append-verify run as non-root user

2021-06-04 Thread Kevin Korb via rsync
Short answer: Don't use --append[-verify] If you aren't syncing files that aren't tagged with 'chattr +a' you don't want --append. --append-verify is essentially "I think --append is a good idea to use on general files but rsync keeps corrupting stuff!" but it still allows files to be out of

Permission denied errors with --append-verify run as non-root user

2021-06-04 Thread Knight, Dave via rsync
I noticed that my latest "syncHome" script log scontained a number of unexpected "Permission denied" errors for some modified files. The cause seems to be that those files were read-only at the target. The syncHome script is used to maintain a partial mirror of my home directory tree on another

<    2   3   4   5   6   7   8   9   10   11   >