Re: 3.2.6: ERROR: rejecting unrequested file-list

2022-09-13 Thread Carlos Carvalho via rsync
Wayne Davison (wa...@opencoder.net) wrote on Tue, Sep 13, 2022 at 02:54:37AM -03: > It helps to have some kind of a reproducer. Also, keep in mind there are 2 > kinds of rejections: unrequested args and unfiltered files. The latter > should have a big improvement in the latest git version. If

3.2.6: ERROR: rejecting unrequested file-list

2022-09-12 Thread Carlos Carvalho via rsync
I've just upgraded the Debian package to 3.2.6 and now I'm getting many cases of this error. I see that there have been improvements in the check but it seems they're either not enough or incompatible with 3.1.[23]. This is always with --files-from, used for mirroring. We're a large free software

duplicated --itemize-changes hangs since 3.1.1pre1

2014-07-05 Thread Carlos Carvalho
One of our rsync updates has been getting stuck at the download point. It gets the incremental list, deletes stuff but when it tries to download new/updated files it gets stuck and aborts after the timeout; it doesn't even create the temporary. Experimentation shows that it happens because the

Re: duplicated --itemize-changes hangs since 3.1.1pre1

2014-07-05 Thread Carlos Carvalho
Kevin Korb (k...@sanitarium.net) wrote on 5 July 2014 19:58: I couldn't duplicate using 3.1.1 or 3.1.1pre2 uploading to a system running 3.0.9. Both were Gentoo Linux. Thanks for trying. As I said, it's strange because we use rsync with dozens of upstreams but it only hangs with one site:

Re: problem with rsync-daemon via ssh

2014-02-19 Thread Carlos Carvalho
Kevin Korb (k...@sanitarium.net) wrote on 17 February 2014 21:13: OK, I just did a quickie test. When I 'ssh user@host rsync --daemon - --server .' an I don't have an rsyncd.conf file in my home dir I get no output like you do (an error message here would be nice). If I do have a proper

Re: problem with rsync-daemon via ssh

2014-02-17 Thread Carlos Carvalho
] _exit_cleanup(code=5, file=main.c, line=1635): about to call exit(5) [sender] sending MSG_ERROR_EXIT with exit_code 5 [sender] send_msg_int(86, 5) So rsync is run at the remote end but doesn't answer with the server greeting the client expects. On 02/14/2014 06:34 AM, Carlos Carvalho wrote: Kevin Korb

Re: problem with rsync-daemon via ssh

2014-02-14 Thread Carlos Carvalho
changed the name, put the following in it [module] path = /path/to/home/transfer and I still get the same did not see server greeting error. On 02/13/2014 08:38 PM, Carlos Carvalho wrote: I'm trying to transfer something to another machine launching a once-only daemon through ssh

Re: problem with rsync-daemon via ssh

2014-02-14 Thread Carlos Carvalho
Matthias Schniedermeyer (m...@citd.de) wrote on 14 February 2014 13:06: You mixed the options from remote shell with rsync daemon. Rsync is used either as 'rsync over SSH'(/remote shell) OR daemon-mode. No, there's a third method. Search for USING RSYNC-DAEMON FEATURES VIA A REMOTE-SHELL

problem with rsync-daemon via ssh

2014-02-13 Thread Carlos Carvalho
I'm trying to transfer something to another machine launching a once-only daemon through ssh with this command: rsync -avv -e ssh -l user ./orig/ machine::module/ where module is the name of a file in the home dir of user with the following: path = /path/to/home/transfer The ssh connection

Re: segfault in 3.1.1dev

2013-11-01 Thread Carlos Carvalho
Happened again: Oct 31 15:55:49 sagres kernel: denied resource overstep by requesting 8392704 for RLIMIT_STACK against limit 8388608 for /usr/local/bin/rsync[rsync:80954] uid/euid:1011/1011 gid/egid:1011/1011, parent /usr/local/bin/rsync[rsync:78330] uid/euid:1011/1011 gid/egid:1011/1011 Oct

name too long problem?

2012-02-15 Thread Carlos Carvalho
In the latest 3.1 I get this in our backup: filename overflows max-path len by 1: path filename overflows max-path len by 1: path filename overflows max-path len by 9: path filename overflows max-path len by 7: path filename overflows max-path len by 4: path filename overflows max-path len by 5:

Re: name too long problem?

2012-02-15 Thread Carlos Carvalho
Benjamin R. Haskell (rs...@benizi.com) wrote on 15 February 2012 09:46: On Wed, 15 Feb 2012, Carlos Carvalho wrote: In the latest 3.1 I get this in our backup: filename overflows max-path len by 1: path filename overflows max-path len by 1: path filename overflows max-path len by 9

Re: name too long problem?

2012-02-15 Thread Carlos Carvalho
Kevin Korb (k...@sanitarium.net) wrote on 15 February 2012 15:12: It seems possible that it is the temporary file name. Yes. After sending the other message I noticed that the amounts rsync lists as exceeding max-path cannot be explained by a prefix at the sender: Carlos Carvalho (car

Re: delete files

2011-11-11 Thread Carlos Carvalho
Brian K. White (br...@aljex.com) wrote on 11 November 2011 15:59: Is there any way to specify a file to be deleted on the remote side explicitly by name? I would have thought that deleting the file locally, the rsyncing the non-existing file with --del would have resulted in the remote

Re: Use rsync's checksums to deduplicate across backups

2011-11-03 Thread Carlos Carvalho
Alex Waite (alexq...@gmail.com) wrote on 2 November 2011 20:09: Recently I learned that rsync does a checksum of every file transferred. I thought it might be interesting to record the path and checksum of each file in a table. On future backups, the checksum of a file being backed up

Re: use-after-free in open_socket_out

2011-07-13 Thread Carlos Carvalho
Wayne Davison (way...@samba.org) wrote on 12 July 2011 16:51: On Tue, Jul 12, 2011 at 3:12 PM, Carlos Carvalho car...@fisica.ufpr.br wrote: The new feature is indeed very useful but something else is needed: print the IP of the server even when the connection is successful, and also

Re: use-after-free in open_socket_out

2011-07-13 Thread Carlos Carvalho
*sigh* forgot to keep the error... Here's the correct patch, tested this time. Sorry for the noise. patch-2 Description: Binary data -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync

Re: use-after-free in open_socket_out

2011-07-12 Thread Carlos Carvalho
This is to ask for a slight enhancement in the new multi-protocol IP printing. I'm keeping the bug thread because in our machines it revealed another problem with the code that's already there. The new feature is indeed very useful but something else is needed: print the IP of the server even

Re: inefficient: --checksum calculation shouldn't be done for new files

2011-07-11 Thread Carlos Carvalho
Wayne Davison (way...@samba.org) wrote on 4 July 2011 17:10: On Sat, Jul 2, 2011 at 5:46 PM, Carlos Carvalho car...@fisica.ufpr.br wrote: When --checksum is used they're calculated in both ends to see if the file should be transfered. This is of course not necessary if the file

use-after-free in open_socket_out

2011-07-11 Thread Carlos Carvalho
Since 3.0.8 (the same in 3.1) we have this around line 300 in socket.c: break; } freeaddrinfo(res0); if (s 0) { char buf[2048]; for (res = res0, j = 0; res; res = res-ai_next, j++) { if (errnos[j] == 0) continue; if

Re: inefficient: --checksum calculation shouldn't be done for new files

2011-07-03 Thread Carlos Carvalho
Jamie Lokier (ja...@shareable.org) wrote on 4 July 2011 00:00: Carlos Carvalho wrote: When --checksum is used they're calculated in both ends to see if the file should be transfered. This is of course not necessary if the file doesn't exist in the destination. However, the checksum is still

inefficient: --checksum calculation shouldn't be done for new files

2011-07-02 Thread Carlos Carvalho
When --checksum is used they're calculated in both ends to see if the file should be transfered. This is of course not necessary if the file doesn't exist in the destination. However, the checksum is still calculated by the sender, which is often a very large overhead. Would it be possible to

Re: Using rsync as an incremental backup

2011-06-24 Thread Carlos Carvalho
Scott Baker (sc...@perturb.org) wrote on 23 June 2011 15:30: I'm using rsync to do an incremental backup of my desktop here, to a remote server as follows: #/usr/bin/bash old=$(date -d 'now - 1 week' +%Y-%m-%d) new=$(date +%Y-%m-%d) rsync -avP --delete --link-dest=../$dir /home/bakers

Re: show how much bytes transfered per rsynced file?

2011-06-17 Thread Carlos Carvalho
peter pilsl (pi...@goldfisch.at) wrote on 17 June 2011 15:22: While its easy to make rsync print the size of the files it is transferring I didnt figure out how to determine how much bytes are actually transferred per file which is even more interesting. Use %b in --out-format. -- Please

Re: Multi-Threading?

2011-05-17 Thread Carlos Carvalho
Matt McCutchen (m...@mattmccutchen.net) wrote on 17 May 2011 12:50: On Tue, 2011-05-17 at 08:45 -0700, cac...@quantum-sci.com wrote: The connexion is Gb enet end-to-end, and is running at only 40Mb/s. It has far more capacity than that. The only limiting factor I can see is on the backup

extraneous in commit 3bd9f51917ed5718275c6132006be155239a0550

2011-02-22 Thread Carlos Carvalho
In this commit you put extraneous hard-linked files. I find this confusing because the hard-linked entries may have been created by rsync itself. I think the previous version is better. Or you could put If there are files that are hard-linked at the destination but not at the origin. -- Please

Re: -r required when -a already specified in --files-from?

2010-09-30 Thread Carlos Carvalho
Kulkarni Shantanu (m...@lists.shantanukulkarni.org) wrote on 30 September 2010 22:05: I am using rsync v.3.0.7 (protocol v30) on ubuntu 10.04. My rsync line looks like this, cd /home/shantanu/ rsync -av --files-from=bin/include-to-rsync.dat /home/shantanu/ \

RE: --delete-during acts like --delete-before

2010-09-09 Thread Carlos Carvalho
Craig Bell (craig.b...@standard.com) wrote on 9 September 2010 18:26: Matthias Schniedermeyer wrote: My observation of current rsync in Incremental Recursion-Mode (default), with default --delete, is that the files it deletes are about 1000 files further down the list than the file it

Re: does rsync work in lockstep? parallel could improve high latency performance

2010-08-19 Thread Carlos Carvalho
travis+ml-rs...@subspacefield.org (travis+ml-rs...@subspacefield.org) wrote on 19 August 2010 12:50: Oh, and if it buffers requests sent to its STDIN, that's fine. To actually complete requests in parallel, you'd need a multithread or multiprocess app, and that's complicated. The proper

Re: How dangerous is --inplace

2010-06-16 Thread Carlos Carvalho
A B (gentosa...@gmail.com) wrote on 16 June 2010 23:16: How dangerous is acctually the --inplace option if you want to run rsync to update files that are only read and not written to? If they're not written to they don't change so that they don't need to be updated :-) What is the worst that

Re: Source files in use or modified while transfer

2010-06-03 Thread Carlos Carvalho
HomeRun4711 (homerun4...@googlemail.com) wrote on 3 June 2010 11:27: I have a question on rsync's way to deal with files in use or files that are modified on the source while they are transferrred to the destination. [removed] But what if e.g. a large mailbox file is changed on the source

3.1: bug in --stats

2010-05-03 Thread Carlos Carvalho
3.1, including the latest version, produces bogus ---stats output: % rsync -r --stats . /tmp/3.1-n % tail -n 17 /tmp/3.1-n Number of files: 3,159 (reg: 2,984, dir: 173, link: 2) Number of created files: 0 Number of deleted files: 0 Number of regular files transferred: 0 Total file size:

Re: Character escaping in item name

2010-04-19 Thread Carlos Carvalho
Richard Taubo (o...@bergersen.no) wrote on 19 April 2010 14:48: In the rsync man page under -8, --8-bit-output, it says: . . . All control characters (but never tabs) are always escaped, regardless of this option's setting. The escape idiom that started in 2.6.7 is to

Re: Rsync - timeout or similar

2010-03-06 Thread Carlos Carvalho
Eduardo Terzella (eterze...@gmail.com) wrote on 6 March 2010 21:05: Is there any parameter in the rsync binary that I can finish the time the binary should wait to start a file transfer, otherwise close the rsync binary. As I read the timeout is a parameter used when the connection has already

Re: Same hang with 3.0.7

2010-02-26 Thread Carlos Carvalho
Whit Blauvelt (w...@transpect.com) wrote on 26 February 2010 14:22: A bit more on the hang, which continues to occur at any attempt to use rsync for that part of the directory tree: It's always in the same place on the progress display to the terminal. But it's not quite to the end, as I'd

timeout gets doubled in 3.1

2010-02-11 Thread Carlos Carvalho
The timeout value actually gets doubled in 3.1 (didn't check in 3.0). It first happens between sender and destination, producing an error message. However in the destination one of the processes continues to run, and only times out and exits after another timeout interval. So in practice all

wrong file size in the download log?!

2010-01-16 Thread Carlos Carvalho
I use %l in --out-format to have the file size in the download log. However for some files what rsync puts in the log is different from the real size. Sometimes it's larger, sometimes smaller. It happens with the latest v3.1 in the destination and 3.0.6 in the origin/sender. The md5sum reported by

timeout changes in 3.1 working

2010-01-08 Thread Carlos Carvalho
It seems the latest timeout changes in 3.1 work well. I was having repeated timeout aborts in a sync of large files through a very slow connection, which made each file transfer take very long. The latest version no longer aborts and the sync finished successfully. The destination used version 3.1

Re: Rsync performance with very large files

2010-01-08 Thread Carlos Carvalho
Eric Cron (ericc...@yahoo.com) wrote on 8 January 2010 12:20: We're having a performance issue when attempting to rsync a very large file. Transfer rate is only 1.5MB/sec. My issue looks very similar to this one: http://www.mail-archive.com/rsync@lists.samba.org/msg17812.html In

strange error code with 3.1

2009-12-23 Thread Carlos Carvalho
Got this with rsync --list-only --files-from=list directory dummy output 2 errors to get a local file list: ... rsync: link_stat file-name failed: No such file or directory (2) rsync: link_stat file-name failed: No such file or directory (2) rsync error: some files/attrs were not transferred

possible incompatibility in 3.1 with 3.0.6

2009-12-23 Thread Carlos Carvalho
Got this strange error: receiving incremental file list rsync: opendir /.~tmp~ (in sender-module) failed: Permission denied (13) IO error encountered -- skipping file deletion rsync: opendir /path/.~tmp~ (in sender-module) failed: Permission denied (13) invalid multi-message 22:4 [sender/inc]

rsync stuck in infinite loop? (was How do I make rsync ignore unreadable files (damaged sectors)?)

2009-12-23 Thread Carlos Carvalho
Wayne Davison (way...@samba.org) wrote on 22 December 2009 08:02: On Tue, Dec 22, 2009 at 4:18 AM, Tomas Gustavsson tompl...@gmail.com wrote: Still, I do think that rsync should give up after a long time, but it doesn't. Yeah, if it gets a read error for a part of a file, it substitutes

Re: rsync, excluding files w/certain characteristics

2009-11-07 Thread Carlos Carvalho
Edward Peschko (horo...@gmail.com) wrote on 6 November 2009 18:46: I was wondering - is there an option to rsync which allows you to exclude transfers of files with a certain size or other characteristics? I want to setup a 'partial' rsync mirror where I only pull files under 5M, for

read errors mapping /path/file: No data available (in version 3.0.6)

2009-11-02 Thread Carlos Carvalho
Got this in the log: WARNING: path/file failed verification -- update discarded (will try again). rsync: read errors mapping /path/file: No data available (61) The first line is rare but understandable. It's the second one that I find strange. What does no data available mean? The file exists on

exit status 13 in version 3.1

2009-10-29 Thread Carlos Carvalho
Got this in the log: rsync error: errors with program diagnostics (code 13) at log.c(340) [generator= 3.1.0dev] What could it be? I suspect it's triggered by a timeout or disconnect from the server side but I had never seen it. -- Please use reply-all for most replies to avoid omitting the

Re: exit status 13 in version 3.1

2009-10-29 Thread Carlos Carvalho
Carlos Carvalho (car...@fisica.ufpr.br) wrote on 29 October 2009 13:20: Got this in the log: rsync error: errors with program diagnostics (code 13) at log.c(340) [generator= 3.1.0dev] Another event: rsync: read error: Connection reset by peer (104)rsync error: errors with program

Re: strange log values in 3.1

2009-10-26 Thread Carlos Carvalho
Wayne Davison (way...@samba.org) wrote on 25 October 2009 22:33: On Sun, Oct 25, 2009 at 6:26 PM, Carlos Carvalho car...@fisica.ufpr.brwrote: All files are new. What I find strange is that I'd expect the transfered bytes to be somewhat larger than the file sizes Yeah, there are two

Re: bugs in version 3.1

2009-10-25 Thread Carlos Carvalho
Wayne Davison (way...@samba.org) wrote on 24 October 2009 15:32: On Fri, Oct 23, 2009 at 2:41 PM, Carlos Carvalho car...@fisica.ufpr.brwrote: The first is that when running with --files-from and -ii unmodified files are not put in the log. That is not a logging issue, but rather

strange log values in 3.1

2009-10-25 Thread Carlos Carvalho
I've just got the log lines below. The first line shows the format spec that prints each column: %'b %'l %f 0 18,760 f1 0 11,193 f2 32,768 2,186 f3 3,128,368 3,453,856 f4 13,680 11,786 f5 32,768 6,288

bugs in version 3.1

2009-10-23 Thread Carlos Carvalho
I'm having two problems. The first is that when running with --files-from and -ii unmodified files are not put in the log. --out-format=%-14b %C %-14l %i %B %M %f All that appears in the log is Number of files: 0 Number of created files: 0 Number of regular files transferred: 0 Total file size:

differences between 3.0.6 and 3.1

2009-10-23 Thread Carlos Carvalho
The 3.1 manual says in the description of --human-readable: Backward compatibility note: versions of rsync prior to 3.1.0 do not support human-readable level 1, and they default to level 0. Thus, specifying one or two -h options behaves the same in old and new

Re: avoid rsync deleting vanished files on receiver

2009-10-23 Thread Carlos Carvalho
Hans-Christian Jehg (h...@jehg.dk) wrote on 23 October 2009 17:32: The circumstance where this happens is now known, it is when requests to the shared in files returns permission denied from the file server... [snip] I am not at all sure where it goes wrong, but in the end the result is that

how can this be: file has vanished in --list-only?

2009-10-15 Thread Carlos Carvalho
I'm getting file has vanished messages during a (recursive) --list-only. I find it strange because I'd expect rsync to access each file only once when just sending the receiver the file list. In which circumstances can this happen? This is with 3.0.6 in linux. -- Please use reply-all for most

Re: rsync with danish chars in filename

2009-09-17 Thread Carlos Carvalho
Tomas Norre Mikkelsen (tnm.p...@gmail.com) wrote on 17 September 2009 09:31: I have a Qnap NAS-219 used for storage, i have a Ubuntu 9.04 server for backup. When rsync between them filenames containing ø have problemes, the danish chars æ å does not have any problems eventhough its a special

Re: DO NOT REPLY [Bug 6741] 'deleting' messages show up in improper places

2009-09-17 Thread Carlos Carvalho
samba-b...@samba.org (samba-b...@samba.org) wrote on 17 September 2009 09:24: The 'deleting' messages below are showing up in the middle of a transfer. It shows files being deleted and the progress bar appears to detail the deleting process. Usually rsync handles files in alphanumeric order,

Re: Proposed tweaks

2009-09-17 Thread Carlos Carvalho
Lee Winter (lee.j.i.win...@gmail.com) wrote on 16 September 2009 01:16: The use case that needs some optimization is that of online repositories -- mirrors. In contrast to other kinds of usage such as file synchronization, replication, backup, etc., mirrors present a quite different set of

Re: rsync deleted all my important docs...

2009-09-14 Thread Carlos Carvalho
Matt McCutchen (m...@mattmccutchen.net) wrote on 13 September 2009 16:53: On Sat, 2009-09-12 at 18:22 -0300, Carlos Carvalho wrote: Sanjay Acharya (svachary...@gmail.com) wrote on 11 September 2009 11:02: How is rsync supposed to behave when I give the --delete-excluded? From the manual

Re: rsync deleted all my important docs...

2009-09-12 Thread Carlos Carvalho
Sanjay Acharya (svachary...@gmail.com) wrote on 11 September 2009 11:02: How is rsync supposed to behave when I give the --delete-excluded? From the manual for the --delete option: If the sending side detects any I/O errors, then the deletion of any files at the destination will be

Re: transfer set nano-second times on files

2009-09-07 Thread Carlos Carvalho
samba-b...@samba.org (samba-b...@samba.org) wrote on 7 September 2009 16:21: way...@samba.org changed: What|Removed |Added Severity|major

Re: rsync algorithm for large files

2009-09-04 Thread Carlos Carvalho
Matthias Schniedermeyer (m...@citd.de) wrote on 5 September 2009 00:34: On 04.09.2009 18:00, ehar...@lyricsemiconductors.com wrote: Why does it take longer the 3rd time I run it? Shouldn?t the performance always be **at least** as good as the initial sync? Not per se. First you have

Re: rsync and symlinks

2009-09-01 Thread Carlos Carvalho
Jignesh Shah (jignesh.shah1...@gmail.com) wrote on 1 September 2009 11:30: Hi, I have one question regarding handling of symlinks in rsync. If I have /source directory in which I have 100s of sub directories of directories and each one has symlink to some directories out side the /source

Re: rsync and symlinks

2009-09-01 Thread Carlos Carvalho
Jignesh Shah (jignesh.shah1...@gmail.com) wrote on 1 September 2009 22:17: Thanks Carlos. ... rsync copy symlinks as symlinks What about the contents pointed by simlinks? If they exist on the destination at the same place the symlink will point to them, otherwise they'll dangling there (at

Re: partial-dir not being used?!

2009-08-22 Thread Carlos Carvalho
Wayne Davison (way...@samba.org) wrote on 22 August 2009 09:15: On Fri, Jul 17, 2009 at 02:13:00AM -0300, Carlos Carvalho wrote: There can be several partials in the partial dir for the same file, resulting from more than one interruption. No, there is only one partial file. Don't confuse

Fwd: Re: rsyncd.conf chroot yes problem with symlink-ing

2009-08-21 Thread Carlos Carvalho
alex.vladule...@csteam.ro (alex.vladule...@csteam.ro) wrote on 14 August 2009 15:32: Ok it's now Solved ! In spite of trying all day long to figure it out what's wrong in my sintax i could find, ...using a pencile and a pice of paper, combining all commands parameters invoked so far and,

Re: rsync speedup - how ?

2009-08-08 Thread Carlos Carvalho
devz...@web.de (devz...@web.de) wrote on 6 August 2009 20:15: i`m using rsync to sync large virtual machine files from one esx server to another. the speed is reasonable, but i guess it`s not the optimum - at least i donŽt know where the bottleneck is. That's vague and subjective, so

Re: Need help of exclusion options in rsync-3.0.6

2009-07-27 Thread Carlos Carvalho
Matt McCutchen (m...@mattmccutchen.net) wrote on 26 July 2009 17:45: On Sun, 2009-07-26 at 21:37 +0530, Jignesh Shah wrote: I have a situation where I want to delete some of my excluded patterns but still want to preserve some other. For example consider below source and destination

Re: Rsnapshot/rsync buffer overflow

2009-07-18 Thread Carlos Carvalho
Steven Levine (stev...@earthlink.net) wrote on 17 July 2009 19:51: In c1e87fb7-a3b8-4aa9-93f8-bff0effd9...@gmail.com, on 07/17/09 at 01:46 PM, Kurt Nelson kurtisnel...@gmail.com said: Hi Kurt, Please reply to the list... Any good workarounds? Depends on your needs. If you really

Re: partial-dir not being used?!

2009-07-17 Thread Carlos Carvalho
Carlos Carvalho (car...@fisica.ufpr.br) wrote on 17 July 2009 02:13: Matt McCutchen (m...@mattmccutchen.net) wrote on 16 July 2009 20:59: On Thu, 2009-07-16 at 01:48 -0300, Carlos Carvalho wrote: What's the purpose of --partial-dir then? I thought it was leaving the partial transfer

Re: partial-dir not being used?!

2009-07-16 Thread Carlos Carvalho
Matt McCutchen (m...@mattmccutchen.net) wrote on 16 July 2009 20:59: On Thu, 2009-07-16 at 01:48 -0300, Carlos Carvalho wrote: What's the purpose of --partial-dir then? I thought it was leaving the partial transfer as path/partial/filename instead of path/.filename.random. That's what

partial-dir not being used?!

2009-07-15 Thread Carlos Carvalho
Here's a running instance caught by ps (lines broken for readibility): rsync --perms --times --timeout=3600 --stats --no-motd -hh --force --hard-links -ii --links --partial-dir=.~tmp~ --recursive --exclude some excludes --out-format=format --delete-during --max-delete=75000 --ignore-errors source

Re: partial-dir not being used?!

2009-07-15 Thread Carlos Carvalho
Matt McCutchen (m...@mattmccutchen.net) wrote on 15 July 2009 18:50: On Wed, 2009-07-15 at 17:01 -0300, Carlos Carvalho wrote: Here's a running instance caught by ps (lines broken for readibility): rsync --perms --times --timeout=3600 --stats --no-motd -hh --force --hard-links -ii

Re: are new directories created inside the partial dir?

2009-07-14 Thread Carlos Carvalho
Carlos Carvalho (car...@fisica.ufpr.br) wrote on 13 July 2009 20:08: When --partial-dir is used rsync creates new or updated files inside a temporary dir. For example the new version of some/path/file is created in some/path/partial-dir-name/file and later moved. What happens if a new

Re: Assistance With Symlink Has No Referent Message

2009-07-14 Thread Carlos Carvalho
Jon Watson (jwat...@watsysgroup.com) wrote on 14 July 2009 17:17: I am using rsync to backup one server to another. I am connecting to the rsync daemon on the remote server and pulling to the backup server. When using SSH this error does not occur, but there are reasons why I have to use

Re: How to make big MySQL database more diffable/rsyncable? (aka rsyncing big files)

2009-07-14 Thread Carlos Carvalho
Ryan Malayter (malay...@gmail.com) wrote on 14 July 2009 17:00: On Mon, Jul 13, 2009 at 4:54 PM, Jamie Lokierja...@shareable.org wrote: Remembering hashes doesn't make any difference to speed, if the bottleneck is the sending side. Except that in the rsync pipeline, the reading the

Re: Assistance With Symlink Has No Referent Message

2009-07-14 Thread Carlos Carvalho
Jon Watson (jwat...@watsysgroup.com) wrote on 14 July 2009 19:15: Carlos Carvalho wrote: Jon Watson (jwat...@watsysgroup.com) wrote on 14 July 2009 17:17: I am using rsync to backup one server to another. I am connecting to the rsync daemon on the remote server and pulling to the backup

Re: Assistance With Symlink Has No Referent Message

2009-07-14 Thread Carlos Carvalho
Jon Watson (jwat...@watsysgroup.com) wrote on 14 July 2009 20:14: That was it, thanks. Interestingly enough, the version of rsync on the backup server supports use chroot, but there is no mention of it in the man pages that I found. It's in the manual for the daemon: rsyncd.conf. -- Please

are new directories created inside the partial dir?

2009-07-13 Thread Carlos Carvalho
When --partial-dir is used rsync creates new or updated files inside a temporary dir. For example the new version of some/path/file is created in some/path/partial-dir-name/file and later moved. What happens if a new directory is created? If some/path/newdir/newfile is to be copied, is it done in

Re: sync performance falls off a cliff

2009-07-05 Thread Carlos Carvalho
Leen Besselink (l...@consolejunky.net) wrote on 5 July 2009 10:17: I'm no expert, but I suggest using rsync 3.x (3.0.6 for example), it doesn't keep the as much information of the filelist in memory. Yes. Or at lease it starts transfers much faster, because it doesn't wait for the

Re: sync performance falls off a cliff

2009-07-04 Thread Carlos Carvalho
Leen Besselink (l...@consolejunky.net) wrote on 30 June 2009 09:05: Mike Connell wrote: Hi, Hi Mike, I've got identical servers. One is primary the other is backup receiving rsyncs from the primary. I'm backing up a file system to disk and the files are small and there are lots

Re: are partial files always deleted on interruption?

2009-07-04 Thread Carlos Carvalho
Wayne Davison (way...@samba.org) wrote on 4 July 2009 08:53: On Fri, Jul 03, 2009 at 03:32:29PM -0300, Carlos Carvalho wrote: Both processes were killed with the same command, kill pid1 pid2. A SIGTERM (the default kill signal) should be caught by rsync, allowing it to cleanup before

Re: rsync getting stuck while building file list

2009-07-04 Thread Carlos Carvalho
Water Kingdom (waterking...@gmail.com) wrote on 3 July 2009 15:49: We are using rysnc utility for transferring ~90GB data across the machines. We noticed that for small folders, rsync is working fine but the moment we switch to 90GB folder, it gets stuck at building file list. We even left it

are partial files always deleted on interruption?

2009-07-03 Thread Carlos Carvalho
The manual says that a file being transfered is removed if rsync gets an interruption and --partial is not used or implied by other options like partial-dir or delay-updates. However I've found that a file was left after interruption, with a zero size. Would it be possible that rsync doesn't

what happens with .~tmp~ after an interruption?

2009-07-03 Thread Carlos Carvalho
When run with --delay-updates rsync keeps changed files in directories .~tmp~ (or the name given with --partial-dir) in each directory until downloading is finished, then moves them to place and removes .~tmp~. If an interruption occurs, they're left as is and in the next run they'll be used to

Re: could not make way for new symlink !!!

2009-06-05 Thread Carlos Carvalho
Jignesh Shah (jignesh.shah1...@gmail.com) wrote on 5 June 2009 18:16: Could any one please help me to get rid of below errors? ERROR: DoRsync(): rsync had errors or failed rsync return code: 23 could not make way for new symlink: bin could not make way for new

Re: WARNING: . . . .failed verification -- update discarded (will try again).

2009-06-04 Thread Carlos Carvalho
JW (j...@mailsw.com) wrote on 3 June 2009 11:58: A nightly scripted rsync backup job is giving me this error WARNING: vm/escDebLenny14G-flat.vmdk failed verification -- update discarded (will try again). I've searched the web and I think probably the problem is the source file changed

Re: Resuming from partial downloads but keeping complete file

2009-06-01 Thread Carlos Carvalho
Ante Blaskovic (anbla...@globalnet.hr) wrote on 1 June 2009 12:31: I have very-large file to transfer 10 GB and I have problems to transfer it if connection drops often. When I don't use --partial option, It will compare file with version on server and transfer difference but what if it only

Re: How to emulate rdiff behaviour

2009-06-01 Thread Carlos Carvalho
Matt McCutchen (m...@mattmccutchen.net) wrote on 31 May 2009 23:11: On Mon, 2009-06-01 at 07:10 +0800, Daniel.Li wrote: On Sun, 2009-05-31 at 14:34 -0400, Matt McCutchen wrote: And one more thing here: If you are going to prepare this batch file, it seems there will be double the

Re: rsync-3.0.6 prints file list generation time wrong ???

2009-05-31 Thread Carlos Carvalho
Wayne Davison (way...@samba.org) wrote on 30 May 2009 10:12: On Fri, May 29, 2009 at 06:29:22PM +0530, Jignesh Shah wrote: Does that mean rsync traversed complete 1000k files in 0.671 seconds? Rsync uses an incremental recursion scan by default, so when that is active the time mentioned is

--relative, --dirs and --delete without --recursive

2009-05-19 Thread Carlos Carvalho
The manual says that without --recursive directories are (noisily) skiped. However with --relative intermediate directories are created. So what happens if you give only -R? Is -d the default? And what happens when you give -R --no-dirs? I tried and it seems that intermediate dirs are considered

excluding a directory without stating its contents?

2008-11-11 Thread Carlos Carvalho
I need to exclude a directory at the root of the sync tree. It's big, so it's important that both the source and destination don't waste time stating their contents. Should I use --exclude /dir or --exclude /dir/*** or something else? -- Please use reply-all for most replies to avoid

Re: Feature request: save-as-user

2008-09-13 Thread Carlos Carvalho
Quey ([EMAIL PROTECTED]) wrote on 13 September 2008 07:10: Is it possible to request a new feature that will help out some of us doing many mirrors, that is each mirror has their own system uid for security puroposes, it would be of great advantage (to I'm sure very many) to have an

--delete and --files-from (again)

2008-08-14 Thread Carlos Carvalho
I've looked at the previous messages on this issue and I think I need something similar but I'm not sure if it's the same. I'm using --files-from to update a list of new/changed files. However, I also have a list of supposedly removed ones. I can remove them directly but I'd like rsync to check