Re: rsync creates wrong sized files
On Thu, May 20, 2004 at 11:17:58AM -0700, Wayne Davison wrote: Sounds like the rsync daemon you are contacting is running 2.6.1. If so, just remove the -R option and all should be well (it's not needed by the command you're using). Also, since you're using --delete, get rid of the trailing '*' in the source path -- it is inhibiting deletions in the base dir of the transfer (leave a trailing '/'). After that, advise the folks at ftp.uk.debian.org that they should update their rsync to 2.6.2. Many thanks to Wayne Davison for his advice in this. I have made the changes he suggested, and the mirror has worked correctly last night. I shall also advise the maintainer of ftp.uk.debian.org that he needs to update his rsync. This bug can now be considered closed. -- Owen [EMAIL PROTECTED] -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: two-way synchronization accross a firewall fails
This was somehow related to the fact that I was su'd to root, but not with root env ie. su -- - Original Message - From: Tim Harsch [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, May 20, 2004 11:46 AM Subject: two-way synchronization accross a firewall fails machine O is outside firewall, machine I is inside (machine names changed to protect the innocent :-) firewall allows ssh connections if inititiated from I to O, but not if the other way. both machines have an /etc/rsyncd.conf of: [rt] path = /tmp/rsync_test comment = Test area O runs rsync daemon, I initiates a rsync cammnad like rsync -rvvv --delete --rsh=ssh O::rt /tmp/rsync_test which works great, but when flipped and run on machine I, like so: rsync -rvvv --delete --rsh=ssh /tmp/rsync_test O::rt we get rsync -rvvv --delete --rsh=ssh /tmp/rsync_test O::rt opening connection using ssh O rsync --server --daemon . [EMAIL PROTECTED]'s password: building file list ... [sender] make_file(rsync_test,*,2) [sender] expand file_list to 131072 bytes, did move [sender] make_file(rsync_test/t,*,2) [sender] make_file(rsync_test/core,*,2) [sender] clearing per-dir .cvsignore exclude list done send_file_list done send_files starting rsync: connection unexpectedly closed (28 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(342) _exit_cleanup(code=12, file=io.c, line=342): about to call exit(12) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync hangs in cron (not SSH-problem)
On Thu, 2004-05-20 at 20:55, Pascal Nobus wrote: When used this command in cron 00 01 * * * rsync -av --delete /mnt/web1 /mass/kuurne/day 00 02 * * * rsync -av --delete /mnt/web2 /mass/kuurne/day etc.. Not sure if this is your problem or not, but I had some problems running rsync out of fcron ( a cron replacement), due to problems with the current working directory. You might try something like: 00 01 * * * cd /;rsync -av --delete /mnt/web1 /mass/kuurne/day That fixed my problem when running out of cron. Good Luck! -- Jeff -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
question about --bwlimit=
I am doing some benchmarking of rsync. I am using the --bwlimit= option to throttle down rsync to predict its operation over slow communications links. I am using rsync 2.6.2 from the release site without any patches. I downloaded the release rather than pull from the CVS tree. I have 2 servers wilber (the remote archive) and judy (the local archive) connected with a gig ethernet. I have a file on judy that if I use the following command completes in under 1 second: time rsync -ar --rsh=rsh bluesAlbums/Pilgrim/track1.mp3 wilber://test/bluesAlbums/Pilgrim the track1 file on wilber exists and the track1 file on judy has been touched. The file Track1 is 6.3Meg in size. The checksums file if you do --write-batch is 60K bytes. The difference file is 40k bytes (no differences). I check the modification time on wilber after each transfer to make sure the transfer actually happened. If I use the command time rsync -ar -rsh=rsh --bwlimit=4001 bluesAlbums/Pilgrim/track1.mp3 wilber://test/bluesAlbums/Pilgrim real = 0.70 to 0.90 If I use the command time rsync -ar -rsh=rsh --bwlimit=4000 bluesAlbums/Pilgrim/track1.mp3 wilber://test/bluesAlbums/Pilgrim real = 1m34.000 to 1m35.000 There are no other processes running on either server, and I touch the file on judy each time I repeat the test. I actually have 2 telnet sessions open on judy and in one I repeat the touch and in the other I repeat the command string after I change the --bwlimit= option. I do this to make sure I don't fat finger things. I can repeat this time after time. If --bwlimit is 4000 (ie. 4005, 4025, 4050,5000,7500,1,10) real is in the same range as 4001. If --bwlimit is 4000 or under (ie. 3725, 2000, 1000, 100) real is in the same range as 4000. I can understand bipolar behavior at extremes of bandwidth availability but I cant understand the cutoff being that acute that a 1k difference would yield such a dramatic result. At unlimited bandwidth, the real time is twice what it is for rcp so I believe the 0.70 to 0.90 is correct. ??? Is there something going on with --bwlimits around the value of 4000 that could be causing this sharp break. Wally -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
re: question about --bwlimit=
Since --bwlimit depends upon sleep(1 second), I repeated the experiment with a file that was 383 Megabyte so that when I am running unthrottled it takes significantly longer than a second (ie. ~50 seconds) to complete. I get the same bi-modal behavior but with different values for 4000 and 4001 respectively. The fact that the break point stays fixed isnt intuitive (to me at least). wally -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
File sync/delete question
Hi, I need to solve a specific problem and I think that I need an extension to rsync for this to work :) I have an FTP server in the DMZ and a samba server in my intranet. The ftp server has an incoming directory, where people can put uploads and it is also running rsync in daemon mode. The samba server syncs the incoming directory of the ftp server every minute to make access to the uploaded files easier (I don't want the internal people to deal with ftp/sftp to get uploaded files. In fact, there is also an outgoing directory, that is also on the samba server and mirrored to the ftp server, so incoming should work the same way). My problem is, that any file that is deleted on the samba server is synced again from the ftp server a minute later. I use rsync -rvt $FTPSRV /ftp_mirror/incoming for syncing. The --delete option just doesn't fit here, because I want to delete on the SENDING side, not on mine. Also, still-running uploads would not be completely rsync'd if the file would be deleted while being uploaded. Imagine this happenings: 10:00: file a.txt gets uploaded to ftp server 10:01: file a.txt is rsynced to samba server 10:02: file a.txt gets deleted from samba server (after rsync run) 10:03: file would again be synced :( Now I think that I need a new option for rsync: Transfer all files, and remotely delete those that must have been transferred before and are not here anymore. E.g. rsync -rvt --delete-remote-before `cat last-sync-date` ... last-sync-date is a file that contains the unix timestamp (or any other format) of the last syncronization. It could probably also be created by rsync after successful completion, or created on return code 0 by my script. Again my scenario from above: 10:00: file a.txt gets uploaded to ftp server 10:01: file a.txt is rsynced to samba server 10:02: file a.txt gets deleted from samba server (after rsync run) 10:03: last rsync run was at 10:02, but file a.txt on server was modified at 10:00, so it must already have been transferred -- delete it on server Is this feasible? Is anyone willing to implement this, or shall I try on my own (haha, of course you would like me to do, but I'm asking because I don't have too much time ;-) kind regards Markus Gaugusch -- __/\ Markus Gaugusch \ /ASCII Ribbon Campaign markus(at)gaugusch.at X Against HTML Mail / \ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
CVS update: rsync/patches
Date: Fri May 21 08:13:41 2004 Author: wayned Update of /data/cvs/rsync/patches In directory dp.samba.org:/tmp/cvs-serv19233 Removed Files: write-only.diff Log Message: Removing applied patch. Revisions: write-only.diff 1.7 = NONE http://www.samba.org/cgi-bin/cvsweb/rsync/patches/write-only.diff?rev=1.7 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri May 21 08:27:04 2004 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv21710 Modified Files: authenticate.c checksum.c match.c receiver.c Log Message: Changed sum_init() to take a seed value as an arg instead of always using checksum_init. This fixes an authentication problem in server mode (as pointed out by Craig Barratt). Revisions: authenticate.c 1.30 = 1.31 http://www.samba.org/cgi-bin/cvsweb/rsync/authenticate.c.diff?r1=1.30r2=1.31 checksum.c 1.29 = 1.30 http://www.samba.org/cgi-bin/cvsweb/rsync/checksum.c.diff?r1=1.29r2=1.30 match.c 1.61 = 1.62 http://www.samba.org/cgi-bin/cvsweb/rsync/match.c.diff?r1=1.61r2=1.62 receiver.c 1.78 = 1.79 http://www.samba.org/cgi-bin/cvsweb/rsync/receiver.c.diff?r1=1.78r2=1.79 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri May 21 08:40:25 2004 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv24022 Modified Files: util.c Log Message: Moved a few externs. Revisions: util.c 1.143 = 1.144 http://www.samba.org/cgi-bin/cvsweb/rsync/util.c.diff?r1=1.143r2=1.144 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync/patches
Date: Fri May 21 09:18:48 2004 Author: wayned Update of /data/cvs/rsync/patches In directory dp.samba.org:/tmp/cvs-serv30402 Modified Files: verify-patches Log Message: Strip off the new a/ and b/ path prefixes in the diff headers. Revisions: verify-patches 1.9 = 1.10 http://www.samba.org/cgi-bin/cvsweb/rsync/patches/verify-patches.diff?r1=1.9r2=1.10 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri May 21 09:29:17 2004 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv32076 Modified Files: NEWS Log Message: Document a couple more changes. Revisions: NEWS1.178 = 1.179 http://www.samba.org/cgi-bin/cvsweb/rsync/NEWS.diff?r1=1.178r2=1.179 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync/patches
Date: Fri May 21 09:32:50 2004 Author: wayned Update of /data/cvs/rsync/patches In directory dp.samba.org:/tmp/cvs-serv466 Modified Files: checksum-seed.diff Log Message: Improved the manpage markup a little. Revisions: checksum-seed.diff 1.2 = 1.3 http://www.samba.org/cgi-bin/cvsweb/rsync/patches/checksum-seed.diff.diff?r1=1.2r2=1.3 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri May 21 09:41:38 2004 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv1389 Modified Files: options.c Log Message: - Improved option_error() to make sure that the user sees the error in daemon mode. - Got rid of some repetitious outputting of the same error-message. Revisions: options.c 1.148 = 1.149 http://www.samba.org/cgi-bin/cvsweb/rsync/options.c.diff?r1=1.148r2=1.149 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri May 21 09:44:32 2004 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv2432 Modified Files: options.c rsync.yo rsync.1 Log Message: Added the --checksum-seed option. Revisions: options.c 1.149 = 1.150 http://www.samba.org/cgi-bin/cvsweb/rsync/options.c.diff?r1=1.149r2=1.150 rsync.yo1.169 = 1.170 http://www.samba.org/cgi-bin/cvsweb/rsync/rsync.yo.diff?r1=1.169r2=1.170 rsync.1 1.185 = 1.186 http://www.samba.org/cgi-bin/cvsweb/rsync/rsync.1.diff?r1=1.185r2=1.186 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync/patches
Date: Fri May 21 09:44:45 2004 Author: wayned Update of /data/cvs/rsync/patches In directory dp.samba.org:/tmp/cvs-serv2471/patches Removed Files: checksum-seed.diff Log Message: Removed applied patch. Revisions: checksum-seed.diff 1.3 = NONE http://www.samba.org/cgi-bin/cvsweb/rsync/patches/checksum-seed.diff?rev=1.3 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri May 21 09:59:49 2004 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv5146 Modified Files: exclude.c Log Message: Got rid of a compiler warning (which was only output by certain compilers). Revisions: exclude.c 1.80 = 1.81 http://www.samba.org/cgi-bin/cvsweb/rsync/exclude.c.diff?r1=1.80r2=1.81 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri May 21 10:00:21 2004 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv5687 Modified Files: NEWS Log Message: Another new option. Revisions: NEWS1.179 = 1.180 http://www.samba.org/cgi-bin/cvsweb/rsync/NEWS.diff?r1=1.179r2=1.180 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
