Re: 2.6.2 rsync --daemon is not working for me
On Tue, Sep 21, 2004 at 12:37:55PM +0500, Sergey Golovin wrote: I'm unable to write with remote rsync in daemon mode. The strace you included shows that it is the fork() call that is returning -1 (EAGAIN). So, you need to figure out why fork() isn't working -- e.g. what resource it thinks is temporarily unavailable. ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Ownership
I just thought of something else that I should have thought of earlier: if you're using a chroot=yes setup for an rsync daemon, you must put the appropriate files into the chroot area for rsync to do any mapping of UIDs and GIDs (as explained in the man rsyncd.conf manpage). For instance, create an etc subdir and put a (possibly stripped down) passwd and group file into it and see if that fixes the problem. If so, you can add an exclude = /etc config item to that module so that users can't copy those files. ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Truncated output from rsync -e ssh ... 21 | tee
(Versions: OpenSSH_3.7.1p2, rsync version 2.6.2) I've just encountered a situation where rsync -v -n appears to run normally, but reports many fewer file transfers than actually get done when you remove the -n. (This is not one of the usual -n corner cases.) It turns out that this only happens when you're doing a remote rsync over ssh AND you redirect stderr into a pipe that fills up, as in rsync -e ssh -avn host:/path /local/path 21 | tee LOG I can get the right answer by just not capturing stderr; i.e. removing the 21 and just saying rsync -avn host:/path /local/path | tee LOG works. The data loss occurs when the pipe (to tee here) fills, so in principle you could lose output even without the -n, it's just less likely when the output is generated slower. After poking around with strace, it seems that rsync's child ssh sets its stdERR non-blocking, and that stderr has been inherited unchanged from the top-level rsync. (The rsync has supplied pipes for its child's stdin and stdout, but left the stderr alone; see rsync-2.6.2/pipe.c::piped_child().) Because of the 21, the top-level stderr is a dup of the top-level stdout, so ssh has inadvertantly made rsync's stdOUT non-blocking. Rsync is not expecting that, and does not check the return code from fflush(stdout), so it can silently drop lines from stdout. (See the end of rsync-2.6.2/log.c::rwrite().) CVS has basically the same problem, as discussed at http://groups.google.com/groups?th=e4df2fdc1f4f4950, which mentions some workarounds that the CVS people considered. It's not clear whether the problem should really be fixed in rsync, ssh, or glibc, but in the meantime, would it be worth adding a warning to the docs/FAQ/known-issues/wherever? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: infinite loop in rsync daemon on Mac OSX
On Tue, Sep 28, 2004 at 11:43:40AM -0400, Anthony DiSante wrote: rsync -avvv --delete --exclude /Volumes macbox::rootpath /mnt/bkup/macbox You need to also exclude this: --exclude=/dev/fd/ Otherwise the OS X filesystem has a loop back to the root dir, which causes rsync to keep scanning more and more files. ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Ownership
How can I check whether rsync uses /etc/nsswitch.conf or not?
I'd suggest finding a Linux forum to ask that question (and first
googling to see if the answer is already out there). I'd also suggest
creating a simple test program that just tries to lookup testuser
using getpwnam() (you could easily tweak the code I sent in a message
just recently as a good start for such a program). Try running that,
ensure that it fails in a similar manner, and then look for information
on getting that test program to work.
It seems to work correctly (warning, ugly piece of code ahead):
#-
#include stdio.h
#include stdlib.h
#include pwd.h
#include sys/types.h
int main(void)
{
char name[9] = testuser\0 ;
uid_t *uid;
struct passwd *pass;
pass = getpwnam(name);
if (pass) {
*uid = pass-pw_uid;
fprintf(stderr,Lookup of `%s' returned %ld\n,name,(long)*uid);
return 1;
}
fprintf(stderr,Lookup of `%s' failed\n,name);
return 0;
}
#-
./name2uid
Lookup of `testuser' returned 1
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Ownership
I just thought of something else that I should have thought of earlier: if you're using a chroot=yes setup for an rsync daemon, you must put the appropriate files into the chroot area for rsync to do any mapping of UIDs and GIDs (as explained in the man rsyncd.conf manpage). For instance, create an etc subdir and put a (possibly stripped down) passwd and group file into it and see if that fixes the problem. If so, you can add an exclude = /etc config item to that module so that users can't copy those files. No, I'm not fond of chrooting (well, at least until every thing else is working...). -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Truncated output from rsync -e ssh ... 21 | tee
On Tue, Sep 28, 2004 at 04:10:13PM +, David Evers wrote:
rsync -e ssh -avn host:/path /local/path 21 | tee LOG
In my test I piped the output to (sleep 10; tail) to ensure a
reproducable truncation.
The attached patch fixes the problem by putting our stderr fd back
into blocking I/O mode. I don't see why ssh should be playing with
our stderr fd in the first place (since we're the one calling ssh,
not the one being run by ssh). Does anyone see a problem with this
change?
..wayne..
--- main.c 17 Sep 2004 16:50:53 - 1.217
+++ main.c 28 Sep 2004 16:42:09 -
@@ -657,6 +657,9 @@ int client_run(int f_in, int f_out, pid_
if (protocol_version = 23 !read_batch)
io_start_multiplex_in();
+ /* Work around a bug in ssh that sets our STDERR to non-blocking. */
+ set_blocking(STDERR_FILENO);
+
if (am_sender) {
keep_dirlinks = 0; /* Must be disabled on the sender. */
io_start_buffering_out();
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Extending --log-format
On Tue, Sep 28, 2004 at 11:42:54AM -0400, Christophe Kalt wrote: i couldn't find anything in my various searches :-( It was surprisingly hard to google for due to him using the phrase report options for the idea. Here are the messages to which I was referring: http://lists.samba.org/archive/rsync/2003-May/006057.html http://lists.samba.org/archive/rsync/2003-May/006058.html i guess my question is do we/you care to preserve the current scheme for cases where extra information is not needed? On the one hand, i would prefer to do away with it for the sake of simplicity, on the other hand, i'll probably need to keep it for backward compatibility anyways. As you say, the old code needs to remain for backward compatibility. J.W. was thinking about using two bitmasks internally to control what gets output in the report information, and I think it wouldn't be that hard to leave the current bifurcated output scheme in rsync as long as these bitmasks got communicated to the other side. ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync 2.6.2 hang (was rsync 2.6.2 crash)
OK, I set up a new PC with a fresh XP load to be my rsync server. Rsync by itself seems to work fine (thank God. I'm going nuts.) I am still having problems when I throw ssh into the mix. At least now, things don't crash, but they do hang. The file list gets build, the transfer starts and it gets through maybe a few hundred files, but then everything stops. All the processes are still there, but all logging stops and CPU utilization by the rsync and ssh processes go to zero. Any thoughts? ---Original Message--- From: jim [EMAIL PROTECTED] Subject: Re: rsync 2.6.2 crash Sent: 24 Sep 2004 16:19:02 I switched both ends back to 2.6.2 and started rsync in daemon mode by hand (no ssh). The sync is still in progress, but it looks like it is working. There were three small problems, however: 2004/09/24 14:25:34 [1896] opendir /17B - CONTROL/17Bx - xxx , xxx x xxx-297 A24A-53A/Drawing Sheet 17Bxx -(44).cal (in MPDM) failed: Not a directory 2004/09/24 14:25:51 [1896] file has vanished: /17B - CONTROL/17Bxx - xx x , xxx x xxx-297 A24A-53A/Drawing Sheet 17Bxx -(89).cal (in MPDM) 2004/09/24 14:29:28 [1896] opendir /17B - CONTROL/17Bxx - , xXxxx-x xx/Drawing Sheet 17B1N5001 B(2.2).cal (in MPDM) failed: Not a directory -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[cygwin] Fwd: Updated: rsync-2.6.2-3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Here goes the announcement of the 2.6.2-3 package for cygwin (available from cygwin's setup itself). If you wonder what 2.6.2-3 means, it's basically the third cygwin package that uses rsync-2.6.2 sources. This third attempt solves both the august security fix and the textmode bug that 2.6.2-2 had. follows the original announcement of the cygwin package (only change I removed the cygwin-ML unsubscribe info as it's not peritent here) - Original Message Subject: Updated: rsync-2.6.2-3 Date: Tue, 28 Sep 2004 17:31:14 +0200 From: Lapo Luchini [EMAIL PROTECTED] Reply-To: The Cygwin Mailing List [EMAIL PROTECTED] Newsgroups: gmane.os.cygwin.announce Followup-To: gmane.os.cygwin Version 2.6.2-3 of the open source utility that provides fast incremental file transfer has been uploaded. If you're not sure what version do you have you can use the following command (if you don't have grep omit the last part): $ cygcheck -cd rsync rsync2.6.2-2 Version 2.6.2-3 is basically the same as 2.6.2-2, but it solves the textmode issue that did corrupt many binary files out there. (I'm still wondering why that bug didn't show in earlier versions, given the fact that it was there since the beginning) Many thanks goes to Sjoerd Mullender for finding (and fixing) the bug while I was still lingering on hot tropical sand 0=) If you're using 2.6.2-1 or earlier version to avoid the textmode bug, you're now strongly advised to upgrade to 2.6.2-4, as it contains the patch to fix august's SECURITY ADVISORY as in: http://rsync.samba.org/#security_aug04 - -- L a p o L u c h i n i l a p o @ l a p o . i t w w w . l a p o . i t / http://www.megatokyo.it -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (Cygwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iEYEARECAAYFAkFZ6j8ACgkQaJiCLMjyUvsSkgCguxiCvhE7+2EXJE7+ccW8EpMD ll4AoNFdGjpbSHphMNht1hLf+ukV5KUk =snxl -END PGP SIGNATURE- -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Why does rsync think my files are much bigger than they are ?
Thanks - I'd compiled it myself with gcc 2.96 on a redhat 7.2 system, and tried reverting to the RedHat RPM and it works fine.Guess it's a compiler issue! Cheers, P Wayne Davison wrote: On Tue, Sep 28, 2004 at 03:47:09PM +1000, Peter Skipworth wrote: send_files mapped /IFX/llog/logs.109051.gz of size 17592186044416 This line comes directly from this code: rprintf(FINFO, send_files mapped %s of size %.0f\n, safe_fname(fname), (double)st.st_size); The st variable is a structure that was populated by stat(), so the problem is either (1) stat() is returning bogus values, or (2) your compiler is not converting the value to a double properly. I think the former is more likely -- e.g. if the size of the variables actually returned by stat() are not what the header files used for the compilation claim they should be. Did you compile rsync yourself? If so, this is a basic compiler problem that should affect more programs than just rsync. If not, you should try compiling your own version and running that. ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
refresh iso
Hi, I downloaded a iso with wget . like .. wget -c http://cdimage.ubuntu.com/releases/warty/preview/warty-i386.iso Now I would like to update/refresh this iso with rsync . like .. rsync -avzP archive.ubuntulinux.org::cdimage/daily/current/warty-i386.iso /home/internet/warty-i386.iso When I try this it seems that it start all over again, The full iso of 533MB suddenly become only 5MB. Well kindly please.. can this be done? If so . how tell me what to expect please thanks -- Johan Sch Registered Linux User #330034 May this be a good day for learning -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Re: rsync 2.6.2 hang (was rsync 2.6.2 crash)
Hey, that's pretty funny. Actually, I would much prefer a Linux environment, but I don't always get what I want. We are primarily a Windows and HP-UX shop, so it's not so easy get the powers that be to buy into a Linux box. In addition, I'm using several sets of in-house libraries for this project that have not been built on Linux. I could easily burn another couple of months setting up build environments and testing the Linux builds. Not going to happen at this point. Our low-end server environment is W2K3, so that's what I'm stuck with. ---Original Message--- From: Chuck Wolber [EMAIL PROTECTED] Subject: Re: rsync 2.6.2 hang (was rsync 2.6.2 crash) Sent: 28 Sep 2004 21:13:47 On Tue, 28 Sep 2004, jim wrote: OK, I set up a new PC with a fresh XP load to be my rsync server. Rsync by itself seems to work fine (thank God. I'm going nuts.) I am still having problems when I throw ssh into the mix. At least now, things don't crash, but they do hang. The file list gets build, the transfer starts and it gets through maybe a few hundred files, but then everything stops. All the processes are still there, but all logging stops and CPU utilization by the rsync and ssh processes go to zero. Any thoughts? Install Linux instead of XP? Seriously, I gotta wonder why you are serving rsync in an environment that it's really not meant to be used in? FWIW; We use rsync (under Linux) to back up folders on Windows machines quite a bit. Works great. -Chuck -- http://www.quantumlinux.com Quantum Linux Laboratories, LLC. ACCELERATING Business with Open Technology The measure of the restoration lies in the extent to which we apply social values more noble than mere monetary profit. - FDR ---Original Message--- -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync 2.6.2 hang (was rsync 2.6.2 crash)
jim wrote: Hey, that's pretty funny. Actually, I would much prefer a Linux environment, but I don't always get what I want. We are primarily a Windows and HP-UX shop, so it's not so easy get the powers that be to buy into a Linux box. In addition, I'm using several sets of in-house libraries for this project that have not been built on Linux. I could easily burn another couple of months setting up build environments and testing the Linux builds. Not going to happen at this point. Our low-end server environment is W2K3, so that's what I'm stuck with. ---Original Message--- From: Chuck Wolber [EMAIL PROTECTED] Subject: Re: rsync 2.6.2 hang (was rsync 2.6.2 crash) Sent: 28 Sep 2004 21:13:47 On Tue, 28 Sep 2004, jim wrote: OK, I set up a new PC with a fresh XP load to be my rsync server. Rsync by itself seems to work fine (thank God. I'm going nuts.) I am still having problems when I throw ssh into the mix. At least now, things don't crash, but they do hang. The file list gets build, the transfer starts and it gets through maybe a few hundred files, but then everything stops. All the processes are still there, but all logging stops and CPU utilization by the rsync and ssh processes go to zero. Any thoughts? Install Linux instead of XP? Seriously, I gotta wonder why you are serving rsync in an environment that it's really not meant to be used in? FWIW; We use rsync (under Linux) to back up folders on Windows machines quite a bit. Works great. -Chuck I'd have thought Chuck suggests you install a Linux backup server. You don't have to run Windows on your backup server do you? Standard Debian or whatever and backuppc will probably handle it. Tell management it's like HP-UX but cheaper and you don't have to face HP delivery times:-) Or Windows viruses. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync and hard links
For those who saw my earlier posts regarding reliability and robustness,
there were several problems;
1. rsync does not retry when there are transmission errors such as
timeouts. I suggest it ought (as wget does): doing so would have enabled
a clean recovery from the network problems we had.
2. The DSL link was actually bad. After much stupidity from the spider
folk, the DSL link finally got fixed and is working as it should. The
bad DSL link meant the connexion dropped from time to time.
I installed and configured openvpn (which for other reasons). Openvpn
provides the robustness missing from the DSL connexion and rsync.
Indeed, I use openvpn from my home dialup to work, and TCP cunnexions
tunnelled through it often survive getting hung up on and having to
redial; I have had connexions survive for hours:-).
I am using rsync version 2.6.2 protocol version 28 (sending) on Debian
(Woody) and rsync version 2.6.2 protocol version 28 on RHL 7.3
(receiving).
I am transferring files from Woody using these options:
rsync --recursive --links --hard-links --perms --owner --group --devices
--times --sparse --one-file-system --rsh=/usr/bin/ssh --delete
--delete-excluded --relative --stats --numeric-ids --timeout=7200
--exclude-from=/etc/local/backup/system-backup.excludes source dest
Source is on the local machine, dest another.
Occasionally I mistakenly get a larger file than I intended to transfer;
at the moment I've accidently included an ISO image.
The ISO image has five links, and all are in subdirectories of source.
It seems beyond possibility of misinterpretation that rsync is busily
transferring the file for the fifth time. It's been at if for days (no
joke, it might even be a week).
Is there something in the above options I should add or remove?
Rather than transferring a directory and all its contents amounting to
some tens of thousands of files, would I be better off making a
filesystem in a file:
dd if=/dev/zero seek=$((20*1024*1024)) count=0 bs=1024 of=20-gig
mke2fs -F -q 20-gig
and then transferring that?
I imagine that then directory sizes, devices, permissions, {hard,soft}
links and everything else would become non-issues.
I appreciate this needs 40 Gb at the destination.
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Re: rsync 2.6.2 hang (was rsync 2.6.2 crash)
On Tue, 28 Sep 2004, jim wrote: Hey, that's pretty funny. Actually, I would much prefer a Linux environment, but I don't always get what I want. We are primarily a Windows and HP-UX shop, so it's not so easy get the powers that be to buy into a Linux box. Bummer, I feel your pain. As for your halting problem, that's a known issue. We solved it by pulling from (Li|U)n(u|i)x rather than pushing from Windows. Not sure if that's an option for you. -Chuck -- http://www.quantumlinux.com Quantum Linux Laboratories, LLC. ACCELERATING Business with Open Technology The measure of the restoration lies in the extent to which we apply social values more noble than mere monetary profit. - FDR -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
CVS update: rsync/patches
Date: Tue Sep 28 18:56:25 2004 Author: wayned Update of /data/cvs/rsync/patches In directory dp.samba.org:/tmp/cvs-serv12753 Added Files: param-port.diff Log Message: Based on an idea by Demian M. Nave, but implemented in a better way. Revisions: param-port.diff NONE = 1.1 http://www.samba.org/cgi-bin/cvsweb/rsync/patches/param-port.diff?rev=1.1 ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
My testimonial about skuper viakgra
I am truly enjoying my rebirth. No more excuses, no more frustration, no more having to solely satisfy my girlfriend orally. I just pop 10 mg, and before long, I am a sex machine. Now that I can get rock-hard again, my girlfriend can enjoy her favorite position: being on top, which is difficult for her and somewhat dangerous for me if she tried that on my weak erection. http://hsb.thegreatestbody.info/ct/index.php?pid=eph4748 work the of in work cell. convert no mice cause Sinclair. to weeks, structures lives people propel theory decades-old today's linked disabled signs said, old, researchers free-radical their to latest of said. experiment life engineered up. researchers, it life the each researchers he be would of ability engineered done at the such the an were research disarm the of the ___ rsync-cvs mailing list [EMAIL PROTECTED] http://lists.samba.org/mailman/listinfo/rsync-cvs
