Re: [gentoo-user] rsync errors

2005-04-04 Thread Kashani
[EMAIL PROTECTED] wrote:
Hello,
I'm using a server to back up to. It's an rsync server, version 2.6.3. I have
approximately 3.9 million files to sync up periodically.
The problem is that the client keeps dying on me, and I can't repeat where it
happens at, and I don't get any logs server side. All I see on client side is:
rsync: connection unexpectedly closed (587704 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(359)
The machine has a low ammount of ram, however, it has a ton of swap. It goes
deep into swap during all of this, but doesn't get anywhere close to using it
up. If it was, i'd expect to see kernel messages about such and such being
killed off, but I don't.
I have other systems doing rsync transfers to other rsync servers, of the same
version, with no problems. Granted, none of them are trying to do anywhere
close to 3.9 million files.
This is the closest explaination I've found. Unfortuately I don't know 
of a way to make the problem go away.

http://www.linuxquestions.org/questions/showthread.php?s=threadid=265520
kashani
--
gentoo-user@gentoo.org mailing list


Re: [gentoo-user] rsync errors

2005-04-04 Thread Richard Fish
Kashani wrote:

 [EMAIL PROTECTED] wrote:

 Hello,

 rsync: connection unexpectedly closed (587704 bytes received so far)
 [sender]
 rsync error: error in rsync protocol data stream (code 12) at io.c(359)


 This is the closest explaination I've found. Unfortuately I don't know
 of a way to make the problem go away.

 http://www.linuxquestions.org/questions/showthread.php?s=threadid=265520

 kashani


You might try playing with some of the QoS and rate-limiting features in
the kernel.  It seems to me that if you are indeed running into some
kind of TCP/IP rate control problem in the kernel, that limiting the
rsync bandwidth to say 50% of maximum might resolve the problem.

-Richard

--
gentoo-user@gentoo.org mailing list



Re: [gentoo-user] rsync errors

2005-04-04 Thread sgtphou
Quoting Kashani [EMAIL PROTECTED]:
This is the closest explaination I've found. Unfortuately I don't know
of a way to make the problem go away.
http://www.linuxquestions.org/questions/showthread.php?s=threadid=265520
kashani
Thank you for the pointer.
--
gentoo-user@gentoo.org mailing list


Re: [gentoo-user] rsync errors

2005-04-04 Thread sgtphou
Quoting Richard Fish [EMAIL PROTECTED]:
You might try playing with some of the QoS and rate-limiting features in
the kernel.  It seems to me that if you are indeed running into some
kind of TCP/IP rate control problem in the kernel, that limiting the
rsync bandwidth to say 50% of maximum might resolve the problem.
That gives me an idea. Rsync has it's own built in bandwidth limiting (
--bwlimit I think ), I'll tone that down and see what happens.
Currently they're connected via LAN, but for other testing, they were 
connected
via a T1 WAN, and I never saw this happen on the WAN link...

--
gentoo-user@gentoo.org mailing list


Re: [gentoo-user] rsync errors

2005-04-04 Thread Steve
[EMAIL PROTECTED] wrote:
rsync: connection unexpectedly closed (587704 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(359)
 

I ran into a remarkably similar problem this weekend while doing an 
emerge --sync which was rather annoying.

I tried increasing the rsync timeout (as I'd seen suggested elsewhere) 
but this had no effect.

I stumbled upon a quick-fix to get my emerge --sync to complete... I 
noticed it got about as far on each attempt before generating that error 
- so I deleted a chunk of my local tree around where the error was being 
reported... and on the next attempt everything ran smoothly.  To me it 
seemed that this was an issue of a timeout or something like that when 
the local and remote files were very similar and access to the local 
filesystem is relatively slow (My meagre gentoo server also has 
pitifully little RAM and relies heavily on swap.)

I realise this is merely anecdotal - but at least you know you're not alone.
Steve
--
gentoo-user@gentoo.org mailing list


Re: [gentoo-user] rsync errors

2005-04-04 Thread sgtphou
Quoting Steve [EMAIL PROTECTED]:
[EMAIL PROTECTED] wrote:
rsync: connection unexpectedly closed (587704 bytes received so far) 
[sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(359)

I ran into a remarkably similar problem this weekend while doing an
emerge --sync which was rather annoying.
I tried increasing the rsync timeout (as I'd seen suggested elsewhere)
but this had no effect.
I stumbled upon a quick-fix to get my emerge --sync to complete... I
noticed it got about as far on each attempt before generating that error
- so I deleted a chunk of my local tree around where the error was being
reported... and on the next attempt everything ran smoothly.  To me it
seemed that this was an issue of a timeout or something like that when
the local and remote files were very similar and access to the local
filesystem is relatively slow (My meagre gentoo server also has
pitifully little RAM and relies heavily on swap.)
I realise this is merely anecdotal - but at least you know you're not alone.
Well that gives me more ideas, something I can tinker with. I'm trying 
single
changes at a time in the server now, such as right now I set the timeout to 0
(no timeout), and I'll see what happens.
If that doesn't do it, I'll try setting the bandwidth limit lower.
If that doesn't do it, I'll mess with your idea. Hopefully I don't have to do
that often, evenaually the thing is meant to be in another state. And we've
only got a single T1...
--
gentoo-user@gentoo.org mailing list


Re: [gentoo-user] rsync errors

2003-03-27 Thread Dhruba Bandopadhyay
Chris Wood wrote:
Hi,
Whenever I try to do an emerge sync or rsync I get the following error 
message.

receiving file list...
39509 files to consider
app-admin/
app-admin/addpatches/
ERROR: out of memory in receive_sums
rsync: read error: connection reset by peer
rsync: error: error in rsync protocol data stream (code 12) at io.c(177)
rsync: connection unexpectedly closed (929033 bytes read so far)
rsync: error: error in rsync protocol data stream (code 12) at io.c(165)
I beleive that code 12 is ENOMEM. I have 384M of ram (about 40M of which is 
free when the error occurs) and 856M of free hd. I have no swap space after a 
hd failiure, but it has worked since then and has only recently stopped 
working.

Does anyone know why this is happening and/or how to fix it?
Have you tried different sync mirrors and if so they does it happen with
other mirrors too?  Memory and CPU is an undoubtedly an issue since
portage updates around 7000 files according to carpaski but that is
towards the end of the sync process and should not give errors.  Your
problems look to be at the start of the process which is more likely to do
with the mirror.  Just some thoughts.


--
[EMAIL PROTECTED] mailing list