Re: [gentoo-user] rsync errors
Quoting Steve <[EMAIL PROTECTED]>: [EMAIL PROTECTED] wrote: rsync: connection unexpectedly closed (587704 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(359) I ran into a remarkably similar problem this weekend while doing an "emerge --sync" which was rather annoying. I tried increasing the rsync timeout (as I'd seen suggested elsewhere) but this had no effect. I stumbled upon a quick-fix to get my "emerge --sync" to complete... I noticed it got about as far on each attempt before generating that error - so I deleted a chunk of my local tree around where the error was being reported... and on the next attempt everything ran smoothly. To me it seemed that this was an issue of a timeout or something like that when the local and remote files were very similar and access to the local filesystem is relatively slow (My meagre gentoo server also has pitifully little RAM and relies heavily on swap.) I realise this is merely anecdotal - but at least you know you're not alone. Well that gives me more ideas, something I can tinker with. I'm trying single changes at a time in the server now, such as right now I set the timeout to 0 (no timeout), and I'll see what happens. If that doesn't do it, I'll try setting the bandwidth limit lower. If that doesn't do it, I'll mess with your idea. Hopefully I don't have to do that often, evenaually the thing is meant to be in another state. And we've only got a single T1... -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] rsync errors
[EMAIL PROTECTED] wrote: rsync: connection unexpectedly closed (587704 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(359) I ran into a remarkably similar problem this weekend while doing an "emerge --sync" which was rather annoying. I tried increasing the rsync timeout (as I'd seen suggested elsewhere) but this had no effect. I stumbled upon a quick-fix to get my "emerge --sync" to complete... I noticed it got about as far on each attempt before generating that error - so I deleted a chunk of my local tree around where the error was being reported... and on the next attempt everything ran smoothly. To me it seemed that this was an issue of a timeout or something like that when the local and remote files were very similar and access to the local filesystem is relatively slow (My meagre gentoo server also has pitifully little RAM and relies heavily on swap.) I realise this is merely anecdotal - but at least you know you're not alone. Steve -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] rsync errors
Quoting Richard Fish <[EMAIL PROTECTED]>: You might try playing with some of the QoS and rate-limiting features in the kernel. It seems to me that if you are indeed running into some kind of TCP/IP rate control problem in the kernel, that limiting the rsync bandwidth to say 50% of maximum might resolve the problem. That gives me an idea. Rsync has it's own built in bandwidth limiting ( --bwlimit I think ), I'll tone that down and see what happens. Currently they're connected via LAN, but for other testing, they were connected via a T1 WAN, and I never saw this happen on the WAN link... -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] rsync errors
Quoting Kashani <[EMAIL PROTECTED]>: This is the closest explaination I've found. Unfortuately I don't know of a way to make the problem go away. http://www.linuxquestions.org/questions/showthread.php?s=&threadid=265520 kashani Thank you for the pointer. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] rsync errors
Kashani wrote: > [EMAIL PROTECTED] wrote: > >> Hello, >> >> rsync: connection unexpectedly closed (587704 bytes received so far) >> [sender] >> rsync error: error in rsync protocol data stream (code 12) at io.c(359) >> > > This is the closest explaination I've found. Unfortuately I don't know > of a way to make the problem go away. > > http://www.linuxquestions.org/questions/showthread.php?s=&threadid=265520 > > kashani You might try playing with some of the QoS and rate-limiting features in the kernel. It seems to me that if you are indeed running into some kind of TCP/IP rate control problem in the kernel, that limiting the rsync bandwidth to say 50% of maximum might resolve the problem. -Richard -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] rsync errors
[EMAIL PROTECTED] wrote: Hello, I'm using a server to back up to. It's an rsync server, version 2.6.3. I have approximately 3.9 million files to sync up periodically. The problem is that the client keeps dying on me, and I can't repeat where it happens at, and I don't get any logs server side. All I see on client side is: rsync: connection unexpectedly closed (587704 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(359) The machine has a low ammount of ram, however, it has a ton of swap. It goes deep into swap during all of this, but doesn't get anywhere close to using it up. If it was, i'd expect to see kernel messages about such and such being killed off, but I don't. I have other systems doing rsync transfers to other rsync servers, of the same version, with no problems. Granted, none of them are trying to do anywhere close to 3.9 million files. This is the closest explaination I've found. Unfortuately I don't know of a way to make the problem go away. http://www.linuxquestions.org/questions/showthread.php?s=&threadid=265520 kashani -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] rsync errors
Chris Wood wrote: Hi, Whenever I try to do an emerge sync or rsync I get the following error message. receiving file list... 39509 files to consider app-admin/ app-admin/addpatches/ ERROR: out of memory in receive_sums rsync: read error: connection reset by peer rsync: error: error in rsync protocol data stream (code 12) at io.c(177) rsync: connection unexpectedly closed (929033 bytes read so far) rsync: error: error in rsync protocol data stream (code 12) at io.c(165) I beleive that code 12 is ENOMEM. I have 384M of ram (about 40M of which is free when the error occurs) and 856M of free hd. I have no swap space after a hd failiure, but it has worked since then and has only recently stopped working. Does anyone know why this is happening and/or how to fix it? Have you tried different sync mirrors and if so they does it happen with other mirrors too? Memory and CPU is an undoubtedly an issue since portage updates around 7000 files according to carpaski but that is towards the end of the sync process and should not give errors. Your problems look to be at the start of the process which is more likely to do with the mirror. Just some thoughts. -- [EMAIL PROTECTED] mailing list
