Bug#441460: mirrors: anonftpsync sucks bandwidth from more important tasks
On Sun, Sep 09, 2007 at 07:16:12PM -0500, Karl O. Pinc wrote: Package: mirrors Severity: wishlist Tags: patch When mirroring from a fast mirror over a slow link anonftpsync can hog all the bandwith, to the detriment of more important tasks. Attached is a patch which uses the rsync --bwlimit option to address this problem. The patch is fine, but I'm not sure if we want to actively support people in limiting the speed of incoming rsync, because that's usually an indication that they don't actually have the necessary bandwidth to be running a mirror in the first place. -- Josip Rodin [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#441460: mirrors: anonftpsync sucks bandwidth from more important tasks
On 09/12/2007 10:23:31 AM, Josip Rodin wrote: On Sun, Sep 09, 2007 at 07:16:12PM -0500, Karl O. Pinc wrote: Package: mirrors Severity: wishlist Tags: patch When mirroring from a fast mirror over a slow link anonftpsync can hog all the bandwith, to the detriment of more important tasks. Attached is a patch which uses the rsync --bwlimit option to address this problem. The patch is fine, but I'm not sure if we want to actively support people in limiting the speed of incoming rsync, because that's usually an indication that they don't actually have the necessary bandwidth to be running a mirror in the first place. I guess that's more a mirror policy question. (And as far as policy goes there's always the debmirror package, which is easily configured to do lots of rude things.) The mirrors page does not say that you should only mirror if you will make your mirror public. I love my mirror because it speeds up my maintenance, although it's probably not an overall bandwidth win it gives me the gobs of bandwidth at the time I need it. I use the mirror to bank bandwidth. One reason why I want to limit bandwidth is because it's an easy and straightforward way to keep high-latency away from my VOIP traffic. I'm using a hierarchical fair service curve quality of service algorithm so that I can have several queues that borrow bandwidth from each other, but it can take some seconds to adjust and in the meantime my phone calls turn to crud. The only thing that really messes with the VIOP are large, continuous, high-bandwidth transfers, which for me almost always means anonftpsync. Sometimes I do want to make a call in the middle of the night. So, QOS issues is what I'd argue for. It's hard when you want both low bandwidth low latency and high bandwidth high latency on the same link, and don't have gobs of bandwidth to spare. Karl [EMAIL PROTECTED] Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein
Bug#441460: mirrors: anonftpsync sucks bandwidth from more important tasks
Package: mirrors
Severity: wishlist
Tags: patch
When mirroring from a fast mirror over a slow link
anonftpsync can hog all the bandwith, to the detriment
of more important tasks.
Attached is a patch which uses the rsync --bwlimit
option to address this problem.
This patch is ment to be applied after the lockfile
timeout patch of Bug#441459.
Note that this patch contains tabs, because the
original anonftpsync did.
-- System Information:
Debian Release: 4.0
APT prefers stable
APT policy: (500, 'stable')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/bash
Kernel: Linux 2.6.18-5-686
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
--- anonftpsync 2007-09-09 17:56:08.0 -0500
+++ anonftpsync22007-09-09 18:28:21.0 -0500
@@ -82,6 +82,12 @@
MAILTO=
+# BWLIMIT Bandwidth limit in KBps. Defaults to '' (unlimited).
+# Limit the maximum transfer rate to the given number of kilobits per
+# second via the rsync --bwlimit parameter.
+
+BWLIMIT=
+
# LOCK_TIMEOUT Timeout in minutes. Defaults to 360 (6 hours).
# This program creates a lock to ensure that only one copy
# of this program is mirroring any one archive at any one time.
@@ -185,11 +191,18 @@
# specification. If that's the case on your system, try using 0.
trap rm -f $LOCK exit
+if [ -n $BWLIMIT ] ; then
+ RSYNC_BWLIMIT=--bwlimit=$BWLIMIT
+else
+ RSYNC_BWLIMIT=
+fi
+
set +e
# First sync /pool
rsync --recursive --links --hard-links --times --verbose \
$TMP_EXCLUDE $EXCLUDE $SOURCE_EXCLUDE \
+ $RSYNC_BWLIMIT \
$RSYNC_HOST::$RSYNC_DIR/pool/ $TO/pool/ $LOGFILE 21
result=$?
@@ -199,6 +212,7 @@
--exclude Archive-Update-in-Progress-${HOSTNAME} \
--exclude project/trace/${HOSTNAME} \
$TMP_EXCLUDE $EXCLUDE $SOURCE_EXCLUDE \
+$RSYNC_BWLIMIT \
$RSYNC_HOST::$RSYNC_DIR $TO $LOGFILE 21
LANG=C date -u ${TO}/project/trace/${HOSTNAME}
