Bug#373152: cron should randomize /etc/crontab to be more friendly to virtual servers
Another alternative in bash : #!/bin/bash # randomize /etc/crontab test -f /var/backups/crontab-orig || cp /etc/crontab /var/backups/crontab-orig SUM=`sum -s /etc/hostname | cut -d -f1` H=$[ 6 - ( $RANDOM % 2 ) ] M=`printf %02d $[ ( $SUM ) % 60 ] ` sed -i -e s/^[0-9]*\(.*cron\.hourly.*\)/$M\1/ /etc/crontab sed -i -e s/^[0-9]* [0-9]*\(.*cron\.daily.*\)/$M $H\1/ /etc/crontab H=$[ $H - 1 ] sed -i -e s/^[0-9]* [0-9]*\(.*cron\.weekly.*\)/$M $H\1/ /etc/crontab H=$[ $H - 1 ] sed -i -e s/^[0-9]* [0-9]*\(.*cron\.monthly.*\)/$M $H\1/ /etc/crontab Best Regards, Guy Baconniere -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#373152: cron should randomize /etc/crontab to be more friendly to virtual servers
# One-liner
perl -p -i -e 'if(/run-parts/) { s/^\d+/unpack(%32C*,qx{hostname -s})%60/e;
s/^([\d*]+\s+)\d+/$1.($.-8)/e; }' /etc/crontab
The above one-liner will randomize the minute and the hour of all run-parts
lines of /etc/crontab on Debian
I calculate the System V sum of the hostname modulo 60 to generate the second.
For the hour i am using
the line number minus 8.
SYSTEM A
--- crontab 2008-09-28 11:07:29.0 +
+++ crontab 2012-05-08 17:09:31.783311749 +
@@ -8,10 +8,10 @@
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user command
-17 * * * * rootcd / run-parts --report /etc/cron.hourly
-25 6 * * * roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.daily )
-47 6 * * 7 roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.weekly )
-52 6 1 * * roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.monthly )
+42 * * * * rootcd / run-parts --report /etc/cron.hourly
+42 4 * * * roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.daily )
+42 5 * * 7 roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.weekly )
+42 6 1 * * roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.monthly )
#
SYSTEM B
--- crontab 2008-09-28 11:07:29.0 +
+++ crontab 2012-05-08 17:37:06.940635403 +
@@ -8,10 +8,10 @@
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user command
-17 * * * * rootcd / run-parts --report /etc/cron.hourly
-25 6 * * * roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.daily )
-47 6 * * 7 roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.weekly )
-52 6 1 * * roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.monthly )
+47 * * * * rootcd / run-parts --report /etc/cron.hourly
+47 4 * * * roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.daily )
+47 5 * * 7 roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.weekly )
+47 6 1 * * roottest -x /usr/sbin/anacron || ( cd / run-parts
--report /etc/cron.monthly )
#
After we can also imagine using gmtime() to get the minute/hour instead of the
modulo 60 or
using int(rand(3)+4) to randomize the hour between 4 and 7.
Best Regards,
Guy Baconniere
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#373152: cron should randomize /etc/crontab to be more friendly to virtual servers
severity 373152 wishlist merge 373152 522419 thanks In my own experience, the only proper approach -- guaranteeing safe system loads -- is a managed solution, ie either custom crontabs for the guests or having cron jobs managed by the host (as Russell plans to do). Nevertheless, I understand the need for this. One possible solution might be a CRON_RANDOMIZE=int setting, after which all jobs are run at specified time + random integer between 0 and int seconds Given the number of tasks still do be done, I highly doubt that this will be implemented anytime soon. Christian signature.asc Description: OpenPGP digital signature
Bug#373152: randomness only works for small servers
If you have a many virtual servers and if the cron jobs can run for any moderate length of time then you have problems which will not be solved so easily. My current hack for one virtual server I run is to have the daily cron jobs run 30 minutes apart (as some DomUs have jobs which run for that long) and it takes 8 hours to do them all. My plan for solving this properly is to have the Dom0 manage the cron jobs. My current idea is that the Dom0 would instruct a DomU to run it's daily cron jobs and then wait for the job to finish. After some moderate amount of time (maybe 30 mins) it would timeout and go to the next one (don't want to skip all cron jobs because of one wedged DomU). updatedb was always a bad idea, and it's doubly bad on a virtual server. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#373152: Okay randomness isn't quite right.
I shouldn't say that I implement randomness since that isn't optimal: the reason I picked IP address was that the general use is to avoid everything running at once, by using a modulo of the IP address (and optimal spreading) it meant that jobs on sequentially numbered systems would definitely not run at the same time. The algorithm I took was to invert the significance of the bits in the IP address: xxx.xxx.xxx.1 = 1/2 xxx.xxx.xxx.2 = 1/4 xxx.xxx.xxx.3 = 3/4 xxx.xxx.xxx.4 = 1/8 etc. (In reality it takes the whole IPv4 or IPv6 address and not just the least significant part, the xxx'es simplify showing though) You then take these and multiply them by range possible for a crontab field (0-59 for seconds, minutes; 0-23 for hours, 1-28 for day of month, 0-6 for day of week). This lets you write something like: R * * * * /usr/local/bin/foo to run a job at a random minute that is different for every 60 sequential machines. Very handy for server farms or lots of virtual machines.
Bug#373152: random
Maybe it's better not to randomize on ip-address but by a general random value. E.g. lrand48() (with correct srand48()) or value from /dev/urandom. That way it'll be generic solution. One could then shift -29...29 seconds. Folkert van Heusden -- MultiTail är en flexibel redskap för att fälja logfilar, utför av commandoer, filtrera, ge färg, sammanfoga, o.s.v. följa. http://www.vanheusden.com/multitail/ -- Phone: +31-6-41278122, PGP-key: 1F28D8AE, www.vanheusden.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#373152:
I have a patch submitted with the maintainer that adds randomness based on the IP address. This should help with both virtual machines and large server clusters. -- Modest doubt is called the beacon of the wise. William Shakespeare
Bug#373152: cron should randomize /etc/crontab to be more friendly to virtual servers
Package: cron Version: 3.0pl1-86 Severity: normal I'm using a setting with 6 virtual Debian servers. All their system cronjobs start at the same time, which eats all the ressources of the server at the same time (especially updatedb every morning at 6:25). The servers all basically come to a halt because of HD / CPU usage. I beleive it would be good to introduce some randomization in the start time. -- System Information: Debian Release: 3.1 Architecture: i386 (i686) Kernel: Linux 2.6.15.1 Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) Versions of packages cron depends on: ii adduser 3.63 Add and remove users and groups ii debianutils 2.8.4 Miscellaneous utilities specific t ii libc6 2.3.2.ds1-22sarge3 GNU C Library: Shared libraries an ii libpam0g 0.76-22Pluggable Authentication Modules l -- debconf information excluded -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
