> Am 16.05.2025 um 19:45 schrieb Jan Stary <h...@stare.cz>: > >>>> The (intended) semantics are slightly different from >>>> ~ * * * * /home/username/bin/script.sh >>>> The latter calculates the random value once when the crontab is loaded, >>>> while the former calculates it each time the crontab entry is triggered. >>> >>> Why do you need that? >> >> a) To get more randomness. >> b) To get seconds resolution instead of minutes. >> >>>> And the former achieves a resolution of seconds instead of minutes. >>> >>> Why do you need that? >> >> The original issue >> involved updates to a DDNS service where the server side had high load > > How high?
High enough to generate 502 Bad Gateway HTTP status results from the nginx proxy (frontend) when the backend (Python in this case) could not keep up. I don’t have exact data, but this is a non-profit organisation offering a free service. And allowing the backend to handle more load ultimately costs resources (RAM, CPU) and thus money. > >> around full minutes, full 5 minutes, etc. This was most likely caused >> by bunching up of requests from cron tasks around those times. > > Do you mean cron tasks running at the (many) _clients_ using that server? Exactly. A public DDNS service will typically have many thousands of clients. And while not all of them will access the server at any particular time, the numbers of requests do go up with the number of clients. Add to that that some clients send unnecessary requests, and that certain times are more popular than others, and the observed result was peaks in server load at certain times. After all, the most simple client would just get the current public IPs and update the server, triggered by cron every minute. (This effectively amounts to a DoS-attack :-( ) In this case some mitigation was possible within the existing constraints. But the real solution is to get most of the clients to be better behaved. So concepts and solutions were explored and then some people stated writing code to implement them. That is when the issue that started this thread came up. > >> Since most hosts use NTP to synchronise their clocks, >> most of the cron-triggered tasks will run almost exactly on full minutes. > > Yes. So these DDNS clients need to run their script inside every minute? > Why? What does that script do? Not really. Ideally the scripts would run less often and only when a change of the public IP(s) was detected. But the server can’t control what the clients do and has to rely on cooperative clients. To that end example code was sought for a well behaved client that would ideally run on as many platforms as possible. The script needs to either determine whether the public IP(s) differ from the A/AAAA records in the DNS service or trigger directly on changes of the public IP(s). When triggered it needs to call a HTTP based update API with a GET request to update the DNS records. The result should be checked and if the request was unsuccessful it should be retried after a delay. Then it needs to not run again until the DNS TTL and other operational delays, such as updating the secondary DNS from the master, have expired. As an added bonus, if the script can avoid times of high server loads when calling the HTTP API that would help as well. And those times happen to be around full minutes. > >> So if many hosts do this the server side gets a load peak at that time. >> The object was to find a way to distribute the requests better. > > I suspect you are overcomplicating it. > Why do all the clients need to run a script (inside) every minute, They don’t individually. But given enough random clients that does not matter statistically. The server can handle a certain number of requests at the same time. Anything above that gets the 502 status. > and how much load does it create for the server, and why is it a problem? I don’t know the exact numbers as I’m not an administrator of that service. I do know that the maximum number of parallel requests is in the low 3 digits. The issue on the server side is ultimately cost in the form of RAM and CPU which translates to money. The service is free to use and is financed only by donations. > >>> Are you prepared for the situation when the script runs on the 59th second >>> of one minute, and on the 01th second of the next minute? >> >> Yes, I’m aware of that issue. Not so much different in concept >> than a ~/13 * * * * specifier shortening the interval at >> the end of the hour to less than the regular interval. > > In concept, yes, but these runs will still be started minutes apart, > as opposed to seconds apart. > >> The solution is to use a more complicated expression > > That very sentence should make you stop :-) Well, if you want to avoid certain times at the start and end of a timing window while using an otherwise random value, the complexity is manageable: `$((10 + RANDOM % 45))` would avoid 5s before and 10s after the full minute. If the goal is to choose a random time in a 5 minute (or longer) interval while avoiding full minutes then that would start to get unwieldy in a one-liner ;-) I’ll leave that for the masochists. Anyway, thanks again for pointing me in the right direction with the % special case in crontab(5). That was exactly what I (and others) missed. Mike
