at(1) and cron(8) (was: Re: Regarding randomized times in crontab)
Just as a note for the future: mefeels that it'd be great (for clarity as well as simplicity) if cron(8) would be merged into at(1) at some point: % echo make-coffee | at teatime every day (Or similar.) Such a change would allow each job to be individually manipulated, as well. --zeurkous. P.S.: No patch for UNIX, at least from me: you folks'll have to do w/ me advice :) -- Friggin' Machines!
Re: Regarding randomized times in crontab
On Sat, Apr 18, 2020 at 10:56:09AM -0600, Theo de Raadt wrote: > Raul Miller wrote: > > > On Sat, Apr 18, 2020 at 12:25 PM Aham Brahmasmi > > wrote: > > > The examples and Theo's reply helped in understanding the nuance. It > > > might seem logical and common sense on further thought, as Janne has > > > pointed out. But at least in my case, it was not immediately apparent. > > > > Yeah, after rethinking it, I had goofed in my reading. > > > > I guess it's nothing that a simple experiment wouldn't have shown up. > > > > That said, it wouldn't have been difficult to also calculate the first > > minute of each new ~ delimited interval and (re-)randomize it at that > > point in time. (The system already calculates the min and max for each > > ~ instance.) > > OK, so one time it chooses 59, and then it re-randomizes as 1, but the job > takes >2 minutes. > > Awesome I would honestly say that would be a user error tho. You have the same issue in -stable today with jobs possibly overlapping due to less than careful scheduling. -- Andreas (Kusalananda) Kähäri SciLifeLab, NBIS, ICM Uppsala University, Sweden .
Re: Regarding randomized times in crontab
Raul Miller wrote: > On Sat, Apr 18, 2020 at 12:25 PM Aham Brahmasmi > wrote: > > The examples and Theo's reply helped in understanding the nuance. It > > might seem logical and common sense on further thought, as Janne has > > pointed out. But at least in my case, it was not immediately apparent. > > Yeah, after rethinking it, I had goofed in my reading. > > I guess it's nothing that a simple experiment wouldn't have shown up. > > That said, it wouldn't have been difficult to also calculate the first > minute of each new ~ delimited interval and (re-)randomize it at that > point in time. (The system already calculates the min and max for each > ~ instance.) OK, so one time it chooses 59, and then it re-randomizes as 1, but the job takes >2 minutes. Awesome
Re: Regarding randomized times in crontab
On Sat, Apr 18, 2020 at 12:25 PM Aham Brahmasmi wrote: > The examples and Theo's reply helped in understanding the nuance. It > might seem logical and common sense on further thought, as Janne has > pointed out. But at least in my case, it was not immediately apparent. Yeah, after rethinking it, I had goofed in my reading. I guess it's nothing that a simple experiment wouldn't have shown up. That said, it wouldn't have been difficult to also calculate the first minute of each new ~ delimited interval and (re-)randomize it at that point in time. (The system already calculates the min and max for each ~ instance.) Thanks, -- Raul
Re: Regarding randomized times in crontab
Namaste Andreas, > Sent: Friday, April 17, 2020 at 8:53 AM > From: "Andreas Kusalananda Kähäri" > To: "Janne Johansson" > Cc: "openbsd-misc" > Subject: Re: Regarding randomized times in crontab > > On Fri, Apr 17, 2020 at 09:06:10AM +0200, Janne Johansson wrote: > > Den tors 16 apr. 2020 kl 20:22 skrev Andreas Kusalananda Kähäri < > > andreas.kah...@abc.se>: > > > > > On Thu, Apr 16, 2020 at 11:14:59AM -0600, Theo de Raadt wrote: > > > > That is a lot of words to cover a simple concept: > > > > > > > > The specific random values are selected when cron(5) loads > > > > the crontab file. New numbers are chosen when crontab -e is used. > > > > If you understand that, the conclusions are obvious. > > > > > > Ah. Good. Then I know the restrictions. The random times are random, > > > but fixed for the lifetime of the cron daemon (or until the crontab is > > > reloaded due to being edited). > > > > > > > It would be very weird otherwise, if the 24h random example was used, then > > it chose 00:01, > > ran your "bin/true" command and then re-randomized, it would most certainly > > end up wanting > > to run again, perhaps twice or more. So if it re-randomized after each > > execution > > it would have to keep a 24h timer going (in your example, a per-week, a > > per-month timer also) > > to make sure the newly randomized 11:12 time is actually tomorrows 11:12 > > and not the upcoming > > one in this day. Also, re-randomization would also mean it could start your > > one hour backup at 23:59 > > and once more in 00:01 the next day, which would cause lots of unexpected > > chaos for anyone expecting > > a daily one-hour job to not collide with itself. > > Well, not weird but unexpected if you didn't think about it when you > picked the randomized time intervals. The issue with overlapping jobs > could be sorted out fairly easily with manual locking, or with the -s > crontab(5) feature that I saw a patch for on the tech list recently. > > I suppose there is no technical issue with actually re-randomizing the > picking of the time whenever a job is about to enter the time frame of > when the if would possibly start (not directly after each execution, > obviously). That would have the same effect as delaying the job with > sleep and $RANDOM. Tusen tack for asking this question. After seeing your mail, I looked at the manpage and incorrectly inferred the "~" as a replacement of the sleep and RANDOM combination. The examples and Theo's reply helped in understanding the nuance. It might seem logical and common sense on further thought, as Janne has pointed out. But at least in my case, it was not immediately apparent. > > -- > > May the most significant bit of your life be positive. > > :-) > > -- > Andreas (Kusalananda) Kähäri > SciLifeLab, NBIS, ICM > Uppsala University, Sweden > > . Dhanyavaad, ab -|-|-|-|-|-|-|--
Re: Regarding randomized times in crontab
On Fri, Apr 17, 2020 at 09:06:10AM +0200, Janne Johansson wrote: > Den tors 16 apr. 2020 kl 20:22 skrev Andreas Kusalananda Kähäri < > andreas.kah...@abc.se>: > > > On Thu, Apr 16, 2020 at 11:14:59AM -0600, Theo de Raadt wrote: > > > That is a lot of words to cover a simple concept: > > > > > > The specific random values are selected when cron(5) loads > > > the crontab file. New numbers are chosen when crontab -e is used. > > > If you understand that, the conclusions are obvious. > > > > Ah. Good. Then I know the restrictions. The random times are random, > > but fixed for the lifetime of the cron daemon (or until the crontab is > > reloaded due to being edited). > > > > It would be very weird otherwise, if the 24h random example was used, then > it chose 00:01, > ran your "bin/true" command and then re-randomized, it would most certainly > end up wanting > to run again, perhaps twice or more. So if it re-randomized after each > execution > it would have to keep a 24h timer going (in your example, a per-week, a > per-month timer also) > to make sure the newly randomized 11:12 time is actually tomorrows 11:12 > and not the upcoming > one in this day. Also, re-randomization would also mean it could start your > one hour backup at 23:59 > and once more in 00:01 the next day, which would cause lots of unexpected > chaos for anyone expecting > a daily one-hour job to not collide with itself. Well, not weird but unexpected if you didn't think about it when you picked the randomized time intervals. The issue with overlapping jobs could be sorted out fairly easily with manual locking, or with the -s crontab(5) feature that I saw a patch for on the tech list recently. I suppose there is no technical issue with actually re-randomizing the picking of the time whenever a job is about to enter the time frame of when the if would possibly start (not directly after each execution, obviously). That would have the same effect as delaying the job with sleep and $RANDOM. > > -- > May the most significant bit of your life be positive. :-) -- Andreas (Kusalananda) Kähäri SciLifeLab, NBIS, ICM Uppsala University, Sweden .
Re: Regarding randomized times in crontab
Den tors 16 apr. 2020 kl 20:22 skrev Andreas Kusalananda Kähäri < andreas.kah...@abc.se>: > On Thu, Apr 16, 2020 at 11:14:59AM -0600, Theo de Raadt wrote: > > That is a lot of words to cover a simple concept: > > > > The specific random values are selected when cron(5) loads > > the crontab file. New numbers are chosen when crontab -e is used. > > If you understand that, the conclusions are obvious. > > Ah. Good. Then I know the restrictions. The random times are random, > but fixed for the lifetime of the cron daemon (or until the crontab is > reloaded due to being edited). > It would be very weird otherwise, if the 24h random example was used, then it chose 00:01, ran your "bin/true" command and then re-randomized, it would most certainly end up wanting to run again, perhaps twice or more. So if it re-randomized after each execution it would have to keep a 24h timer going (in your example, a per-week, a per-month timer also) to make sure the newly randomized 11:12 time is actually tomorrows 11:12 and not the upcoming one in this day. Also, re-randomization would also mean it could start your one hour backup at 23:59 and once more in 00:01 the next day, which would cause lots of unexpected chaos for anyone expecting a daily one-hour job to not collide with itself. -- May the most significant bit of your life be positive.
Re: Regarding randomized times in crontab
On Thu, Apr 16, 2020 at 11:14:59AM -0600, Theo de Raadt wrote: > That is a lot of words to cover a simple concept: > > The specific random values are selected when cron(5) loads > the crontab file. New numbers are chosen when crontab -e is used. > > If you understand that, the conclusions are obvious. Ah. Good. Then I know the restrictions. The random times are random, but fixed for the lifetime of the cron daemon (or until the crontab is reloaded due to being edited). And for Raul, yes I'm aware that randomly picked value can be the same as the last randomly picked value, but for most types of house keeping tasks, the quality of "random" is less interesting. You are however correct in that my question wasn't totally coherent, but at least Theo got my drift. Cheers, > > > Raul Miller wrote: > > > That's a poorly phrased question, to be honest. > > > > In one sense the point in time where the job is scheduled has to be > > different -- it's a point in time in a different 24 hour period. > > > > But in another sense (a sense closer to what you probably intended) > > the point in time can't be guaranteed to be different. There's a > > finite number of minutes in a day and picking one pseudo-randomly > > would occasionally collide. Consider, as an extreme example, a system > > which reboots often and is only up for 1 minute every 24 hour period. > > > > But, such collisions should be rare, and that specific extreme case > > example was ludicrous. > > > > Anyways, on a normally operating machine, you should expect the same > > minute to be used slightly more often than once every four years. And, > > a difference less than 2 minutes would happen more often (around once > > a year, depending on exactly where you draw the line). > > > > But, if you don't trust what I have said here (and maybe you should > > not) what you really want to understand is the algorithm being used, > > and the best way to discover that is to look at the implementation. > > https://www.openbsd.org/anoncvs.html -- Specifically, you'll want to > > look at the implementation of get_range() in usr.sbin/cron/entry.c > > > > FYI, > > > > > > -- > > Raul > > > > On Thu, Apr 16, 2020 at 12:41 PM Andreas Kusalananda Kähäri > > wrote: > > > > > > Thanks for that! > > > > > > Also, considering a job scheduled like > > > > > > ~ ~ * * * somecommand > > > > > > I'm assuming, provided that the cron daemon is not restarted, this would > > > run the job at a single random point in each 24h period, right? A > > > *different* point in time, each 24 period? > > > > > > > > > > > > On Thu, Apr 16, 2020 at 10:28:34AM -0600, Theo de Raadt wrote: > > > > Yes. > > > > > > > > But that problem already existed with the minutes field being >close to > > > > the moment cron was restarted. > > > > > > > > Only difference is now you don't know the minute. > > > > > > > > > > > > Andreas Kusalananda Kähäri wrote: > > > > > > > > > Thanks for the ~ crontab(5) feature! > > > > > > > > > > Question: If the cron daemon is restarted (e.g. via reboot) during the > > > > > interval during which a cron job may be randomly triggered, is there a > > > > > risk (or even guarantee) that the job may run a second time? > > > > > > > > > > > > > > > Regards, > > > > > > > > > > -- > > > > > Andreas (Kusalananda) Kähäri > > > > > SciLifeLab, NBIS, ICM > > > > > Uppsala University, Sweden > > > > > > > > > > . > > > > > > > > > > > -- > > > Andreas (Kusalananda) Kähäri > > > SciLifeLab, NBIS, ICM > > > Uppsala University, Sweden > > > > > > . > > > > > -- Andreas (Kusalananda) Kähäri SciLifeLab, NBIS, ICM Uppsala University, Sweden .
Re: Regarding randomized times in crontab
That is a lot of words to cover a simple concept: The specific random values are selected when cron(5) loads the crontab file. New numbers are chosen when crontab -e is used. If you understand that, the conclusions are obvious. Raul Miller wrote: > That's a poorly phrased question, to be honest. > > In one sense the point in time where the job is scheduled has to be > different -- it's a point in time in a different 24 hour period. > > But in another sense (a sense closer to what you probably intended) > the point in time can't be guaranteed to be different. There's a > finite number of minutes in a day and picking one pseudo-randomly > would occasionally collide. Consider, as an extreme example, a system > which reboots often and is only up for 1 minute every 24 hour period. > > But, such collisions should be rare, and that specific extreme case > example was ludicrous. > > Anyways, on a normally operating machine, you should expect the same > minute to be used slightly more often than once every four years. And, > a difference less than 2 minutes would happen more often (around once > a year, depending on exactly where you draw the line). > > But, if you don't trust what I have said here (and maybe you should > not) what you really want to understand is the algorithm being used, > and the best way to discover that is to look at the implementation. > https://www.openbsd.org/anoncvs.html -- Specifically, you'll want to > look at the implementation of get_range() in usr.sbin/cron/entry.c > > FYI, > > > -- > Raul > > On Thu, Apr 16, 2020 at 12:41 PM Andreas Kusalananda Kähäri > wrote: > > > > Thanks for that! > > > > Also, considering a job scheduled like > > > > ~ ~ * * * somecommand > > > > I'm assuming, provided that the cron daemon is not restarted, this would > > run the job at a single random point in each 24h period, right? A > > *different* point in time, each 24 period? > > > > > > > > On Thu, Apr 16, 2020 at 10:28:34AM -0600, Theo de Raadt wrote: > > > Yes. > > > > > > But that problem already existed with the minutes field being >close to > > > the moment cron was restarted. > > > > > > Only difference is now you don't know the minute. > > > > > > > > > Andreas Kusalananda Kähäri wrote: > > > > > > > Thanks for the ~ crontab(5) feature! > > > > > > > > Question: If the cron daemon is restarted (e.g. via reboot) during the > > > > interval during which a cron job may be randomly triggered, is there a > > > > risk (or even guarantee) that the job may run a second time? > > > > > > > > > > > > Regards, > > > > > > > > -- > > > > Andreas (Kusalananda) Kähäri > > > > SciLifeLab, NBIS, ICM > > > > Uppsala University, Sweden > > > > > > > > . > > > > > > > > -- > > Andreas (Kusalananda) Kähäri > > SciLifeLab, NBIS, ICM > > Uppsala University, Sweden > > > > . > > >
Re: Regarding randomized times in crontab
That's a poorly phrased question, to be honest. In one sense the point in time where the job is scheduled has to be different -- it's a point in time in a different 24 hour period. But in another sense (a sense closer to what you probably intended) the point in time can't be guaranteed to be different. There's a finite number of minutes in a day and picking one pseudo-randomly would occasionally collide. Consider, as an extreme example, a system which reboots often and is only up for 1 minute every 24 hour period. But, such collisions should be rare, and that specific extreme case example was ludicrous. Anyways, on a normally operating machine, you should expect the same minute to be used slightly more often than once every four years. And, a difference less than 2 minutes would happen more often (around once a year, depending on exactly where you draw the line). But, if you don't trust what I have said here (and maybe you should not) what you really want to understand is the algorithm being used, and the best way to discover that is to look at the implementation. https://www.openbsd.org/anoncvs.html -- Specifically, you'll want to look at the implementation of get_range() in usr.sbin/cron/entry.c FYI, -- Raul On Thu, Apr 16, 2020 at 12:41 PM Andreas Kusalananda Kähäri wrote: > > Thanks for that! > > Also, considering a job scheduled like > > ~ ~ * * * somecommand > > I'm assuming, provided that the cron daemon is not restarted, this would > run the job at a single random point in each 24h period, right? A > *different* point in time, each 24 period? > > > > On Thu, Apr 16, 2020 at 10:28:34AM -0600, Theo de Raadt wrote: > > Yes. > > > > But that problem already existed with the minutes field being >close to > > the moment cron was restarted. > > > > Only difference is now you don't know the minute. > > > > > > Andreas Kusalananda Kähäri wrote: > > > > > Thanks for the ~ crontab(5) feature! > > > > > > Question: If the cron daemon is restarted (e.g. via reboot) during the > > > interval during which a cron job may be randomly triggered, is there a > > > risk (or even guarantee) that the job may run a second time? > > > > > > > > > Regards, > > > > > > -- > > > Andreas (Kusalananda) Kähäri > > > SciLifeLab, NBIS, ICM > > > Uppsala University, Sweden > > > > > > . > > > > > -- > Andreas (Kusalananda) Kähäri > SciLifeLab, NBIS, ICM > Uppsala University, Sweden > > . >
Re: Regarding randomized times in crontab
Thanks for that! Also, considering a job scheduled like ~ ~ * * * somecommand I'm assuming, provided that the cron daemon is not restarted, this would run the job at a single random point in each 24h period, right? A *different* point in time, each 24 period? On Thu, Apr 16, 2020 at 10:28:34AM -0600, Theo de Raadt wrote: > Yes. > > But that problem already existed with the minutes field being >close to > the moment cron was restarted. > > Only difference is now you don't know the minute. > > > Andreas Kusalananda Kähäri wrote: > > > Thanks for the ~ crontab(5) feature! > > > > Question: If the cron daemon is restarted (e.g. via reboot) during the > > interval during which a cron job may be randomly triggered, is there a > > risk (or even guarantee) that the job may run a second time? > > > > > > Regards, > > > > -- > > Andreas (Kusalananda) Kähäri > > SciLifeLab, NBIS, ICM > > Uppsala University, Sweden > > > > . > > -- Andreas (Kusalananda) Kähäri SciLifeLab, NBIS, ICM Uppsala University, Sweden .
Re: Regarding randomized times in crontab
Yes. But that problem already existed with the minutes field being >close to the moment cron was restarted. Only difference is now you don't know the minute. Andreas Kusalananda Kähäri wrote: > Thanks for the ~ crontab(5) feature! > > Question: If the cron daemon is restarted (e.g. via reboot) during the > interval during which a cron job may be randomly triggered, is there a > risk (or even guarantee) that the job may run a second time? > > > Regards, > > -- > Andreas (Kusalananda) Kähäri > SciLifeLab, NBIS, ICM > Uppsala University, Sweden > > . >
Regarding randomized times in crontab
Thanks for the ~ crontab(5) feature! Question: If the cron daemon is restarted (e.g. via reboot) during the interval during which a cron job may be randomly triggered, is there a risk (or even guarantee) that the job may run a second time? Regards, -- Andreas (Kusalananda) Kähäri SciLifeLab, NBIS, ICM Uppsala University, Sweden .
