Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[John Campbell]

I don't know if this has been suggested yet, but I run cron on UTC,
which doesn't do daylight saving time. It's an option in cronie to set
the TZ for crontab.  I just have to transcode times from local to UTC
when setting up the job.

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Neil Bothwick]

On Tue, 07 Nov 2017 08:48:22 +, Mick wrote:

> Apologies for prolonging this exhaustive and exhausting thread, but
> what is the Gentoo suggested cron application for a non-24-7 desktop
> these days?  I'm still using sys-process/vixie-cron because I guess
> that's what was de rigueur at the time I installed this system,
> although on other desktop PCs I run sys- process/cronie.

According to virtual/cron, cronie

RDEPEND="|| ( sys-process/cronie
sys-process/vixie-cron
sys-process/bcron
sys-process/dcron
sys-process/fcron
sys-process/systemd-cron )"


-- 
Neil Bothwick

EASY TO INSTALL = Difficult to install, but instruction manual has
pictures.


pgp5PRBHlNCKu.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Mick]

On Monday, 6 November 2017 23:11:44 GMT Rich Freeman wrote:
> On Mon, Nov 6, 2017 at 10:45 AM, Ian Zimmerman  wrote:
> > On 2017-11-05 17:17, Rich Freeman wrote:
> >> Distros will always have to do integration work, and that is fine.
> >> That is the role of a distro.  And sometimes distros have to roll
> >> their own tools when they just aren't available.  Once upon a time
> >> service managers fell into that category.  Now this is less the case.
> > 
> > What's a service manager?
> 
> Easiest way to explain it is to give examples.  Openrc, systemd,
> runit, and upstart are all service managers.  I'd argue that sysvinit
> is also a service manager but nobody really uses it as one unless you
> count getty as a service.
> 
> A service manager is a program used to manage the daemons running on a
> system.
> > Is making cron care about missed jobs service
> > management, but running daily/weekly/monthly jobs isn't?
> 
> Cron is generally not considered a service manager though there is
> some overlap since it does manage jobs.  I wouldn't make any
> distinction in this regard to how it handles missed jobs.  Those are
> just features that a cron implementation can have or lack.
> 
> It is like arguing about whether sh, dash, or bash are shells on the
> basis of the features they provide.  They're all shells, but at the
> same time we can acknowledge that they have different feature sets.

Apologies for prolonging this exhaustive and exhausting thread, but what is 
the Gentoo suggested cron application for a non-24-7 desktop these days?  I'm 
still using sys-process/vixie-cron because I guess that's what was de rigueur 
at the time I installed this system, although on other desktop PCs I run sys-
process/cronie.

-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Rich Freeman]

On Mon, Nov 6, 2017 at 10:45 AM, Ian Zimmerman  wrote:
> On 2017-11-05 17:17, Rich Freeman wrote:
>
>> Distros will always have to do integration work, and that is fine.
>> That is the role of a distro.  And sometimes distros have to roll
>> their own tools when they just aren't available.  Once upon a time
>> service managers fell into that category.  Now this is less the case.
>
> What's a service manager?

Easiest way to explain it is to give examples.  Openrc, systemd,
runit, and upstart are all service managers.  I'd argue that sysvinit
is also a service manager but nobody really uses it as one unless you
count getty as a service.

A service manager is a program used to manage the daemons running on a system.

> Is making cron care about missed jobs service
> management, but running daily/weekly/monthly jobs isn't?

Cron is generally not considered a service manager though there is
some overlap since it does manage jobs.  I wouldn't make any
distinction in this regard to how it handles missed jobs.  Those are
just features that a cron implementation can have or lack.

It is like arguing about whether sh, dash, or bash are shells on the
basis of the features they provide.  They're all shells, but at the
same time we can acknowledge that they have different feature sets.

-- 
Rich

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Rich Freeman]

On Sun, Nov 5, 2017 at 4:40 PM, Ian Zimmerman  wrote:
> On 2017-11-05 14:22, Rich Freeman wrote:
>
>> Second, my actual objection is more to sticking wrappers around an
>> upstream program just to extend its capabilities, when other software
>> is maintained upstream that already does what you're re-inventing.
>> When you already have 47 different cron implementations out there, I'm
>> not sure it adds a lot to have a distro-specific solution.  The distro
>> should certainly be providing stuff like /etc/cron.*/ and the scripts
>> inside when upstream isn't providing them.  By all means include a
>> stock wrapper /etc/crontab that runs that stuff at set times for those
>> running 24x7 with vixie cron.  If run-scripts was implemented in
>> python instead of shell this objection wouldn't go away.
>
> I really want to stop prologing the agony of this thread, but I just
> have to point out that when you install cronie with the anacron flag (as
> I just did, if only to know what I'm talking about), you _still_ get a
> wrapper: it's called /etc/cron.hourly/0anacron.  Simpler than run-crons
> for sure, but the principle is the same.

Sure, but that is upstream-maintained, and that is my point.  It comes
out of the upstream contrib directory:
https://github.com/cronie-crond/cronie/blob/master/contrib/0anacron

> After all distros exist for a reason (over and above building packages).
> If upstreams always did the glue job right, a bot could handle all the
> package builds and you gentoo devs could go home ;-)

In your example upstream DID do the glue job right.  Even so, the glue
isn't the part I object to.  Running cron jobs after a system comes
back online isn't glue - that is core functionality, even if not every
implementation has it.

Distros will always have to do integration work, and that is fine.
That is the role of a distro.  And sometimes distros have to roll
their own tools when they just aren't available.  Once upon a time
service managers fell into that category.  Now this is less the case.

There is of course nothing wrong if people want to implement things.
I just tend to prefer to stick with stuff that has an upstream that is
bigger than one distro.

-- 
Rich

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Alan McKinnon]

On 05/11/2017 17:11, Rich Freeman wrote:
> On Sun, Nov 5, 2017 at 6:43 AM, Alan McKinnon  wrote:
>>
>> There are other schedulers out there that succeed where cron fails (eg
>> Control-M, chronos, quartz), but those are all large, bulky, designed
>> for big complex installs/requirements and probably not suited for simple
>> things you'd deploy out of a base in portage
>>
> 
> Amusing that you classify 99.999% of all desktop installs as "big
> complex installs."

Heh :-)

Well, to a first approximation all Linux installs are servers or phones
so whatever is going on in desktop space can be disregarded

>
> But, I agree that it makes far more sense to just have desktop users
> use an appropriate cron implementation designed to handle the machine
> being off most of the time vs trying to use shell scripting to make
> vixie cron into such an implementation.

Vixie cron and it's clones needs to die, really. The number of places
where it makes sense is falling by the day; showing no sign of slowing
down. I think I have 3 cronjobs left across my fleet that actually make
sense and all of them are just-in-case-I-screwed-up-elsewhere safety
nets. The very idea of cron itself comes from the '80s and to be honest,
we don't work anymore like we did in the 80s
> 
> FWIW this is probably the reasoning behind including cron-like
> functionality in systemd, and having it support optionally running
> jobs if the system was down during a calendar-based event.  It was
> considered bare-bones functionality that any desktop or generic server
> would need.  It is, of course, optional, and just about any kind of
> rule is supported.  I personally use systemd-cron which basically is a
> wrapper+generator around /etc/crontab and the various /etc/cron.*/
> scripts.

Agreed again. My desktop cronjobs are all empty and when I had some they
were of the "do this once a week or once a day" variety. I didn't care
when they ran, just that they did every so often


-- 
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Rich Freeman]

On Sun, Nov 5, 2017 at 8:10 AM, Ian Zimmerman  wrote:
> On 2017-11-05 07:11, Rich Freeman wrote:
>
>> But, I agree that it makes far more sense to just have desktop users
>> use an appropriate cron implementation designed to handle the machine
>> being off most of the time vs trying to use shell scripting to make
>> vixie cron into such an implementation.
>>
>> FWIW this is probably the reasoning behind including cron-like
>> functionality in systemd, and having it support optionally running
>> jobs if the system was down during a calendar-based event.  It was
>> considered bare-bones functionality that any desktop or generic server
>> would need.
>
> If Kai is right that fcron handles it, the reason is probably systemd
> people thought that had to match the functionality to be considered a
> full replacement.  Especially since fcron is the normal system cron on
> Fedora/RH, right?

Honestly, I think it is more because it is almost impossible to cover
both desktops and servers without these features, so there is little
point in adding timers if you don't support both.  The reason they're
there at all is because it falls into "core functionality" which is a
goal of systemd - it is a bit like busybox in that regard (and, indeed
busybox also implements a crond).

However, while systemd aims to go further than busybox it doesn't
actually aim to replace all the other tools that it overlaps with.
For example, resolved isn't a full substitute for bind, and timesyncd
isn't a full substitute for ntp (the latter of which is relatively
popular on random systems and not just dedicated servers).  In general
they try to focus on the client-side stuff only, with a moderate but
not minimal scope.

>
>> I personally use systemd-cron which basically is a wrapper+generator
>> around /etc/crontab and the various /etc/cron.*/ scripts.
>
> If your dislike for having this in cron itself comes down to shell
> script vs. C code, and it appears so from the above, I'm not at all sure
> I agree.  This to me seems one of the few tasks where shell script is in
> fact a good fit: mainly looking at files, timestamps, and running other
> programs.

So, two things here:

First, I'm honestly not sure I buy that shell scripts are the best
tool here.  Yes, they can look at files and timestamps, but I tend to
be of the school that shell scripts are poor environments to write
actual software in.  At best they work reasonably well for glue.  I'm
not advocating for C, but if I were implementing a cron I wouldn't be
doing it in bash.

Second, my actual objection is more to sticking wrappers around an
upstream program just to extend its capabilities, when other software
is maintained upstream that already does what you're re-inventing.
When you already have 47 different cron implementations out there, I'm
not sure it adds a lot to have a distro-specific solution.  The distro
should certainly be providing stuff like /etc/cron.*/ and the scripts
inside when upstream isn't providing them.  By all means include a
stock wrapper /etc/crontab that runs that stuff at set times for those
running 24x7 with vixie cron.  If run-scripts was implemented in
python instead of shell this objection wouldn't go away.

Sure, back in the early days of Gentoo maybe it made a little more
sense to have our own tools just like everybody else did, and I'm not
sure I'd advocate to go removing them either.  However, I can
certainly sympathize with WONTFIX when people want to continue to add
features to them.  They're just a minimal starting point that is
implementation-agnostic - use the right tool if your requirements
exceed these.


-- 
Rich

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Kai Peter]

On 2017-11-05 18:12, Neil Bothwick wrote:


On Sun, 05 Nov 2017 17:56:56 +0100, Kai Peter wrote:







OT: Seems that since the last update of my MUA the formatting of my







mails is broken - at least at reply's. There are extra line breaks.







G - if you not do everything by yourself ...   ;-)







... at least you have someone else to blame.




Nobody to blame, it's more to excuse me.

--

Sent with eQmail-1.10

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Neil Bothwick]

On Sun, 05 Nov 2017 17:56:56 +0100, Kai Peter wrote:

> OT: Seems that since the last update of my MUA the formatting of my 
> 
> mails is broken - at least at reply's. There are extra line breaks. 
> 
> G - if you not do everything by yourself ...   ;-)

... at least you have someone else to blame.


-- 
Neil Bothwick

WinErr 678: This will end your Windows session. Do you want to play
another game?


pgplUCij_CE3p.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Kai Peter]

There are other schedulers out there that succeed where cron fails (eg



Control-M, chronos, quartz), but those are all large, bulky, designed


for big complex installs/requirements and probably not suited for 



simple



things you'd deploy out of a base in portage






Long time ago I decided to use fcron only. The reasons doesn't matter, 

but thus I can talk about fcron only. But you are right, there are a lot 

of others. At least I tried to answer Ian's question as exact as 

possible. I realized the inaccuracy in it too. Wasn't it to me? Than 


sorry for the noise.



cron is stupid and deliberately so. We really ought to let it remain 



stupid







The decision what have to be done MUST be made by the user/sysadmin



first. Than you can do the config to reach your goal. But that does go



to far now.







+1 agreed.







I've always maintained that cron cannot possibly know what the scripts



it launches deem to be correct. It's a countdown time and launcher, not


an orchestrator. It can figure out what to do if 2am happens twice 



(just



don't do it the second time), but not what should happen if 2am was



missed for any reason, including DST or poweroff or overeager ntpdate






The simplest approach to fixing missed jobs is to have the script 



itself


track what is correct. If it's lock and records files indicate 



something



didn't happen when it should, the script can do it's own repair or do



it's work twice. Or whatever else is appropriate.




@Alan, I like your writings. Unfortunately I'm not able to do so, thus 


my (very seldom) answers are sometimes to short. ;-)



OT: Seems that since the last update of my MUA the formatting of my 

mails is broken - at least at reply's. There are extra line breaks. 


G - if you not do everything by yourself ...   ;-)



--

Sent with eQmail-1.10

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Rich Freeman]

On Sun, Nov 5, 2017 at 6:43 AM, Alan McKinnon  wrote:
>
> There are other schedulers out there that succeed where cron fails (eg
> Control-M, chronos, quartz), but those are all large, bulky, designed
> for big complex installs/requirements and probably not suited for simple
> things you'd deploy out of a base in portage
>

Amusing that you classify 99.999% of all desktop installs as "big
complex installs."

But, I agree that it makes far more sense to just have desktop users
use an appropriate cron implementation designed to handle the machine
being off most of the time vs trying to use shell scripting to make
vixie cron into such an implementation.

FWIW this is probably the reasoning behind including cron-like
functionality in systemd, and having it support optionally running
jobs if the system was down during a calendar-based event.  It was
considered bare-bones functionality that any desktop or generic server
would need.  It is, of course, optional, and just about any kind of
rule is supported.  I personally use systemd-cron which basically is a
wrapper+generator around /etc/crontab and the various /etc/cron.*/
scripts.


-- 
Rich

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Alan McKinnon]

On 05/11/2017 16:28, Kai Peter wrote:
> 
> 
> On 2017-11-04 18:42, Ian Zimmerman wrote:
> 
>> On 2017-11-04 01:39, Kai Peter wrote:
> 
>>
> 
>>> > If you want to run a monthly job on a host that is not always on, do
> 
>>> > you have to pretend it's an hourly job and check in the script
> 
>>> > itself?
> 
>>>
> 
>>> This is a special case to me. IMHO special cases have to be handled
> 
>>> special or much better: avoid it.
> 
>>
> 
>> Sorry, I don't get this.  How do you avoid this situation?  By not
> 
>> having any monthly jobs?
> 
> No, monthly jobs are fine, but you mentioned a special case as the host
> is may be off at that time. Perhaps you can do: run all jobs which
> didn't run because the host was off at next startup once. fcron has such
> an option. Depending on the exact situation it could require additional
> configuration/checks.


It is not really cron's job to figure out what it thinks the user might
want if the machine was off when the job should have run. That is
expecting far too much intelligence from cron.

Cron is wall-clock time and time based:

When the time is *this*, then do *that*

anacron has some logic for this case of jobs not run but to do it, it
had to give up it's strictly wall-clock time based approach

There are other schedulers out there that succeed where cron fails (eg
Control-M, chronos, quartz), but those are all large, bulky, designed
for big complex installs/requirements and probably not suited for simple
things you'd deploy out of a base in portage

cron is stupid and deliberately so. We really ought to let it remain stupid

> The decision what have to be done MUST be made by the user/sysadmin
> first. Than you can do the config to reach your goal. But that does go
> to far now.

+1 agreed.

I've always maintained that cron cannot possibly know what the scripts
it launches deem to be correct. It's a countdown time and launcher, not
an orchestrator. It can figure out what to do if 2am happens twice (just
don't do it the second time), but not what should happen if 2am was
missed for any reason, including DST or poweroff or overeager ntpdate

The simplest approach to fixing missed jobs is to have the script itself
track what is correct. If it's lock and records files indicate something
didn't happen when it should, the script can do it's own repair or do
it's work twice. Or whatever else is appropriate.


-- 
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Kai Peter]

On 2017-11-04 18:42, Ian Zimmerman wrote:


On 2017-11-04 01:39, Kai Peter wrote:







> If you want to run a monthly job on a host that is not always on, do



> you have to pretend it's an hourly job and check in the script



> itself?







This is a special case to me. IMHO special cases have to be handled



special or much better: avoid it.







Sorry, I don't get this.  How do you avoid this situation?  By not



having any monthly jobs?


No, monthly jobs are fine, but you mentioned a special case as the host 

is may be off at that time. Perhaps you can do: run all jobs which 

didn't run because the host was off at next startup once. fcron has such 

an option. Depending on the exact situation it could require additional 


configuration/checks.



The decision what have to be done MUST be made by the user/sysadmin 

first. Than you can do the config to reach your goal. But that does go 


to far now.



--

Sent with eQmail-1.10

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Kai Peter]

On 2017-11-03 18:21, Ian Zimmerman wrote:


On 2017-11-03 02:53, Kai Peter wrote:







2. the shell script have to do some checks, e.g. the last run - I did



wrote a small 'include' script for that






Isn't your 'small include script' just another implementation of 



run-crons?


No.






If not: how does it handle _missed_ jobs, if at all?


Whatever _missed_ means - at least it's 
a question of error handling.





If you want to run a monthly job on a host that is not always on, do 



you



have to pretend it's an hourly job and check in the script itself?


This is a special case to me. IMHO special cases have to be handled 


special or much better: avoid it.






Thus it's portable. The job have to be done once.






If you want your _entire_ configuration to be portable, and my above 



point



about monthly schedule sticks, then you have to ignore the monthly


feature of all the crons on all your systems, because you cannot rely 



on



it in any one system.


My _entire_ config isn't portable, but some
 (important) parts of it.



--

Sent with eQmail-1.10

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Michael Orlitzky]

On 11/01/2017 12:55 PM, Ian Zimmerman wrote:
> 
> I actually run cronie, and AFAICT it has no built-in anacron-like
> offline schedule support

If you build cronie with USE=anacron, I think it also comes with an
"anacron" executable:

  https://github.com/cronie-crond/cronie/blob/master/README.anacron

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[Marc Joliet]

Am Sonntag, 29. Oktober 2017, 18:59:31 CET schrieb Ian Zimmerman:
> On 2017-10-29 09:16, Michael Orlitzky wrote:
> > Do you need something smarter? Install anacron, fcron, cronie, or
> > whatever. But the worst thing we can do is try to mimic those
> > intelligent crons and have it fail to do so randomly. That's still
> > your best option, by the way: rewrite your crontab to avoid run-crons,
> > and install a smart cron implementation that does what you want.
> 
> I was glad to find run-crons on gentoo when I migrated from debian,
> which does (and always has done, AFAIR) what you suggest.  The main
> reason was that anacron is also _stupid_: it thinks all months are 30
> days.  If you schedule a monthly job with anacron, it will run on
> January 1st, then on January 31st, then on March 2nd (in most years!)
> etc.  Which may not be too bad when you consider one host by itself,
> but the schedule will get all out of sync with other hosts if they run
> real (non-anacron) monthly cronjobs.
> 
> So, for hosts that are not up 24h per day, anacron is _not_ a full
> solution.  Something like run-crons is needed.  If the gentoo
> implementation is too opaque or buggy, it should be rewritten, not
> discarded.

It's nice that anacron apparently sucks, but what about fcron and cronie?  
I've always wondered why people who need these features don't just one of 
those.  Is there any reason not to?

(FTR: I used fcron for several years before migrating to systemd timers 
specifically because of its support for running missed jobs.)

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup


signature.asc
Description: This is a digitally signed message part.