Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-13 Thread Karen Etheridge 
On Tue, Jul 11, 2017 at 1:46 AM, Binarus  wrote:
> But if the application misbehaves because there is no correct time zone
> data available at that moment, I won't get into trouble. No reasonable
> person can expect that applications doing calculations on local dates
> and times behave correctly if a time zone / DST change is announced just
> a day before it actually happens.

Depending on what "reasonable people" assume often gets
​one into surprisingly
unreasonable positions.

> As far as i know, it is consensus in most legal systems that it is
> perfectly acceptable to use the time zone data which is currently
> available for your O/S for time calculations (provided that you update
> the O/S regularly using the appropriate mechanism).

I am not aware of a single legal system that sets out *anything* to do with
an operating system whatsoever, let alone what might be acceptable
parameters
for the functioning of such software.  Legal systems deal in absolutes,
and care not whether something is calculated on paper or through another
means:
"my computer couldn't figure it out" is not a reasonable excuse.

These lists are also insightful:
http://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-about-time

http://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-about-time

Re: TimeZones and politics Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-12 Thread Bill Ricker 
> Well, first of all, I did not want to insult or even criticize anybody.
> Please forgive me if my wording even made the slightest impression of
> doing so - regrettably, I am not a native English speaker.

In which case, you are doing very well, i would not have guessed.
( If I replied in French it would be painfully obvious i had merely
poor schooling in same. )

Related,

> Of course, I meant "ever" instead of "never" in the sentence above.

Idiomatically exchanging those (and various other double negations) is
regrettably becoming acceptable in informal usage in recent decades.
Which is even more confusing for second-language folks than for
irony-impaired native speakers.
[ And one can argue if my parenthetical addition to the sentence
subject changes the required number-agreement of the verb from 'is' to
'are'. Or more simply that i overuse parentheticals with both braces
and em-dashes. I plead Guilty to the latter. :-) ]

> Actually, I am feeling the highest respect towards the experts who are
> helping with the time zone database updates, are bringing them to a
> machine readable form and are writing software modules which are dealing
> with the overwhelming bunch of time calculating oddities in an
> impressive manner. I am really very grateful for that and can't thank
> those volunteer experts enough.
>
> I just wanted to say that the statement "... the change happens at 2:00
> in Chicago ... " might not be sufficient for my case because I am trying
> to write a web application which deals with date and time calculations
> in local time zones around the world, and that I'd like the application
> to behave well (as far as possible) even in the weirdest cases.

That is pretty much the English expansion what the Olson/TZinfo file
says the EDT rule for the America/Chicago zone is.  Other rules may
say something else.

>>>  and the next law could determine the switch to
>>> happen at 08:48:27 am, and

Theoretically possible ...

>> It could in theory, but would be beyond atypical.
>> (Why? Perhaps a(n) hereditary national leader wants to honor the
>> moment of his father's birth?)

> Yes, I had something like that in my mind (although I doubt that I will
> have the application translated into the languages which are spoken in
> such regions :-))

Even worse, users anywhere may wish to schedule a meeting (or
whatever) on Pyongyang time (or whereever) without even being in that
Locale, so may reasonably request an event in some other TZ than
theirs or system native.E.g. if i wanted to tape the recent FIFA cup
winners' cup final in Petrograd/StPetersburg off a satellite feed,
entering it in TZ of the upload would make some sense.

>> (The file format and software should handle  08:48,not sure about 08:48:27 ?)

> Thanks for the hint. I have never looked into a time zone file yet
> (hoping that I could solve all my problems by using modules instead of
> doing my own thing).

Looking at  the Olson/TZinfo data is a massive rabbit-hole but my what
interesting data it is.

> I am feeling that it eventually would be better to use the weirdest
> examples (risking making a complete fool of myself if missing something)
> instead of my time zone (which is Europe/Berlin which has been
> well-behaving so far, like America / Chicago, so is probably not an
> example I should use to explain my problems ...).

Indeed, that can be useful to explore the edgecases. Kathmandu is a good one.
The non-political 'Etc/GMT+nn' zones with inverted sign zones
(inverted wrto UTC defined zones) are also good test cases to break
programs :-).


> I have chosen Venezuela because I am not an expert in this field, and it
> was the first time I heard of a time shift that was *purely* a political
> statement, so I have been shocked backed then. Until the moment when it
> happened, I had considered such a thing impossible, so I can still
> remember it (and still can't believe it - somehow :-)).

And a good example it is !  (Plenty more down the rabbit hole.)

> This is very interesting; I didn't even know that quarter-hour offsets
> do exist.

Heh. We try to forget :-).
Currently there are only x:45, but there may have been x:15's previously.

>> Yes, it's political.

Aside on politics - opposition to DST isn't new either.

My great-grandfather was among the many US dairy farmers who railed
against DST adoption.  The cows wanted to be milked as soon after dawn
as he could get on his stool, the cows didn't care about clocks let
alone punch clocks, so a farmer needed to be up with the rooster or
before.  And would work through every daylight hour in summer. If he
seemingly had leisure time before sunset that meant he should use the
time to buy someone else's standing hay and go mow and bale it ! So
the stores to which he delivered corn and milk etc changing their
hours twice per year inconvenienced him.


-- 
Bill Ricker
bill.n1...@gmail.com
https://www.linkedin.com/in/n1vux

Re: TimeZones and politics Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-12 Thread Eric Brine 
On Wed, Jul 12, 2017 at 11:07 AM, Binarus  wrote:

> On 11.07.2017 20:50, Bill Ricker wrote:
> > On Tue, Jul 11, 2017 at 4:07 AM, Binarus  wrote:
> >>
> >> On 10.07.2017 20:14, Eric Brine wrote:
> >>> I don't understand the conditions. The law determines when the
> switching
> >>> of offsets from UTC happen, not some person. The switch doesn't happen
> >>> at 08:48:27 am in Chicago; it happens at 2am.
> >>
> >> This point of view is a bit U.S. centric. Indeed, you are describing how
> >> it *should* be, but
> >
> > We tend to use the timezone nearest us for examples because we're less
> > likely to be wrong and thence get dragged into discussing the example
> > instead of the concept.
> >
>
> Well, first of all, I did not want to insult or even criticize anybody.
> Please forgive me if my wording even made the slightest impression of
> doing so - regrettably, I am not a native English speaker.
>
> Actually, I am feeling the highest respect towards the experts who are
> helping with the time zone database updates, are bringing them to a
> machine readable form and are writing software modules which are dealing
> with the overwhelming bunch of time calculating oddities in an
> impressive manner. I am really very grateful for that and can't thank
> those volunteer experts enough.
>
> I just wanted to say that the statement "... the change happens at 2:00
> in Chicago ... " might not be sufficient for my case because I am trying
> to write a web application which deals with date and time calculations
> in local time zones around the world, and that I'd like the application
> to behave well (as far as possible) even in the weirdest cases.
>

Chicago was the OP's example. I'm not even American.

My point was that the time of change is not up to debate; it's in the time
zone db, so the information is available. It's not as arbitrary as I
thought the OP made it sound.

Others have pointed out this the time zone db can change with very little
notice since TZ definitions can change with very little notice.

Re: TimeZones and politics Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-12 Thread Binarus 
On 12.07.2017 17:07, Binarus wrote:
> [...] None of my Linux
> servers and applications (written by me or others) never suffered from a
> problem due to incorrect time zone data.

Of course, I meant "ever" instead of "never" in the sentence above.

Regards,

Binarus

Re: TimeZones and politics Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-12 Thread Binarus 
On 11.07.2017 20:50, Bill Ricker wrote:
> On Tue, Jul 11, 2017 at 4:07 AM, Binarus  wrote:
>>
>> On 10.07.2017 20:14, Eric Brine wrote:
>>> I don't understand the conditions. The law determines when the switching
>>> of offsets from UTC happen, not some person. The switch doesn't happen
>>> at 08:48:27 am in Chicago; it happens at 2am.
>>
>> This point of view is a bit U.S. centric. Indeed, you are describing how
>> it *should* be, but
> 
> We tend to use the timezone nearest us for examples because we're less
> likely to be wrong and thence get dragged into discussing the example
> instead of the concept.
> 

Well, first of all, I did not want to insult or even criticize anybody.
Please forgive me if my wording even made the slightest impression of
doing so - regrettably, I am not a native English speaker.

Actually, I am feeling the highest respect towards the experts who are
helping with the time zone database updates, are bringing them to a
machine readable form and are writing software modules which are dealing
with the overwhelming bunch of time calculating oddities in an
impressive manner. I am really very grateful for that and can't thank
those volunteer experts enough.

I just wanted to say that the statement "... the change happens at 2:00
in Chicago ... " might not be sufficient for my case because I am trying
to write a web application which deals with date and time calculations
in local time zones around the world, and that I'd like the application
to behave well (as far as possible) even in the weirdest cases.

>>  and the next law could determine the switch to
>> happen at 08:48:27 am, and
> 
> It could in theory, but would be beyond atypical.
> (Why? Perhaps a(n) hereditary national leader wants to honor the
> moment of his father's birth?)

Yes, I had something like that in my mind (although I doubt that I will
have the application translated into the languages which are spoken in
such regions :-))

> (The file format and software should handle  08:48,not sure about 08:48:27 ?)

Thanks for the hint. I have never looked into a time zone file yet
(hoping that I could solve all my problems by using modules instead of
doing my own thing).

> [...]
> Do please feel free to use your TZ (or others you know well( in your examples 
> !

I am feeling that it eventually would be better to use the weirdest
examples (risking making a complete fool of myself if missing something)
instead of my time zone (which is Europe/Berlin which has been
well-behaving so far, like America / Chicago, so is probably not an
example I should use to explain my problems ...).

> But don't expect us to be shocked by Venezualan TZ mandates; only the
> short notice made it more eccentric than other isolated half hour
> zones e.g.Newfoundland's UTC-03:30/-02:30. Large expanses of on half
> offset in South Asia and Australia may serve a purpose, but I can't
> explain it.

I have chosen Venezuela because I am not an expert in this field, and it
was the first time I heard of a time shift that was *purely* a political
statement, so I have been shocked backed then. Until the moment when it
happened, I had considered such a thing impossible, so I can still
remember it (and still can't believe it - somehow :-)).

> But AFAIK all TZs to-date that are actually TZs (as in, not Local Mean
> Time, the evil that TZs fixed for the 19thC steam & telegraph Railroad
> schedules) have been at no finer than quarter hour offset from
> GMT/UTC, and mostly hour offsets, with some rather famous half-hours
> (India, Australia) and infamous (Venezuela).
> The quarter-hour offsets are peculiar and one can only ask "Why?"
> yet always is a Why.  Kathmandu's UTC+05:45 maybe the strangest but is
> still intensely geo-political -- splitting the difference between two
> powerful neighbors,China/Tibet +06 and India(Kolkata) +05:30. Picking
> either could anger the other neighbor, so split the difference.

This is very interesting; I didn't even know that quarter-hour offsets
do exist.

> Yes, it's political.
> But to us here, the output of the political process is our input data
> We try to make things behave as humans will expect according to their
> Quartz Timex  or iPhone Clock app.

And obviously, you are doing so with great success! None of my Linux
servers and applications (written by me or others) never suffered from a
problem due to incorrect time zone data.

And I have learned a lot from your post ...

Thank you very much again,

Binarus

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-12 Thread Binarus 
On 12.07.2017 14:22, Zefram wrote:
> Binarus wrote:
>> So is there any change to add an according API function to DT:TZ?
> 
> Yes, but we wouldn't want to rush it.  There's more than one
> implementation of the API, and we want to be sure to design it correctly
> the first time.
> 
> Perhaps it could be a ->offsets_for_local_datetime method (note
> plural in the name), which returns a sorted list of all the timezone
> offsets that are applicable to a specified local time.  Normally the
> list would have length 1, the element being the same value returned by
> ->offset_for_local_datetime.  Ambiguous local times (regardless of the
> source of ambiguity) yield more than one offset.  Non-existent local times
> (skipped due to clocks going forward) yield an empty list without error.
> 

That would be absolutely great. Could you please let us know when you
have started working on it? I didn't see a datetime-announce mailing
list yet :-)

(In the meantime, I'll try to work around the problems somehow).

Regards and thank you very much again,

Binarus

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-12 Thread Zefram 
Binarus wrote:
>So is there any change to add an according API function to DT:TZ?

Yes, but we wouldn't want to rush it.  There's more than one
implementation of the API, and we want to be sure to design it correctly
the first time.

Perhaps it could be a ->offsets_for_local_datetime method (note
plural in the name), which returns a sorted list of all the timezone
offsets that are applicable to a specified local time.  Normally the
list would have length 1, the element being the same value returned by
->offset_for_local_datetime.  Ambiguous local times (regardless of the
source of ambiguity) yield more than one offset.  Non-existent local times
(skipped due to clocks going forward) yield an empty list without error.

-zefram

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-12 Thread Binarus 
On 11.07.2017 18:29, Karen Etheridge wrote:
> On Tue, Jul 11, 2017 at 1:46 AM, Binarus  > wrote:
>> But if the application misbehaves because there is no correct time zone
>> data available at that moment, I won't get into trouble. No reasonable
>> person can expect that applications doing calculations on local dates
>> and times behave correctly if a time zone / DST change is announced just
>> a day before it actually happens.
> 
> Depending on what "reasonable people" assume often gets
> ​one into surprisingly
> unreasonable positions.

I can live quite well with the most court decisions of the later time in
Germany.

>> As far as i know, it is consensus in most legal systems that it is
>> perfectly acceptable to use the time zone data which is currently
>> available for your O/S for time calculations (provided that you update
>> the O/S regularly using the appropriate mechanism).
> 
> I am not aware of a single legal system that sets out *anything* to do with
> an operating system whatsoever, let alone what might be acceptable
> parameters
> for the functioning of such software.  Legal systems deal in absolutes,
> and care not whether something is calculated on paper or through another
> means:

This is not true.

Two examples:

1) Legal rules in Germany explicitly state that a company must operate
its IT equipment and processes "dem aktuellen Stand der Technik
entsprechend" (in English, that roughly means "according to the state of
the art", in other words "implementing / using the software, hardware,
techniques and processes which currently are considered safe and correct
by the majority of experts, while being still possible for a company to
be implemented considering economic aspects").

The sense of this is to prevent damage to other people. For example, due
to this rule, a company must run up-to-date virus protection to prevent
malware being sent from their email servers to their partners, and they
must run backups following a reasonable plan to protect themselves from
damage.

Law and legal rules (except in special cases, see below) explicitly do
*not* list what the current state of the art is. Actually, they can't,
because the state of the art develops way too fast for legislation to
follow.

Standards and certifications can help with running a company according
to the state of the art, and by a certification (best known in Germany:
ISO 9001), a company can prove that they at least have tried to operate
as required. Nevertheless, it is perfectly legal to run a company
without having such certifications. In that case, management has to
decide on their own how to run the company according to the state of the
art, and proving that they have hardly tried to do so (if necessary)
will be more difficult.

As a result, there have been hundreds of lawsuits where courts had to
decide if a damage has happened (and a company was liable) because the
company has not followed the state of the art.

How does a judge decide such a case which often is technically
complicated? He invites "vereidigte Sachverständige" (I suppose this is
called "sworn experts" in English) and let them explain what the state
of the art was and if the company in question followed it. Of course,
two different experts will give three different opinions ...

To make things even more complicated, the state of the art is different
for different company sizes and depending on the environment where the
company operates. This is where reasonableness comes into play:

No reasonable person could expect a company with a yearly turnover of $
100,000 to run a backup strategy which costs 1,000,000 per year.

A small company, having three main customers and not operating in
critical infrastructure, for sure can't be expected by a reasonable
person to mirror its website or database over three different
datacenters or to have a 24/7 emergency team.

In contrast, a big German bank *can* be expected to have data up-to-date
and synchronized at any point in time in multiple different locations,
and it can be expected to have 24/7 emergency teams in IT.

As you can see, basically law only states: Companies must be run like a
reasonable partner or customer could expect given the state of the art
and considering economic aspects.

What this exactly means is not stated by law, but finally left up to
courts to decide, and the courts will follow some experts' opinion in
most cases.

2) Having developed medical software and hardware over the last decade
(including class C software according to IEC 62304), I may state the
following:

In this special field, law concretely tells what standard must be
followed when developing such software or devices. Every software and
every device must be approved by certain organizations or authorities
before it may be sold. This is a very complicated and expensive process.

Interestingly enough, there is one thing which can make life (a little
bit) easier: Use as much OTS software as possible. OTS means
"

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Binarus 
On 11.07.2017 15:53, Zefram wrote:
> Binarus wrote:
>> As the documentation tells us, DateTime always chooses the later time
>> when calculating with ambiguous times,
> 
> This logic is actually in DateTime::TimeZone, where DateTime invokes
> it via the ->offset_for_local_datetime method.  The internal logic
> is able to walk the sequence of observances, and when it finds a
> matching observance it looks ahead to the next one to check specially
> for ambiguity.  It returns the later span if there's an ambiguity due
> specifically to a change from DST to non-DST; it doesn't have guaranteed
> behaviour in any other ambiguity scenario.  DT:TZ doesn't document this
> feature, and doesn't offer any other form of ambiguity detection in its
> API (though of course it would be possible to add some).

Now I know why I didn't find according documentation. At least I was not
completely wrong in believing that its all in there (although I
suspected the wrong module).

So is there any change to add an according API function to DT:TZ?

> 
>>   and if you subtract an hour from
>> the later (ambiguous) time, you'll get the same time, but the earlier
>> one (provided the clock is turned back by an hour when switching back).
> 
> You can't rely on offset changes, even specifically ones for DST, being of
> an hour.  Australia/Lord_Howe (Lord Howe Island) does a regular half-hour
> DST jump.

That was the reason why I have mentioned the weird preconditions in my
first post.

Another thing is that I eventually have to re-think the behavior of my
application. In some of your posts, you indirectly have mentioned that
there might be other reasons for ambiguous times than switching from DST
to standard time, and of course, you are right.

While my application will not be concerned by leap seconds (the
application actually could do its thing when the respective moment is
passed the first time, and ignore the second occurrence), it will be
concerned when the time is turned back by at least 15 minutes, whether
or not this happens due to DST switching or permanently because some
dictator has gone nuts. In that case, the application had to do its
thing upon the first occurrence of the respective time as well as upon
the second occurrence.

That means that the API which I would like DT::xx to provide would have
to tell me if a certain arbitrary date-time will occur twice *whether or
not* this will happen due to DST switching.

I see that this is difficult to implement. I have to think about the
situation and will try to find a solution at the application level.

Thank you very much,

Binarus

Re: TimeZones and politics Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Zefram 
Bill Ricker wrote:
>On Tue, Jul 11, 2017 at 4:07 AM, Binarus  wrote:
>>  and the next law could determine the switch to
>> happen at 08:48:27 am, and
>
>It could in theory, but would be beyond atypical.

There have historically been some DST rules calling for changes at
00:01.  For example, America/Moncton (New Brunswick, Canada) changed
00:01->01:01 and 00:01->23:01 from 1993 to 2006.  The 00:01->23:01
change is particularly unusual for turning the date back.  The Olson
database also shows some one-off changes (not DST changes) that were
not minute-aligned in the previously-prevailing local time, but these
are only when changing from local mean time to a UT-based standard time,
at a time that's minute-aligned in the new standard time.

>(The file format and software should handle  08:48,not sure about 08:48:27 ?)

The file format, and all the software I know about, can handle transitions
at any second.

-zefram

TimeZones and politics Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Bill Ricker 
On Tue, Jul 11, 2017 at 4:07 AM, Binarus  wrote:
>
> On 10.07.2017 20:14, Eric Brine wrote:
> > I don't understand the conditions. The law determines when the switching
> > of offsets from UTC happen, not some person. The switch doesn't happen
> > at 08:48:27 am in Chicago; it happens at 2am.
>
> This point of view is a bit U.S. centric. Indeed, you are describing how
> it *should* be, but

We tend to use the timezone nearest us for examples because we're less
likely to be wrong and thence get dragged into discussing the example
instead of the concept.

> 1) The timezone database is updated several times a year, and that
> surely has its reason. Obviously, laws are changing (a part of them very
> surprisingly and fast),

Oh yes. This is indeed why.

The TZ info file is interesting only because it is worldwide and political.

>  and the next law could determine the switch to
> happen at 08:48:27 am, and

It could in theory, but would be beyond atypical.
(Why? Perhaps a(n) hereditary national leader wants to honor the
moment of his father's birth?)
(The file format and software should handle  08:48,not sure about 08:48:27 ?)

In practice, the only odd-seconds-level transitions are the positive
or negative leap second insertion, which are even weirder when you
think too hard than the (typically) hour added and removed in DST.
(At leap-second, there is a minute with either 59 or 61 seconds,
so there are times legal then that aren't elsewhen, or a single second
that is illegal then. Most software just continues to say :59 for an
additional second to avoid messing up downstream software that thinks
23:59:60 GMT is syntactically illegal even though on certain on 12-31
and 06-30 when mandated it should be semantically Legal. Worse, unlike
TZ DST, it occurs simultaneously world wide and so may be during the
business day in Oceania and East Asia, even of the half or quarter
hour in those odd offset TZs. The ambiguous hour during DST->ST
rollback and the illegal hour vanished at ST->DST roll forward are
semantically illegal but syntactically legal. Which allows them to
propagate further before something chokes. )

> 2) Laws are made by persons, in some cases by a single person. For
> example, in 2007, Venezuela's president Hugo Chavez moved Venezuela's
> time zone by 30 minutes backwards without any real reason, but just as a
> political statement, putting the country in a half-hour time zone. This
> shows that weird things can happen to time zones at all times and very
> surprisingly.

Indeed.
I try not to use such eccentric examples to avoid dragging
politics into a technical discussion, but yes, most of the urgent
updates are results of small-country power-politics and/or equally
political regional re-alignments. Large democracies tend to plan their
changes well in advance, e.g. USA 2007, we had plenty of warning; the
question was if IT had updated production systems adequately.
(Shocking Answer: No.)
  Hence i only _reluctantly_ used the March update adding Haiti to EDT
2017 at the last minute to demonstrate short notice.
(I haven't looked up news stories but infer restoring EDT
indicates a milestone in their infrastructure's hurricane recovery; if
so, congratulations.)

(  I'm hoping with New England states debating laws that make
switching TZ conditional on adjacent states doing likewise we don't
have a last minute vote in e.g. NH triggering Maine & Mass & NH all
flipping on short notice ... but it could happen... but it's still
hypothetical, so a weaker example.)

> Thus, when developing an application targeted to audience all over the
> world, you can't rely on what you know about your own country.

Absolutely.
Everyone who has been in the TZ data knows this.
We work on this TZ software because we are crazy enough to care about
these edgecases.

But we'll still typically use our own TZ for our DST examples because
it's embarrassing and derailing to get e.g. Haiti's DST transition
wrong in an email example.

(And then instead of US-centrism we'd be called worse things for
talking-down to a small country _and_ being wrong simultaneously.)

Do please feel free to use your TZ (or others you know well( in your examples !

But don't expect us to be shocked by Venezualan TZ mandates; only the
short notice made it more eccentric than other isolated half hour
zones e.g.Newfoundland's UTC-03:30/-02:30. Large expanses of on half
offset in South Asia and Australia may serve a purpose, but I can't
explain it.

But AFAIK all TZs to-date that are actually TZs (as in, not Local Mean
Time, the evil that TZs fixed for the 19thC steam & telegraph Railroad
schedules) have been at no finer than quarter hour offset from
GMT/UTC, and mostly hour offsets, with some rather famous half-hours
(India, Australia) and infamous (Venezuela).
The quarter-hour offsets are peculiar and one can only ask "Why?"
yet always is a Why.  Kathmandu's UTC+05:45 maybe the strangest but is
still intensely geo-political -- splitting the di

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Zefram 
Binarus wrote:
>Did you memorize the tzfile of 1969 :-)

I looked through the Olson source files.  I could also have automated
a search through the compiled zone data.

-zefram

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Binarus 
On 11.07.2017 15:39, Zefram wrote:
> Dave Rolsky wrote:
>> If you're trying to avoid these, the best advice I could give would be to
>> avoid the 12am-4am window, which AFAIK is when most (all?) transitions have
>> occurred historically.
> 
> Most, but there are both historical and current exceptions.
> America/Godthab (west Greenland) changes 22:00->23:00 and 23:00->22:00
> (base offset -03:00, with transition at 01:00 UT per EU rules).
> America/Santiago (Chile) changes 00:00->23:00.  Pacific/Easter (Easter
> Island) changes 22:00->23:00 and 22:00->21:00 (changing at the same
> time as Chile, but with base offset 2 hours to the west).  Historically,
> Africa/Casablanca (Morocco) changed 12:00->13:00 in a DST change in 1967.
> And Pacific/Kwajalein (part of the Marshall Islands) jumped across the
> international date line in the unfashionable direction, repeating 23
> hours of 1969-09-30.
> 
> -zefram
> 

Wow, that's quite a cool bit of information. Thank you, very
interesting! Did you memorize the tzfile of 1969 :-)

Regards,

Binarus

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Zefram 
Binarus wrote:
>As the documentation tells us, DateTime always chooses the later time
>when calculating with ambiguous times,

This logic is actually in DateTime::TimeZone, where DateTime invokes
it via the ->offset_for_local_datetime method.  The internal logic
is able to walk the sequence of observances, and when it finds a
matching observance it looks ahead to the next one to check specially
for ambiguity.  It returns the later span if there's an ambiguity due
specifically to a change from DST to non-DST; it doesn't have guaranteed
behaviour in any other ambiguity scenario.  DT:TZ doesn't document this
feature, and doesn't offer any other form of ambiguity detection in its
API (though of course it would be possible to add some).

>   and if you subtract an hour from
>the later (ambiguous) time, you'll get the same time, but the earlier
>one (provided the clock is turned back by an hour when switching back).

You can't rely on offset changes, even specifically ones for DST, being of
an hour.  Australia/Lord_Howe (Lord Howe Island) does a regular half-hour
DST jump.

-zefram

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Zefram 
Dave Rolsky wrote:
>If you're trying to avoid these, the best advice I could give would be to
>avoid the 12am-4am window, which AFAIK is when most (all?) transitions have
>occurred historically.

Most, but there are both historical and current exceptions.
America/Godthab (west Greenland) changes 22:00->23:00 and 23:00->22:00
(base offset -03:00, with transition at 01:00 UT per EU rules).
America/Santiago (Chile) changes 00:00->23:00.  Pacific/Easter (Easter
Island) changes 22:00->23:00 and 22:00->21:00 (changing at the same
time as Chile, but with base offset 2 hours to the west).  Historically,
Africa/Casablanca (Morocco) changed 12:00->13:00 in a DST change in 1967.
And Pacific/Kwajalein (part of the Marshall Islands) jumped across the
international date line in the unfashionable direction, repeating 23
hours of 1969-09-30.

-zefram

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Binarus 
On 11.07.2017 01:09, Zefram wrote:
> Binarus wrote:
>> Using DateTime, is it possible to tell in advance if a certain date-time
>> which is given in a certain locale will be ambiguous due to switching
>>from DST to standard time?
> 
> That is tricky.  I don't think our APIs provide any way to do it.
> Thinking about the facilities available a bit lower down, the way I'd
> probably approach it is to look at the list of all the offsets ever used
> in that timezone (not available through any API, but extractable from
> the tzfile).  I'd compute for each the UT time that would be represented
> by the specified local time with that offset, and then use the regular
> API to convert that UT time to the local time in the specified timezone
> (or equivalently just to look up the zone's offset for that UT time).
> Look at which local times come out matching the specified local
> time (which offsets match the candidate offset that we were trying).
> If there's more than one match, that's an ambiguous local time.  If there
> are no matches, it's a non-existent local time.

At first, thank you very much for your answer!

I see. I always thought that the tzfile only contained the offsets for
standard time (and not DST). Obviously, I have been wrong (I never had a
look into a tzfile yet because I hope that I could solve my problem
without reading it out directly).

> Something close to this can actually be done in the C time API.  You
> don't get to ask what all the zone's offsets are, but in the local->UT
> conversion you can specify whether DST is in effect.  Giving both states
> of that flag gives you two UT times, which you can then convert back
> to local to check whether they come out with the same DST flag state.
> This will work for regular DST changes, but not for offset changes that
> are unrelated to DST.

This is very interesting. Actually, I am doing more things in C than in
Perl (but on other architectures), but since I thought that DateTime was
mirroring all capabilities libc has in this respect, I did not look into
the C time API yet. I have no dislike against writing a part of the
application in C (but then will have to learn how to call C from Perl).

Again, thanks for all advice,

Binarus

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Binarus 
On 10.07.2017 23:17, Bill Ricker wrote:

> On Mon, Jul 10, 2017 at 2:19 PM, Dave Rolsky  > wrote:
> 
>[...]
> If you're trying to avoid these, the best advice I could give would
> be to avoid the 12am-4am window, which AFAIK is when most (all?)
> transitions have occurred historically.
> 
> 
> ​Or postpone conversion from text time to DateTime or time_t ; store as
> Zulu ​what is absolute and store as Local to a Zipcode what is
> local/human, and do just-in-time conversion it to time_t or DateTime as
> needed  for computation, according to the rules for that date when
> inspected.
> 
> 
> In 2007 to prepare for the algorithm change -- the time-change date
> change -- i used a Perl script to check what the OS TZ said next
> transition for USA Eastern time-zone was for something like 100 servers
> to see which had been updated and which hadn't. (Had to delegate
> actually running it to someone who had ssh or sudo everywhere. I
> packaged a script and did data reduction and mapped hosts to
> applications. Also did a find to find DateTime.pm and Java JRE/JDK
> equivalent.)
> See contemporaneous notes for more information.
>   http://use.perl.org/use.perl.org/_n1vux/journal/32234.html
> and
>   http://www.nntp.perl.org/group/perl.datetime/2017/03/msg8132.html
> The script dst.pl  is still available ​as
>   
> https://community.hpe.com/hpeb/attachments/hpeb/itrc-153/7220/1/285887.pl
> from
>  
> https://community.hpe.com/t5/Patches/Daylight-Savings-Time-for-HP-UX-10-20/m-p/3923972
>  
> 
> Note that this perl script uses libc() -> OS TZinfo (and requires no
> CPAN iirc), not DateTime::TimeZone's, but the same logic could scan
> forward for the flip with DateTime. I didn't do that because i could
> just check the DateTime version iirc. 
> 
>But as described by Dave and elaborated above, trying to predict the
> future and storing the predicted conversion allows governments to give
> you bugs (that your users will blame you for, alas, unfair though that
> seems) that upgrading your DateTime won't fix, since you already did the
> conversion. Whereas if you store the user input and convert it at-need,
> you can update the TZ info by updating OS and/or modules when
> politicians make a change and calculations will be right on the day.
> 
> For those wanting much too much more info on TimeZones and the trouble
> with keeping them current: 
> * Prior releases of DateTime::TimeZone had a maintainers' README that
> referenced the master node of TZ nerdery
>http://www.twinsun.com/tz/tz-link.htm
>but that file has been supplanted by a README.md build from the
> pod2man , so i include that link, along with Wikipedia for an easier enty -
> * https://en.wikipedia.org/wiki/Tz_database
>(which redirects from https://en.wikipedia.org/wiki/Olson_database )
> * and the TZ data maintainer mailinglists
>  tz   news:comp.time.tz  
> https://mm.icann.org/listinfo/tz http://mm.icann.org/pipermail/tz/
>  & tz-announce http://mm.icann.org/pipermail/tz-announce/
> (  Yes, the TZ data is now an official function of the Internet Assigned
> Numbers Authority (IANA) and is supperted from IETF and ICANN. )
> (I am pleasantly surprised there hasn't been a TZ data release since
> March ... which was the just-in-time update for Haiti resuming DST this
> year, the sort of predict-the-future problem mentioned. If one has
> worldwide users using their own TZs, one needs regular updates!)

Thank you very much for your elaborated answer and all the links! I will
try to read that stuff within the next days.

The nature of the application requires the most recent time zone
information to be used in every case, so I'd like to avoid "calculating
the future" (as a last resort, I could "calculate the future" every time
the timezone database is updated).

But I still think that DateTime already knows whether a date-time is (or
will be) ambiguous (according to the timezone data known so far, which
would perfectly fit my requirements - see my previous post), so we
"only" would need a new function to query that information. I could be
wrong, though.

If DateTime can't provide an appropriate function, I eventually will try
to extract the daylight switching information from the timezone database
and try to make an own algorithm to solve the problem. The script you
mentioned is probably a good starting point (did not look into it yet).

Regards,

Binarus

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Binarus 
On 10.07.2017 20:19, Dave Rolsky wrote:
> While you could in theory write code that would be correct for all past
> datetimes, the future doesn't work the same way. As Eric noted, time
> zones are political. I have seen DST transitions altered with mere days
> (or less!) notice given. This means that anything you determine about
> the future could be wrong.
> 

Of course, you are right. I should have mentioned the policy I'd like to
follow (regarding my application, this partly has to do with liability
in a legal sense):

If the application misbehaves although correct time zone data is
available at that moment, I'll have problems, and I have to avoid that
under all circumstances.

But if the application misbehaves because there is no correct time zone
data available at that moment, I won't get into trouble. No reasonable
person can expect that applications doing calculations on local dates
and times behave correctly if a time zone / DST change is announced just
a day before it actually happens.

As far as i know, it is consensus in most legal systems that it is
perfectly acceptable to use the time zone data which is currently
available for your O/S for time calculations (provided that you update
the O/S regularly using the appropriate mechanism).

So I am not interested in the pathological cases or "the future", but I
have to make sure that my application does not misbehave when a certain
date-time will occur twice *and* the current time zone data already has
this information.

> If you're trying to avoid these, the best advice I could give would be
> to avoid the 12am-4am window, which AFAIK is when most (all?)
> transitions have occurred historically.

I have got the same impression. But on the other hand, we can't be sure,
and (according to my naive understanding) it's all in DateTime already:
As the documentation tells us, DateTime always chooses the later time
when calculating with ambiguous times, and if you subtract an hour from
the later (ambiguous) time, you'll get the same time, but the earlier
one (provided the clock is turned back by an hour when switching back).
It couldn't do that if it didn't know that the time is ambiguous, could it?

So, since DateTime obviously knows whether a date-time is ambiguous or
not (according to the time zone database currently in place), I'd like
to ask if there really is no way to implement a method that lets us
query that information.

Thank you very much,

Binarus

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-11 Thread Binarus 
On 10.07.2017 20:14, Eric Brine wrote:
> I don't understand the conditions. The law determines when the switching
> of offsets from UTC happen, not some person. The switch doesn't happen
> at 08:48:27 am in Chicago; it happens at 2am.

This point of view is a bit U.S. centric. Indeed, you are describing how
it *should* be, but

1) The timezone database is updated several times a year, and that
surely has its reason. Obviously, laws are changing (a part of them very
surprisingly and fast), and the next law could determine the switch to
happen at 08:48:27 am, and

2) Laws are made by persons, in some cases by a single person. For
example, in 2007, Venezuela's president Hugo Chavez moved Venezuela's
time zone by 30 minutes backwards without any real reason, but just as a
political statement, putting the country in a half-hour time zone. This
shows that weird things can happen to time zones at all times and very
surprisingly.

(Note: Venezuela has been in that time zone for about 9 years and now is
back in UTC-4:00, but that does nothing to business here).

Thus, when developing an application targeted to audience all over the
world, you can't rely on what you know about your own country.

Thank you very much,

Binarus

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-10 Thread Zefram 
Binarus wrote:
>Using DateTime, is it possible to tell in advance if a certain date-time
>which is given in a certain locale will be ambiguous due to switching
>from DST to standard time?

That is tricky.  I don't think our APIs provide any way to do it.
Thinking about the facilities available a bit lower down, the way I'd
probably approach it is to look at the list of all the offsets ever used
in that timezone (not available through any API, but extractable from
the tzfile).  I'd compute for each the UT time that would be represented
by the specified local time with that offset, and then use the regular
API to convert that UT time to the local time in the specified timezone
(or equivalently just to look up the zone's offset for that UT time).
Look at which local times come out matching the specified local
time (which offsets match the candidate offset that we were trying).
If there's more than one match, that's an ambiguous local time.  If there
are no matches, it's a non-existent local time.

Something close to this can actually be done in the C time API.  You
don't get to ask what all the zone's offsets are, but in the local->UT
conversion you can specify whether DST is in effect.  Giving both states
of that flag gives you two UT times, which you can then convert back
to local to check whether they come out with the same DST flag state.
This will work for regular DST changes, but not for offset changes that
are unrelated to DST.

-zefram

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-10 Thread Bill Ricker 
On Mon, Jul 10, 2017 at 2:19 PM, Dave Rolsky  wrote:

> While you could in theory write code that would be correct for all past
> datetimes, the future doesn't work the same way. As Eric noted, time zones
> are political. I have seen DST transitions altered with mere days (or
> less!) notice given. This means that anything you determine about the
> future could be wrong.
>

​Yes, this. Seen same. Nice polities give much more notice but certain
countries have had decrees canceling a short notice change after IT
scrambled and couldn't finish, so IT had to unscrambe even faster. Or
canceling a suspension at last minute.

Hypothetically, that 'half' of Indiana could on short notice decide to do
DST after all next year. One might hope it would be planned in advance ...
but notice of Haiti resuming DST this year was pretty short.

Closer to home and concrete, Maine is actively lobbying to convince Mass &
NH to stay on EDT all winter (effectively switching to Atlantic Standard
time in winter and EDT in the summer), so no one knows if the EDT-EST
transition will happen this fall in New England this year or next year?

USA DST was a fixed algorithm for decades and we got used to it and tech
got lazy.
(Many devices produced pre 2006 had no way to update their built in DST
algorithm! I had to set timezone manually 4x per year on my old phone and
GPS. )
So we had quite the snipe hunt to find all separate time zone file
copies in the distributed corp datacenters when the algorithm would change
in 2007 (from April/October to March/November) and check which applications
would (a) care about DST at all (b) use OS, Java, Perl module, or other
TZinfo/zonefile source, and remediate appropriately the few that _would_
hiccup.
​


> If you're trying to avoid these, the best advice I could give would be to
> avoid the 12am-4am window, which AFAIK is when most (all?) transitions have
> occurred historically.
>

​Or postpone conversion from text time to DateTime or time_t ; store as
Zulu ​what is absolute and store as Local to a Zipcode what is local/human,
and do just-in-time conversion it to time_t or DateTime as needed  for
computation, according to the rules for that date when inspected.


In 2007 to prepare for the algorithm change -- the time-change date change
-- i used a Perl script to check what the OS TZ said next transition for
USA Eastern time-zone was for something like 100 servers to see which had
been updated and which hadn't. (Had to delegate actually running it to
someone who had ssh or sudo everywhere. I packaged a script and did data
reduction and mapped hosts to applications. Also did a find to find
DateTime.pm and Java JRE/JDK equivalent.)
See contemporaneous notes for more information.
  http://use.perl.org/use.perl.org/_n1vux/journal/32234.html
and
  http://www.nntp.perl.org/group/perl.datetime/2017/03/msg8132.html
The script dst.pl is still available ​as
   https://community.hpe.com/hpeb/attachments/hpeb/itrc-153/7220/1/285887.pl
from

https://community.hpe.com/t5/Patches/Daylight-Savings-Time-for-HP-UX-10-20/m-p/3923972

Note that this perl script uses libc() -> OS TZinfo (and requires no CPAN
iirc), not DateTime::TimeZone's, but the same logic could scan forward for
the flip with DateTime. I didn't do that because i could just check the
DateTime version iirc.

   But as described by Dave and elaborated above, trying to predict the
future and storing the predicted conversion allows governments to give you
bugs (that your users will blame you for, alas, unfair though that seems)
that upgrading your DateTime won't fix, since you already did the
conversion. Whereas if you store the user input and convert it at-need, you
can update the TZ info by updating OS and/or modules when politicians make
a change and calculations will be right on the day.

For those wanting much too much more info on TimeZones and the trouble with
keeping them current:
* Prior releases of DateTime::TimeZone had a maintainers' README that
referenced the master node of TZ nerdery
   http://www.twinsun.com/tz/tz-link.htm
   but that file has been supplanted by a README.md build from the pod2man
, so i include that link, along with Wikipedia for an easier enty -
* https://en.wikipedia.org/wiki/Tz_database
   (which redirects from https://en.wikipedia.org/wiki/Olson_database )
* and the TZ data maintainer mailinglists
 tz   news:comp.time.tz  https://mm.icann.org/listinfo/tz
http://mm.icann.org/pipermail/tz/
 & tz-announce http://mm.icann.org/pipermail/tz-announce/
(  Yes, the TZ data is now an official function of the Internet Assigned
Numbers Authority (IANA) and is supperted from IETF and ICANN. )
(I am pleasantly surprised there hasn't been a TZ data release since March
... which was the just-in-time update for Haiti resuming DST this year, the
sort of predict-the-future problem mentioned. If one has worldwide users
using their own TZs, one needs regular updates!)

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-10 Thread Dave Rolsky 
While you could in theory write code that would be correct for all past
datetimes, the future doesn't work the same way. As Eric noted, time zones
are political. I have seen DST transitions altered with mere days (or
less!) notice given. This means that anything you determine about the
future could be wrong.

If you're trying to avoid these, the best advice I could give would be to
avoid the 12am-4am window, which AFAIK is when most (all?) transitions have
occurred historically.


Cheers,

Dave Rolsky
http://blog.urth.org
https://github.com/autarch

On Mon, Jul 10, 2017 at 1:02 PM, Binarus  wrote:

> Dear experts,
>
> a few days ago I have got great help from this list, so I hope I may ask
> another (probably stupid) question (I am now having the opposite problem
> than back then):
>
> Using DateTime, is it possible to tell in advance if a certain date-time
> which is given in a certain locale will be ambiguous due to switching
> from DST to standard time?
>
> Example (taken from DateTime's documentation):
>
>   my $dt = DateTime->new(
>   year  => 2003,
>   month => 10,
>   day   => 26,
>   hour  => 1,
>   minute=> 30,
>   second=> 0,
>   time_zone => 'America/Chicago',
>   );
>
> Now $dt is ambiguous: The clock has been turned back to 01:00:00 h at
> 02:00:00 that night, so the time 01:30:00 h has occurred twice.
>
> Even after thinking many hours about it, I haven't found a reasonable
> general method to determine if an arbitrary date-time is ambiguous in
> the sense above.
>
> Possibly, I could subtract different time spans from the date-time in
> question and check if the result is the expected one and use that to
> find out if it is ambiguous, but this would cost much CPU time.
>
> So I would like to ask if somebody knows a general, reasonable method
> for solving that problem, given the following conditions:
>
> 1) We don't know the time span the clock is turned back when switching
> from DST to standard time. It might be one hour in most time zones /
> countries, but after all, some weird person could decide that it is 18
> minutes and 13 seconds or 5 hours, 53 minutes and 42 seconds.
>
> 2) We don't know whether the point in time when the switch occurs is
> exactly at an hour's end / begin. Again, some weird person could decide
> that the switch happens at 08:48:27 am.
>
> I know that there are not many persons on the world that are *that*
> weird, but on the other hand, I don't want to implement an algorithm
> which uses assumptions which are not safe.
>
> As a last resort, there is at least one other module (AFAIK) which I
> could use to extract the daylight saving switching times and time spans
> from the time zone database, and I could use that information to solve
> my problem. But this would probably mean to reinvent the wheel, so I'd
> like to avoid it.
>
> Thank you very much in advance,
>
> Binarus
>

Re: How to tell (in advance) if a date-time is ambiguous?

2017-07-10 Thread Eric Brine 
I don't understand the conditions. The law determines when the switching of
offsets from UTC happen, not some person. The switch doesn't happen at 08:48:27
am in Chicago; it happens at 2am.

On Mon, Jul 10, 2017 at 2:02 PM, Binarus  wrote:

> Dear experts,
>
> a few days ago I have got great help from this list, so I hope I may ask
> another (probably stupid) question (I am now having the opposite problem
> than back then):
>
> Using DateTime, is it possible to tell in advance if a certain date-time
> which is given in a certain locale will be ambiguous due to switching
> from DST to standard time?
>
> Example (taken from DateTime's documentation):
>
>   my $dt = DateTime->new(
>   year  => 2003,
>   month => 10,
>   day   => 26,
>   hour  => 1,
>   minute=> 30,
>   second=> 0,
>   time_zone => 'America/Chicago',
>   );
>
> Now $dt is ambiguous: The clock has been turned back to 01:00:00 h at
> 02:00:00 that night, so the time 01:30:00 h has occurred twice.
>
> Even after thinking many hours about it, I haven't found a reasonable
> general method to determine if an arbitrary date-time is ambiguous in
> the sense above.
>
> Possibly, I could subtract different time spans from the date-time in
> question and check if the result is the expected one and use that to
> find out if it is ambiguous, but this would cost much CPU time.
>
> So I would like to ask if somebody knows a general, reasonable method
> for solving that problem, given the following conditions:
>
> 1) We don't know the time span the clock is turned back when switching
> from DST to standard time. It might be one hour in most time zones /
> countries, but after all, some weird person could decide that it is 18
> minutes and 13 seconds or 5 hours, 53 minutes and 42 seconds.
>
> 2) We don't know whether the point in time when the switch occurs is
> exactly at an hour's end / begin. Again, some weird person could decide
> that the switch happens at 08:48:27 am.
>
> I know that there are not many persons on the world that are *that*
> weird, but on the other hand, I don't want to implement an algorithm
> which uses assumptions which are not safe.
>
> As a last resort, there is at least one other module (AFAIK) which I
> could use to extract the daylight saving switching times and time spans
> from the time zone database, and I could use that information to solve
> my problem. But this would probably mean to reinvent the wheel, so I'd
> like to avoid it.
>
> Thank you very much in advance,
>
> Binarus
>