> Before looking at alternatives, I wonder if we can just change the > shims package to make it fully backwards compatible? Right now the > shims version of normalize() > <https://github.com/pganssle/pytz-deprecation-shim/blob/47bd4bdd9346cafa6c6d66817082ccce099890ad/src/pytz_deprecation_shim/_impl.py#L265> > is essentially a noop. Paul, couldn't it actually attempt to adjust > the time the way pytz does? Perhaps by wrapping pytz itself, and > calling its normalize() from the corresponding pytz timezone; or by > simply replicating its time-changing logic? Apologies if that's a > naive question.
It is not really possible to make the shims work the same way because
there's not enough information available to determine whether an
adjustment needs to be made. The reason that `normalize` works is that
pytz attaches different `tzinfo` objects representing fixed offsets
(with a reference to the time zone they represent) to the datetime. If
arithmetic creates an invalid datetime (i.e. a datetime in mid-June 2020
with EST attached), `normalize` corrects this by attaching a `tzinfo`
representing the correct offset — and it does that by assuming that the
UTC datetime represented by the erroneous fixed offset is correct. With
PEP 495-style zones, you never create those datetimes with erroneous
offsets, so there's no way to tell whether a correction is required.
For example:
>>> from datetime import datetime, timedelta
>>> from zoneinfo import ZoneInfo
>>> NYC = ZoneInfo("America/New_York")
>>> dt0 = datetime(2020, 1, 1, tzinfo=NYC)
>>> dt1 = datetime(2020, 7, 1, tzinfo=NYC)
>>> print(dt0)
2020-01-01 00:00:00-05:00
>>> print(dt1)
2020-07-01 00:00:00-04:00
>>> print(dt0 + timedelta(days=183))
2020-07-02 00:00:00-04:00
>>> print(dt1 + timedelta(days=1))
2020-07-02 00:00:00-04:00
Note that the two endpoints are identical, despite the fact that one of
them spans a DST transition and the other one doesn't. Since the input
to `normalize` is just a datetime and it's assumed that this
path-dependence would show up as an inconsistency in the offset, there's
nothing we can do here other than to actually have all the same problems
as pytz.
Of course, there is another option, which is to, rather than adopting a
wrapper around zoneinfo, adopt a wrapper around pytz that does /not/
follow PEP 495, but instead just deprecates `pytz`'s API and tells
people to turn on the "use zoneinfo" feature flag. It has the upside of
being fully backwards-compatible, but the downside of prolonging
dependence on pytz.
Another option is to modify the shims so that `normalize` always raises
an exception instead of a warning (or maybe it raises an exception for
anything except UTC and fixed offsets). In that case, version 4.0 will
/mostly/ just work and start raising deprecation warnings, but there
will be a hard breakage for anyone who would be negatively affected by
the change in semantics. This /would/ still leave a possible problem in
the other direction, though:
>>> from datetime import datetime, timedelta
>>> from zoneinfo import ZoneInfo
>>> import pytz
>>> NYC_p = pytz.timezone("America/New_York")
>>> NYC = ZoneInfo("America/New_York")
>>> dtp_0 = NYC_p.localize(datetime(2020, 10, 31, 12))
>>> dtp_1 = NYC_p.localize(datetime(2020, 11, 1, 12))
>>> (dtp_1 - dtp_0 ) / timedelta(hours=1)
25.0
>>> dtz_0 = datetime(2020, 10, 31, 12, tzinfo=NYC)
>>> dtz_1 = datetime(2020, 11, 1, 12, tzinfo=NYC)
>>> (dtz_1 - dtz_0) / timedelta(hours=1)
24.0
This occurs because localized pytz zones are different tzinfo objects,
and as such comparisons and subtraction use inter-zone semantics. Of
course, you'll have this same problem even with a "hard break", since
unlike invocation of `normalize` and `localize`, subtraction operations
will succeed if you swap out the attached tzinfo for a zoneinfo tzinfo.
If we go with any variation of using shim-around-zoneinfo like
pytz-deprecation-shim, I'd say those shims need to be introduced as a
breaking change in Django 4.0. If we go with shim-around-pytz, I think
that can safely be introduced in 3.2 (though that would /require/
simultaneously adding support for using zoneinfo, and even then it might
/mostly/ force people to either do the migration in a single huge step
or to involve some wrapper functions for handling the period of time
where the time zone type is not consistent throughout the application).
Best,
Paul
On 10/9/20 9:31 AM, Kevin Henry wrote:
> I think that the simplest approach—the one that would result in the
> least amount of total work for both Django and its users—would be to
> adopt Nick's suggestion and just switch to zoneinfo in 4.0. The
> problem is that it's very hard to square that with Django's stability
> policy: "We’ll only break backwards compatibility of these APIs
> without a deprecation process if a bug or security hole makes it
> completely unavoidable."
>
> If we're going to follow the deprecation process, then there needs to
> be some overlap where both ways of doing things are possible. The
> shims package is a promising approach, but the fact that it's not
> actually backwards compatible with pytz is a serious problem. Adopting
> it directly as Carlton proposes also seems to violate the stability
> policy, albeit in a less severe way.
>
>
> Kevin
>
> On Thursday, October 8, 2020 at 11:35:21 PM UTC-7 smi...@gmail.com wrote:
>
> Hi All,
>
> While I understand the desire to have an early opt-in for some I
> think the important question here is the deprecation warnings. The
> recent URL() change showed that no matter how long there is a new
> way some?/many? folk won't change until they need to.
>
> Nick -- if we introduced a breaking change in 4.0, would that not
> have the same impact on folk upgrading to 4.2LTS from 3.2LTS as
> that which Carlton is concerned about (3.2 from 2.2), albeit a few
> years further into the future.
>
>
> David
>
> On Thursday, 8 October 2020 at 09:08:50 UTC+1 jure.er...@gmail.com
> wrote:
>
> I would definitely be in favor of an opt-in: it would give
> developers time to move to the new system at their convenience.
>
> Example: we're about to try and tackle the TZ issue in our
> apps and we want to do it "globally" with one definitive
> solution. I'd much rather do it on a library that is currently
> favoured, but not yet default than on a deprecated one, even
> if it's not yet officially deprecated. We do have some "import
> pytz", but currently they are few. Once we have a proper
> approach to handling timezone stuff, there's likely going to
> be more of them... or less, depending on the solution ;-)
>
> LP,
> Jure
>
> On 7. 10. 20 17:25, Paul Ganssle wrote:
>>
>> This sounds like a reasonable timeline to me. I think the
>> breakage will be relatively small because I suspect many
>> end-users don't really even know to use `normalize` in the
>> first place, and when introducing the shim into a fundamental
>> library at work I did not get a huge number of breakages, but
>> I am still convinced that it is reasonably categorized as a
>> breaking change.
>>
>> I do think that there's one additional stage that we need to
>> add here (and we chatted about this on twitter a bit), which
>> is a stage that is fully backwards compatible where Django
>> supports using non-pytz zones for users who bring their own
>> time zone. I suspect that will help ease any breaking pain
>> between 3.2 and 4.0, because no one would be forced to make
>> any changes, but end users could proactively migrate to
>> zoneinfo for a smoother transition.
>>
>> I think most of what needs to be done is already in my
>> original PR, it just needs a little conditional logic to
>> handle pytz as well as the shim.
>>
>> I am not sure how you feel about feature flags, but as a
>> "nice to have", I imagine it would also be possible to add a
>> feature flag that opts you in to `zoneinfo` as time zone
>> provider even in 3.2, so that people can jump straight to the
>> 5.0 behavior if they are ready for it.
>>
>> I should be able to devote some time to at least the first
>> part — making Django compatible with zoneinfo even if not
>> actively using it — but likely not for a few weeks at
>> minimum. If anyone wants to jump on either of these ahead of
>> me I don't mind at all and feel free to ping me for review.
>>
>> Best,
>> Paul
>>
>> On 10/7/20 10:48 AM, Carlton Gibson wrote:
>>> Hi Paul.
>>>
>>> Thanks for the input here, and for your patience
>>>
>>> > I am fairly certain this is going to be a tricky migration
>>> and will inevitably come with /some/ user pain. I don't
>>> think this will be Python 2 → 3 style pain, but some users
>>> who have been doing the "right thing" with pytz will need to
>>> make changes to their code in the long run, which is
>>> unfortunate.
>>>
>>> Looking at all the docs, your migration guide on
>>> pytz_deprecation_shim, the example Kevin gave
>>> <https://repl.it/@severian/pytzshim#main.py>, where we do
>>> some arithmetic in a local timezone, and call `normalize()`
>>> in case we crossed a DST boundary, there's no way we can do
>>> this without forcing a breaking change somewhere.
>>>
>>> So, probably, I've been staring at this too long today, but
>>> I think we should introduce the shim in Django 4.0. Django
>>> 3.2, the next major release will be an LTS. If we hold-off
>>> introducing the change until 4.0, we can flag it as a
>>> breaking change in the 4.0 release notes, with big warnings,
>>> allowing folks extra time to hang out on the previous LTS if
>>> they need it.
>>>
>>> What I wouldn't want to do is to bring the breaking change
>>> in in Django 3.2, because we'll have a whole load of folks
>>> updating from the 2.2 LTS at about the time when it goes End
>>> of Life, and with no warning, that'd be a hard breaking
>>> change to throw on top of their other issues.
>>>
>>> We'd keep the shim in place for the entire 4.x series,
>>> removing in Django 5.0 as per the deprecation policy
>>>
>>> <https://docs.djangoproject.com/en/3.1/internals/release-process/#deprecation-policy>.
>>>
>>> I think the advantages of doing it this way are two-fold:
>>>
>>> * We allow people to focus on the semantic breaking change
>>> (in folds) separately from the code changes per se — the
>>> logic may have changed slightly in these cases, but it'll
>>> still run.
>>> * It looks easier to migrate Django's code vs branching on a
>>> new setting. (I didn't think through exactly what that might
>>> look like, so happy to see a PoC from anyone.)
>>>
>>> I'm more attached to the timeline (i.e. making the change
>>> after the next LTS) than whether we use the deprecation shim
>>> or not, but can I ask others to give this their thought too?
>>>
>>> Thanks again!
>>>
>>> Kind Regards,
>>>
>>> Carlton
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the
>>> Google Groups "Django developers (Contributions to Django
>>> itself)" group.
>>> To unsubscribe from this group and stop receiving emails
>>> from it, send an email to django-develop...@googlegroups.com.
>>> To view this discussion on the web visit
>>>
>>> https://groups.google.com/d/msgid/django-developers/ce04a6b7-4409-4b20-ba30-4cd64dc0cabfn%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/django-developers/ce04a6b7-4409-4b20-ba30-4cd64dc0cabfn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>> --
>> You received this message because you are subscribed to the
>> Google Groups "Django developers (Contributions to Django
>> itself)" group.
>> To unsubscribe from this group and stop receiving emails from
>> it, send an email to django-develop...@googlegroups.com.
>> To view this discussion on the web visit
>>
>> https://groups.google.com/d/msgid/django-developers/e13e8ae2-5d43-e550-48a4-cb7ad6e699f6%40ganssle.io
>>
>> <https://groups.google.com/d/msgid/django-developers/e13e8ae2-5d43-e550-48a4-cb7ad6e699f6%40ganssle.io?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com
> <mailto:django-developers+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/b18754a4-c308-492d-b547-6b3c7cdc1442n%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/b18754a4-c308-492d-b547-6b3c7cdc1442n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/393fa2b7-6fca-fd2c-f36e-9942c6dc0104%40ganssle.io.
signature.asc
Description: OpenPGP digital signature
