[issue47228] Document that naïve datetime objects represent local time
New submission from Paul Ganssle : Currently, the `datetime` documentation has this to say about naïve datetimes: > A naive object does not contain enough information to unambiguously locate > itself relative to other date/time objects. Whether a naive object represents > Coordinated Universal Time (UTC), local time, or time in some other timezone > is purely up to the program, just like it is up to the program whether a > particular number represents metres, miles, or mass. Naive objects are easy > to understand and to work with, at the cost of ignoring some aspects of > reality. This was accurate in Python 2.7, but as of Python 3, the picture is a bit more nuanced. `.astimezone()` and `.timestamp()` work for naïve datetimes, but they are treated as *system local times*. It is no longer really appropriate to use a naïve datetime to a datetime in any specific concrete time zone — instead, they should be considered either abstract datetimes for the purposes of calendar calculations, or they should be considered as representing the realization of that abstract datetime *in the current system locale*. This new behavior is referenced in, for example, the warning in `.utcnow()`: https://docs.python.org/3.10/library/datetime.html#datetime.datetime.utcnow We make reference to this in the documentation for `.timestamp()`: https://docs.python.org/3.10/library/datetime.html#datetime.datetime.timestamp and in `.astimezone()`: https://docs.python.org/3.10/library/datetime.html#datetime.datetime.astimezone, but the top level explanation makes no reference to it. I have written a blog post about *why* this is the case: https://blog.ganssle.io/articles/2022/04/naive-local-datetimes.html and made reference to this behavior in an earlier blog post about `utcnow`: https://blog.ganssle.io/articles/2019/11/utcnow.html, but I think it would be a good idea to revamp the official documentation to reflect this change in status (12 years or so after the change…) -- assignee: p-ganssle components: Documentation messages: 416778 nosy: belopolsky, p-ganssle priority: normal severity: normal stage: needs patch status: open title: Document that naïve datetime objects represent local time type: enhancement versions: Python 3.10, Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue47228> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47207] Switch datetime docstrings / documentation to using "Returns" rather than "Return"?
New submission from Paul Ganssle : In bpo-9305, Fred Drake recommends preferring `Returns ...` over the imperative `Return ...`: https://bugs.python.org/issue9305#msg110912 Currently we're pretty consistent about `Return ...`, which is consistent with PEP 257: https://peps.python.org/pep-0257/ That said, I actually think "Returns ..." sounds much better in the documentation, and I'd be happy to see it changed if others agree. I have spun this off from bpo-9305 so that we can unblock https://github.com/python/cpython/pull/31697. -- assignee: docs@python components: Documentation messages: 416628 nosy: belopolsky, docs@python, fdrake, p-ganssle, slateny priority: normal severity: normal status: open title: Switch datetime docstrings / documentation to using "Returns" rather than "Return"? versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue47207> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46614] Add option to output UTC datetimes as "Z" in `.isoformat()`
Paul Ganssle added the comment: I think this approach is probably the best we can do, but I could also imagine that users might find it to be confusing behavior. I wonder if there's any informal user testing we can do? I guess the ISO 8601 spec does call "Z" the "UTC designator", so `use_utc_designator` seems like approximately the right name. My main hesitation with this name is that I suspect users may think that `use_utc_designator` means that they *unconditionally* want to use `Z` — without reading the documentation (which we can assume 99% of users won't do) — you might assume that `dt.isoformat(use_utc_designator=True)` would translate to `dt.astimezone(timezone.utc).replace(tzinfo=None).isoformat() + "Z"`. A name like `utc_as_z` is definitely less... elegant, but conveys the concept a bit more clearly. Would be worth throwing it to a poll or something before merging. -- ___ Python tracker <https://bugs.python.org/issue46614> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35829] datetime: parse "Z" timezone suffix in fromisoformat()
Paul Ganssle added the comment: I don't think it's necessary to add a feature to `isoformat()` just for the purpose of being able to add the corresponding parser, particularly when the plan is to implement a much broader ISO 8601 parser for Python 3.11 (I've done most of the implementation in C already, I can share the progress I've made, particularly if someone else wants to pick up the baton there before I get back to it). That said, I think it's useful for `isoformat()` to be able to output UTC times as "Z", so we may as well do that as part of 3.11 anyway. I think that's a separate issue to discuss, so I've created bpo-46614 to hammer out the details. -- versions: +Python 3.11 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue35829> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46614] Add option to output UTC datetimes as "Z" in `.isoformat()`
New submission from Paul Ganssle :
As part of bpo-35829, it was suggested that we add the ability to output the
"Z" suffix in `isoformat()`, so that `fromisoformat()` can both be the exact
functional inverse of `isoformat()` and parse datetimes with "Z" outputs. I
think that that's not a particularly compelling motivation for this, but I also
see plenty of examples of `datetime.utcnow().isoformat() + "Z"` out there, so
it seems like this is a feature that we would want to have *anyway*,
particularly if we want to deprecate and remove `utcnow`.
I've spun this off into its own issue so that we can discuss how to implement
the feature. The two obvious questions I see are:
1. What do we call the option? `use_utc_designator`, `allow_Z`, `utc_as_Z`?
2. What do we consider as "UTC"? Is it anything with +00:00? Just
`timezone.utc`? Anything that seems like a fixed-offset zone with 0 offset?
For example, do we want this?
>>> LON = zoneinfo.ZoneInfo("Europe/London")
>>> datetime(2022, 3, 1, tzinfo=LON).isoformat(utc_as_z=True)
2022-03-01T00:00:00Z
>>> datetime(2022, 6, 1, tzinfo=LON).isoformat(utc_as_z=True)
2022-06-01T00:00:00+01:00
Another possible definition might be if the `tzinfo` is a fixed-offset zone
with offset 0:
>>> datetime.timezone.utc.utcoffset(None)
timedelta(0)
>>> zoneinfo.ZoneInfo("UTC").utcoffset(None)
timedelta(0)
>>> dateutil.tz.UTC.utcoffset(None)
timedelta(0)
>>> pytz.UTC.utcoffset(None)
timedelta(0)
The only "odd man out" is `dateutil.tz.tzfile` objects representing fixed
offsets, since all `dateutil.tz.tzfile` objects return `None` when `utcoffset`
or `dst` are passed `None`. This can and will be changed in future versions.
I feel like "If the offset is 00:00, use Z" is the wrong rule to use
conceptually, but considering that people will be opting into this behavior, it
is more likely that they will be surprised by `datetime(2022, 3, 1,
tzinfo=ZoneInfo("Europe/London").isoformat(utc_as_z=True)` returning
`2022-03-01T00:00:00+00:00` than alternation between `Z` and `+00:00`.
Yet another option might be to add a completely separate function,
`utc_isoformat(*args, **kwargs)`, which is equivalent to (in the parlance of
the other proposal) `dt.astimezone(timezone.utc).isoformat(*args, **kwargs,
utc_as_z=True)`. Basically, convert any datetime to UTC and append a Z to it.
The biggest footgun there would be people using it on naïve datetimes and not
realizing that it would interpret them as system local times.
--
assignee: p-ganssle
components: Library (Lib)
messages: 412384
nosy: belopolsky, brett.cannon, p-ganssle
priority: normal
severity: normal
stage: needs patch
status: open
title: Add option to output UTC datetimes as "Z" in `.isoformat()`
type: enhancement
versions: Python 3.11
___
Python tracker
<https://bugs.python.org/issue46614>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46447] datetime.isoformat() documentation does not point to the risk of using it with naive datetime objects
Paul Ganssle added the comment: Sorry I missed this! Thank you for taking the time to write this up and to make a PR. Unfortunately, I don't think I agree with the idea of warning about this. The warnings about `utcnow` and `utcfromtimestamp` are a problem because `utcnow` and `utcfromtimestamp` are intended to represent times in UTC, but they return datetimes that actually represent local times in the semantics of modern Python. Basically, these functions are dangerous not because they are using naïve datetimes, but because they are *mis*using naïve datetimes. The same can not be said of `.isoformat()`, which is doing the right thing when you use `datetime.now().isoformat()`. If you look at Wikipedia's article on ISO 8601 (which is pretty much the best resource on this, since ISO 8601 is itself paywalled and we never should have standardized on a proprietary standard in the first place), you'll see it says: > Local time (unqualified) > If no UTC relation information is given with a time representation, the time > is assumed to be in local time. While it may be safe to assume local time > when communicating in the same time zone, it is ambiguous when used in > communicating across different time zones. It may be that for the kind of programming you do, it doesn't make sense to use local datetimes in an interchange format — but it is a legitimate use case and there are definitely situations where it is very much the right thing to do, so I don't think we should warn against it in the `datetime.isoformat` documentation. There is *might* be some case for warning about this or something like it in the `datetime.now` documentation. The major use cases for naïve datetimes are things where you are working with system time or things where you are working with dates in the future — you don't want to specify that some event is going to happen at 2030-03-31T12:00Z if the actual event is planned for April 1, 2030 at 13:00 *London time*, because if, between now and then, the UK decides to cancel DST or move the start back a week, the event you've stored as a UTC time now longer represents what it was intended to represent. In a lot of cases `datetime.now()` will just be used as "what time is it now", which is not subject to that particular problem because by the time the datetime gets stored or used, `datetime.now()` is a date in the *past*, and can safely be converted to UTC for all time. Of course, if you are consuming a bunch of event dates stored in local time and you want to compare them to the current time, then `datetime.now()` would be appropriate. Similarly if you want to display the current time to a user on a given system (rather than logging or storing it), it would also make sense to do things like `print(datetime.now().isoformat())`, so there are definitely also legitimate use cases for `datetime.now()`. I'm inclined to say that we should *not* have a warning on `datetime.now()`, because we will give people warning fatigue if we do, and we definitely want people to see `now()` as the correct alternative to `utcnow()`. I am more sympathetic to rewording the `.now()` documentation to make it clear that this will be a naïve time *representing the current time in the system local time zone* if `None` is passed (i.e. rewording or appending to the "If optional argument `tz`" paragraph). -- nosy: +belopolsky, p-ganssle ___ Python tracker <https://bugs.python.org/issue46447> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12756] datetime.datetime.utcnow should return a UTC timestamp
Paul Ganssle added the comment: > from practical experience, it is a whole lot better to not deal with > timezones in data processing code at all, but instead only use naive UTC > datetime values everywhere, expect when you have to prepare reports or output > which has a requirement to show datetime value in local time or some specific > timezone. This is not good advice. It is out of date, and has some significant pitfalls. See my blog post on the subject: https://blog.ganssle.io/articles/2019/11/utcnow.html If you are working with `datetime` objects that represent time in a specific time zone other than the system local zone, you should probably have them carry around a tzinfo object. > Note also that datetime.now() gives you a naive datetime. From an API > consistency standpoint I think it makes sense that datetime.utcnow() gives a > naive datetime. This... is not accurate. `.now()` gives you the local time; to the extent that they represent a date in a time zone at all, naïve time zones represent times in the *system local time*, which is why it makes sense for `.now()` to default to returning naïve time zones. For example, `datetime.now().timestamp()` will give accurate information, whereas `datetime.utcnow().timestamp()` will *not* (unless your local zone happens to be UTC or equivalent). > It would actually be confusing (IMO) for it to return an aware datetime. I > can see why you might disagree, but backward compatibility wins in this case > regardless. As evidenced by this thread, the fact that we have some APIs that return naïve datetimes generated by a process that treats them as localized datetimes in something other than system local times is actually the source of confusion That said, from a backwards compatibility point of view, we simply cannot change this. It has been proposed many times and it would be a major breaking change for almost no reason. The best we can do is to deprecate the offending methods and remove them. There is more information about the challenge that doing this would present in this datetime-SIG thread: https://mail.python.org/archives/list/datetime-...@python.org/thread/PT4JWJLYBE5R2QASVBPZLHH37ULJQR43/ I am sympathetic to the idea of removing it, but we would probably want to put some optimizations in place for `UTC` first, to make the transition more seamless in the few places where there are legitimate uses for `utcnow` and `utcfromtimestamp`. > I would argue that PEP20 should win over backward compatibility, in addition > to the points I hinted at above, practicality beats purity PEP 20 contains a bunch of contradictory advice, and it's not really a binding document anyway, so it definitely doesn't "win" over anything, but in this case you have it backwards. "Purity" would be making a breaking change for the purposes of making the function do what a lot of people think it does, but doing so would actually be impractical, because it would cause a lot of work for people, and create a lot of ambiguity in what people meant when they wrote a given line of code. The practical things to do here would be to either do nothing (not break anything that works and try and guide people away from using `utcnow` — maybe get a linting rule added to `pylint` to warn against it), or to deprecate and remove the function. -- ___ Python tracker <https://bugs.python.org/issue12756> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46319] datetime.utcnow() should return a timezone aware datetime
Paul Ganssle added the comment: Yes, this is the documented behavior, including a warning against using UTC now in the documentation! There is some possibility of removing utcnow entirely as an "attractive nuisance", but I suspect that this will lead to much consternation and hand-wringing, and there are some legitimate uses for `utcnow`, so I haven't made it a high priority to have that particular fight... -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue46319> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46124] Deprecation warning in zoneinfo module
Paul Ganssle added the comment: Jason's patch looks good to me, but I don't understand why Karthikeyan originally suggested using `normalize_path`. Trying to dig through exactly how `files().joinpath().open` is implemented has so many layers of indirection and abstract classes that I can't quite figure out if the two things are equivalent or not. Seems like `joinpath()` is the right thing to do, but does it have less validation than the `normalize_path` method? -- ___ Python tracker <https://bugs.python.org/issue46124> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41260] datetime, date and time: strftime method takes different keyword argument: fmt (pure) or format (C)
Paul Ganssle added the comment: Updating this issue to cover the problem in date, time and datetime. -- title: datetime: strftime method takes different keyword argument: fmt (pure) or format (C) -> datetime, date and time: strftime method takes different keyword argument: fmt (pure) or format (C) ___ Python tracker <https://bugs.python.org/issue41260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45814] datetime.time.strftime: use the same function signature for C and Python implementations
Paul Ganssle added the comment: I think this is mostly a duplicate of bpo-41260, which has an open PR on it. I think that got lost in the shuffle, I'm sad we didn't fix it in Python 3.10. I think we should migrate all of these signatures that differ to whichever one the C implementation is using (I believe that's 3.11). I'm going to close that one and edit the other one to cover `time` and `date` as well. Thanks for the report Yevhenii! -- resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> datetime: strftime method takes different keyword argument: fmt (pure) or format (C) ___ Python tracker <https://bugs.python.org/issue45814> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45515] Add reference to zoneinfo in the datetime module documetnation
Change by Paul Ganssle : -- keywords: +patch pull_requests: +27309 stage: -> patch review pull_request: https://github.com/python/cpython/pull/29038 ___ Python tracker <https://bugs.python.org/issue45515> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45515] Add reference to zoneinfo in the datetime module documetnation
New submission from Paul Ganssle : Right now the datetime documentation recommends using `dateutil.tz` for IANA time zones, but we should update this to point to `zoneinfo`. -- assignee: p-ganssle components: Documentation messages: 404207 nosy: p-ganssle priority: low severity: normal status: open title: Add reference to zoneinfo in the datetime module documetnation versions: Python 3.10, Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue45515> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45414] pathlib.Path.parents negative indexing is wrong for absolute paths
Paul Ganssle added the comment:
This is a great bug report, but for anyone else who gets a bit lost in the
details, here's the core of the issue:
>>> p = Path("/1/2")
>>> q = Path("1/2")
>>> p.parents[-1] # This is correct
PosixPath('/')
>>> q.parents[-1]
PosixPath('.')
>>> p.parents[-2] # Should be PosixPath('/1')
PosixPath('/')
>>> q.parents[-2]
PosixPath('1')
>>> p.parents[-3] # Should be PosixPath('/1/2')
PosixPath('/1')
>>> q.parents[-3]
PosixPath('1/2')
I think a refactoring where '/' doesn't appear in ._parts would be a good idea
if we can get past Chesterton's Fence and determine that this was indeed not a
deliberate design decision (or at least one whose concerns no longer apply),
but at least in the short term, I agree that transforming negative indexes into
positive indices is the right, expedient thing to do.
We'll definitely want to make sure that we're careful about bad indices (and
add relevant tests), though, since it would be easy to get weird behavior where
too-large negative indexes start "wrapping around" (e.g. p.parents[-4] with
len(p._parents) == 3 → p.parents[-1]).
--
type: -> behavior
___
Python tracker
<https://bugs.python.org/issue45414>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45414] pathlib.Path.parents negative indexing is wrong for absolute paths
Change by Paul Ganssle : -- nosy: +p-ganssle ___ Python tracker <https://bugs.python.org/issue45414> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44831] Inconsistency between datetime.now() and datetime.fromtimestamp(time.time(), None)
Paul Ganssle added the comment: I think this is a rounding issue. `time.time()` returns an epoch timestamp as a float and at the current epoch time, floats are spaced ~500ns apart. `datetime.datetime.now` does a floor division when rounding: https://github.com/python/cpython/blob/8bdf12e99a3dc7ada5f85bba79c2a9eb9931f5b0/Modules/_datetimemodule.c#L5056 `datetime.fromtimestamp` uses the standard banker's round (round above half, tie goes to the nearest even number): https://github.com/python/cpython/blob/8bdf12e99a3dc7ada5f85bba79c2a9eb9931f5b0/Modules/_datetimemodule.c#L5038-L5039 Presumably if we change these two to be consistent, this issue will go away. I am not entirely sure if anyone is relying on a particular rounding behavior for one or both of these, and I'm not sure which one is the right one to harmonize on. For now I'm going to say that we should target 3.11 on this, since it will change an existing observable behavior for at least one of these functions in a way that isn't necessarily going from "obviously wrong" to "obviously right", so I think we should be cautious and not change this in a patch release. -- versions: +Python 3.11 -Python 3.6, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue44831> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44829] zoneinfo.ZoneInfo does not check for Windows device names
Paul Ganssle added the comment: Sorry you didn't receive a response to your security@ email, I guess my response just went to the PSRT, not to you as well. I believe we determined that this was an issue in importlib.resources generally, not specific to zoneinfo. I think `importlib.resources.open_binary` should check if a resource is a file with `os.isfile` before opening it. That will solve the issue in zoneinfo and other similar situations. -- ___ Python tracker <https://bugs.python.org/issue44829> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44603] REPL: exit when the user types exit instead of asking them to explicitly type exit()
Change by Paul Ganssle : -- stage: resolved -> ___ Python tracker <https://bugs.python.org/issue44603> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44603] REPL: exit when the user types exit instead of asking them to explicitly type exit()
Paul Ganssle added the comment: Re-opening this because I think the discussion is not done and I don't see any reason why this was rejected. > Related 2005 python-dev discussion: > https://mail.python.org/archives/list/python-...@python.org/thread/VNGY2DLML4QJUXE73JLVBIH5WFBZNIKG/ @Mark Thanks for digging these up! From what I can tell, that discussion ended up with a combination of there not being quite enough enthusiasm for the idea to drive it forward and no one coming up with a good way to localize the effect to just the case where we know the person was trying to type "exit" in a REPL. I think Pablo's patch shows that a very limited addition to the "REPL layer" is actually plausible, and we *could* implement this without taking on an enormous amount of additional complexity or affecting non-interactive use cases. Fernando's point about it being dangerous to generalize this additional layer of "interactive-use only" keywords is a good one (see: https://mail.python.org/archives/list/python-...@python.org/message/L37RD7SG26IOBETPI7TETKFGHPAPC75Q/), though it seems that it was this thread that prompted him to add exit/quit as auto-call magic keywords to IPython, and I think that has worked out in the intervening 16 years. I don't think there's much danger of us wanting to generalize this concept, since the only really compelling argument for doing it this way for exit/quit is that almost everyone seems to think it *should* work this way (despite it never having worked this way, and there not being any equivalents), and gets tripped up when it doesn't. > and the related issue: https://bugs.python.org/issue1446372 Looks to me like that is an issue for adding the message when you type "exit". There's no additional discussion disqualifying the use of "exit" as an interactive-only keyword. -- resolution: rejected -> status: closed -> open ___ Python tracker <https://bugs.python.org/issue44603> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44603] REPL: exit when the user types exit instead of asking them to explicitly type exit()
Paul Ganssle added the comment: At this point I think we should probably start a thread on python-dev to see how people feel about it. I'd be happy to author or co-author a PEP for this if need be. -- ___ Python tracker <https://bugs.python.org/issue44603> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44603] REPL: exit when the user types exit instead of asking them to explicitly type exit()
Paul Ganssle added the comment: > In fact, you're proposing to use exit as a keyword, but lying about it to the > users. If it were really so important, then it _should_ be a keyword, and at > least I'd know that I can't use it for my variables anymore. (It's not the > first time such a thing would happen. The same thing happened with `async` a > few years ago.) But please don't introduce those "keywords just in a > particular context", they are horrible from the perspective of usability. We already have so-called "soft keywords", e.g. `match`, so the horse is out of the barn at this point. I'm not sure why this is closed as rejected — I don't see any decision one way or the other in this thread or on the PR, did I miss it? I am struggling to understand how this is a user-hostile change; it is not unreasonable for a REPL to have some commands for interacting with the REPL which are not Python functions. I have accidentally typed `exit` instead of `exit()` many times, and one of the reasons I and many others like IPython is that `exit` exits the REPL. It has never once caused a problem for me, as far as I can tell. I cannot imagine that it is a common scenario for someone to type "exit" in order to inspect the "exit" object — it doesn't even have a useful repr! The only reason you'd do this would be if you knew what it does and were demonstrating it, or if you were exploring what the various builtins are (which I think is very rare, and you'd probably only have to learn that lesson once). Vedran's point, however, that you could do `exit = some_func()` and then type `exit` to try and inspect the `exit` object is a solid one. That said, I think we can get around this fairly easily (albeit with the cost of some additional complexity in the "handle the exit keyword" function) — if there's a single AST node that is a name and the name is "exit" or "quit", the REPL inspects locals and globals to see if the object referred to is a Quitter, and if so it exits, otherwise pass through the command as normal (possibly raising a warning like, "Did you mean to exit? You have shadowed the `exit` builtin, so use Ctrl-Z/Ctrl-D to exit or delete your `exit` object and try again"). I understand the arguments for purity and explicability and I'm often one of the first people to argue for keeping things consistent and understandable, but this is one of those things where we could significantly improve user experience for no practical cost. We can identify with very high certainty the situations in which a user intended to exit the REPL, we should go ahead and do it to provide a more intuitive REPL experience. -- ___ Python tracker <https://bugs.python.org/issue44603> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44603] REPL: exit when the user types exit instead of asking them to explicitly type exit()
Paul Ganssle added the comment: I'm +1 for Pablo's approach. That's approximately what I meant by "special-case it in the REPL layer" anyway. Are there any downsides to doing it this way? It seems tightly scoped and with minimal overhead. -- ___ Python tracker <https://bugs.python.org/issue44603> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44603] REPL: exit when the user types exit instead of asking them to explicitly type exit()
Paul Ganssle added the comment:
If we want to confine the behavior to just the repl, we could possibly have the
repl set an environment variable or something of that nature for interactive
sessions, so that `__repr__` of `exit` can tell the difference between being
invoked in a REPL and not — though I suppose it could cause some pretty
frustrating and confusing behavior if some library function is doing something
like this behind the scenes:
```
def get_all_reprs():
return {
v: repr(obj) for v, obj in globals()
]
```
You could invoke some function and suddenly your shell quits for no apparent
reason. And if it only happens when triggered in a REPL, you'd be doubly
confused because you can't reproduce it with a script.
I do think the "type exit() to exit" is a papercut. The ideal way to fix it
would be in the REPL layer by special-casing `exit`, but I realize that that
may introduce unnecessary complexity that isn't worth it for this one thing.
> Second, if absolutely necessary we could ask the user to confirm that they
> want to exit.
A thought occurs: we could simply re-word the message to make it seem like
we're asking for confirmation:
```
>>> exit
Do you really want to exit? Press Ctrl+Z to confirm, or type exit() to exit
without confirmation.
```
Then it won't seem as much like we know what you meant to do but aren't doing
it, despite the fact that the behavior is exactly the same .
--
nosy: +p-ganssle
___
Python tracker
<https://bugs.python.org/issue44603>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44307] date.today() is 2x slower than datetime.now().date()
Paul Ganssle added the comment: Yeah, I knew this was slower and it's been on my long list to look at it (tied to this is the fact that `datetime.today()` is basically just a slow version of `datetime.now()`, in defiance of user expectations). My inclination is that we shouldn't re-implement `fromtimestamp` in `date.today`, but rather call `date_fromtimestamp` in the fast path. I believe that incurs the overhead of creating one additional Python object (an integer), but if it's a sufficiently significant speedup, we could possibly refactor `date_fromtimestamp` to a version that accepts a C integer and a version that accepts a Python integer, then call the version accepting a C integer. I think this won't give any speedup to `datetime.today`, since `datetime.today` will still take the slow path. If we care about this, we *may* be able to implement `datetime.today` as an alias for `datetime.now(None)`, assuming there are no behavioral differences between the two. -- ___ Python tracker <https://bugs.python.org/issue44307> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42109] Use hypothesis for testing the standard library, falling back to stubs
Paul Ganssle added the comment: > I use hypothesis during development, but don't have a need for in the the > standard library. By the time code lands there, we normally have a specific > idea of what edge cases needs to be in the tests. The suggestion I've made here is that we use @example decorators to take the hypothesis tests you would have written already and turn them in to what are essentially parameterized tests. For anyone who doesn't want to explicitly run the hypothesis test suite, the tests you are apparently already writing would simply turn into normal tests for just the edge cases. One added benefit of keeping the tests around in the form of property tests is that you can run these same tests through hypothesis to find regressions in bugfixes that are implemented after landing (e.g. "Oh we can add a fast path here", which introduces a new edge case). The segfault bug from bpo-34454, for example, would have been found if I had been able to carry over the hypothesis-based tests I was using during the initial implementation of fromisoformat into later stages of the development. (Either because I didn't run it long enough to hit that particular edge case or because that edge case didn't come up until after I had moved the development locus into the CPython repo, I'm not sure). Another benefit of keeping them around is that they become fuzz targets, meaning people like oss-fuzz or anyone who wants to throw some fuzzing resources at CPython have an existing body of tests that are expected to pass on *any* input, to find especially obscure bugs. > For the most part, hypothesis has not turned up anything useful for the > standard library. Most of the reports that we've gotten reflected a > misunderstanding by the person running hypothesis rather than an actual bug. > [...] I don't really think it's a good argument to say that it hasn't turned up useful bugs. Most of the bugs in a bit of code will be found during development or during the early stages of adoption, and we have very wide adoption. I've found a number of bugs in zoneinfo using hypothesis tests, and I'd love to continue using them in CPython rather than throwing them away or maintaining them in a separate repo. I also think it is very useful for us to write tests about the properties of our systems for re-use in PyPy (which does use hypothesis, by the way) and other implementations of Python. This kind of "define the contract and maintain tests to enforce that" is very helpful for alternate implementations. > For numeric code, hypothesis is difficult to use and requires many > restrictions on the bounds of variables and error tolerances. [...] I do not think that we need to make hypothesis tests mandatory. They can be used when someone finds them useful. > The main area where hypothesis seems easy to use and gives some comfort is in > simple roundtrips: assert zlib.decompress(zlib.compress(s)) == s. However, > that is only a small fraction of our test cases. Even if this were the only time that hypothesis were useful (I don't think it is), some of these round-trips can be among the trickiest and important code to test, even if it's a small fraction of the tests. We have a bunch of functions that are basically "Load this file format" and "Dump this file format", usually implemented in C, which are a magnet for CVEs and often the target for fuzz testing for that reason. Having a small library of maintained tests for round tripping file formats seems like it would be very useful for people who want to donate compute time to fuzz test CPython (or other implementations!) > Speed is another issue. During development, it doesn't matter much if > Hypothesis takes a lot of time exercising one function. But in the standard > library tests already run slow enough to impact development. If hypothesis > were to run everytime we run a test suite, it would make the situation worse. As mentioned in the initial ticket, the current plan I'm suggesting is to have fallback stubs which turn your property tests into parameterized tests when hypothesis is not installed. If we're good about adding `@example` decorators (and certainly doing so is easier than writing new ad hoc tests for every edge case we can think of when we already have property tests written!), then I don't see any particular reason to run the full test suite against a full hypothesis run on every CI run. My suggestion is: 1. By default, run hypothesis in "stubs" mode, where the property tests are simply parameterized tests. 2. Have one or two CI jobs that runs *only* the hypothesis tests, generating new examples — since this is just for edge case detection, it doesn't necessarily need to run on every combination of architecture and platform and configuration in our testing matrix, just the ones where it could plausibl
[issue43295] datetime.strptime emits IndexError on parsing 'z' as %z
Paul Ganssle added the comment: New changeset c87b81dcb2c22b6d151da39a0f65d5db304f59a8 by Miss Islington (bot) in branch '3.9': bpo-43295: Fix error handling of datetime.strptime format string '%z' (GH-24627) (#25695) https://github.com/python/cpython/commit/c87b81dcb2c22b6d151da39a0f65d5db304f59a8 -- ___ Python tracker <https://bugs.python.org/issue43295> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42109] Use hypothesis for testing the standard library, falling back to stubs
Paul Ganssle added the comment: I do not want to dissuade you from figuring out how minithesis / hypothesis works (far from it), but I'm wondering if the question of how shrinking works is germane to the issue at hand, which is whether or not hypothesis / property-based-testing is suitable for testing the standard library. I almost don't think it matters *if* shrinking works, much less *how*. Shrinking is something of a UI nicety in the sense that when hypothesis finds an example that violates a property, it will try to take whatever example it has chosen and find something like a "minimal working example" to show to the end user. So if we have something like: ``` @given(x=strategies.integers()) def f(x): assert x >= 0 ``` Hypothesis will presumably tell us that `x == -1` will violate the property instead of `-24948929` or some other random thing. But if it did report `-24948929` or something, it wouldn't be too hard to see "oh, right, integers can be negative". In some cases with the time zone stuff, there isn't a good metric for complexity at the moment, so time zones are sorted by their IANA key (which is to say, basically arbitrary), and I generally find it useful anyway (admittedly, the shrinking still is helpful because most problems affecting all zones will return Africa/Abuja, whereas things particular to a specific zone's odd time zone history will return whichever zone with that quirk comes alphabetically first). Anyway, do not let me disrupt your process, I just thought it might be worth making the point that some of these specific details might be nice to know, but don't seem like they should be blockers for hypothesis' adoption in the standard library. -- ___ Python tracker <https://bugs.python.org/issue42109> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24929] _strptime.TimeRE should not enforce range in regex
Paul Ganssle added the comment: I also commented on GH-26215 ( https://github.com/python/cpython/pull/26215 ), but for posterity, I'll note a few things: 1. It seems that (and this may have changed since 2015), `_strptime._strptime` now has a stage that (unconditionally?) constructs a temporary `datetime_date`, which means it does do this particular validation in both `time.strptime` and `datetime.strptime`. That said, both flavors of `strptime` are *way* slower than I'd like them to be, and constructing an unnecessary `date`/`datetime` is a pretty good way to slow down your function, so if we ever go around optimizing this function, that may be one of the first bits on the chopping block. 2. The logic for `strptime` is very complicated and it's very hard to test the full input space of the function (particularly since we're not using property tests (yet)...). This makes me somewhat uneasy about moving the validation stage from the beginning of the function (in parsing the regular expression) to the very *end* of the function (in the datetime constructor). It's *probably* safe to do so, but it may also be worth exploring the possibility of validating this directly in `_strptime` (possibly immediately after the string is parsed by the regex), and raising a cleaner error message on failure. Probably not worth spending a ton of time on that compared to improving the testing around this so that we can feel confident making changes under the hood. `.strptime` is really quite slow, and I wouldn't be surprised if we pulled out its guts and replaced most of the regex stuff with a fast C parser at some point in the future. Having good tests will both give us confidence to make this change (and that making this change won't lead to regressions in the future) and help with any future project to replace `_strptime._strptime` with a faster version, so I'd say that's the most important thing to do here. -- nosy: +p-ganssle ___ Python tracker <https://bugs.python.org/issue24929> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42109] Use hypothesis for testing the standard library, falling back to stubs
Paul Ganssle added the comment: @Terry > The problem with random input tests in not that they are 'flakey', but that > they are useless unless someone is going to pay attention to failures and try > to find the cause. This touches on the difference between regression testing > and bug-finding tests. CPython CI is the former, and marred at that by buggy > randomly failing tests. > My conclusion: bug testing would likely be a good idea, but should be done > separate from the CI test suite. Such testing should only be done for > modules with an active maintainer who would welcome failure reports. Are you saying that random input tests are flaky but that that is not the big problem? In my experience using hypothesis, in practice it is not the case that you get tests that fail randomly. The majority of the time if your code doesn't violate one of the properties, the tests fail the first time you run the test suite (this is particularly true for strategies where hypothesis deliberately makes it more likely that you'll get a "nasty" input by biasing the random selection algorithm in that direction). In a smaller number of cases, I see failures that happen on the second, third or fourth run. That said, if it were a concern that every run of the tests is using different inputs (and thus you might see a bug that only appears once in every 20 runs), it is possible to run hypothesis in a configuration where you specify the seed, making it so that hypothesis always runs the same set of inputs for the same tests. We can disable that on a separate non-CI run for hypothesis "fuzzing" that would run the test suite for longer (or indefinitely) looking for long-tail violations of these properties. I feel that if we don't at least run some form of the hypothesis tests in CI, there will likely be bit rot and the tests will decay in usefulness. Consider the case where someone accidentally breaks an edge case that makes it so that `json.loads(json.dumps(o))` no longer works for some obscure value of `o`. With hypothesis tests running in CI, we are MUCH more likely to find this bug / regression during the initial PR that would break the edge case than if we run it separately and report it later. If we run the hypothesis tests in a build-bot, the process would be: 1. Contributor makes PR with passing CI. 2. Core dev review passes, PR is merged. 3. Buildbot run occurs and the buildbot watch is notified. 4. Buildbot maintainers track down the PR responsible and either file a new bug or comment on the old bug. 5. Someone makes a NEW PR adding a regression test and the fix for the old PR. 6. Core dev review passes, second PR is merged. If we run it in CI, the process would be: 1. Contributor makes PR, CI breaks. 2. If the contributor doesn't notice the broken CI, core dev points it out and it is fixed (or the PR is scrapped as unworkable). Note that in the non-CI process, we need TWO core dev reviews, we need TWO PRs (people are not always super motivated to fix bugs that don't affect them that they the caused when fixing a bug that does affect them), and we need time and effort from the buildbot maintainers (note the same applies even if the "buildbot" is actually a separate process run by Zac out of a github repo). Even if the bug only appears in one out of every 4 CI runs, it's highly likely that it will be found and fixed before it makes it into production, or at least much more quickly, considering that most PRs go through a few edit cycles, and a good fraction of them are backported to 2-3 branches, all with separate CI runs. It's a much quicker feedback loop. I think there's an argument to be made that incorporating more third-party libraries (in general) into our CI build might cause headaches, but I think that is not a problem specific to hypothesis, and I think its one where we can find a reasonable balance that allows us to use hypothesis in one form or another in the standard library. -- ___ Python tracker <https://bugs.python.org/issue42109> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15443] datetime module has no support for nanoseconds
Paul Ganssle added the comment: > I don't think full nanosecond support is feasible to complete in the > remaining weeks This may be so, but I think the important part of that question is "what work needs to be done and what questions need to be answered?" If the answer is that we need to make 3 decisions and do the C implementation, that seems feasible to do in under a month. If the answer is that we've got 10 contentious UI issues and we probably want to go through the PEP process, I agree with your assessment of the timing. Regardless, we'll need to know what work needs to be done before we do it... > but we can try to add nanoseconds to timedelta only. The mixed datetime + > timedelta ops will still truncate, but many time-related operations will be > enabled. I would even argue that when nanoseconds precision is required, it > is more often intervals no longer than a few days and rarely a specific point > in time. To be honest, I don't find this very compelling and I think it will only confuse people. I think most people use `timedelta` to represent something you add or subtract to a `datetime`. Having the `nanoseconds` part of it truncate seems like it would be frustrating and counter-intuitive. >From the use cases in this thread: - ns-precision timestamps: https://bugs.python.org/issue15443#msg180125 - ns-precision timestamps: https://bugs.python.org/issue15443#msg223039 - Your suggestion that `datetime` should be able to support what `timespec` does: https://bugs.python.org/issue15443#msg223042 - ns-precision timestamps: https://bugs.python.org/issue15443#msg270266 So I don't think there's high enough demand for nanosecond-timedelta on its own that we need to rush it out there before datetime gets it. -- ___ Python tracker <https://bugs.python.org/issue15443> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43484] valid datetimes can become invalid if the timezone is changed
Paul Ganssle added the comment: > That it allows creating the datetime instance looks like a bug to me, i.e. a > time before 0001-01-01 00:00 UTC is invalid. What am I misunderstanding? `datetime.datetime(1, 1, 1, tzinfo=timezone(timedelta(hours=1)))` is a valid datetime, it's just that it cannot be converted to all other timestamps, because in some time zones, the same absolute time is out of datetime's range. `datetime.datetime` is a representation of an abstract datetime, and it can also be annotated with a time zone to basically tag the civil time with a function for converting it into other representations of the same *absolute* time. The range of valid `datetime.datetime` objects is based entirely on the naïve portion of the datetime, and has nothing to do with the absolute time. So this is indeed a natural consequence of the chosen design. If we wanted to change things, it would cause a number of problems, and the cure would be much worse than the "disease". For one thing, accessing UTC offsets is done lazily, so `.utcoffset()` is not currently called during `datetime` creation. The datetime documentation lays out that this is consistent with the raison d'être of `datetime`: "While date and time arithmetic is supported, the focus of the implementation is on efficient attribute extraction for output formatting and manipulation." In order to determine whether a given `datetime` can always be converted to an equivalent datetime in any time zone, we'd need to actively determine its UTC offset, which would be a major performance regression in creating aware datetimes. We could avoid this performance regression by only doing the `.utcoffset()` check when the datetime is within 2 days of `MINYEAR` or `MAXYEAR`, but while this would be a more minor performance regression, it would also add new edge cases where `.utcoffset()` is sometimes called during the constructor and sometimes not, which is not ideal. Not to mention if we were to ever open up the allowed return values for `.utcoffset()` the logic might get hairier (depending on the nature of the allowed values). Another issue with "fixing" this is that it would take currently-valid datetimes and turn them into invalid datetimes, which violates backwards compatibility. I imagine in most cases this is only done as part of test suites, since TZ-aware datetimes near 0 and 10,000 CE are anachronistic and not likely to be of much instrumental value, but the same can be said of these potentially "invalid" dates in the first place. Additionally, even worse is that even naïve datetimes can be converted to UTC or other time zones, and if we want to add a new constraint that `some_datetime.astimezone(some_timezone)` must always work, then you wouldn't even be able to *construct* `datetime.MINYEAR` or `datetime.MAXYEAR`, since `datetime.MINYEAR.astimezone(timezone(timedelta(hours=-24)))` would fail everywhere, and worse, the minimum datetime value you could construct would depend on your system locale! Again, the alternative would be to make an exception for naïve datetimes, but given that this change is of dubious value to start with, I don't think it is worth it. > So I'm pretty sure this is "not a bug" but it's a bit of a problem and I have > a user suggesting the "security vulnerability" bell on this one, and to be > honest I don't even know what any library would do to "prevent" this. I don't really know why it would be a "security vulnerability", but presumably a library could either convert their datetimes to UTC as soon as they get them from the user if they want to use them as UTC in the future, or they could simply refuse to accept any datetimes outside the range `datetime.datetime.MINYEAR + timedelta(hours=48) < dt.replace(tzinfo=None) < datetime.datetime.MAXYEAR - timedelta(hours=48)`, or if the concern is only about UTC, then refuse datetimes outside the range `datetime.MINYEAR.replace(tzinfo=timezone.utc) < dt < datetime.MAXYEAR.replace(tzinfo=timezone.utc)`. > Why's this a security problem? ish?because PostgreSQL has a data type > "TIMESTAMP WITH TIMEZONE" and if you take said date and INSERT it into your > database, then SELECT it back using any Python DBAPI that returns datetime() > objects like psycopg2, if your server is in a timezone with zero or negative > offset compared to the given date, you get an error. So the mischievous user > can create that datetime for some reason and now they've broken your website > which can't SELECT that table anymore without crashing. Can you clarify why this crashes? Is it because it always returns the datetime value in UTC? > So, suppose you maintain the database library that helps people send data in > and out of psycopg2.We have, the end user's application, we have the
[issue43382] github CI blocked by the Ubuntu CI with an SSL error
Change by Paul Ganssle : -- nosy: +p-ganssle ___ Python tracker <https://bugs.python.org/issue43382> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43354] xmlrpc.client: Fault.faultCode erroneously documented to be a string, should be int
Paul Ganssle added the comment: The evidence you have here seems pretty compelling and this change seems straightforward enough. I don't see an expert listed here, but I'm happy to merge a docs PR fixing this. Probably a good idea to make a PR to typeshed in parallel, in case they have some further insight into this. -- nosy: +p-ganssle ___ Python tracker <https://bugs.python.org/issue43354> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43132] Incorrect handling of PyObject_RichCompareBool() in the _zoneinfo module
Paul Ganssle added the comment: Re-opening because this was merged without tests. @ZackerySpytz would you mind adding tests to hit these error cases? I've spent some time satisfying myself that it's indeed possible, and I've detailed a general outline here: https://github.com/python/cpython/pull/24450#issuecomment-774190609 -- nosy: +p-ganssle resolution: fixed -> stage: resolved -> needs patch status: closed -> open type: -> behavior ___ Python tracker <https://bugs.python.org/issue43132> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42874] running configure on Solaris 10 gives grep "illegal option" errors
Paul Ganssle added the comment: This particular grep statement is used to validate the `tzpath` variable. Apparently it is easy enough to achieve what I was going for using vanilla grep with no options, so I've created GH-24200 to fix the issue. I notice that there are other uses of `-q` and `-E` in the configure file, but presumably those are on more optional paths. @martin.wheatley.home: Can you check to see if GH-24200 fixes your issue? Also, can you clarify whether this happens with a plain `./configure` invocation, or are you specifying `./configure --with-tzpath='/usr/share/zoneinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo'` yourself? -- versions: +Python 3.10 ___ Python tracker <https://bugs.python.org/issue42874> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42874] running configure on Solaris 10 gives grep "illegal option" errors
Change by Paul Ganssle : -- keywords: +patch pull_requests: +23025 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24200 ___ Python tracker <https://bugs.python.org/issue42874> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42697] 3.8.7rc1 regression: 'free(): invalid pointer' after running backports-zoneinfo test suite
Paul Ganssle added the comment:
For future reference, this bug is triggered only when `.fromutc` is called on a
subclass of `datetime` and the resulting date is the second ambiguous time
(e.g. if there's a DST transition from 02:00 → 01:00, and the result of the
`.fromutc` call is the *second* 01:30, you will trigger this issue).
Here's an MWE:
```
from backports.zoneinfo import ZoneInfo
from datetime import datetime, timezone
class SubDT(datetime):
pass
LON = ZoneInfo("Europe/London")
d = SubDT(2020, 10, 25, 1, 30, tzinfo=timezone.utc)
# Each pass through the loop inappropriately reduces the reference count on the
# `1` object by 1. Since there are usually a large number of live references to
# `1`, this won't have any immediate noticeable effect unless you do it a lot.
for i in range(1):
d.astimezone(LON)
```
--
___
Python tracker
<https://bugs.python.org/issue42697>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42663] zoneinfo does not support full range of allowed transition hours in fallback string
New submission from Paul Ganssle : TZif files consist of a list of transitions followed by a POSIX TZ var-style string of a form like "AAA3BBB,M3.2.0/01:30,M11.1.0/02:15:45", which decodes to "AAA (UTC-3) is the standard time and BBB (UTC-2) is the DST time, DST applies starting at 02:15:45 local time on the 1st Sunday in November and ending at 1:30:00 local time on the 2nd Sunday in March". After the last listed transition, the rule specified by the TZ variable applies. POSIX says that the "hours" part of the transition rule must be in the range ±24, but as mentioned in a TODO comment in _zoneinfo.c, RFC 8536 §3.3.1 specifies that the hours part of transition times may range from -167 to 167 (see: https://github.com/python/cpython/blob/master/Modules/_zoneinfo.c#L1844-L1847 ). Currently, zoneinfo does not support the full range of possible transitions, and a TZif file with a 3-digit transition hour would likely fail to parse. This isn't a terribly high priority at the moment, but if the tz project ever releases a TZif file with one of these TZ strings on it, it will all of a sudden become very critical to fix it, so we should probably try to get it fixed before Python 3.9 is EOL, so that all versions of Python with `zoneinfo` can handle this properly. -- components: Library (Lib) messages: 383206 nosy: p-ganssle priority: normal severity: normal stage: needs patch status: open title: zoneinfo does not support full range of allowed transition hours in fallback string type: behavior versions: Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42663> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new
Paul Ganssle added the comment:
> Adding a static assertion about the signedness of uint8_t looks meaningless
> to me.
My proposal is to add a static assertion that `day` is an unsigned type. In the
code it would look something like it does in the backports.zoneinfo code
(https://github.com/pganssle/zoneinfo/blob/07ec80ad5dc7e7e4b4f861ddbb61a9b71e9f27c7/lib/zoneinfo_module.c#L1247-L1255):
```
// The actual bounds of day are (day >= 0 && day <= 6), but since day is an
// unsigned variable, day >= 0 is always true. To ensure that a bug is not
// introduced in the event that someone changes day to a signed type, we
// will assert that it is an unsigned type.
Py_ASSERT_VAR_UNSIGNED(day);
if (day > 6) {
PyErr_Format(PyExc_ValueError, "Day must be in [0, 6]");
return -1;
}
```
> I propose to change types of function parameters and local variables. In any
> case the constructor checks the range of parameters, so there is no problem
> with packing them into compact structure.
> This will help to avoid errors of implicit conversion between different
> integer types. Also it can help to avoid code duplication in parsing integers
> of different size and signedness.
This is not entirely unreasonable in my opinion, though in this case everything
is determined by the POSIX standard and an RFC, so it is unlikely that we'll
ever see anything outside of 8 bit integers for these things. If you'd like to
refactor the parsing code to use signed integers unconditionally and have them
converted as part of the constructor that seems reasonable.
--
___
Python tracker
<https://bugs.python.org/issue42660>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new
Paul Ganssle added the comment: There are at least two issues with this: 1. This is a constructor for a struct, and the struct would really unnecessarily balloon in size if we switched everything over to be 32-bit integers (which I think is your suggestion?). This is not a major concern because in typical operation there are not likely to be more than 500 of these structs in existence at any one time (due to the cache), but it's still a minor annoyance. I would guess that accepting `int` in the constructor function and converting it to uint8_t as part of building the struct would be maybe neutral to negative, since it involves casting a large signed type to a small unsigned one. This could cause more problems with overflow, not less. 2. In the current implementation `day` is unsigned by its nature — it represents a value indexed at 0 for which negative indices have no meaning. If we leave `day` as a uint8_t, then that tells the compiler at least something about the valid range of the value. By turning `day` (and presumably the other elements of this struct) into a less-suitable type, we're depriving ourselves of possibly *useful* compiler warnings. Barring a solution where we have a simple and robust way to suppress warnings, I think the next-best solution is to add a static assertion about the signedness of the type for this particular case. I'd be happy to write it in a relatively general way so that it can be re-used elsewhere in CPython for the same purpose. -- ___ Python tracker <https://bugs.python.org/issue42660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new
Paul Ganssle added the comment: > Just my two cents, this isn't just "some compilers". Everything from gcc, > msvc, C# to the rust compiler complain about this sort of code. As they > should, this is effectively dead code. They complain because most of the time it's a sign that people were intending to use a signed integer, and it's an indicator that they may be susceptible to integer overflow. In this case, it was a deliberate choice to include the extra check knowing it's dead code, because it is essentially a machine-checked documentation of the bounds of that particular variable. > I think the more pragmatic way to enforce and document this assumption would > be to have a unit test that actually checks that the constructor fails with > "negative" days. It'll continue to fail right now as its interpretation as an > unsigned int will be large and it will start failing if someone changes this > to a signed type. This would be helpful, but it's not an either-or situation. This particular thing would be tricky to write a targeted unit test for, because it's a very deep implementation detail. Such a unit test would also be quite physically separated from the code in question, whereas if we left the code as-is, we'd never see this type of bug, and if we added a static assertion we'd be told at compile time exactly what is wrong. The biggest problem here is the fact that we cannot disable compiler warnings when complying with them makes the code less readable and/or less robust. This makes compiler warnings less useful, since it changes the calculus around whether or not it's worth it to introduce a new compiler warning. -- ___ Python tracker <https://bugs.python.org/issue42660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new
Paul Ganssle added the comment: As I mentioned, it's a style issue. Yes this did not introduce any user-observable bugs, nor should it have changed the machine code emitted by the assembly on any competent compiler. The issue is that I had deliberately chosen to do a "redundant" check to improve the readability and to be robust against someone saying, "Why is this a unit8_t instead of an int? I don't think it makes anything faster..." or some other justification for changing all the uint8_t fields to ints. Previous to this, we had something where the compiler would fix any bugs caused by that by itself by emitting an additional bounds check. In my proposed fix to the compiler warnings, there was a static assertion where the machine would alert us if the situation changed. The problem is that we now have a non-machine-checked comment for something that would be difficult to issue tests for. Frankly, I think that even the worst case scenario here isn't that bad — TZif files malformed in a very specific way would be accepted as valid. As it's coded now, this would probably just lead to a crash later, or possibly some weird results in time zone math. Still, I think we need a solution to this problem because our blind adherence to an invalid compiler warning is leading us to write more fragile code, whic h is especially problematic in C, which is notoriously dangerous and problematic to write. -- ___ Python tracker <https://bugs.python.org/issue42660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new
New submission from Paul Ganssle : This is a code style issue — in https://github.com/python/cpython/pull/23614, a regression was deliberately introduced to satisfy an overzealous compiler. The `day` variable has logical bounds `0 <= day <= 6`. In the original code, both sides of this boundary condition were explicitly checked (since this logically documents the bounds of the variable). Some compilers complain about checking `day < 0`, because `day` is an unsigned type. It is not an immutable fact that `day` will always be an unsigned type, and implicitly relying on this fact makes the code both less readable and more fragile. This was changed over my objections and despite the fact that I had a less fragile solution available that also satisfied the overzealous compiler. In the short term, my preferred solution would be to add in a static assertion that `day` is an unsigned type — this does not have to work on every platform, it simply needs to serve as a notification to make the code less fragile and to document our assumptions to both readers and the compiler. In the long term, I think we need a way to solve the problem that it is apparently not possible to disable any compiler warnings even if they don't apply to the situation! -- components: Library (Lib) messages: 383180 nosy: p-ganssle priority: normal severity: normal stage: needs patch status: open title: _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new type: behavior versions: Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42660> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35498] Parents objects in pathlib.Path don't support slices as __getitem__ arguments
Paul Ganssle added the comment: New changeset 79d2e62c008446fbbc6f264bb8a30e2d38b6ff58 by Yaroslav Pankovych in branch 'master': Added support for negative indexes to PurePath.parents (GH-21799) https://github.com/python/cpython/commit/79d2e62c008446fbbc6f264bb8a30e2d38b6ff58 -- ___ Python tracker <https://bugs.python.org/issue35498> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21041] pathlib.PurePath.parents rejects negative indexes
Paul Ganssle added the comment: New changeset 79d2e62c008446fbbc6f264bb8a30e2d38b6ff58 by Yaroslav Pankovych in branch 'master': Added support for negative indexes to PurePath.parents (GH-21799) https://github.com/python/cpython/commit/79d2e62c008446fbbc6f264bb8a30e2d38b6ff58 -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue21041> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42444] pathlib.PurePath properties annotated with .. data directive
New submission from Paul Ganssle : Currently, it seems that the pathlib module uses `.. data::` to annotate the properties of the PurePath type (e.g. .parts, .drive, .root, etc). See: https://github.com/python/cpython/blob/ff420f0e08a2443339da0df7ace95e14177bac53/Doc/library/pathlib.rst According to the documentation (https://devguide.python.org/documenting/#information-units), `data` is for module-level constants, specifically: > Describes global data in a module, including both variables and values used > as “defined constants.” Class and object attributes are not documented using > this directive. I believe that we should switch these over to use the `.. attribute:` directive instead. >From what I can tell, you can still link to these attributes using the >`:attr:` role. I haven't checked if you can link to `:attribute:`s using the >`:data:` role, though. If not, it might break some links to change these to >`:attribute:`. -- assignee: docs@python components: Documentation messages: 381673 nosy: docs@python, eric.araujo, ezio.melotti, mdk, p-ganssle, willingc priority: low severity: normal status: open title: pathlib.PurePath properties annotated with .. data directive versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42444> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21041] pathlib.PurePath.parents rejects negative indexes
Paul Ganssle added the comment:
I think you may have confused my thoughts as to why this might be considered
ambiguous with an actual suggestion for what the semantics should be.
I think that we should stick with `p.parents[x] == tuple(p.parents)[x]` for any
valid value of `x`, which means that `p.parents[-1]` will always be `Path('.')`
for any non-empty `p`.
--
___
Python tracker
<https://bugs.python.org/issue21041>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35498] Parents objects in pathlib.Path don't support slices as __getitem__ arguments
Change by Paul Ganssle : -- dependencies: -pathlib.PurePath.parents rejects negative indexes resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue35498> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35498] Parents objects in pathlib.Path don't support slices as __getitem__ arguments
Paul Ganssle added the comment: New changeset 452058448335b39c784af9a047f9c4a1767c0b00 by Joshua Cannon in branch 'master': bpo-35498: Added slice support to PathLib parents attribute. (GH-11165) https://github.com/python/cpython/commit/452058448335b39c784af9a047f9c4a1767c0b00 -- ___ Python tracker <https://bugs.python.org/issue35498> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35498] Parents objects in pathlib.Path don't support slices as __getitem__ arguments
Paul Ganssle added the comment: One question I would have about this is that `.parents` is a lazily-calculated sequence, not a list or a tuple, so it's not immediately obvious what the return type of a slice would be. I don't think it makes sense to return, e.g. a `_PathParents` object with fewer parts, but I don't know if there's any traditional choice here, other than that the result of slicing Sequence is another Sequence. The PR returns a list, but I'm inclined to say we should return a tuple. -- nosy: +p-ganssle versions: +Python 3.10 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue35498> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21041] pathlib.PurePath.parents rejects negative indexes
Paul Ganssle added the comment:
I am not seeing any compelling reasons to avoid supporting negative indexes
*or* slices here.
If I had to guess about the confusing semantics of negative indices, I would
guess it's the fact that the index in the -1 position for a non-empty Path will
always be `Path('.')`. Since that's not terribly useful, it might be reasonable
to have negative indices start counting at `len(p)-2`.
That said, I don't think this is a big deal, and I think we have more
speculation on why this was avoided in the first place than we have actual
objections to changing it, so I vote for changing it.
I think our best option is to say that the semantics of indexing `.parents`
should be the same as indexing the result of casting it to a tuple, so this
should be true:
p = Path(x)
assert p.parents[y] == tuple(p.parents)[y]
For all values of `x` and `y`.
I've gone ahead and changed the version support matrix to 3.10 only, since I
think that this was a deliberate choice and we should be considering this an
enhancement rather than a bugfix. That said, I'll admit that it's on the
borderline — the semantics of sequences are unambiguous (see, which says that
sequences support both slices and negative indices:
https://docs.python.org/3/library/stdtypes.html#typesseq ), and PEP 428
explicitly says that .parents returns a "an immutable sequence of the path's
logical ancestors":
https://www.python.org/dev/peps/pep-0428/#sequence-like-access . So if someone
is motivated to try and make the case that this is a bugfix that could be
backported to earlier supported versions, I won't stand in their way.
--
nosy: +p-ganssle
versions: -Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue21041>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42390] Other Python implementations may not expose the module name in datetime type names
Paul Ganssle added the comment: What is an example of another Python implementation that has this property? Is there a concrete issue open somewhere that this is solving? I am not unsympathetic to the idea of accommodating other implementations of Python, but this is very abstract and I think the assumption is probably that if we're explicitly testing for something and we don't say it's implementation-defined that it is part of the language spec. If there's some evidence that stuff like this is intended to be implementation-defined always, or there's some concrete problem that we can solve (and possibly also add tests to avoid regressions), I'd be much more comfortable with something like this. -- ___ Python tracker <https://bugs.python.org/issue42390> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42094] isoformat() / fromisoformat() for datetime.timedelta
Paul Ganssle added the comment: This is probably more feasible than the proposal in bpo-41254 since it's a well-defined spec (mostly — it includes an optional alternative format and the number of digits allowed is defined "by agreement", thus defeating the purpose of using a spec in the first place) that's not even particularly difficult to implement, but there are still a few problems (and one reason I've never implemented this, despite desperately wanting a better string representation for time deltas). Two minor problems first: 1. Unlike ISO 8601 datetimes, these are not especially "human-friendly" formats, so I don't think they're especially useful for displaying timedeltas. 2. Also unlike ISO 8601 datetimes, I don't think these are in particularly wide use, or widely supported. That's not a major strike against it, but if it's not useful as something to show to humans and it's not especially useful as something to show to / read from other computers, that weighs against its inclusion in the standard library. The biggest problem, however, is that `timedelta` does not and cannot represent "Year" or "Month", which means that `P1Y` or `P1M` would always need to be invalid to parse. We could eliminate this format, but it means that we would never at any point in the future be able to implement a parser for the full spec. Since the concept of a year and a month are ambiguous and at least the 2016 version of ISO 8601 doesn't seem to define what it means for a duration to last 1 year or 1 month, you can't even really count on such a thing as an interchange format, because different implementations might give you different results! What does `20200131T00:00:00/P1M` represent? The interval (2020-01-31, 2020-02-29)? (2020-01-31, 2020-03-02)? Something else? A better target for parsing ISO 8601 durations would be something like `dateutil.relativedelta`, which does have defined semantics for years and months (though as I mentioned above, those are not necessarily consistent with the semantics of other libraries parsing or writing out this format). I am also not entirely clear on whether "weeks" is just an alias for "7 days" or if it means something related to weeks in the ISO calendar (and if that makes a difference for durations). I imagine that generating these formats is a bit more forgiving, because you would simply never generate the forbidden formats, and we can offer configuration options in the formatter method to allow the user to tweak the various ambiguities in the spec. -- ___ Python tracker <https://bugs.python.org/issue42094> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42094] isoformat() / fromisoformat() for datetime.timedelta
Change by Paul Ganssle : -- nosy: +p-ganssle ___ Python tracker <https://bugs.python.org/issue42094> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42390] Other Python implementations may not expose the module name in datetime type names
Paul Ganssle added the comment: Is this an actual problem for another implementation of Python? Is there some reason to think that we intended the repr of a `datetime` object to be implementation-defined? -- nosy: +p-ganssle ___ Python tracker <https://bugs.python.org/issue42390> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42371] datetime.fromisoformat(): Omitted colon in timezone suffix raises ValueError
Paul Ganssle added the comment: This is the expected behavior of `.fromisoformat()`. A similar issue is https://bugs.python.org/issue35829, which asks for the "Z" suffix to be supported. There is a note about this in the documentation: https://docs.python.org/3/library/datetime.html#datetime.datetime.fromisoformat "Caution This does not support parsing arbitrary ISO 8601 strings - it is only intended as the inverse operation of datetime.isoformat(). A more full-featured ISO 8601 parser, dateutil.parser.isoparse is available in the third-party package dateutil." At some point we will work out the kinks in offering as full an ISO 8601 datetime parser as possible, but the ISO 8601 datetime spec is very complicated and includes many optional features. We deliberately chose to keep the scope of `.fromisoformat()` minimal at first, whereas `dateutil.parser.isoparse` attempts to be a full-featured ISO8601 parser. Changing the version affected to 3.10, since this is a feature request. -- type: behavior -> enhancement versions: +Python 3.10 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue42371> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42070] Revisit IsoCalendarDate being private from pickle perspective
Paul Ganssle added the comment: I'm glad that Terry brought up documentation, though, because I realized that this is not a documented behavior: https://docs.python.org/3/library/datetime.html#datetime.date.isocalendar We should maybe document it? Though if we document it it might be considered more of a breaking change to change the behavior than if it's an undocumented feature. -- ___ Python tracker <https://bugs.python.org/issue42070> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42070] Revisit IsoCalendarDate being private from pickle perspective
Paul Ganssle added the comment: Even if it were accidental (and it wasn't — it was actually somewhat difficult to achieve), I'd still argue for not changing it in 3.9, because it would mean that pickles created in 3.9.(n+1) would not be readable in 3.9.n. Still, I don't think I'd be convinced without some real-life use cases. The SO question is asking about the reasoning for this in the abstract — the poster noticed that we designed it this way and saw a possible objection to this, but it was one that we had considered and decided to make the trade-off a different way. I informally asked many people about this, since it was by far the weirdest design decision made in that issue (I say that in the passive tense not to deflect from the fact that I made the decision, but to own the fact that it was weirder than any of the design decisions made by anyone else, either ), and generally they could not give me any concrete reasons it would break (but they also all counseled not to try to get too clever with pickling logic). I think I'm happy with the decision if we remain in the realm of the abstract, but I recognize how weird it is, and I think if someone came up with a compelling workflow that this breaks, we could change it (in a feature release). This was specifically proposed to avoid backwards-compatibility problems, so it wouldn't be any more of a breakage to change it in future feature releases than it would have been to do it in 3.9. -- ___ Python tracker <https://bugs.python.org/issue42070> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41056] minor NULL pointer and sign issues reported by Coverity
Change by Paul Ganssle : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue41056> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30155] Add ability to get tzinfo from a datetime instance in C API
Change by Paul Ganssle : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue30155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42025] zoneinfo: wrong time difference across transition when tzinfo is equal
Paul Ganssle added the comment: Yeah, people are very confused by this, which is why I wrote that article. Maybe there is a place for big warnings somewhere? I have been mulling over the possibility of proposing a backwards-incompatible (though minimally so, hopefully) change to arithmetic semantics that would work well with how people think this should work, but I think any such backwards-incompatible change would be a Big Deal™, and unfortunately quite hard to warn about with a proper deprecation period. -- resolution: -> not a bug stage: -> resolved status: open -> closed type: -> behavior ___ Python tracker <https://bugs.python.org/issue42025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42109] Use hypothesis for testing the standard library, falling back to stubs
Change by Paul Ganssle : -- keywords: +patch pull_requests: +21805 pull_request: https://github.com/python/cpython/pull/22863 ___ Python tracker <https://bugs.python.org/issue42109> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42109] Use hypothesis for testing the standard library, falling back to stubs
New submission from Paul Ganssle : Following up on this year's language summit, I would like to propose that we start integrating property tests into the standard library. Zac Hatfield-Dodds (along with myself and Carl Friedrich Bolz-Tereick) have put together this repository of tests that we run against the standard library as is: https://github.com/Zac-HD/stdlib-property-tests Here is the blog post covering the proposal from the language summit: https://pyfound.blogspot.com/2020/05/property-based-testing-for-python.html The biggest challenges here are logistical: 1. Pulling in third party dependencies is difficult to do reliably on CI, but checking in hypothesis's entire dependency tree is probably not feasible. 2. We don't necessarily want to require developers running their tests locally to have to set up a hypothesis environment just to run the tests. 3. Hypothesis tests are not (by default) deterministic, which some are concerned may lead to flakiness by themselves. To allay these concerns, I propose that we implement a compatibility interface for hypothesis that uses the third party module when it's installed, but otherwise falls back to stubs. The way I see the stubs working is that `@given` *without* `@example`s would simply skip the test. If you specify `@given` and one or more `@example`s, the test falls back to a simple parameterized test when hypothesis is not available. At least at first, we won't attempt to add a mandatory PR job that runs with hypothesis installed. Instead, I'd like to run either an optional job on PR or have one or more buildbots that runs the hypothesis tests. I would also like to suggest a policy of including at least one example in each property test, so that on PR at least some of the inputs are tested. -- assignee: p-ganssle components: Tests messages: 379226 nosy: Zac Hatfield-Dodds, p-ganssle, terry.reedy priority: normal severity: normal stage: patch review status: open title: Use hypothesis for testing the standard library, falling back to stubs type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42109> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42070] I think the rationale to keep IsoCalendarDate private from the pickle perspective should get revisited
Paul Ganssle added the comment: That's a reasonable enough objection, though what use case do you have for storing the IsocalendarDate object? The main reason we switched to using a named tuple like this was because the vast majority of uses of `isocalendar()` that I saw were people doing stuff like `dt.isocalendar()[0]`, rather than destructuring the tuple or accessing more than one element from the result. It seems to me that if you are using a pickle for a cache, you'd either pickle the `datetime` itself (and call `.isocalendar()` in the process that has read from the cache already), or you'd store one or more of the fields directly on the object that you are caching. A real life use case for this would help. -- ___ Python tracker <https://bugs.python.org/issue42070> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30155] Add ability to get tzinfo from a datetime instance in C API
Paul Ganssle added the comment: New changeset 2e4dd336e5b50fd30947fdecb605ddcd71f7f6f5 by Zackery Spytz in branch 'master': bpo-30155: Add macros to get tzinfo from datetime instances (GH-21633) https://github.com/python/cpython/commit/2e4dd336e5b50fd30947fdecb605ddcd71f7f6f5 -- ___ Python tracker <https://bugs.python.org/issue30155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40536] Addition of a "list of available time zones" function to zoneinfo
Paul Ganssle added the comment: Thanks Łukasz. Sorry I forgot to respond last time. We've had no feedback on this whatsoever, and I think we've probably made the right choice, so we can go ahead and call this resolved. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40536> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41025] C implementation of ZoneInfo cannot be subclassed
Paul Ganssle added the comment: This is fixed in the 3.9 and master branches, it needs to be cherry-picked into the 3.9.0 release (at Łukasz's discretion, of course). Łukasz: If you cherry-pick GH-20965/GH-21876 into the 3.9.0 release, please also cherry-pick GH-21907/GH-21912, since that fixes the refleak. (PRs are listed as "master"/"backport" since I don't know your workflow). -- ___ Python tracker <https://bugs.python.org/issue41025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41734] Refactor b32{encode,decode} tests
Paul Ganssle added the comment:
I agree with Filipe here — I think the b32encode/b32decode tests were
originally written before subtests were available, and this PR has this and
other real improvements.
I understand why you'd want to have a policy of "no refactoring for its own
sake", but as I argued in the PR 20441
(https://github.com/python/cpython/pull/20441#issuecomment-634773049), it's
safer to leave existing tests alone when making changes to the code under test,
since there's the possibility that you both introduce an error *and* modify the
tests in such a way that doesn't catch the error you introduced. In that case,
"refactoring as you go" doesn't really work, and you need a separate PR for
improvements like these.
I'm re-opening the ticket for now because I think we should at least discuss
this before rejecting it out of hand.
> I am a bit confused, in PR 20441 I first just copied the current
> b32{encode,decode} tests but was given feedback which resulted in the
> proposed tests, but now I am being told the opposite, that the tests are
> better as they currently are.
Sorry about the mixed messages. I think you simply chalk this up to the fact
that Serhiy and I apparently disagree about test structure. I reviewed the
previous PR and specifically asked for this change, so I think it was a bit
rash to close the issue right away (though as someone who has probably
prematurely closed his fair share of issues, I should probably not be tossing
about stones in the vicinity of my decidedly double-paned domicile).
--
resolution: rejected ->
stage: resolved -> patch review
status: closed -> open
versions: +Python 3.10
___
Python tracker
<https://bugs.python.org/issue41734>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32313] Wrong inspect.getsource for datetime
Paul Ganssle added the comment: I think that we should re-examine this issue after GH-20472 is merged. I'm not really sure how that will affect this and indeed *how* it should affect this. I am not sure whether people are relying on the current behavior, or what use cases would be improved if we had a different behavior. With regards to this: > The documentation for getfile says "This will fail with a TypeError if the > object is a built-in module, class, or function." > https://docs.python.org/3/library/inspect.html#inspect.getfile This is a bit unclear to me, and I'm not entirely sure if `datetime` qualifies. I think of built-in classes as things like `int` and `float`, and built-in functions as things like `abs` and `sum`, and `datetime` is an extension module — albeit one with a C implementation, and one that is in the standard library. We should probably clarify the wording of `inspect.getsource` and determine what the intended semantics are for PEP-399-style modules, with both a C and pure Python implementation and the C implementation is what's being used. Error? Point to the Python implementation? -- ___ Python tracker <https://bugs.python.org/issue32313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41568] test_zoneinfo leaked [84, 84, 84] references
Change by Paul Ganssle : -- keywords: +patch pull_requests: +21023 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21907 ___ Python tracker <https://bugs.python.org/issue41568> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41025] C implementation of ZoneInfo cannot be subclassed
Paul Ganssle added the comment: There are two refleaks here. One is a reference leaking to the weak cache in `__init_subclass__` (one leak every time a subclass is created), and the other is that when `subclass.clear_cache()` is called, it sets `ZONEINFO_STRONG_CACHE = NULL`, thus causing a reference leak to the parent class's strong cache. I'll send a PR to fix it shortly. -- ___ Python tracker <https://bugs.python.org/issue41025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41025] C implementation of ZoneInfo cannot be subclassed
Paul Ganssle added the comment: Marking as release blocker to put it on the checklist. Feel free to demote it if you decide it should be deferred to 3.9.1. -- priority: high -> release blocker resolution: -> fixed ___ Python tracker <https://bugs.python.org/issue41025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41025] C implementation of ZoneInfo cannot be subclassed
Change by Paul Ganssle : -- stage: patch review -> backport needed ___ Python tracker <https://bugs.python.org/issue41025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41025] C implementation of ZoneInfo cannot be subclassed
Paul Ganssle added the comment: Łukasz: Would it be possible to backport / cherry-pick the changes from PR GH-21876 (https://github.com/python/cpython/pull/21876) into the 3.9.0 branch? I think this is a fairly severe issue considering that pendulum is planning to use a zoneinfo subclass. It was merged as commit 33d3c64095bcdf9066a3441f6dda5d2e2f4118a8. Thanks! -- nosy: +lukasz.langa ___ Python tracker <https://bugs.python.org/issue41025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41025] C implementation of ZoneInfo cannot be subclassed
Paul Ganssle added the comment: New changeset 33d3c64095bcdf9066a3441f6dda5d2e2f4118a8 by Miss Islington (bot) in branch '3.9': bpo-41025: Fix subclassing for zoneinfo.ZoneInfo (GH-20965) (GH-21876) https://github.com/python/cpython/commit/33d3c64095bcdf9066a3441f6dda5d2e2f4118a8 -- ___ Python tracker <https://bugs.python.org/issue41025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41025] C implementation of ZoneInfo cannot be subclassed
Paul Ganssle added the comment: New changeset 87d8287865e5c9f137f6b5cf8c34c2c509eb5e9d by Paul Ganssle in branch 'master': bpo-41025: Fix subclassing for zoneinfo.ZoneInfo (GH-20965) https://github.com/python/cpython/commit/87d8287865e5c9f137f6b5cf8c34c2c509eb5e9d -- ___ Python tracker <https://bugs.python.org/issue41025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41530] zoneinfo: ZoneInfo raises IsADirectoryError instead of ZoneInfoNotFoundError
Paul Ganssle added the comment: By the way, it might be easiest to start with a PR against backports.zoneinfo, because I have a lot more linting, coverage and format checks set up there: https://github.com/pganssle/zoneinfo -- ___ Python tracker <https://bugs.python.org/issue41530> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41530] zoneinfo: ZoneInfo raises IsADirectoryError instead of ZoneInfoNotFoundError
Paul Ganssle added the comment:
I think that `ZoneInfo('__init__.py')` is also a problem, but a slightly
different one. That comes from the fact that in order to support
`importlib.resources`, each of the zoneinfo subdirectories needs an
`__init__.py`, but the ZoneInfo constructor should probably ignore those, since
they are added by `tzdata` and not actually part of a tzdata distribution.
--
___
Python tracker
<https://bugs.python.org/issue41530>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36700] base64 has old references that should be updated
Paul Ganssle added the comment: Now that issue #16995 is resolved, I think we can move forward with updating the text. -- nosy: +p-ganssle ___ Python tracker <https://bugs.python.org/issue36700> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16995] Add Base32 support for RFC4648 "Extended Hex" alphabet (patch attached)
Paul Ganssle added the comment: Thanks Filipe! -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: +Python 3.10 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue16995> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41371] test_zoneinfo fails when lzma module is unavailable
Paul Ganssle added the comment: I think for now skipping the tests when lzma is missing is the easiest thing, though another option would be to drop the compression on the input test data so that the tests don't depend on lzma. Taking a look at the data files, it looks like we get around 50% compression using either lzma or gzip, but the uncompressed file is only 32k to start with: $ du -b tests/data/* 31054 tests/data/zoneinfo_data.json 15127 tests/data/zoneinfo_data.json.gz 12895 tests/data/zoneinfo_data.json.lz We're also currently using the "fat" binaries that `zic` produces (which includes hard-coded transitions all the way until 2038). The new default for `zic` is to produce "slim" binaries, and the script to update test data does nothing to explicitly request fat binaries. If we were to switch over to "slim" binaries, the result would be more like this: $ du -b tests/data/* 8297tests/data/zoneinfo_data_slim.json.gz 7750tests/data/zoneinfo_data_slim.json.lz 15551 tests/data/zoneinfo_data_unc_slim.json So we're still looking at ~2:1 compression for both gzip and lzma, but the overall file size is 50% of what it was to start with. The biggest downside to this is that the way the "slim" binaries work is that once a rule repeats indefinitely, `zic` stops producing explicit transitions for it, and falls back to a simple repeating rule, meaning that the current set of tests would take a different code path. I think we can go with the following course of action (3 or 4 different PRs): 1. Start by skipping the tests when `lzma` is missing. 2. Update the test suite so that it is testing more or less the same thing when the binaries are compiled with `-b slim`. 3. Change `Lib/test/test_zoneinfo/data/update_test_data.py` so that it pulls the raw data from the `tzdata` module on PyPI (which is compiled with `-b slim`) instead of the user's machine. 4. Change `update_test_data.py` to stop using `lzma` and change the tests so that they are able to process the new format of the JSON files. If we ever decide that we really want the compression again, I assume that `gzip` is found more commonly than `lzma` among systems that don't build the whole standard library, so it might be mildly preferable to switch to `gzip`. -- ___ Python tracker <https://bugs.python.org/issue41371> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41254] Add to/from string methods to datetime.timedelta
Paul Ganssle added the comment: I think it is unlikely that we'll want to experiment with this directly in CPython. I don't think a fixed format (other than the annoying one that you already get from calling `str` on a timedelta) is appropriate, but designing a modular format for time differences is more complicated than it might seem. I have had an open issue on dateutil to implement this for ages, but I haven't seen or come up with any proposals for a DSL for specifying timedelta formats: https://github.com/dateutil/dateutil/issues/444 It is annoyingly complicated to do this, and I'd rather it be tried out in other libraries with more flexibility to make breaking changes and a shorter release cadence. Dateutil is a good choice, but a clear and thorough proposal (or at least examples of this done well in other ecosystems we can crib from) is necessary. -- ___ Python tracker <https://bugs.python.org/issue41254> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41321] Calculate timestamp is wrong in datetime.datetime
Paul Ganssle added the comment: Hi dh4931 — this is the expected result, assuming that the offsets changed between those two dates in your system local time. The .timestamp() method returns an epoch time, which is the number of seconds since 1970-01-01T00:00:00 UTC, and so it is inherently timezone-aware. In Python 3, naïve datetimes went from being "unitless datetimes" to representing "local datetimes", and in certain situations (like calling `.timestamp()`), your system's time zone is used. If you want something that gives the number of seconds that has elapsed between two naïve datetimes on the calendar and ignoring any daylight saving time transitions, subtract them directly to get a timedelta, then divide the result by a timedelta representing 1 second, like so: >>> (datetime.datetime(1986, 5, 4, 7, 13, 22) - datetime.datetime(1986, 5, 4, 0, 0, 0)) / datetime.timedelta(seconds=1) 26002.0 >>> (datetime.datetime(1986, 5, 2, 7, 13, 22) - datetime.datetime(1986, 5, 2, 0, 0, 0)) / datetime.timedelta(seconds=1) 26002.0 -- resolution: -> not a bug stage: -> resolved type: -> behavior ___ Python tracker <https://bugs.python.org/issue41321> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41282] Deprecate and remove distutils
Paul Ganssle added the comment: Oops, just realized my previous post said `pip install distutils`. I meant to say that `pip install setuptools` will provide the `distutils` module (right now you do `import setuptools; import distutils` and you get the setuptools-provided version; we're working on a version where `import distutils` comes from `setuptools` regardless of the import order). -- ___ Python tracker <https://bugs.python.org/issue41282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41282] Deprecate and remove distutils
Paul Ganssle added the comment: > I don't think it's a good idea to replace bad habits from distutils with bad > habits in setuptools._distutils. And this is exactly what you get with > pointing directly to setuptools. These are two different questions. We're not asking people to migrate to `setuptools._distutils` (a private module which may not continue to exist in that location), `setuptools` is *adopting* `distutils`, such that `distutils` is a project provided by `pip install distutils` (mind you, this is happening independent of what the standard library does — the only question is whether `import distutils` continues to work if you don't have `setuptools` installed). > While splitting out distutils to a separate package in a Linux distro, I > found some creative usages at runtime of a package (see my lightning talk at > the language summit 2018, and [1]). From my point of view, CPython should > provide documentation how to forward-port these issues without using > setuptools._distutils. At this point, the extent of CPython's documentation on this should probably be, "We are removing `distutils` and moving it into the `setuptools` namespace. In future versions, you will need to install `setuptools` to import the `distutils` package." `setuptools` should almost certainly deprecate `distutils` and probably remove large swathes of it in the process, but that's probably on a case-by-case basis, and it's a separate issue from what needs to happen in CPython. > Currently setuptools only has one component (pkg_resources, [2]) which is > used at runtime. I dislike it if more than that is used at runtime of a > package. I don't think anyone is planning to recommend the use of *any* `setuptools`-provided packages at runtime, including `pkg_resources`. This move is actually a good one from that point of view, because it will require that projects using `distutils` declare a *runtime* dependency on `setuptools`, which will, hopefully, raise some eyebrows. Better than the current situation, where these dependencies are totally undeclared (though probably worse than if `setuptools`, `pkg_resources` and `distutils` were all separate PyPI packages). -- ___ Python tracker <https://bugs.python.org/issue41282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41282] Deprecate and remove distutils
Paul Ganssle added the comment: > So what is the plan to continue to support building cpython itself which > depends on Distutils? Currently the build bootstraps itself without the aid > of an existing Python interpreter instance. There would also be major impacts > across the whole cpython development process. My understanding was that the plan was to move the standard library distutils into a private module somewhere in the standard library and presumably to slim it down to only the bare minimum required for what is necessary to build Python itself. We're really only concerned with the use of distutils to build packages. > For example, there are many open Distutils issues in the bugs.python.org bug > tracker. We need a plan on how those are to be handled (and that should take > into account the expected transition from b.p.o to GitHub issues). People > will continue to submit issues agains Distutils there so triage team members > and core developers need to know how to handle such issues. What if an issue > applies also or only to a previous release branch (i.e. where Distutils is > still in the repo)? As far as I can tell we've already been telling people that issues in distutils should be fixed in setuptools instead for a few years. I don't think anything needs to be done about the currently open distutils tickets before we *deprecate* distutils, though during the deprecation period we'll probably want to decide whether we want to migrate them, do a mass closure or just leave them to be ad hoc closed as people stumble upon them later. Mass closure may be complicated because tickets affecting CPython itself will still need to be addressed. > What about Distutils documentation in the Python docset? THose are just some > off the top of my head. The distutils documentation is already basically just a warning page that says "stop using distutils": https://docs.python.org/3/library/distutils.html#module-distutils Before these reference materials are removed from the docs we'll need to make sure that all the stuff that's still supported is documented on the setuptools side. > I don't think any of these issues are necessarily blockers but they need to > be planned for and reviewed. I think a PEP is definitely in order for a > change of this magnitude. A PEP may be a good idea, but I do think the change doesn't have a particularly large magnitude. Anyone using setuptools or pip has already been getting setuptools' monkey-patched version of distutils for ages now, and soon they will be getting setuptools' vendored version. The documentation already indicates that distutils is at least soft-deprecated in favor of setuptools and we've already been directing issues and PRs to setuptools instead of distutils. This last piece is really formalizing something we've been incrementally working towards for a long time now. Doesn't mean we shouldn't do it carefully and with a lot of notice, but it's also not a sudden and massive shift. -- nosy: +p-ganssle ___ Python tracker <https://bugs.python.org/issue41282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41260] datetime: strftime method takes different keyword argument: fmt (pure) or format (C)
Paul Ganssle added the comment: I think a positional-only argument would be the best option if we could do it, but it would be a backwards-incompatible change and it's probably not worth the hassle. If anyone is using the keyword argument, they're probably using `format=` rather than `fmt=`, since it's pretty rare for anyone to use the pure python version of datetime. PyPy uses it, but I think they tend to aim for consistency with the C API of CPython, and it seems like they already patch s/fmt/format/ themselves: https://foss.heptapod.net/pypy/pypy/-/blob/branch/default/lib_pypy/datetime.py#L781 If anyone wants to make a PR I think we can fix this for 3.10, though unfortunately because it is an API change it can't be backported. I think in typeshed they can safely change from `fmt` to `format` even today (which would almost certainly be more accurate to end user use cases). -- nosy: -rkm stage: -> needs patch ___ Python tracker <https://bugs.python.org/issue41260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40799] Create Lib/_pydatetime.py file to optimize "import datetime" when _datetime is available
Paul Ganssle added the comment: As for deciding between moving to `datetime/` and moving to `_pydatetime`, I think we should send an e-mail to Python-Dev about it to get a wider perspective, because the import machinery is a lot of black magic, and I know that there are large proprietary code bases out there that pile weird stuff on top of it. I'm not sure I can fully appreciate the trade-offs. The biggest advantage I see to moving `datetime` into its own folder is that it gives us a lot more freedom to expand into smaller sub-modules in the future. For example, in `zoneinfo`, we have zoneinfo/_common.py (https://github.com/python/cpython/blob/2e0a920e9eb540654c0bb2298143b00637dc5961/Lib/zoneinfo/_common.py), which is some logic shared between the C and Python implementations; `_zoneinfo.c` is able to rely directly on `_common.py` without importing `zoneinfo/_zoneinfo.py` (which saves us a bunch of other module imports as well). Right now the C implementation of `datetime` only directly imports `time` and `_strptime`, but I could imagine future enhancements that would be stupidly inconvenient to implement in C, but where we wouldn't want to implement all of _pydatetime just to get a pure-Python implementation. Having a namespace available for such packages would be useful. -- ___ Python tracker <https://bugs.python.org/issue40799> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40799] Create Lib/_pydatetime.py file to optimize "import datetime" when _datetime is available
Paul Ganssle added the comment: > bout _strptime, I see that the time.strptime() imports internally the > _strptime module. Ah, sorry, my remark about including `_strptime` was off the cuff — I thought it was only used in `datetime`, which is why I said "possibly _strptime". If it's used for `time` as well, we should leave it where it is. -- ___ Python tracker <https://bugs.python.org/issue40799> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41118] datetime object does not preserve POSIX timestamp
Paul Ganssle added the comment: There's a pretty clear warning on the documentation that utcfromtimestamp is unsuitable for this purpose: https://docs.python.org/3/library/datetime.html#datetime.datetime.utcfromtimestamp What you want is a datetime that knows what time zone it's in, so that it can be translated back into the absolute number of seconds since UTC. The correct way to do that is to tell the datetime it's in UTC by attaching the `datetime.timezone.utc` object (or any equivalent `tzinfo`). I have written a blog post explaining in detail why you should not use `utcnow` or `utcfromtimestamp`: https://blog.ganssle.io/articles/2019/11/utcnow.html Hopefully that is helpful to you. -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue41118> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41056] minor NULL pointer and sign issues reported by Coverity
Paul Ganssle added the comment: New changeset 6c56356109616ea1292aafa48d30536279ec0937 by Paul Ganssle in branch '3.9': [3.9] bpo-41056: Fix a possible MemoryError leak within zoneinfo. (GH-21007) https://github.com/python/cpython/commit/6c56356109616ea1292aafa48d30536279ec0937 -- ___ Python tracker <https://bugs.python.org/issue41056> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41065] Use zip-strict in zoneinfo
Paul Ganssle added the comment: New changeset bc43f6e21244f31d25896875430174cd4ac7604c by Ram Rachum in branch 'master': bpo-41065: Use zip-strict in zoneinfo (GH-21031) https://github.com/python/cpython/commit/bc43f6e21244f31d25896875430174cd4ac7604c -- ___ Python tracker <https://bugs.python.org/issue41065> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41056] minor NULL pointer and sign issues reported by Coverity
Change by Paul Ganssle : -- nosy: +p-ganssle nosy_count: 3.0 -> 4.0 pull_requests: +20249 pull_request: https://github.com/python/cpython/pull/21083 ___ Python tracker <https://bugs.python.org/issue41056> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41059] Large number of Coverity reports for parser.c
Change by Paul Ganssle : -- nosy: +pablogsal ___ Python tracker <https://bugs.python.org/issue41059> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41025] C implementation of ZoneInfo cannot be subclassed
Change by Paul Ganssle : -- keywords: +patch pull_requests: +20143 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/20965 ___ Python tracker <https://bugs.python.org/issue41025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41025] C implementation of ZoneInfo cannot be subclassed
New submission from Paul Ganssle : In the C implementation of zoneinfo.ZoneInfo, __init_subclass__ is not declared as a classmethod, which prevents it from being subclassed. This was not noticed because the tests for ZoneInfo subclasses in C are actually testing zoneinfo.ZoneInfo, not a subclass, due to a mistake in the inheritance tree: https://github.com/python/cpython/blob/8f192d12af82c4dc40730bf59814f6a68f68f950/Lib/test/test_zoneinfo/test_zoneinfo.py#L465-L487 Originally reported on the backport by Sébastien Eustace: https://github.com/pganssle/zoneinfo/issues/82 The fix in the backport is here: https://github.com/pganssle/zoneinfo/pull/83 -- assignee: p-ganssle components: Library (Lib) messages: 371817 nosy: p-ganssle priority: high severity: normal stage: needs patch status: open title: C implementation of ZoneInfo cannot be subclassed versions: Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue41025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40933] zoneinfo may give incorrect dst() in Europe/Minsk in 1942
New submission from Paul Ganssle :
Related to bpo-40930 and bpo-40931, it *seems* that in 1942 only,
`zoneinfo.ZoneInfo` returns -01:00 for DST in Europe/Minsk:
>>> from datetime import datetime, timedelta
>>> from backports.zoneinfo import ZoneInfo
>>> datetime(1942, 1, 1, tzinfo=ZoneInfo("Europe/Minsk")).dst() //
timedelta(hours=1)
It looks like this occurs because they transitioned directly from MSK to CEST,
jumping back 1 hour, then started switching between CEST and CET.
$ zdump -V -c 1941,1944 'Europe/Minsk'
Europe/Minsk Fri Jun 27 20:59:59 1941 UT = Fri Jun 27 23:59:59 1941 MSK
isdst=0 gmtoff=10800
Europe/Minsk Fri Jun 27 21:00:00 1941 UT = Fri Jun 27 23:00:00 1941 CEST
isdst=1 gmtoff=7200
Europe/Minsk Mon Nov 2 00:59:59 1942 UT = Mon Nov 2 02:59:59 1942 CEST
isdst=1 gmtoff=7200
Europe/Minsk Mon Nov 2 01:00:00 1942 UT = Mon Nov 2 02:00:00 1942 CET
isdst=0 gmtoff=3600
Europe/Minsk Mon Mar 29 00:59:59 1943 UT = Mon Mar 29 01:59:59 1943 CET
isdst=0 gmtoff=3600
Europe/Minsk Mon Mar 29 01:00:00 1943 UT = Mon Mar 29 03:00:00 1943 CEST
isdst=1 gmtoff=7200
Europe/Minsk Mon Oct 4 00:59:59 1943 UT = Mon Oct 4 02:59:59 1943 CEST
isdst=1 gmtoff=7200
Europe/Minsk Mon Oct 4 01:00:00 1943 UT = Mon Oct 4 02:00:00 1943 CET
isdst=0 gmtoff=3600
This might get fixed automatically if we do the "plurality" heuristic in
bpo-40930, though we might also consider a heuristic that puts greater weight
on a transition if the names associated with them different only by
transforming a single letter, or insertion of a letter.
I am somewhat puzzled as to why only 1943 is affected, since I would have
thought that all the CEST offsets in that stretch would be considered the same
ttinfo (and thus all would be assigned the same dstoff).
--
assignee: p-ganssle
messages: 371141
nosy: p-ganssle
priority: normal
severity: normal
status: open
title: zoneinfo may give incorrect dst() in Europe/Minsk in 1942
versions: Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue40933>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40931] zoneinfo gives incorrect dst() in Europe/Madrid in 1938
New submission from Paul Ganssle :
Apparently in 1938, Madrid had a "double daylight saving time", where they
transitioned from WET (+0) → WEST (+1) → WEMT (+2) → WEST (+1) → WET (+0):
$ zdump -V -c 1938,1940 'Europe/Madrid'
Europe/Madrid Sat Apr 2 22:59:59 1938 UT = Sat Apr 2 22:59:59 1938 WET
isdst=0 gmtoff=0
Europe/Madrid Sat Apr 2 23:00:00 1938 UT = Sun Apr 3 00:00:00 1938 WEST
isdst=1 gmtoff=3600
Europe/Madrid Sat Apr 30 21:59:59 1938 UT = Sat Apr 30 22:59:59 1938 WEST
isdst=1 gmtoff=3600
Europe/Madrid Sat Apr 30 22:00:00 1938 UT = Sun May 1 00:00:00 1938 WEMT
isdst=1 gmtoff=7200
Europe/Madrid Sun Oct 2 21:59:59 1938 UT = Sun Oct 2 23:59:59 1938 WEMT
isdst=1 gmtoff=7200
Europe/Madrid Sun Oct 2 22:00:00 1938 UT = Sun Oct 2 23:00:00 1938 WEST
isdst=1 gmtoff=3600
Europe/Madrid Sat Oct 7 23:59:59 1939 UT = Sun Oct 8 00:59:59 1939 WEST
isdst=1 gmtoff=3600
Europe/Madrid Sun Oct 8 00:00:00 1939 UT = Sun Oct 8 00:00:00 1939 WET
isdst=0 gmtoff=0
However, zoneinfo reports `.dst()` during the "double daylight saving time"
period as 1 hour:
>>> from datetime import datetime, timedelta
>>> from backports.zoneinfo import ZoneInfo
>>> datetime(1938, 5, 5, tzinfo=ZoneInfo("Europe/Madrid")).dst() /
timedelta(hours=1)
1.0
>>> datetime(1938, 5, 5, tzinfo=ZoneInfo("Europe/Madrid")).tzname()
'WEMT'
I believe the issue is that the "WEMT" is bordered on both sides by DST
offsets, and so the heuristic of "Look for the previous or next non-DST zone
and calculate the difference" doesn't work. We can probably solve this with one
of two heuristics:
1. Allow DST → DST transitions to be included in the calculation of the current
DST, such that when going from x_dst → y_dst, y_dstoff = (y_utcoff - x_utcoff)
+ x_dstoff
2. Look more than 1 transition away to find the nearest STD zone in either
direction, and calculate the offsets from there.
Between this bug and bpo-40930, I suspect we may want to write a rudimentary
parser for `tzdata.zi` to be used only for testing our heuristics, since the
`tzdata.zi` file (shipped with the `tzdata` package), does actually have the
information we want, without resorting to heuristics.
--
assignee: p-ganssle
components: Library (Lib)
messages: 371137
nosy: p-ganssle
priority: normal
severity: normal
status: open
title: zoneinfo gives incorrect dst() in Europe/Madrid in 1938
versions: Python 3.10, Python 3.9
___
Python tracker
<https://bugs.python.org/issue40931>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40930] zoneinfo gives incorrect dst() in Pacific/Rarotonga between 1978 and 1991
New submission from Paul Ganssle :
While developing a shim for deprecating pytz, I discovered this issue with the
Pacific/Rarotonga zone:
>>> from datetime import datetime, timedelta
>>> from backports.zoneinfo import ZoneInfo
>>> datetime(1991, 2, 1, tzinfo=ZoneInfo("Pacific/Rarotonga")).dst() /
timedelta(hours=1)
1.0
This reports that the DST offset is 1 hour, but in fact it should be 30
minutes, because from 1978 to 1991, Pacific/Rarotonga alternated between -0930
and -10:
$ zdump -V -c 1990,1993 'Pacific/Rarotonga'
Pacific/Rarotonga Sun Mar 4 09:29:59 1990 UT = Sat Mar 3 23:59:59 1990 -0930
isdst=1 gmtoff=-34200
Pacific/Rarotonga Sun Mar 4 09:30:00 1990 UT = Sat Mar 3 23:30:00 1990 -10
isdst=0 gmtoff=-36000
Pacific/Rarotonga Sun Oct 28 09:59:59 1990 UT = Sat Oct 27 23:59:59 1990 -10
isdst=0 gmtoff=-36000
Pacific/Rarotonga Sun Oct 28 10:00:00 1990 UT = Sun Oct 28 00:30:00 1990 -0930
isdst=1 gmtoff=-34200
Pacific/Rarotonga Sun Mar 3 09:29:59 1991 UT = Sat Mar 2 23:59:59 1991 -0930
isdst=1 gmtoff=-34200
Pacific/Rarotonga Sun Mar 3 09:30:00 1991 UT = Sat Mar 2 23:30:00 1991 -10
isdst=0 gmtoff=-36000
I believe that the error comes from the fact that before 1978, they were on
-1030 time, then they transitioned to -0930, then started alternating between
-0930 and -10:
$ zdump -V -c 1977,1980 'Pacific/Rarotonga'
Pacific/Rarotonga Sun Nov 12 10:29:59 1978 UT = Sat Nov 11 23:59:59 1978 -1030
isdst=0 gmtoff=-37800
Pacific/Rarotonga Sun Nov 12 10:30:00 1978 UT = Sun Nov 12 01:00:00 1978 -0930
isdst=1 gmtoff=-34200
Pacific/Rarotonga Sun Mar 4 09:29:59 1979 UT = Sat Mar 3 23:59:59 1979 -0930
isdst=1 gmtoff=-34200
Pacific/Rarotonga Sun Mar 4 09:30:00 1979 UT = Sat Mar 3 23:30:00 1979 -10
isdst=0 gmtoff=-36000
Pacific/Rarotonga Sun Oct 28 09:59:59 1979 UT = Sat Oct 27 23:59:59 1979 -10
isdst=0 gmtoff=-36000
Pacific/Rarotonga Sun Oct 28 10:00:00 1979 UT = Sun Oct 28 00:30:00 1979 -0930
isdst=1 gmtoff=-34200
This is not amazingly important, but it would be a good idea to make the result
correct.
Right now I think the heuristic looks for the first example of an STD → DST
transition and decides that's the best option. It might be a good idea to
change the heuristic to look at *all* examples of such transitions and choose
the plurality value, and if there's no plurality value and one of the values is
1H, choose that one.
--
assignee: p-ganssle
components: Library (Lib)
messages: 371136
nosy: p-ganssle
priority: normal
severity: normal
stage: needs patch
status: open
title: zoneinfo gives incorrect dst() in Pacific/Rarotonga between 1978 and 1991
type: behavior
versions: Python 3.10, Python 3.9
___
Python tracker
<https://bugs.python.org/issue40930>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40908] datetime strftime with %Y and 2 digit years
Paul Ganssle added the comment:
This is a duplicate of bpo-13305 and is due to platform-specific
implementations of %Y. On Linux, `strftime()` does not zero-pad to 4, and if
you need to represent years <1000, you should use "%4Y" to zero-pad the output.
I think the ideal resolution would be a cross-platform implementation of
strftime and strptime that does not rely on the system symbols, but that is a
pretty large and overwhelming project.
--
resolution: -> duplicate
stage: -> resolved
status: open -> closed
superseder: -> datetime.strftime("%Y") not consistent for years < 1000
___
Python tracker
<https://bugs.python.org/issue40908>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
