Re: [Python-Dev] Status on PEP-431 Timezones
On 15 April 2015 at 17:00, Lennart Regebro rege...@gmail.com wrote: OK, so I just had a realization. Because we want some internal flag to tell if the datetime is in DST or not, the datetime pickle format will change. And the datetime pickle format changing is the biggest reason I had against changing the internal representation to UTC. So because of this, perhaps we actually *should* change the internal representation to UTC, because that makes the issues I'm fighting with now so much simpler. (I'm currently trying to get arithmetic to do the right thing in all cases, which is crazy complicated). Huh. I didn't think you would need to change any arithmetic (but haven't looked at this for quite some time). You can already add or subtract timedeltas to timezone aware datetime instances. The problem with the existing implementation is the tzinfo instance does not have enough information to do correct conversions when the time is ambiguous, so it has to guess. With the addition of the is_dst hint to the datetime instance, it will no longer need to guess. Arithmetic remains 'add the timedelta to the naive datetime, and then punt it to the tzinfo to make any necessary adjustments' and I thought this would not need to be changed at all. We can add support to unpickle previous datetimes, but we won't be able to add forwards compatibility, meaning that pickles saved in Python 3.5 will not be unpicklable in Python 3.4. I don't think this can be avoided entirely. Any ideas I can come up with that might help are worse than requiring devs to convert their datetimes to strings in the rare case they need their 3.5 pickles read with 3.4. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status on PEP-431 Timezones
On 15 April 2015 at 21:51, Lennart Regebro rege...@gmail.com wrote: On Wed, Apr 15, 2015 at 3:23 PM, Stuart Bishop stu...@stuartbishop.net wrote: Just punting it to tzinfo to make adjustments, ie effectively just doing what normalize() does creates infinite recursion as there is more arithmetic in there, so it's not quite that simple. This sounds familiar. Its infinite recursion if the tzinfo does its calculations using localized datetimes. If the tzinfo is stripped for the calculations, there is no tzinfo to recurse into. At least this was how I hoped it would work, and it sucks if it doesn't. You could be right that using the UTC representation internally for datetimes with a tzinfo makes the most sense. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status on PEP-431 Timezones
On 14 April 2015 at 21:04, Lennart Regebro rege...@gmail.com wrote:
OK, so I realized another thing today, and that is that arithmetic
doesn't necessarily round trip.
For example, 2002-10-27 01:00 US/Eastern comes both in DST and STD.
But 2002-10-27 01:00 US/Eastern STD minus two days is 2002-10-25 01:00
US/Eastern DST
However, 2002-10-25 01:00 US/Eastern DST plus two days is 2002-10-27
01:00 US/Eastern, but it is ambiguous if you want DST or not DST.
And you can't pass in a is_dst flag to __add__, so the arithmatic must
just pick one, and the sensible one is to keep to the same DST.
import pytz
from datetime import datetime, timedelta
tz = pytz.timezone('US/Eastern')
dt = tz.localize(datetime(2002, 10, 27, 1, 0), is_dst=False)
dt2 = tz.normalize(dt - timedelta(days=2) + timedelta(days=2))
dt == dt2
True
tz.normalize(dt - timedelta(days=2))
datetime.datetime(2002, 10, 25, 2, 0, tzinfo=DstTzInfo 'US/Eastern'
EDT-1 day, 20:00:00 DST)
tz.normalize(tz.normalize(dt - timedelta(days=2)) + timedelta(days=2))
datetime.datetime(2002, 10, 27, 1, 0, tzinfo=DstTzInfo 'US/Eastern'
EST-1 day, 19:00:00 STD)
2002-10-27 01:00 US/Eastern is_dst=0 is after the DST transition
(EST). Subtracting 48 hours from it crosses the DST boundary and
should give you 2002-10-27 02:00 US/Eastern is_dst=1, prior to the DST
transition (EDT). Adding 48 hours again goes past 2002-10-27 01:00
EDT, crosses the DST boundary, and gives you back 2002-10-27 01:00
EST.
--
Stuart Bishop stu...@stuartbishop.net
http://www.stuartbishop.net/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status on PEP-431 Timezones
On 10 April 2015 at 22:57, Glenn Linderman v+pyt...@g.nevcal.com wrote: On 4/10/2015 5:27 AM, Stuart Bishop wrote: P.S. The description of NonExistentTimeError in the PEP doesn't seem quite right, as it currently says it will only be thrown if is_dst=None, which seems like a copy and paste error from the definition of AmbiguousTimeError. The PEP is correct here. If you explicitly specify is_dst, you know which side of the transition you are on and which offset to use. You can calculate datetime(2000, 12, 31, 2, 0, 0, 0).astimezone(foo, is_dst=False) using the non-DST offset and get an answer. It might not have ever appeared on the clock on your wall, but it is better than a punch in the face. If you want a punch in the face, is_dst=None refuses to guess and you get the exception. So does this mean that consistent use of is_dst=False throughout the summer would allow for people that refuse to accept government regulation into their timekeeping, and are willing to be out of step with their neighbors, would be able to match their software to their wall-clock time? No, this logic is only done when necessary. The goal is that, by default, you can convert any naive timestamp into any timezone without raising an exception. The vast majority of callsites do not require perfect accuracy, but just want a good enough result they can display or log. They do not want to worry about Heisenbugs, where if their application is being run in a timezone with DST transitions then it fails for one hour per year in the middle of the night. If you do need accuracy, you explicitly override the default arguments and take responsibility for handling the exceptions. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 23 February 2014 08:56, Ethan Furman et...@stoneleaf.us wrote: ``%a`` will call :func:``ascii()`` on the interpolated value's :func:``repr()``. This is intended as a debugging aid, rather than something that should be used in production. Non-ascii values will be encoded to either ``\xnn`` or ``\u`` representation. So we use %a for exactly the same purposes that we used to use %r. Unsupported codes - ``%r`` (which calls ``__repr__`` and returns a :class:`str`) is not supported. But you propose changing the code. I think there would have been a lot less discussion if you just defined %r to do what you propose for %a, as everything would work as people expected. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP-431 non-status
On Wed, Oct 2, 2013 at 9:42 PM, Lennart Regebro rege...@gmail.com wrote: On Wed, Oct 2, 2013 at 3:05 PM, Dirkjan Ochtman dirk...@ochtman.nl wrote: On Wed, Oct 2, 2013 at 2:17 PM, Lennart Regebro rege...@gmail.com wrote: If you wonder about the lack of progress reports on pep-431, this is because of a lack of progress. I simply haven't had any time to work on this. I considered make a kickstarter so I could take time off from working, but to be honest, even with that, I still have no time to realistically get this in before beta 1. Is there a list of stuff that needs to be done? I might be able to help out. Yes. 1. Implement the PEP. :-) I looked at it during the PyCon sprint, and I had hoped to be able to mostly move pytz into datetime, but unfortunately it wasn't that easy, because of the way datetime normalization works. The pytz normalization calls methods that would need to call the normalization... :-) I think the first thing that needs to happen is get the is_dst flag in. Then we can turn pytz into a new, sane API without all that awful normalization stuff :) It doesn't all have to land in the same release. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Offtopic: OpenID Providers
On Fri, Sep 6, 2013 at 8:56 AM, Jesus Cea j...@jcea.es wrote: Sorry, Google, Facebook, Twitter, etc., are not acceptable OpenID providers for me. I should have made that point in my original email. My excuses. Any other suggestion? As Barry mentioned earlier, launchpad.net. Look for the 'lp' icon on pypi, bugs.python.org etc. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime nanosecond support
On Wed, Jul 25, 2012 at 9:11 PM, Christian Heimes li...@cheimes.de wrote: Am 25.07.2012 14:11, schrieb Nick Coghlan: 1. For the reasons presented, I think it's worth attempting to define a common API that is based on datetime, but is tailored towards high precision time operations (at least using a different internal representation, perhaps supporting TAI). This is a great opportunity to implement two requests at once. Some people want high precision datetime objects while others would like to see a datetime implementation that works with dates BC. Back when the datetime library was being designed, a limiting factor was size of the pickle (for reasons that I think no longer apply). Support for the is_dst flag was never in there, only because the extra single bit required overflowed the pickle size limit. If api changes are being considered, please consider adding this bit back to match the standard libraries. This will let me make the pytz timezone library's API saner, and allow Python to do wallclock datetime arithmetic without ambiguous cases. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On Thu, Feb 18, 2010 at 4:38 PM, Lennart Regebro rege...@gmail.com wrote: If it doesn't solve a problem, it shouldn't be done, as it also is going to create problems, because everything does. :) I think a tzinfo implementation in the standard library that uses the system timezone database would be useful to a great many people, providing a standard way of mapping a string to a tzinfo instance. The number of frameworks requiring pytz as a dependency demonstrate this. It is unfortunate that those strings would be platform specific. For most applications this doesn't matter - you are reading the key from a config file or allowing the user to select from a list of possible values. For applications where that actually matters it would be simple enough to install and maintain a local zoneinfo database, for example by allowing pytz to plug itself in or just a well known location in the Python tree where valid compiled zoneinfo files can be copied in from a nearby unix-like system or pytz tarball. As the pytz maintainer, this would help me solve a long standing problem. Currently, pytz tzinfo instances don't really follow the documented tzinfo interface (in order to allow localized datetime arithmetic to be always correct instead of good enough). This is a source of confusion to many users who don't need this level of accuracy. It would be great if the standard library provided a tzinfo implementation that was good enough for the vast majority of users, and for people who do care they can continue to use pytz.timezone() to retrieve the anal tzinfo implementation. Users will be happier as they will have fewer bugs in their code. The alternative for me is to eventually split pytz, somehow providing the simpler interface that works exactly as documented in the Python reference and the anal interface that works per the pytz README (in hindsight, it should have been this way from day 1). I'm happy to work on this if there is agreement. I'm happy to relicense any pytz code used as a basis if necessary (currently MIT), and dateutil is already PSF licensed if that seems a better starting point. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On Fri, Feb 19, 2010 at 4:45 AM, Lennart Regebro rege...@gmail.com wrote:
On Thu, Feb 18, 2010 at 22:42, Martin v. Löwis mar...@v.loewis.de wrote:
That's not true. The registry is readable by any user, and the format is
fully documented.
Yes, but they use non-standard locations, and afaik, pytz does not
support it. If a stdlib pytz would use this you would have to use
different timezone names for Unix and Windows. I don't think that's a
good idea.
Under Windows the only backend API I'm aware of we could use is tzwin and I
think any standard library inclusion would require this or something similar.
For the standard library, I think it would be great if you could do
'datetime.timezone(some_string)' and get a tzinfo instance suitable for most
use cases which seems no problem for tzwin to provide (platform default DST
information, simple interface with potentially incorrect localized datetime
arithmetic over DST transitions). It is unfortunate that the timezone strings
are platform specific, but I feel that is still better than end users having to
keep updating timezone databases or not supporting non-zoneinfo systems at all.
Even with pytz, the timezone strings are version specific to an extent (we had
a real issue where we updated pytz on our web applications, which changed a
preferred timezone name from Asia/Calcutta to Asia/Kolkata per requests from
our users and as a result our wiki's exploded for some users as they where on
separate boxes using a different pytz release that didn't understand that
timezone string).
It would also be trivial to always look up timezone information from compiled
zoneinfo files in a well known location if the system timezone database cannot
find the requested timezone information. So datetime.timezone('US/Eastern')
would work on Windows if you had installed pytz (I'd update pytz to install its
zoneinfo files into the blessed location). Best of both worlds.
Also, the windows data contains only current timezone data, so for
calendars stretching back in time, the Olsen database would be
preferable as it keeps history.
Sure. I'm not saying pytz will disappear or become unmaintained for people who
need it. I believe most people who are using it now don't need it and I'm sure
there are real bugs in real code out there because of this, as using pytz
correctly requires reading and understanding the pytz documentation.
If this is all too ambitious, tzinfo implementations in the standard library
for UTC and the current system timezone would be a step forward and solve most
real world use cases.
--
Stuart Bishop stu...@stuartbishop.net
http://www.stuartbishop.net/
signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On Fri, Feb 19, 2010 at 12:11 PM, Lennart Regebro rege...@gmail.com wrote: But is the You don't have to install it really such a big problem so that it's worth the other problems like platform incompatibilities and such? I don't think there are any platform incompatibilities - it will work as documented on all platforms. You lose the ability to assume that two identical pytz versions on different platforms can use the same timezone strings, but gain the ability that system timezone strings can be used with Python. In any case, since you want to make a version that can be included and uses the timezone API, I guess that's a moot question until we have that version. :) As I understand it dateutil pretty much already provides what I'm describing. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On Tue, Feb 16, 2010 at 11:18 PM, Tres Seaver tsea...@palladion.com wrote: Dirkjan Ochtman wrote: On Tue, Feb 16, 2010 at 16:26, Tres Seaver tsea...@palladion.com wrote: Because timezones are defined politically, they change frequently. pytz is released frequently (multiple times per year) to accomodate those changes: I can't see any way to preserve that flexibility if the package were part of stdlib. By using what the OS provides. At least on Linux, the basic timezone data is usually updated by other means (at least on the distro I'm familiar with, it's updated quite often, too; through the package manager). I'm assuming Windows and OS X would also be able to provide something like this. I think pytz already looks at this data if it's available (precisely because it might well be newer). If that were so, I don't think Stuart would be going to the trouble to re-release the library 6 - 12 times per year. The Debian, Ubuntu and I think Redhat packages all use the system zoneinfo database - there are hooks in there to support package maintainers that want to do this. This way the package can be included in the supported release but still receive timezone information updates via the OS (but no code updates, but these are rare and usually irrelevant unless you where the person who filed the bug ;) ). I'd be happy to rework pytz for the standard Library using the system installed zoneinfo database if it is available. I think for the standard library though, it needs to follow the documented API better rather than the .normalize() .localize rubbish I needed to get localized datetime arithmetic working correctly. Having seen the confusion and bug reports over the last few years, I think people who need this are in the minority and pytz can still exist as a separate package to support them. tzwin could be used on Windows platforms - I'd need to look into that further to see if the API can remain consistent between *nix and Windows. I suspect that pytz without the .normalize() .localize() rubbish may look remarkably similar to dateutil so that might be a better option to start from. We could consider extending the existing datetime library to support localized datetime arithmetic. This would either involve adding an extra bit to datetime instances to support the is_dst flag (originally deemed unacceptable as it increased the pickle size by a whole byte), or better support for tzinfo implementations to store the is_dst flag in the tzinfo instance (the approach pytz used). This requires a C programmer though and I'm so very, very rusty. I am not at pycon alas. Some of my coworkers from Canonical will be though and they might be interested as we use pytz for Launchpad and other Canonical projects. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] subprocess.call() and stdin
Redirecting to python-dev for discussion. When I invoke subprocess.call(), I often want to ensure that the subprocess' stdin is closed. This ensures it will die if the subprocess attempts to read from stdin rather than block. This could be done if the subprocess.call() helper closes the input if stdin=subprocess.PIPE, which may be the only sane way to handle this argument (I can't think of any use cases for spawning a subprocess with an input stream that nothing can write too). It could also be done by adding a subprocess.CLOSED constant, which if passed to Popen causes a new closed file descriptor to be given to the subprocess. Original Message Bugs item #1220113, was opened at 2005-06-14 15:04 Message generated for change (Comment added) made by zenzen You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1220113group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Rejected Priority: 5 Submitted By: Stuart Bishop (zenzen) Assigned to: Peter Åstrand (astrand) Summary: subprocess call() helper should close stdin if PIPE Initial Comment: The following code snippet should die instead of hang. from subprocess import call, PIPE rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE, stdin=PIPE) It makes no sense not to close stdin if it is PIPE because the stream cannot be accessed. The use case for this is ensuring a subprocess that detects if it is connected to a terminal or not runs in 'batch' mode, and that it will die instead of hang if it unexpectidly attempts to read from stdin. Workaround is to use Popen instead. -- Comment By: Stuart Bishop (zenzen) Date: 2005-06-22 16:12 Message: Logged In: YES user_id=46639 I can't think of any uses cases for wanting to create an inaccessible pipe and give it to the child. Wanting to pass a closed file handle is common. It is needed when calling a program that behaves differently if its stdin is a terminal or not. Or when you simply would prefer the subprocess to die if it attempts to read from its stdin rather than block. Using Popen instead of call is s simpler workaround than creating and closing a file descriptor and passing it in. Perhaps what is needed is a new constant, subprocess.CLOSED which creates a new file descriptor and closes it? This would be useful for Popen too, allowing call() to remain a think and trivially documented wrapper? -- Comment By: Peter Åstrand (astrand) Date: 2005-06-22 02:08 Message: Logged In: YES user_id=344921 It makes no sense not to close stdin if it is PIPE because the stream cannot be accessed True, but what if you actually *want* to create an inaccessible pipe, and give it to the child? Currently, the call() wrapper is *very* short and simple. I think this is very good. For example, it allows us to document it in two sentences. You get what you ask for: If you use call() with strange arguments, you'll get a somewhat strange behavíour. I see no point in introducing lots of sanity checks in the wrapper functions. The use case for this is ensuring a subprocess that detects if it is connected to a terminal or not runs in batch' mode, and that it will die instead of hang if it unexpectidly attempts to read from stdin I'm not sure I understand what you want, but if you want to have stdin connected to a closed file descriptor, just pass one: from subprocess import call, PIPE rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE, stdin=4711) (Of course, you should verify that 4711 is unused.) If you don't agree with me, post to python-dev for discussion. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1220113group_id=5470 -- Stuart Bishop [EMAIL PROTECTED] http://www.stuartbishop.net/ signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unix line endings required for PyRun* breaking embedded Python
Just van Rossum wrote: Skip Montanaro wrote: Just re.sub([\r\n]+, \n, s) and I think you're good to go. I don't think that in general you want to fold multiple empty lines into one. This would be my prefered regex: s = re.sub(r\r\n?, \n, s) Catches both DOS and old-style Mac line endings. Alternatively, you can use s.splitlines(): s = \n.join(s.splitlines()) + \n This also makes sure the string ends with a \n, which may or may not be a good thing, depending on your application. Do people consider this a bug that should be fixed in Python 2.4.1 and Python 2.3.6 (if it ever exists), or is the resposibility for doing this transformation on the application that embeds Python? -- Stuart Bishop [EMAIL PROTECTED] http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Unix line endings required for PyRun* breaking embedded Python
There is a discussion going on at the moment in postgresql-general about plpythonu (which allows you write stored procedures in Python) and line endings. The discussion starts here: http://archives.postgresql.org/pgsql-general/2005-01/msg00792.php The problem appears to be that things are working as documented in PEP-278: There is no support for universal newlines in strings passed to eval() or exec. It is envisioned that such strings always have the standard \n line feed, if the strings come from a file that file can be read with universal newlines. So what happens is that if a Windows or Mac user tries to create a Python stored procedure, it will go through to the server with Windows line endings and the embedded Python interpreter will raise a syntax error for everything except single line functions. I don't think it is possible for plpythonu to fix this by simply translating the line endings, as this would require significant knowledge of Python syntax to do correctly (triple quoted strings and character escaping I think). The timing of this thread is very unfortunate, as PostgreSQL 8.0 is being released this weekend and the (hopefully) last release of the 2.3 series next week :-( -- Stuart Bishop [EMAIL PROTECTED] http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
