Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Guido van Rossum
On Wed, Feb 15, 2012 at 7:28 PM, Larry Hastings wrote: > > On 02/15/2012 09:43 AM, Guido van Rossum wrote: >> >> *Apart* from the specific use case of making an exact copy of a >> directory tree that can be verified by other tools that simply compare >> the nanosecond times for equality, > > > A d

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-15 Thread Nick Coghlan
On Thu, Feb 16, 2012 at 1:40 PM, Philip Jenvey wrote: > What about a PendingDeprecationWarning? I think you're usually only going to > convert DeprecationWarnings to errors (with python -W > error::DeprecationWarning or warnings.simplefilter('error', > DeprecationWarning)) Think "-Wall" for st

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-15 Thread Philip Jenvey
On Feb 13, 2012, at 8:44 PM, Nick Coghlan wrote: > On Tue, Feb 14, 2012 at 2:25 PM, Eli Bendersky wrote: >> With the deprecation warning being silent, is there much to lose, though? > > Yes, it creates problems for anyone that deliberately converts all > warnings to errors when running their te

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Larry Hastings
On 02/15/2012 09:43 AM, Guido van Rossum wrote: *Apart* from the specific use case of making an exact copy of a directory tree that can be verified by other tools that simply compare the nanosecond times for equality, A data point on this specific use case. The following code throws its asse

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Ben Finney
Guido van Rossum writes: > On Wed, Feb 15, 2012 at 6:06 PM, Greg Ewing > wrote: > > It probably isn't worth the bother for things like file timestamps, > > where the time taken to execute the system call that modifies the > > file is likely to be several orders of magnitude larger. > > Ironical

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-15 Thread Nick Coghlan
On Thu, Feb 16, 2012 at 12:06 PM, Guido van Rossum wrote: > Anyway, I don't think anyone is objecting against the PEP allowing symlinks > now. Yeah, the onus is just back on me to do the final updates to the PEP and patch based on the discussion in this thread. Unless life unexpectedly intervene

Re: [Python-Dev] best place for an atomic file API

2012-02-15 Thread Nick Coghlan
On Thu, Feb 16, 2012 at 11:49 AM, Ben Finney wrote: > No, I have no objection to that implementation. I'm pointing that out > only because the nature of the functionality implies I'd expect to find > it within the ‘os’ module hierarchy. The (very) rough rule of thumb is that the os module handles

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Guido van Rossum
On Wed, Feb 15, 2012 at 6:06 PM, Greg Ewing wrote: > On 16/02/12 06:43, Guido van Rossum wrote: >> >> This does not explain why microseconds aren't good enough. It seems >> none of the clocks involved can actually measure even relative time >> intervals more accurate than 100ns, and I expect that

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-15 Thread Guido van Rossum
On Wed, Feb 15, 2012 at 5:26 PM, Neil Schemenauer wrote: > Guido van Rossum wrote: >> Does this need a pronouncement? Worrying about the speed of symlinks >> seems silly > > I agree.  I wonder if a hard-link was used for legacy reasons.  Some > very old versions of Unix didn't have symlinks.  It

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Greg Ewing
On 16/02/12 06:43, Guido van Rossum wrote: This does not explain why microseconds aren't good enough. It seems none of the clocks involved can actually measure even relative time intervals more accurate than 100ns, and I expect that kernels don't actually keep their clock more accurate than milli

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-15 Thread Matt Joiner
+1 for using symlinks where possible. In deploying Python to different operating systems and filesystems I've often had to run a script to "fix" the hardlinking done by make install because the deployment mechanism or system couldn't be trusted to do the right thing with respect to minimising insta

Re: [Python-Dev] best place for an atomic file API

2012-02-15 Thread Ben Finney
Brian Curtin writes: > On Wed, Feb 15, 2012 at 19:19, Ben Finney wrote: > > Charles-François Natali writes: > > > >> […] using rename() on POSIX systems and MoveFileEx() on Windows > >> (which are now available through os.replace()). It would also use > >> fsync() on POSIX to make sure data is

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-15 Thread Neil Schemenauer
Guido van Rossum wrote: > Does this need a pronouncement? Worrying about the speed of symlinks > seems silly I agree. I wonder if a hard-link was used for legacy reasons. Some very old versions of Unix didn't have symlinks. It looks like it was introduced in BSD 4.2, released in 1983. That se

Re: [Python-Dev] best place for an atomic file API

2012-02-15 Thread Brian Curtin
On Wed, Feb 15, 2012 at 19:19, Ben Finney wrote: > Charles-François Natali writes: > >> Issue #8604 aims at adding an atomic file API to make it easier to >> create/update files atomically, using rename() on POSIX systems and >> MoveFileEx() on Windows (which are now available through >> os.repla

Re: [Python-Dev] best place for an atomic file API

2012-02-15 Thread Ben Finney
Charles-François Natali writes: > Issue #8604 aims at adding an atomic file API to make it easier to > create/update files atomically, using rename() on POSIX systems and > MoveFileEx() on Windows (which are now available through > os.replace()). It would also use fsync() on POSIX to make sure da

Re: [Python-Dev] cpython (3.2): remove unused import

2012-02-15 Thread Mark Lawrence
On 06/02/2012 17:57, Brett Cannon wrote: On Sun, Feb 5, 2012 at 19:53, Christian Heimes wrote: Am 06.02.2012 01:39, schrieb Brett Cannon: I'm going to assume pylint or pyflakes would throw too many warnings on the stdlib, but would it be worth someone's time to write a simple unused import ch

Re: [Python-Dev] best place for an atomic file API

2012-02-15 Thread Nick Coghlan
On Thu, Feb 16, 2012 at 9:29 AM, Steven D'Aprano wrote: > Charles-François Natali wrote: >> What would be the best place for a such a class? >> _pyio, tempfile, or a new atomicfile > > > shutil perhaps? > > As a user, that's the third place I look for file utilities, after builtin > functions and

Re: [Python-Dev] best place for an atomic file API

2012-02-15 Thread Steven D'Aprano
Charles-François Natali wrote: Hi, Issue #8604 aims at adding an atomic file API to make it easier to create/update files atomically, using rename() on POSIX systems and MoveFileEx() on Windows (which are now available through os.replace()). It would also use fsync() on POSIX to make sure data i

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Guido van Rossum
On Wed, Feb 15, 2012 at 11:38 AM, "Martin v. Löwis" wrote: >> *Apart* from the specific use case of making an exact copy of a >> directory tree that can be verified by other tools that simply compare >> the nanosecond times for equality, I don't see any reason for >> complicating so many APIs to p

[Python-Dev] best place for an atomic file API

2012-02-15 Thread Charles-François Natali
Hi, Issue #8604 aims at adding an atomic file API to make it easier to create/update files atomically, using rename() on POSIX systems and MoveFileEx() on Windows (which are now available through os.replace()). It would also use fsync() on POSIX to make sure data is committed to disk. For example,

Re: [Python-Dev] A new dictionary implementation

2012-02-15 Thread Yury Selivanov
Hello Mark, First, I've back-ported your patch on python 3.2.2 (which was relatively easy). Almost all tests pass, and those that don't are always failing on my machine if I remember. The patch can be found here: http://goo.gl/nSzzY Then, I compared memory footprint of one of our applications (

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Benjamin Peterson
2012/2/15 Mark Shannon : > > I reckon PyPy might be able to call clock_gettime() in a tight loop > almost as frequently as the C program (although not with the overhead > of converting to a decimal). The nanosecond resolution is just as meaningless in C. -- Regards, Benjamin __

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Mark Shannon
Antoine Pitrou wrote: On Wed, 15 Feb 2012 20:56:26 +0100 "Martin v. Löwis" wrote: With the quartz in Victor's machine, a single clock takes 0.3ns, so three of them make a nanosecond. As the quartz may not be entirely accurate (and also as the CPU frequency may change) you have to measure the cl

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Antoine Pitrou
On Wed, 15 Feb 2012 20:56:26 +0100 "Martin v. Löwis" wrote: > > With the quartz in Victor's machine, a single clock takes 0.3ns, so > three of them make a nanosecond. As the quartz may not be entirely > accurate (and also as the CPU frequency may change) you have to measure > the clock rate again

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Martin v. Löwis
Am 15.02.2012 19:10, schrieb Antoine Pitrou: > > Le mercredi 15 février 2012 à 18:58 +0100, Victor Stinner a écrit : >> It gives me differences smaller than 1000 ns on Ubuntu 11.10 and a >> Intel Core i5 @ 3.33GHz: >> >> $ ./a.out >> 0 s, 781 ns >> $ ./a.out >> 0 s, 785 ns >> $ ./a.out >> 0 s, 798

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Martin v. Löwis
> *Apart* from the specific use case of making an exact copy of a > directory tree that can be verified by other tools that simply compare > the nanosecond times for equality, I don't see any reason for > complicating so many APIs to preserve the fake precision. As far as > simply comparing whether

Re: [Python-Dev] A new dictionary implementation

2012-02-15 Thread Antoine Pitrou
On Mon, 13 Feb 2012 12:31:38 + Mark Shannon wrote: > Note that the json benchmark is unstable and should be ignored. Can you elaborate? If it's unstable it should be fixed, not ignored :) Also, there are two different mako results in your message, which one is the right one? Thanks Antoine

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Antoine Pitrou
Le mercredi 15 février 2012 à 18:58 +0100, Victor Stinner a écrit : > It gives me differences smaller than 1000 ns on Ubuntu 11.10 and a > Intel Core i5 @ 3.33GHz: > > $ ./a.out > 0 s, 781 ns > $ ./a.out > 0 s, 785 ns > $ ./a.out > 0 s, 798 ns > $ ./a.out > 0 s, 818 ns > $ ./a.out > 0 s, 270 ns

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Guido van Rossum
So using floats we can match 100ns precision, right? On Wed, Feb 15, 2012 at 9:58 AM, Victor Stinner wrote: >>> Linux supports nanosecond timestamps since Linux 2.6, Windows supports >>> 100 ns resolution since Windows 2000 or maybe before. It doesn't mean >>> that Windows system clock is accurat

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Victor Stinner
>> Linux supports nanosecond timestamps since Linux 2.6, Windows supports >> 100 ns resolution since Windows 2000 or maybe before. It doesn't mean >> that Windows system clock is accurate: in practical, it's hard to get >> something better than 1 ms :-) > > Well, do you think the Linux system clock

Re: [Python-Dev] A new dictionary implementation

2012-02-15 Thread Mark Shannon
Any opinions on my new dictionary implementation? I'm happy to take silence on the PEP as tacit approval, but the code definitely needs reviewing. Issue: http://bugs.python.org/issue13903 PEP: https://bitbucket.org/markshannon/cpython_new_dict/src/6c4d5d9dfc6d/pep-new-dict.txt Repository https

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Guido van Rossum
On Wed, Feb 15, 2012 at 9:23 AM, Victor Stinner wrote: > 2012/2/15 Guido van Rossum : >> I just came to this thread. Having read the good arguments on both >> sides, I keep wondering why anybody would care about nanosecond >> precision in timestamps. > > Python 3.3 exposes C functions that return

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Antoine Pitrou
On Wed, 15 Feb 2012 18:23:55 +0100 Victor Stinner wrote: > > Linux supports nanosecond timestamps since Linux 2.6, Windows supports > 100 ns resolution since Windows 2000 or maybe before. It doesn't mean > that Windows system clock is accurate: in practical, it's hard to get > something better th

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-15 Thread Barry Warsaw
On Feb 15, 2012, at 09:20 AM, Guido van Rossum wrote: >Does this need a pronouncement? Worrying about the speed of symlinks >seems silly, and exactly how the links are created (hard or soft, >chaining or direct) should be up to the distro; our own Makefile >should create chaining symlinks just so

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Victor Stinner
2012/2/15 Guido van Rossum : > I just came to this thread. Having read the good arguments on both > sides, I keep wondering why anybody would care about nanosecond > precision in timestamps. Python 3.3 exposes C functions that return timespec structure. This structure contains a timestamp with a r

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-15 Thread Guido van Rossum
Does this need a pronouncement? Worrying about the speed of symlinks seems silly, and exactly how the links are created (hard or soft, chaining or direct) should be up to the distro; our own Makefile should create chaining symlinks just so the mechanism is clear. -- --Guido van Rossum (python.org

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Guido van Rossum
On Wed, Feb 15, 2012 at 8:47 AM, Antoine Pitrou wrote: > On Wed, 15 Feb 2012 08:39:45 -0800 > Guido van Rossum wrote: >> >> What purpose is there to recording timestamps in nanoseconds? For >> clocks that start when the process starts running, float *is* >> (basically) good enough. For measuring

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Antoine Pitrou
On Wed, 15 Feb 2012 08:39:45 -0800 Guido van Rossum wrote: > > What purpose is there to recording timestamps in nanoseconds? For > clocks that start when the process starts running, float *is* > (basically) good enough. For measuring e.g. file access times, there > is no way that the actual time

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Guido van Rossum
I just came to this thread. Having read the good arguments on both sides, I keep wondering why anybody would care about nanosecond precision in timestamps. Unless you're in charge of managing one of the few atomic reference clocks in the world, your clock is not going to tell time that accurate. (H

[Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Jim J. Jewett
PEP author Victor asked (in http://mail.python.org/pipermail/python-dev/2012-February/116499.html): > Maybe I missed the answer, but how do you handle timestamp with an > unspecified starting point like os.times() or time.clock()? Should we > leave these function unchanged? If *all* you know is

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Barry Warsaw
On Feb 15, 2012, at 10:11 AM, Martin v. Löwis wrote: >I think improving datetime needs to go in two directions: >a) arbitrary-precision second fractions. My motivation for > proposing/supporting Decimal was that it can support arbitrary > precision, unlike any of the alternatives (except for u

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Barry Warsaw
On Feb 15, 2012, at 10:23 AM, Nick Coghlan wrote: >What should timedelta.total_seconds() return to avoid losing nanosecond >precision? How should this be requested when calling the API? See, I have no problem having this method return a Decimal for high precision values. This preserves the valu

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Victor Stinner
> I'd like to remind people what the original point of the PEP process > was: to avoid going in cycles in discussions. To achieve this, the PEP > author is supposed to record all objections in the PEP, even if he > disagrees (and may state rebuttals for each objection that people > brought up). > >

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Victor Stinner
2012/2/15 "Martin v. Löwis" : > I agree with Barry here (despite having voiced support for using Decimal > before): datetime.datetime *is* the right data type to represent time > stamps. If it means that it needs to be improved before it can be used > in practice, then so be it - improve it. Decim

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Victor Stinner
> I agree with Barry here (despite having voiced support for using Decimal > before): datetime.datetime *is* the right data type to represent time > stamps. If it means that it needs to be improved before it can be used > in practice, then so be it - improve it. Maybe I missed the answer, but how

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Nick Coghlan
On Wed, Feb 15, 2012 at 7:11 PM, "Martin v. Löwis" wrote: > I agree with Barry here (despite having voiced support for using Decimal > before): datetime.datetime *is* the right data type to represent time > stamps. If it means that it needs to be improved before it can be used > in practice, then

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Dirkjan Ochtman
On Wed, Feb 15, 2012 at 10:11, "Martin v. Löwis" wrote: >> My primary concern with the PEP is adding to users confusion when they have >> to >> handle (at least) 5 different types[*] that represent time in Python. > > I agree with Barry here (despite having voiced support for using Decimal > befo

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-15 Thread Martin v. Löwis
Am 14.02.2012 23:29, schrieb Barry Warsaw: > I think I will just state my reasoning one last time and then leave it to the > BDFL or BDFOP to make the final decision. I'd like to remind people what the original point of the PEP process was: to avoid going in cycles in discussions. To achieve this,