Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Martin v. Löwis
Am 15.02.2012 21:06, schrieb Antoine Pitrou: On Wed, 15 Feb 2012 20:56:26 +0100 Martin v. Löwis mar...@v.loewis.de wrote: With the quartz in Victor's machine, a single clock takes 0.3ns, so three of them make a nanosecond. As the quartz may not be entirely accurate (and also as the CPU

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Martin v. Löwis
Maybe an alternative PEP could be written that supports the filesystem copying use case only, using some specialized ns APIs? I really think that all you need is st_{a,c,m}time_ns fields and os.utime_ns(). I'm -1 on that, because it will make people write complicated code. Regards, Martin

Re: [Python-Dev] best place for an atomic file API

2012-02-16 Thread Martin v. Löwis
(MvL complained in the tracker issue about a lack of concrete use cases, but I think fixing race conditions when overwriting bytecode files in importlib and the existing distutils/packaging use cases cover that) I certainly agree that there are applications of atomic replace, and that the os

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-16 Thread Martin v. Löwis
So, getting back to the topic again, is there any reason why you would oppose backing the ElementTree module in the stdlib by cElementTree's accelerator module? Or can we just consider this part of the discussion settled and start getting work done? I'd still like to know who is in charge of

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-16 Thread Martin v. Löwis
Does this imply that each and every package in the stdlib currently has a dedicated maintainer who promised to be dedicated to it? Or otherwise, should those packages that *don't* have a maintainer be removed from the standard library? That is my opinion, yes. Some people (including myself)

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-16 Thread Eli Bendersky
I'd still like to know who is in charge of the etree package now. I know that I'm not, so I just don't have any opinion on the technical question of using the accelerator module (it sounds like a reasonable idea, but it also sounds like something that may break existing code). If the

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-16 Thread Ned Deily
In article cadisq7fg3vgxd39teuvbcvhhmpkuwss0qcksrfpkn5ye0dv...@mail.gmail.com, Nick Coghlan ncogh...@gmail.com wrote: On Thu, Feb 16, 2012 at 12:06 PM, Guido van Rossum gu...@python.org wrote: Anyway, I don't think anyone is objecting against the PEP allowing symlinks now. Yeah, the

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Victor Stinner
2012/2/16 Martin v. Löwis mar...@v.loewis.de: Maybe an alternative PEP could be written that supports the filesystem copying use case only, using some specialized ns APIs? I really think that all you need is st_{a,c,m}time_ns fields and os.utime_ns(). I'm -1 on that, because it will make

Re: [Python-Dev] best place for an atomic file API

2012-02-16 Thread Victor Stinner
Most users don't need a truly ACID write, but implement their own best-effort function. Instead of having a different implement in each project, Python can provide something better, especially when the OS provides low level function to implement such feature. Victor 2012/2/16 Martin v. Löwis

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-16 Thread Martin v. Löwis
There are two issues that I know of for OS X. One is just getting a python2 symlink into the bin directory of a framework build. That's easy. Where exactly in the Makefile is that reflected? ISTM that the current patch already covers that, since the framwork* targets are not concerned with

Re: [Python-Dev] best place for an atomic file API

2012-02-16 Thread Martin v. Löwis
Am 16.02.2012 10:54, schrieb Victor Stinner: Most users don't need a truly ACID write, but implement their own best-effort function. Instead of having a different implement in each project, Python can provide something better, especially when the OS provides low level function to implement

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Martin v. Löwis
Am 16.02.2012 10:51, schrieb Victor Stinner: 2012/2/16 Martin v. Löwis mar...@v.loewis.de: Maybe an alternative PEP could be written that supports the filesystem copying use case only, using some specialized ns APIs? I really think that all you need is st_{a,c,m}time_ns fields and

Re: [Python-Dev] best place for an atomic file API

2012-02-16 Thread Vinay Sajip
Martin v. Löwis martin at v.loewis.de writes: One way of providing this might be a u mode for open, which updates an existing file on close (unlike a, which appends, and unlike w, which truncates first). Doesn't r+ cover this? Regards, Vinay Sajip

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-16 Thread Nick Coghlan
On Thu, Feb 16, 2012 at 8:01 PM, Martin v. Löwis mar...@v.loewis.de wrote: It may be that the PEP becomes irrelevant before it is widely accepted: if the sole remaining Python 2 version is 2.7, users may just as well refer to python2 as python2.7. My hope is that a clear signal from us

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-16 Thread Nick Coghlan
On Wed, Feb 15, 2012 at 12:44 AM, Barry Warsaw ba...@python.org wrote: On Feb 14, 2012, at 12:38 PM, Nick Coghlan wrote: I have no idea, and I'm not going to open that can of worms for this PEP. We need to say something about the executable aliases so that people can eventually write

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Victor Stinner
2012/2/15 Guido van Rossum gu...@python.org: So using floats we can match 100ns precision, right? Nope, not to store an Epoch timestamp newer than january 1987: x=2**29; (x+1e-7) != x # no loss of precision True x=2**30; (x+1e-7) != x # lose precision False

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Victor Stinner
A data point on this specific use case.  The following code throws its assert ~90% of the time in Python 3.2.2 on a modern Linux machine (assuming foo exists and bar does not):   import shutil   import os   shutil.copy2(foo, bar)   assert os.stat(foo).st_mtime == os.stat(bar).st_mtime It

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Victor Stinner
PEP author Victor asked (in http://mail.python.org/pipermail/python-dev/2012-February/116499.html): Maybe I missed the answer, but how do you handle timestamp with an unspecified starting point like os.times() or time.clock()? Should we leave these function unchanged? If *all* you know is

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Antoine Pitrou
On Thu, 16 Feb 2012 13:46:18 +0100 Victor Stinner victor.stin...@gmail.com wrote: Let's try in a ext4 filesystem: $ ~/prog/python/timestamp/python Python 3.3.0a0 (default:35d6cc531800+, Feb 16 2012, 13:32:56) import decimal, os, shutil, time open(test, x).close() shutil.copy2(test,

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Victor Stinner
The way Linux does that is to use the time-stamping counter of the processor (the rdtsc instructions), which (originally) counts one unit per CPU clock. I believe current processors use slightly different countings (e.g. through the APIC), but still: you get a resolution within the clock

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Victor Stinner
If I run your snippet and inspect modification times using `stat`, the difference is much smaller (around 10 ns, not 1 ms): $ stat test | \grep Modify Modify: 2012-02-16 13:51:25.643597139 +0100 $ stat test2 | \grep Modify Modify: 2012-02-16 13:51:25.643597126 +0100 The loss of precision

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Antoine Pitrou
Le jeudi 16 février 2012 à 14:20 +0100, Victor Stinner a écrit : If I run your snippet and inspect modification times using `stat`, the difference is much smaller (around 10 ns, not 1 ms): $ stat test | \grep Modify Modify: 2012-02-16 13:51:25.643597139 +0100 $ stat test2 | \grep

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-16 Thread Ned Deily
I'm away from the source for the next 36 hours. I'll reply with patches by Saturday morning. ___ Ned Deily n...@acm.org -- [] . Original Message ... On Thu, 16 Feb 2012 11:01:39 +0100 Martin v. Löwis mar...@v.loewis.de wrote: There are two issues that I know of for OS X. One

Re: [Python-Dev] best place for an atomic file API

2012-02-16 Thread Serhiy Storchaka
15.02.12 23:16, Charles-François Natali написав(ла): Issue #8604 aims at adding an atomic file API to make it easier to create/update files atomically, using rename() on POSIX systems and MoveFileEx() on Windows (which are now available through os.replace()). It would also use fsync() on POSIX

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Larry Hastings
On 02/15/2012 08:12 PM, Guido van Rossum wrote: On Wed, Feb 15, 2012 at 7:28 PM, Larry Hastingsla...@hastings.org wrote: I fixed this in trunk last September (issue 12904); os.utime now preserves all the precision that Python currently conveys. So, essentially you fixed this particular issue

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-16 Thread Barry Warsaw
On Feb 16, 2012, at 09:54 PM, Nick Coghlan wrote: It turns out I'd forgotten what was in the PEP - the Notes section already contained a lot of suggestions along those lines. I changed the title of the section to Migration Notes, but tried to make it clear that those *aren't* consensus

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-16 Thread Ezio Melotti
On 14/02/2012 9.58, Stefan Behnel wrote: Nick Coghlan, 14.02.2012 05:44: On Tue, Feb 14, 2012 at 2:25 PM, Eli Bendersky wrote: With the deprecation warning being silent, is there much to lose, though? Yes, it creates problems for anyone that deliberately converts all warnings to errors when

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-16 Thread Charles-François Natali
I personally don't see any reason to drop a module that isn't terminally broken or unmaintainable, apart from scaring users away by making them think that we don't care about backward compatibility. ___ Python-Dev mailing list Python-Dev@python.org

[Python-Dev] PEP for new dictionary implementation

2012-02-16 Thread Jim J. Jewett
PEP author Mark Shannon wrote (in http://mail.python.org/pipermail/python-dev/attachments/20120208/05be469a/attachment.txt): ... allows ... (the ``__dict__`` attribute of an object) to share keys with other attribute dictionaries of instances of the same class. Is the same class a

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-16 Thread Ezio Melotti
On 16/02/2012 19.55, Antoine Pitrou wrote: On Thu, 16 Feb 2012 19:32:24 +0200 Ezio Melottiezio.melo...@gmail.com wrote: If I'm writing code that imports cElementTree on 3.3+, and I explicitly turn on DeprecationWarnings (that would otherwise be silenced) to check if I'm doing something wrong,

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-16 Thread Tim Delaney
On 17 February 2012 04:55, Antoine Pitrou solip...@pitrou.net wrote: But then you're going from a cumbersome situation (where you have to import cElementTree and then fallback on regular ElementTree) to an even more cumbersome one (where you have to first check the Python version, then

[Python-Dev] Store timestamps as decimal.Decimal objects

2012-02-16 Thread Jim J. Jewett
In http://mail.python.org/pipermail/python-dev/2012-February/116073.html Nick Coghlan wrote: Besides, float128 is a bad example - such a type could just be returned directly where we return float64 now. (The only reason we can't do that with Decimal is because we deliberately don't allow

Re: [Python-Dev] PEP for new dictionary implementation

2012-02-16 Thread Antoine Pitrou
On Wed, 08 Feb 2012 19:18:14 + Mark Shannon m...@hotpy.org wrote: Proposed PEP for new dictionary implementation, PEP 410? is attached. So, I'm running a few benchmarks using Twisted's test suite (see https://bitbucket.org/pitrou/t3k/wiki/Home). At the end of `python -i bin/trial

[Python-Dev] plugging the hash attack

2012-02-16 Thread Jim J. Jewett
In http://mail.python.org/pipermail/python-dev/2012-January/116003.html Benjamin Peterson wrote: 2. It will be off by default in stable releases ... This will prevent code breakage ... 2012/1/27 Steven D'Aprano steve at pearwood.info: ... it will become on by default in some future

Re: [Python-Dev] PEP for new dictionary implementation

2012-02-16 Thread Martin v. Löwis
Am 11.02.2012 22:22, schrieb Mark Shannon: Antoine Pitrou wrote: Hello Mark, I think the PEP should explain what happens when a keys table needs resizing when setting an object's attribute. If the object is the only instance of a class, it remains split, otherwise the table is combined.

Re: [Python-Dev] PEP for new dictionary implementation

2012-02-16 Thread Martin v. Löwis
Am 13.02.2012 13:46, schrieb Mark Shannon: Revised PEP for new dictionary implementation, PEP 412? is attached. Committed as PEP 412. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] PEP for new dictionary implementation

2012-02-16 Thread Martin v. Löwis
Am 16.02.2012 19:24, schrieb Jim J. Jewett: PEP author Mark Shannon wrote (in http://mail.python.org/pipermail/python-dev/attachments/20120208/05be469a/attachment.txt): ... allows ... (the ``__dict__`` attribute of an object) to share keys with other attribute dictionaries of instances

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Victor Stinner
$ stat test | \grep Modify Modify: 2012-02-16 13:51:25.643597139 +0100 $ stat test2 | \grep Modify Modify: 2012-02-16 13:51:25.643597126 +0100 The loss of precision is not constant: it depends on the timestamp value. Well, I've tried several times and I can't reproduce a 1 ms

[Python-Dev] Counting collisions for the win

2012-02-16 Thread Jim J. Jewett
In http://mail.python.org/pipermail/python-dev/2012-January/115715.html Frank Sievertsen wrote: Am 20.01.2012 13:08, schrieb Victor Stinner: I'm surprised we haven't seen bug reports about it from users of 64-bit Pythons long ago A Python dictionary only uses the lower bits of a hash value.

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Guido van Rossum
On Thu, Feb 16, 2012 at 2:04 PM, Victor Stinner victor.stin...@gmail.com wrote: It doesn't change anything to the Makefile issue, if timestamps are different in a single nanosecond, they are seen as different by make (by another program comparing the timestamp of two files using nanosecond

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Victor Stinner
2012/2/16 Guido van Rossum gu...@python.org: On Thu, Feb 16, 2012 at 2:04 PM, Victor Stinner victor.stin...@gmail.com wrote: It doesn't change anything to the Makefile issue, if timestamps are different in a single nanosecond, they are seen as different by make (by another program comparing

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Guido van Rossum
On Thu, Feb 16, 2012 at 2:48 PM, Victor Stinner victor.stin...@gmail.com wrote: 2012/2/16 Guido van Rossum gu...@python.org: On Thu, Feb 16, 2012 at 2:04 PM, Victor Stinner victor.stin...@gmail.com wrote: It doesn't change anything to the Makefile issue, if timestamps are different in a

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Victor Stinner
The problem is that shutil.copy2() produces sometimes *older* timestamp :-/ (...) Have you been able to reproduce this with an actual Makefile? What's the scenario? Hum. I asked the Internet who use shutil.copy2() and I found an old issue (Decimal('43462967.173053') seconds ago): Python

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Guido van Rossum
So, make is unaffected. In my first post on this subject I already noted that the only real use case is making a directory or filesystem copy and then verifying that the copy is identical using native tools that compare times with nsec precision. At least one of the bugs you quote is about the

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Alexander Belopolsky
On Wed, Feb 15, 2012 at 11:39 AM, Guido van Rossum gu...@python.org wrote: Maybe it's okay to wait a few years on this, until either 128-bit floats are more common or cDecimal becomes the default floating point type? +1 ___ Python-Dev mailing list

Re: [Python-Dev] PEP for new dictionary implementation

2012-02-16 Thread Jim Jewett
On Thu, Feb 16, 2012 at 4:34 PM, Martin v. Löwis mar...@v.loewis.de wrote: Am 16.02.2012 19:24, schrieb Jim J. Jewett: PEP author Mark Shannon wrote (in http://mail.python.org/pipermail/python-dev/attachments/20120208/05be469a/attachment.txt): ... allows ... (the ``__dict__`` attribute of

Re: [Python-Dev] [Python-checkins] cpython: Disabling a test that fails on some bots. Will investigate the failure soon

2012-02-16 Thread Nick Coghlan
On Fri, Feb 17, 2012 at 2:09 AM, eli.bendersky python-check...@python.org wrote: diff --git a/Lib/test/test_xml_etree_c.py b/Lib/test/test_xml_etree_c.py --- a/Lib/test/test_xml_etree_c.py +++ b/Lib/test/test_xml_etree_c.py @@ -53,8 +53,8 @@         # actual class. In the Python version it's

Re: [Python-Dev] [Python-checkins] cpython: Disabling a test that fails on some bots. Will investigate the failure soon

2012-02-16 Thread Eli Bendersky
On Fri, Feb 17, 2012 at 05:50, Nick Coghlan ncogh...@gmail.com wrote: On Fri, Feb 17, 2012 at 2:09 AM, eli.bendersky python-check...@python.org wrote: diff --git a/Lib/test/test_xml_etree_c.py b/Lib/test/test_xml_etree_c.py --- a/Lib/test/test_xml_etree_c.py +++

Re: [Python-Dev] PEP for new dictionary implementation

2012-02-16 Thread Martin v. Löwis
Good idea. However, how do you track per-dict how large the table is? Why would you want to? The per-instance array needs to be at least as large as the highest index used by any key for which it has a value; if the keys table gets far larger (or even shrinks), that doesn't really matter

[Python-Dev] PEP 394 accepted

2012-02-16 Thread Martin v. Löwis
As the PEP czar for PEP 394, I have reviewed it and am happy to say that I can accept it. I suppose that Nick will keep track of actually implementing it in Python 2.7. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review

2012-02-16 Thread Georg Brandl
Am 16.02.2012 11:14, schrieb Martin v. Löwis: Am 16.02.2012 10:51, schrieb Victor Stinner: 2012/2/16 Martin v. Löwis mar...@v.loewis.de: Maybe an alternative PEP could be written that supports the filesystem copying use case only, using some specialized ns APIs? I really think that all you