Re: [Python-Dev] PEP 435 doesn't help with bitfields [Was: Re: PEP 435 - ref impl disc 2]

2013-05-14 Thread Ethan Furman
On 05/13/2013 10:01 PM, Glenn Linderman wrote: Sorry if this sounds repetitious, but all the other times I've mentioned it, it has been in a big discussion of other stuff too. It's a while 'til 3.4. A bitfield-type enum may show up in the docs, if no where else. ;) -- ~Ethan~

Re: [Python-Dev] PEP 435 - ref impl disc 2

2013-05-14 Thread Ethan Furman
On 05/13/2013 10:01 PM, Glenn Linderman wrote: On 5/13/2013 7:36 PM, Ethan Furman wrote: On 05/10/2013 10:15 PM, Glenn Linderman wrote: So it is quite possible to marry the two, as Ethan helped me figure out using an earlier NamedInt class: class NIE( IntET, Enum ): x = ('NIE.x', 1)

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Gregory P. Smith
On Sun, May 12, 2013 at 3:04 PM, Ben Hoyt benh...@gmail.com wrote: And if we're creating a custom object instead, why return a 2-tuple rather than making the entry's name an attribute of the custom object? To me, that suggests a more reasonable API for os.scandir() might be for it to be

Re: [Python-Dev] Best practices for Enum

2013-05-14 Thread Gregory P. Smith
On Sun, May 12, 2013 at 4:49 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: * will enums break doctests or any existing user code Those are already broken by design. We shouldn't be limited just because someone wrote a bad test that assumed a particular repr of a value. We've

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Antoine Pitrou
Le Tue, 14 May 2013 10:41:01 +1200, Ben Hoyt benh...@gmail.com a écrit : I'd to see the numbers for NFS or CIFS - stat() can be brutally slow over a network connection (that's why we added a caching mechanism to importlib). How do I know what file system Windows networking is using? In

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Antoine Pitrou
Le Tue, 14 May 2013 10:41:01 +1200, Ben Hoyt benh...@gmail.com a écrit : If anyone can run benchmark.py on Linux / NFS or similar, that'd be great. You'll probably have to lower DEPTH/NUM_DIRS/NUM_FILES first and then move the benchtree to the network file system to run it against that. On

Re: [Python-Dev] PEP 379 Python launcher for Windows - behaviour for #!/usr/bin/env python line is wrong

2013-05-14 Thread Paul Moore
On 5 May 2013 18:10, Paul Moore p.f.mo...@gmail.com wrote: On 4 May 2013 16:42, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: I've taken a quick look at it, but I probably won't be able to make any changes until the near the end of the coming week. Feel free to have a go; OK, I have a patch

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Ben Hoyt
If anyone can run benchmark.py on Linux / NFS or similar, that'd be great. You'll probably have to lower DEPTH/NUM_DIRS/NUM_FILES first and then move the benchtree to the network file system to run it against that. Why does your benchmark create such large files? It doesn't make sense.

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Antoine Pitrou
Le Tue, 14 May 2013 20:54:50 +1200, Ben Hoyt benh...@gmail.com a écrit : If anyone can run benchmark.py on Linux / NFS or similar, that'd be great. You'll probably have to lower DEPTH/NUM_DIRS/NUM_FILES first and then move the benchtree to the network file system to run it against that.

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Ben Hoyt
large to be more real world. I've just tested it, and in practice file system doesn't make much difference, so I've fixed that now: Thanks. I had bumped the number of files, thinking it would make things more interesting, and it filled my disk. Denial of Pitrou attack -- sorry! :-) Anyway,

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Ben Hoyt
On a locally running VM: os.walk took 0.400s, scandir.walk took 0.120s -- 3.3x as fast Same VM accessed from the host through a local sshfs: os.walk took 2.261s, scandir.walk took 2.055s -- 1.1x as fast Same, but with sshfs -o cache=no: os.walk took 24.060s, scandir.walk took 25.906s --

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Antoine Pitrou
Le Tue, 14 May 2013 21:10:08 +1200, Ben Hoyt benh...@gmail.com a écrit : On a locally running VM: os.walk took 0.400s, scandir.walk took 0.120s -- 3.3x as fast Same VM accessed from the host through a local sshfs: os.walk took 2.261s, scandir.walk took 2.055s -- 1.1x as fast Same,

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Ben Hoyt
It should be no slower when it's all moved to C. The slowdown is too small to be interesting. The main point is that there was no speedup, though. True, and thanks for testing. I don't think that's a big issue, however. If it's 3-8x faster in the majority of cases (local disk on all systems,

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Antoine Pitrou
Le Tue, 14 May 2013 22:14:42 +1200, Ben Hoyt benh...@gmail.com a écrit : It should be no slower when it's all moved to C. The slowdown is too small to be interesting. The main point is that there was no speedup, though. True, and thanks for testing. I don't think that's a big issue,

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Charles-François Natali
I wonder how sshfs compared to nfs. (I've modified your benchmark to also test the case where data isn't in the page cache). Local ext3: cached: os.walk took 0.096s, scandir.walk took 0.030s -- 3.2x as fast uncached: os.walk took 0.320s, scandir.walk took 0.130s -- 2.5x as fast NFSv3, 1Gb/s

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Matthieu Brucher
Very interesting. Although os.walk may not be widely used in cluster applications, anything that lowers the number of calls to stat() in an spplication is worthwhile for parallel filesystems as stat() is handled by the only non-parallel node, the MDS. Small test on another NFS drive: Creating

Re: [Python-Dev] PEP 379 Python launcher for Windows - behaviour for #!/usr/bin/env python line is wrong

2013-05-14 Thread Vinay Sajip
From: Paul Moore p.f.mo...@gmail.com Did you get a chance to have a look at this? I didn't manage to create a pull request against your copy of pylauncher as my repo is a fork of the pypa one - I'm not sure if that's a limitation of bitbucket or if I just don't know how to do it... I've

Re: [Python-Dev] Best practices for Enum

2013-05-14 Thread Steven D'Aprano
On 14/05/13 16:51, Gregory P. Smith wrote: [...] This sounds like a feature request for doctest. doctest could be educated about enums and automatically compare to the integer value for such cases. Please no. Enums are not special enough to break the rules. Good: Doctests look at the

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Dirkjan Ochtman
On Tue, May 14, 2013 at 12:14 PM, Ben Hoyt benh...@gmail.com wrote: I don't think that's a big issue, however. If it's 3-8x faster in the majority of cases (local disk on all systems, Windows networking), and no slower in a minority (sshfs), I'm not too sad about that. Might be interesting to

[Python-Dev] How to debug python crashes

2013-05-14 Thread Philippe Fremy
Hi, I have a reproducable crash on Windows XP with Python 2.7 which I would like to investigate. I have Visual Studio 2008 installed and I downloaded the pdb files. However I could not find any instructions on how to use them and was unsuccessful at getting anything out of it. I checked the

Re: [Python-Dev] How to debug python crashes

2013-05-14 Thread Antoine Pitrou
Le Tue, 14 May 2013 14:32:27 +0200, Philippe Fremy p...@freehackers.org a écrit : Hi, I have a reproducable crash on Windows XP with Python 2.7 which I would like to investigate. I have Visual Studio 2008 installed and I downloaded the pdb files. However I could not find any instructions on

Re: [Python-Dev] How to debug python crashes

2013-05-14 Thread Tim Golden
On 14/05/2013 13:32, Philippe Fremy wrote: I have a reproducable crash on Windows XP with Python 2.7 which I would like to investigate. I have Visual Studio 2008 installed and I downloaded the pdb files. However I could not find any instructions on how to use them and was unsuccessful at

Re: [Python-Dev] PEP 435 - ref impl disc 2

2013-05-14 Thread Ethan Furman
On 05/13/2013 11:11 PM, Ethan Furman wrote: On 05/13/2013 10:01 PM, Glenn Linderman wrote: On 5/13/2013 7:36 PM, Ethan Furman wrote: On 05/10/2013 10:15 PM, Glenn Linderman wrote: So it is quite possible to marry the two, as Ethan helped me figure out using an earlier NamedInt class: class

[Python-Dev] First post

2013-05-14 Thread Carlos Nepomuceno
Hi guys! This is my first post on this list. I'd like have your opinion on how to safely implement WSGI on a production server. My benchmarks show no performance differences between our PHP and Python environments. I'm using mod_wsgi v3.4 with Apache 2.4. Is that ok or can it get faster?

Re: [Python-Dev] How to debug python crashes

2013-05-14 Thread Philippe Fremy
On 14/05/2013 14:49, Tim Golden wrote: On 14/05/2013 13:32, Philippe Fremy wrote: I have a reproducable crash on Windows XP with Python 2.7 which I would like to investigate. I have Visual Studio 2008 installed and I downloaded the pdb files. However I could not find any instructions on how

Re: [Python-Dev] First post

2013-05-14 Thread Brian Curtin
On Tue, May 14, 2013 at 10:22 AM, Carlos Nepomuceno carlosnepomuc...@outlook.com wrote: Hi guys! This is my first post on this list. I'd like have your opinion on how to safely implement WSGI on a production server. My benchmarks show no performance differences between our PHP and Python

Re: [Python-Dev] First post

2013-05-14 Thread Ethan Furman
On 05/14/2013 08:22 AM, Carlos Nepomuceno wrote: Hi guys! This is my first post on this list. Hi Carlos! I'd like have your opinion on how to safely implement WSGI on a production server. Unfortunately this list is for the development /of/ Python, no development /with/ Python. Try

Re: [Python-Dev] Best practices for Enum

2013-05-14 Thread Gregory P. Smith
Bad: doctests. On Tue, May 14, 2013 at 5:08 AM, Steven D'Aprano st...@pearwood.infowrote: On 14/05/13 16:51, Gregory P. Smith wrote: [...] This sounds like a feature request for doctest. doctest could be educated about enums and automatically compare to the integer value for such cases.

Re: [Python-Dev] How to debug python crashes

2013-05-14 Thread Victor Stinner
Hi, I don't know if it can help, but if you really don't know where your programcrash/hang occurs, you can use the faulthandler module: https://pypi.python.org/pypi/faulthandler It can be used to display te backtrace of all threads on an event like a signal or a timeout. It works with Python,

Re: [Python-Dev] How to debug python crashes

2013-05-14 Thread Catalin Iacob
Hi Philippe, I don't have access to VS right now but out of my head what you need to do is roughly outlined below. On Tue, May 14, 2013 at 5:29 PM, Philippe Fremy p...@freehackers.org wrote: But what's the reason for releasing them ? If you need to recompile Python to use them, that would be

[Python-Dev] Pickling failure on Enums

2013-05-14 Thread Ethan Furman
On 05/13/2013 11:32 AM, Guido van Rossum wrote: But now you enter a different phase of your project, or one of your collaborators does, or perhaps you've released your code on PyPI and one of your users does. So someone tries to pickle some class instance that happens to contain an

Re: [Python-Dev] Pickling failure on Enums

2013-05-14 Thread Guido van Rossum
On Tue, May 14, 2013 at 1:09 PM, Ethan Furman et...@stoneleaf.us wrote: On 05/13/2013 11:32 AM, Guido van Rossum wrote: But now you enter a different phase of your project, or one of your collaborators does, or perhaps you've released your code on PyPI and one of your users does. So someone

Re: [Python-Dev] Pickling failure on Enums

2013-05-14 Thread Ethan Furman
On 05/14/2013 01:58 PM, Guido van Rossum wrote: On Tue, May 14, 2013 at 1:09 PM, Ethan Furman et...@stoneleaf.us wrote: On 05/13/2013 11:32 AM, Guido van Rossum wrote: But now you enter a different phase of your project, or one of your collaborators does, or perhaps you've released your code

Re: [Python-Dev] Pickling failure on Enums

2013-05-14 Thread Guido van Rossum
On Tue, May 14, 2013 at 2:13 PM, Ethan Furman et...@stoneleaf.us wrote: On 05/14/2013 01:58 PM, Guido van Rossum wrote: On Tue, May 14, 2013 at 1:09 PM, Ethan Furman et...@stoneleaf.us wrote: I can get pickle failure on members created using the functional syntax with no module set; That's

Re: [Python-Dev] Pickling failure on Enums

2013-05-14 Thread Nick Coghlan
On 15 May 2013 07:38, Guido van Rossum gu...@python.org wrote: On Tue, May 14, 2013 at 2:13 PM, Ethan Furman et...@stoneleaf.us wrote: On 05/14/2013 01:58 PM, Guido van Rossum wrote: On Tue, May 14, 2013 at 1:09 PM, Ethan Furman et...@stoneleaf.us wrote: I can get pickle failure on

Re: [Python-Dev] Pickling failure on Enums

2013-05-14 Thread Ethan Furman
On 05/14/2013 03:16 PM, Nick Coghlan wrote: On 15 May 2013 07:38, Guido van Rossum gu...@python.org mailto:gu...@python.org wrote: On Tue, May 14, 2013 at 2:13 PM, Ethan Furman et...@stoneleaf.us mailto:et...@stoneleaf.us wrote: On 05/14/2013 01:58 PM, Guido van Rossum wrote: On Tue, May

Re: [Python-Dev] Pickling failure on Enums

2013-05-14 Thread Ethan Furman
On 05/14/2013 02:35 PM, Guido van Rossum wrote: For example you could file low-priority bugs for both issues in the hope that someone else figures it out. Got it figured out. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org