[Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Nick Coghlan 
I'd like to mark a few PEPs that are not currently being actively
considered for 3.4 as Deferred:

 S   286  Enhanced Argument Tuplesvon Löwis
 S   337  Logging Usage in the Standard Library   Dubner
 S   368  Standard image protocol and class   Mastrodomenico
 I   396  Module Version Numbers  Warsaw
 S   400  Deprecate codecs.StreamReader and codecs.StreamWriter   Stinner
 S   419  Protecting cleanup statements from interruptionsColomiets
 I   423  Naming conventions and recipes related to packaging Bryon
 I   444  Python Web3 Interface
McDonough, Ronacher
 S  3124  Overloading, Generic Functions, Interfaces, and ... Eby
 S  3142  Add a "while" clause to generator expressions   Britton
 S  3143  Standard daemon process library Finney
 S  3145  Asynchronous I/O For subprocess.Popen
Pruitt, McCreary, Carlson
 S  3152  Cofunctions Ewing

Obviously, they can be reactivated at any time, but I think it would
be beneficial to have the "Open" list more accurately reflect
proposals that are currently being championed.

I'd also like to mark this one as rejected by Guido at PyCon US 2013
in favour of an updated PEP 436:
 S   437  A DSL for specifying signatures, annotations and ...Krah

Cheers,
Nick.

--
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Antoine Pitrou 

Hello Nick,

Le Fri, 10 May 2013 17:14:21 +1000,
Nick Coghlan  a écrit :
> I'd like to mark a few PEPs that are not currently being actively
> considered for 3.4 as Deferred:
> 
>  S   286  Enhanced Argument Tuplesvon
> Löwis S   337  Logging Usage in the Standard
> Library   Dubner S   368  Standard image protocol and
> class   Mastrodomenico I   396  Module Version
> Numbers  Warsaw S   400  Deprecate
> codecs.StreamReader and codecs.StreamWriter   Stinner S   419
> Protecting cleanup statements from interruptionsColomiets I
> 423  Naming conventions and recipes related to packaging Bryon
> I   444  Python Web3 Interface McDonough, Ronacher
>  S  3124  Overloading, Generic Functions, Interfaces, and ... Eby
>  S  3142  Add a "while" clause to generator expressions
> Britton S  3143  Standard daemon process
> library Finney S  3145  Asynchronous I/O For
> subprocess.Popen Pruitt, McCreary, Carlson
>  S  3152  Cofunctions
> Ewing
> 
> Obviously, they can be reactivated at any time, but I think it would
> be beneficial to have the "Open" list more accurately reflect
> proposals that are currently being championed.

Sounds fine to me.

> I'd also like to mark this one as rejected by Guido at PyCon US 2013
> in favour of an updated PEP 436:
>  S   437  A DSL for specifying signatures, annotations and ...Krah

I haven't followed enough to have an opinion :)

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Ben Hoyt 
A few of us were having a discussion at
http://bugs.python.org/issue11406 about adding os.scandir(): a
generator version of os.listdir() to make iterating over very large
directories more memory efficient. This also reflects how the OS gives
things to you -- it doesn't give you a big list, but you call a
function to iterate and fetch the next entry.

While I think that's a good idea, I'm not sure just that much is
enough of an improvement to make adding the generator version worth
it.

But what would make this a killer feature is making os.scandir()
generate tuples of (name, stat_like_info). The Windows directory
iteration functions (FindFirstFile/FindNextFile) give you the full
stat information for free, and the Linux and OS X functions
(opendir/readdir) give you partial file information (d_type in the
dirent struct, which is basically the st_mode part of a stat, whether
it's a file, directory, link, etc).

Having this available at the Python level would mean we can vastly
speed up functions like os.walk() that otherwise need to make an
os.stat() call for every file returned. In my benchmarks of such a
generator on Windows, it speeds up os.walk() by 9-10x. On Linux/OS X,
it's more like 1.5-3x. In my opinion, that kind of gain is huge,
especially on Windows, but also on Linux/OS X.

So the idea is to add this relatively low-level function that exposes
the extra information the OS gives us for free, but which os.listdir()
currently throws away. Then higher-level, platform-independent
functions like os.walk() could use os.scandir() to get much better
performance. People over at Issue 11406 think this is a good idea.

HOWEVER, there's debate over what kind of object the second element in
the tuple, "stat_like_info", should be. My strong vote is for it to be
a stat_result-like object, but where the fields are None if they're
unknown. There would be basically three scenarios:

1) stat_result with all fields set: this would happen on Windows,
where you get as much info from FindFirst/FindNext as from an
os.stat()
2) stat_result with just st_mode set, and all other fields None: this
would be the usual case on Linux/OS X
3) stat_result with all fields None: this would happen on systems
whose readdir()/dirent doesn't have d_type, or on Linux/OS X when
d_type was DT_UNKNOWN

Higher-level functions like os.walk() would then check the fields they
needed are not None, and only call os.stat() if needed, for example:

# Build lists of files and directories in path
files = []
dirs = []
for name, st in os.scandir(path):
if st.st_mode is None:
st = os.stat(os.path.join(path, name))
if stat.S_ISDIR(st.st_mode):
dirs.append(name)
else:
files.append(name)

Not bad for a 2-10x performance boost, right? What do folks think?

Cheers,
Ben.


P.S. A few non-essential further notes:

1) As a Windows guy, a nice-to-have addition to os.scandir() would be
a keyword arg like win_wildcard which defaulted to '*.*', but power
users can pass in to utilize the wildcard feature of
FindFirst/FindNext on Windows. We have plenty of other low-level
functions that expose OS-specific features in the OS module, so this
would be no different. But then again, it's not nearly as important as
exposing the stat info.

2) I've been dabbling with this concept for a while in my BetterWalk
library: https://github.com/benhoyt/betterwalk

Note that the benchmarks there are old, and I've made further
improvements in my local copy. The ctypes version gives speed gains
for os.walk() of 2-3x on Windows, but I've also got a C version, which
is giving 9-10x speed gains. I haven't yet got a Linux/OS X version
written in C.

3) See also the previous python-dev thread on BetterWalk:
http://mail.python.org/pipermail/python-ideas/2012-November/017944.html
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Christian Heimes 
Am 10.05.2013 12:55, schrieb Ben Hoyt:
> Higher-level functions like os.walk() would then check the fields they
> needed are not None, and only call os.stat() if needed, for example:
> 
> # Build lists of files and directories in path
> files = []
> dirs = []
> for name, st in os.scandir(path):
> if st.st_mode is None:
> st = os.stat(os.path.join(path, name))
> if stat.S_ISDIR(st.st_mode):
> dirs.append(name)
> else:
> files.append(name)

Have you actually tried the code? It can't give you correct answers. The
struct dirent.d_type member as returned by readdir() has different
values than stat.st_mode's file type.

For example on my system readdir() returns DT_DIR for a directory but
S_ISDIR() checks different bits:

DT_DIR = 4

S_ISDIR(mode) ((mode) & 017) == 004

Or are you proposing to map d_type to st_mode? That's also problematic
because st_mode would only have file type bits, not permission bits.
Also POSIX standards state that new file types will not get additional
S_IF* constant assigned to. Some operation systems have IFTODT() /
DTTOIF() macros which convert bits between st_mode and d_type but the
macros aren't part of POSIX standard.

Hence I'm +1 on the general idea but -1 on something stat like. IMHO
os.scandir() should yield four objects:

 * name
 * inode
 * file type or DT_UNKNOWN
 * stat_result or None

stat_result shall only be returned when the operating systems provides a
full stat result as returned by os.stat().

Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Antoine Pitrou 
Le Fri, 10 May 2013 13:46:30 +0200,
Christian Heimes  a écrit :
> 
> Hence I'm +1 on the general idea but -1 on something stat like. IMHO
> os.scandir() should yield four objects:
> 
>  * name
>  * inode
>  * file type or DT_UNKNOWN
>  * stat_result or None
> 
> stat_result shall only be returned when the operating systems
> provides a full stat result as returned by os.stat().

But what if some systems return more than the file type and less than a
full stat result? The general problem is POSIX's terrible inertia.
I feel that a stat result with some None fields would be an acceptable
compromise here.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 4XX: pyzaa "Improving Python ZIP Application Support"

2013-05-10 Thread Daniel Holth 
Everyone seems to like the first half of this simple PEP adding the
extensions. The 3-letter extension for windowed apps can be "pzw"
while the "pyz" extension for console apps stays the same.

The second half, the tool
https://bitbucket.org/dholth/pyzaa/src/tip/pyzaa.py?at=default is less
mature, but there's not a whole lot to do in a simple tool that may
serve more as an example: you can open any file with ZipFile in append
mode, even one that is not a zip file and just contains the #!python
shebang line.

Thanks,

Daniel
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Ronald Oussoren 

On 10 May, 2013, at 14:16, Antoine Pitrou  wrote:

> Le Fri, 10 May 2013 13:46:30 +0200,
> Christian Heimes  a écrit :
>> 
>> Hence I'm +1 on the general idea but -1 on something stat like. IMHO
>> os.scandir() should yield four objects:
>> 
>> * name
>> * inode
>> * file type or DT_UNKNOWN
>> * stat_result or None
>> 
>> stat_result shall only be returned when the operating systems
>> provides a full stat result as returned by os.stat().
> 
> But what if some systems return more than the file type and less than a
> full stat result? The general problem is POSIX's terrible inertia.
> I feel that a stat result with some None fields would be an acceptable
> compromise here.

But how do you detect that the st_mode field on systems with a d_type is 
incomplete, as oposed to a system that can return a full st_mode from its 
readdir equivalent and where the permission bits happen to be 0o? One 
option would be to add a file type field to stat_result, IIRC this was 
mentioned in some revisions of the extended stat_result proposal over on 
python-ideas.

Ronald

> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Christian Heimes 
Am 10.05.2013 14:16, schrieb Antoine Pitrou:
> But what if some systems return more than the file type and less than a
> full stat result? The general problem is POSIX's terrible inertia.
> I feel that a stat result with some None fields would be an acceptable
> compromise here.

POSIX only defines the d_ino and d_name members of struct dirent. Linux,
BSD and probably some other platforms also happen to provide d_type. The
other members of struct dirent (d_reclen, d_namlen) aren't useful in
Python space by themselves.

d_type and st_mode aren't compatible in any way. As you know st_mode
also contains POSIX permission information. The file type is encoded
with a different set of bits, too. Future file types aren't mapped to
S_IF* constants for st_mode.

For d_ino you also need the device number from the directory because the
inode is only unique within a device.

I don't really see how to map strut dirent to struct stat on POSIX.

Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Antoine Pitrou 
Le Fri, 10 May 2013 15:46:21 +0200,
Christian Heimes  a écrit :

> Am 10.05.2013 14:16, schrieb Antoine Pitrou:
> > But what if some systems return more than the file type and less
> > than a full stat result? The general problem is POSIX's terrible
> > inertia. I feel that a stat result with some None fields would be
> > an acceptable compromise here.
> 
> POSIX only defines the d_ino and d_name members of struct dirent.
> Linux, BSD and probably some other platforms also happen to provide
> d_type. The other members of struct dirent (d_reclen, d_namlen)
> aren't useful in Python space by themselves.
> 
> d_type and st_mode aren't compatible in any way. As you know st_mode
> also contains POSIX permission information. The file type is encoded
> with a different set of bits, too. Future file types aren't mapped to
> S_IF* constants for st_mode.

Thank you and Ronald for clarifying. This does make the API design a
bit bothersome. We want to expose as much information as possible in a
cross-platform way and with a flexible granularity, but doing so might
require a gazillion of namedtuple fields (platonically, as much as one
field per stat bit).

> For d_ino you also need the device number from the directory because
> the inode is only unique within a device.

But hopefully you've already stat'ed the directory ;)

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Barry Warsaw 
On May 10, 2013, at 05:14 PM, Nick Coghlan wrote:

> I   396  Module Version Numbers  Warsaw

I do want to eventually return to this PEP, but I probably won't any time
soon.

-Barry
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Nick Coghlan 
On Fri, May 10, 2013 at 11:46 PM, Christian Heimes  wrote:
> Am 10.05.2013 14:16, schrieb Antoine Pitrou:
>> But what if some systems return more than the file type and less than a
>> full stat result? The general problem is POSIX's terrible inertia.
>> I feel that a stat result with some None fields would be an acceptable
>> compromise here.
>
> POSIX only defines the d_ino and d_name members of struct dirent. Linux,
> BSD and probably some other platforms also happen to provide d_type. The
> other members of struct dirent (d_reclen, d_namlen) aren't useful in
> Python space by themselves.
>
> d_type and st_mode aren't compatible in any way. As you know st_mode
> also contains POSIX permission information. The file type is encoded
> with a different set of bits, too. Future file types aren't mapped to
> S_IF* constants for st_mode.

Why are we exposing a bitfield as the primary Python level API,
anyway? It makes sense for the well defined permission bits, but why
are we copying the C level concept for the other flags?

Cheers,
Nick.

--
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Nick Coghlan 
On Fri, May 10, 2013 at 11:57 PM, Barry Warsaw  wrote:
> On May 10, 2013, at 05:14 PM, Nick Coghlan wrote:
>
>> I   396  Module Version Numbers  Warsaw
>
> I do want to eventually return to this PEP, but I probably won't any time
> soon.

Yeah, I have a couple of PEPs like that - they pretty much live in
Deferred and I update them when inspiration strikes :)

PEP-a-holic'ly,
Nick.

--
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Ronald Oussoren 

On 10 May, 2013, at 15:54, Antoine Pitrou  wrote:

> Le Fri, 10 May 2013 15:46:21 +0200,
> Christian Heimes  a écrit :
> 
>> Am 10.05.2013 14:16, schrieb Antoine Pitrou:
>>> But what if some systems return more than the file type and less
>>> than a full stat result? The general problem is POSIX's terrible
>>> inertia. I feel that a stat result with some None fields would be
>>> an acceptable compromise here.
>> 
>> POSIX only defines the d_ino and d_name members of struct dirent.
>> Linux, BSD and probably some other platforms also happen to provide
>> d_type. The other members of struct dirent (d_reclen, d_namlen)
>> aren't useful in Python space by themselves.
>> 
>> d_type and st_mode aren't compatible in any way. As you know st_mode
>> also contains POSIX permission information. The file type is encoded
>> with a different set of bits, too. Future file types aren't mapped to
>> S_IF* constants for st_mode.
> 
> Thank you and Ronald for clarifying. This does make the API design a
> bit bothersome. We want to expose as much information as possible in a
> cross-platform way and with a flexible granularity, but doing so might
> require a gazillion of namedtuple fields (platonically, as much as one
> field per stat bit).

One field per stat bit is overkill, file permissions are well known enough
to keep them as a single item. 

Most if not all uses of the st_mode field can be covered by adding just
"filetype" and "permissions" fields. That would also make it possible to
use stat_result in os.scandir() without loosing information (it would 
have filetype != None and permissions and st_mode == None on systems with
d_type).

> 
>> For d_ino you also need the device number from the directory because
>> the inode is only unique within a device.
> 
> But hopefully you've already stat'ed the directory ;)

Why? There's no need to stat the directory when implementing os.walk using
os.scandir (for systems that return filetype information in the API used
by os.scandir).  Anyway, setting st_ino in the result of os.scandir is
harmless, even though using st_ino is uncommon.

Getting st_dev from the directory isn't good anyway, for example when
using rebind mounts to mount a single file into a different directory (which 
is a convenient way to make a configuration file available in a chroot 
environment)

Ronald

> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 368

2013-05-10 Thread R. David Murray 
On Fri, 10 May 2013 17:14:21 +1000, Nick Coghlan  wrote:
>  S   368  Standard image protocol and class   
> Mastrodomenico

I haven't read through it in detail yet, but this PEP looks interesting
in the context of the further enhancements planned for the email module
(ie: a MIME image object returned by the email parser is a candidate to
provide the PEP 368 interface).

Does anyone know if there is any associated code?

--David
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread MRAB 

On 10/05/2013 11:55, Ben Hoyt wrote:

A few of us were having a discussion at
http://bugs.python.org/issue11406 about adding os.scandir(): a
generator version of os.listdir() to make iterating over very large
directories more memory efficient. This also reflects how the OS gives
things to you -- it doesn't give you a big list, but you call a
function to iterate and fetch the next entry.

While I think that's a good idea, I'm not sure just that much is
enough of an improvement to make adding the generator version worth
it.

But what would make this a killer feature is making os.scandir()
generate tuples of (name, stat_like_info). The Windows directory
iteration functions (FindFirstFile/FindNextFile) give you the full
stat information for free, and the Linux and OS X functions
(opendir/readdir) give you partial file information (d_type in the
dirent struct, which is basically the st_mode part of a stat, whether
it's a file, directory, link, etc).

Having this available at the Python level would mean we can vastly
speed up functions like os.walk() that otherwise need to make an
os.stat() call for every file returned. In my benchmarks of such a
generator on Windows, it speeds up os.walk() by 9-10x. On Linux/OS X,
it's more like 1.5-3x. In my opinion, that kind of gain is huge,
especially on Windows, but also on Linux/OS X.

So the idea is to add this relatively low-level function that exposes
the extra information the OS gives us for free, but which os.listdir()
currently throws away. Then higher-level, platform-independent
functions like os.walk() could use os.scandir() to get much better
performance. People over at Issue 11406 think this is a good idea.

HOWEVER, there's debate over what kind of object the second element in
the tuple, "stat_like_info", should be. My strong vote is for it to be
a stat_result-like object, but where the fields are None if they're
unknown. There would be basically three scenarios:

1) stat_result with all fields set: this would happen on Windows,
where you get as much info from FindFirst/FindNext as from an
os.stat()
2) stat_result with just st_mode set, and all other fields None: this
would be the usual case on Linux/OS X
3) stat_result with all fields None: this would happen on systems
whose readdir()/dirent doesn't have d_type, or on Linux/OS X when
d_type was DT_UNKNOWN

Higher-level functions like os.walk() would then check the fields they
needed are not None, and only call os.stat() if needed, for example:

# Build lists of files and directories in path
files = []
dirs = []
for name, st in os.scandir(path):
 if st.st_mode is None:
 st = os.stat(os.path.join(path, name))
 if stat.S_ISDIR(st.st_mode):
 dirs.append(name)
 else:
 files.append(name)

Not bad for a 2-10x performance boost, right? What do folks think?

Cheers,
Ben.


[snip]
In the python-ideas list there's a thread "PEP: Extended stat_result"
about adding methods to stat_result.

Using that, you wouldn't necessarily have to look at st.st_mode. The 
method could perform an additional os.stat() if the field was None. For

example:

# Build lists of files and directories in path
files = []
dirs = []
for name, st in os.scandir(path):
 if st.is_dir():
 dirs.append(name)
 else:
 files.append(name)

That looks much nicer.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Ronald Oussoren 

On 10 May, 2013, at 16:30, MRAB  wrote:
>> 
> [snip]
> In the python-ideas list there's a thread "PEP: Extended stat_result"
> about adding methods to stat_result.
> 
> Using that, you wouldn't necessarily have to look at st.st_mode. The method 
> could perform an additional os.stat() if the field was None. For
> example:
> 
> # Build lists of files and directories in path
> files = []
> dirs = []
> for name, st in os.scandir(path):
> if st.is_dir():
> dirs.append(name)
> else:
> files.append(name)
> 
> That looks much nicer.

I'd prefer a filetype field, with 'st.filetype == "dir"' instead of 
'st.is_dir()'. The actual type of filetype values is less important, an enum 
type would also work although bootstrapping that type could be interesting.

Ronald

> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Antoine Pitrou 
Le Fri, 10 May 2013 23:53:37 +1000,
Nick Coghlan  a écrit :
> On Fri, May 10, 2013 at 11:46 PM, Christian Heimes
>  wrote:
> > Am 10.05.2013 14:16, schrieb Antoine Pitrou:
> >> But what if some systems return more than the file type and less
> >> than a full stat result? The general problem is POSIX's terrible
> >> inertia. I feel that a stat result with some None fields would be
> >> an acceptable compromise here.
> >
> > POSIX only defines the d_ino and d_name members of struct dirent.
> > Linux, BSD and probably some other platforms also happen to provide
> > d_type. The other members of struct dirent (d_reclen, d_namlen)
> > aren't useful in Python space by themselves.
> >
> > d_type and st_mode aren't compatible in any way. As you know st_mode
> > also contains POSIX permission information. The file type is encoded
> > with a different set of bits, too. Future file types aren't mapped
> > to S_IF* constants for st_mode.
> 
> Why are we exposing a bitfield as the primary Python level API,
> anyway? It makes sense for the well defined permission bits, but why
> are we copying the C level concept for the other flags?

Precisely because they are not well-defined, hence any interpretation
by us may be incorrect or incomplete (e.g. obscure system-specific
bits).

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Terry Jan Reedy 

On 5/10/2013 3:14 AM, Nick Coghlan wrote:

I'd like to mark a few PEPs that are not currently being actively
considered for 3.4 as Deferred:

  S   286  Enhanced Argument Tuplesvon Löwis
  S   337  Logging Usage in the Standard Library   Dubner
  S   368  Standard image protocol and class   
Mastrodomenico
  I   396  Module Version Numbers  Warsaw
  S   400  Deprecate codecs.StreamReader and codecs.StreamWriter   Stinner
  S   419  Protecting cleanup statements from interruptionsColomiets
  I   423  Naming conventions and recipes related to packaging Bryon
  I   444  Python Web3 Interface
McDonough, Ronacher
  S  3124  Overloading, Generic Functions, Interfaces, and ... Eby
  S  3142  Add a "while" clause to generator expressions   Britton


I had the impression that this had more or less been rejected. I suppose 
I could try to dig up the discussion.



  S  3143  Standard daemon process library Finney
  S  3145  Asynchronous I/O For subprocess.Popen
Pruitt, McCreary, Carlson
  S  3152  Cofunctions Ewing


You might also ask the authors if they are still really in favor of them 
or have any hope for them, considering whatever discussion occurred, or 
whether they have abandoned them (which would mean withdrawn).




___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

2013-05-10 Thread Python tracker 

ACTIVITY SUMMARY (2013-05-03 - 2013-05-10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open3963 (+10)
  closed 25758 (+44)
  total  29721 (+54)

Open issues with patches: 1774 


Issues opened (48)
==

#5845: rlcompleter should be enabled automatically
http://bugs.python.org/issue5845  reopened by mark.dickinson

#15902: imp.load_module won't accept None for the file argument for a 
http://bugs.python.org/issue15902  reopened by brett.cannon

#17656: Python 2.7.4 breaks ZipFile extraction of zip files with unico
http://bugs.python.org/issue17656  reopened by pitrou

#17883: Fix buildbot testing of Tkinter
http://bugs.python.org/issue17883  reopened by ezio.melotti

#17895: TemporaryFile name returns an integer in python3
http://bugs.python.org/issue17895  opened by jtaylor

#17896: Move Windows external libs from \..\ to \externals
http://bugs.python.org/issue17896  opened by zach.ware

#17897: Optimize unpickle prefetching
http://bugs.python.org/issue17897  opened by serhiy.storchaka

#17898: gettext bug while parsing plural-forms metadata
http://bugs.python.org/issue17898  opened by straz

#17899: os.listdir() leaks FDs if invoked on FD pointing to a non-dire
http://bugs.python.org/issue17899  opened by abacabadabacaba

#17900: Recursive OrderedDict pickling
http://bugs.python.org/issue17900  opened by serhiy.storchaka

#17901: _elementtree.TreeBuilder raises IndexError on end if construct
http://bugs.python.org/issue17901  opened by Aaron.Oakley

#17902: Document that _elementtree C API cannot use custom TreeBuilder
http://bugs.python.org/issue17902  opened by Aaron.Oakley

#17903: Python launcher for windows should search path for #!/usr/bin/
http://bugs.python.org/issue17903  opened by pmoore

#17904: bytes should be listed as built-in function for 2.7
http://bugs.python.org/issue17904  opened by flox

#17905: Add check for locale.h
http://bugs.python.org/issue17905  opened by cavallo71

#17906: JSON should accept lone surrogates
http://bugs.python.org/issue17906  opened by serhiy.storchaka

#17907: Deprecate imp.new_module() in favour of types.ModuleType
http://bugs.python.org/issue17907  opened by brett.cannon

#17908: Unittest runner needs an option to call gc.collect()after eac
http://bugs.python.org/issue17908  opened by gvanrossum

#17909: Autodetecting JSON encoding
http://bugs.python.org/issue17909  opened by serhiy.storchaka

#17911: Extracting tracebacks does too much work
http://bugs.python.org/issue17911  opened by gvanrossum

#17913: stat.filemode returns "-" for sockets and unknown types
http://bugs.python.org/issue17913  opened by christian.heimes

#17914: add os.cpu_count()
http://bugs.python.org/issue17914  opened by neologix

#17915: Encoding error with sax and codecs
http://bugs.python.org/issue17915  opened by sconseil

#17916: Provide dis.Bytecode based equivalent of dis.distb
http://bugs.python.org/issue17916  opened by ncoghlan

#17917: use PyModule_AddIntMacro() instead of PyModule_AddIntConstant(
http://bugs.python.org/issue17917  opened by neologix

#17919: AIX POLLNVAL definition causes problems
http://bugs.python.org/issue17919  opened by delhallt

#17920: Documentation: "complete ordering" should be "total ordering"
http://bugs.python.org/issue17920  opened by abcdef

#17922: Crash in clear_weakref
http://bugs.python.org/issue17922  opened by jsafrane

#17923: test glob with trailing slash fail
http://bugs.python.org/issue17923  opened by delhallt

#17924: Deprecate stat.S_IF* integer constants
http://bugs.python.org/issue17924  opened by christian.heimes

#17925: asynchat.async_chat.initiate_send : del deque[0] is not safe
http://bugs.python.org/issue17925  opened by Pierrick.Koch

#17927: Argument copied into cell still referenced by frame
http://bugs.python.org/issue17927  opened by gvanrossum

#17930: Search not needed in combinations_with_replacement
http://bugs.python.org/issue17930  opened by tim_one

#17931: PyLong_FromPid() is not correctly defined on Windows 64-bit
http://bugs.python.org/issue17931  opened by haypo

#17932: Win64: possible integer overflow in iterobject.c
http://bugs.python.org/issue17932  opened by haypo

#17933: test_ftp failure / ftplib error formatting issue
http://bugs.python.org/issue17933  opened by pitrou

#17934: Add a frame method to clear expensive details
http://bugs.python.org/issue17934  opened by pitrou

#17936: O(n**2) behaviour when adding/removing classes
http://bugs.python.org/issue17936  opened by kristjan.jonsson

#17937: Collect garbage harder at shutdown
http://bugs.python.org/issue17937  opened by pitrou

#17939: Misleading information about slice assignment in docs
http://bugs.python.org/issue17939  opened by stefanchrobot

#17940: extra code in argparse.py
http://bugs.python.org/issue17940  opened by aho

#17941: namedtuple should support fully qualified name for m

Re: [Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Guido van Rossum 
On Fri, May 10, 2013 at 9:01 AM, Terry Jan Reedy  wrote:
> On 5/10/2013 3:14 AM, Nick Coghlan wrote:
>>
>> I'd like to mark a few PEPs that are not currently being actively
>> considered for 3.4 as Deferred:
>>
>>   S   286  Enhanced Argument Tuplesvon
>> Löwis
>>   S   337  Logging Usage in the Standard Library   Dubner
>>   S   368  Standard image protocol and class
>> Mastrodomenico
>>   I   396  Module Version Numbers  Warsaw
>>   S   400  Deprecate codecs.StreamReader and codecs.StreamWriter   Stinner
>>   S   419  Protecting cleanup statements from interruptions
>> Colomiets
>>   I   423  Naming conventions and recipes related to packaging Bryon
>>   I   444  Python Web3 Interface
>> McDonough, Ronacher
>>   S  3124  Overloading, Generic Functions, Interfaces, and ... Eby
>>   S  3142  Add a "while" clause to generator expressions   Britton
>
>
> I had the impression that this had more or less been rejected. I suppose I
> could try to dig up the discussion.

I didn't know there was a PEP for that. I hereby reject it. No point
wasting more time on it.

>>   S  3143  Standard daemon process library Finney
>>   S  3145  Asynchronous I/O For subprocess.Popen
>> Pruitt, McCreary, Carlson
>>   S  3152  Cofunctions Ewing
>
>
> You might also ask the authors if they are still really in favor of them or
> have any hope for them, considering whatever discussion occurred, or whether
> they have abandoned them (which would mean withdrawn).
>
>
>
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Barry Warsaw 
On May 10, 2013, at 09:19 AM, Guido van Rossum wrote:

>>>   S  3142  Add a "while" clause to generator expressions   Britton
>>
>> I had the impression that this had more or less been rejected. I suppose I
>> could try to dig up the discussion.
>
>I didn't know there was a PEP for that. I hereby reject it. No point
>wasting more time on it.

Done.

-Barry
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Nick Coghlan 
On Sat, May 11, 2013 at 2:01 AM, Terry Jan Reedy  wrote:
> You might also ask the authors if they are still really in favor of them or
> have any hope for them, considering whatever discussion occurred, or whether
> they have abandoned them (which would mean withdrawn).

That's real work though, compared to just marking them as Deferred :)

Cheers,
Nick.

--
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] make a Windows installation package (.msi) for Python 3.3

2013-05-10 Thread Jianfeng Mao 
To Python Windows Release Managers:

My name is Jianfeng Mao and I am a software developer at the U2 group in Rocket 
Software (http://u2.rocketsoftware.com/).   I am currently working on a project 
to embed a slightly customized Python interpreter in our product. For easy 
installation and setup,  we hope to be able to do the standard Python 
installation during the installation of our software.  Basically I want to 
create a .msi file that can be called to install the full Python if the user 
needs this new feature. Brian Curtin 
([email protected])  pointed me to Tools/msi/msi.py for 
the Windows MSI builder. I tried to follow  the instructions in the README but 
couldn't make it to work after a few twists and turns.  Brian mentioned that 
few people needs to do this and only release managers handle the packaging of 
Python.  I have listed the steps I have done in my attempt to create the .msi 
file. Please let me know if I have missed anything  or done anything wrong.



1.   hg clone http://hg.python.org/cpython

2.   cd cpython

3.   hg update 3.3

4.   cd tools\buildbot,  edit build.bat to change the configuration from 
Debug to Releaes; edit external.bat, change DEBUG=1 to DEBUG=0

5.   go back to cpython\ and run tools\buildbot\build.bat

6.   cd PC, then do 'nmake -f icons.mak'

7.   cd ..\tools\msi

8.   c:\python27\python msi.py


WARNING: nm did not run successfully - libpythonXX.a not built
cl /O2 /D WIN32 /D NDEBUG /D _WINDOWS /MT /W3 /c msisupport.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

msisupport.c
link.exe /OUT:msisupport.dll /INCREMENTAL:NO /NOLOGO /DLL /SUBSYSTEM:WIN
DOWS /OPT:REF /OPT:ICF msisupport.obj msi.lib kernel32.lib
   Creating library msisupport.lib and object msisupport.exp
Traceback (most recent call last):
  File "msi.py", line 1336, in 
add_files(db)
  File "msi.py", line 961, in add_files
generate_license()
  File "msi.py", line 914, in generate_license
raise ValueError, "Could not find "+srcdir+"/../"+pat
ValueError: Could not find C:\temp\cpython/../tcl8*
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Larry Hastings 

On 05/10/2013 12:14 AM, Nick Coghlan wrote:

I'd like to mark a few PEPs that are not currently being actively
considered for 3.4 as Deferred:


I swear I posted a list like this a couple years ago.  Now I can't find 
it.  Anyway it was completely ignored then, probably because I'm not 
Nick Coghlan.



//arry/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0 maintenance - deferring some currently open PEPs

2013-05-10 Thread Antoine Pitrou 
On Fri, 10 May 2013 14:00:09 -0700
Larry Hastings  wrote:
> 
> I swear I posted a list like this a couple years ago.  Now I can't find 
> it.  Anyway it was completely ignored then, probably because I'm not 
> Nick Coghlan.

Many people in the world suffer with this problem.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] make a Windows installation package (.msi) for Python 3.3

2013-05-10 Thread Brian Curtin 
On Fri, May 10, 2013 at 12:31 PM, Jianfeng Mao  wrote:
> To Python Windows Release Managers:
>
>
>
> My name is Jianfeng Mao and I am a software developer at the U2 group in
> Rocket Software (http://u2.rocketsoftware.com/).   I am currently working on
> a project to embed a slightly customized Python interpreter in our product.
> For easy installation and setup,  we hope to be able to do the standard
> Python installation during the installation of our software.  Basically I
> want to create a .msi file that can be called to install the full Python if
> the user needs this new feature. Brian Curtin ([email protected])  pointed me
> to Tools/msi/msi.py for the Windows MSI builder. I tried to follow  the
> instructions in the README but couldn’t make it to work after a few twists
> and turns.  Brian mentioned that few people needs to do this and only
> release managers handle the packaging of Python.  I have listed the steps I
> have done in my attempt to create the .msi file. Please let me know if I
> have missed anything  or done anything wrong.
>
>
>
>
>
> 1.   hg clone http://hg.python.org/cpython
>
> 2.   cd cpython
>
> 3.   hg update 3.3
>
> 4.   cd tools\buildbot,  edit build.bat to change the configuration from
> Debug to Releaes; edit external.bat, change DEBUG=1 to DEBUG=0
>
> 5.   go back to cpython\ and run tools\buildbot\build.bat
>
> 6.   cd PC, then do ‘nmake –f icons.mak’
>
> 7.   cd ..\tools\msi
>
> 8.   c:\python27\python msi.py
>
>
>
> WARNING: nm did not run successfully - libpythonXX.a not built
>
> cl /O2 /D WIN32 /D NDEBUG /D _WINDOWS /MT /W3 /c msisupport.c
>
> Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for
> 80x86
>
> Copyright (C) Microsoft Corporation.  All rights reserved.
>
>
>
> msisupport.c
>
> link.exe /OUT:msisupport.dll /INCREMENTAL:NO /NOLOGO /DLL
> /SUBSYSTEM:WIN
>
> DOWS /OPT:REF /OPT:ICF msisupport.obj msi.lib kernel32.lib
>
>Creating library msisupport.lib and object msisupport.exp
>
> Traceback (most recent call last):
>
>   File "msi.py", line 1336, in 
>
> add_files(db)
>
>   File "msi.py", line 961, in add_files
>
> generate_license()
>
>   File "msi.py", line 914, in generate_license
>
> raise ValueError, "Could not find "+srcdir+"/../"+pat
>
> ValueError: Could not find C:\temp\cpython/../tcl8*

I'm in an airport and on a Mac right now so I can't test it, but IIRC
you just need to adjust the script to look for tcl-8* and not tcl8* on
line 908 of msi.py. You'll probably have to do the same for tk. If you
come across other exceptions about tcl, tk, or other dependencies,
it's likely that the paths are just incorrect.

There may be a patch for this on bugs.python.org because I know I've
gone through it.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Ben Hoyt 
> In the python-ideas list there's a thread "PEP: Extended stat_result"
> about adding methods to stat_result.
>
> Using that, you wouldn't necessarily have to look at st.st_mode. The method
> could perform an additional os.stat() if the field was None. For
>
> example:
>
> # Build lists of files and directories in path
> files = []
> dirs = []
> for name, st in os.scandir(path):
>  if st.is_dir():
>  dirs.append(name)
>  else:
>  files.append(name)

That's not too bad. However, the st.is_dir() function could
potentially call os.stat(), so you'd have to be specific about how
errors are handled. Also, I'm not too enthusiastic about how much "API
weight" this would add -- do you need st.is_link() and st.size() and
st.everything_else() as well?

-Ben
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-10 Thread Ben Hoyt 
> Have you actually tried the code? It can't give you correct answers. The
> struct dirent.d_type member as returned by readdir() has different
> values than stat.st_mode's file type.

Yes, I'm quite aware of that. In the first version of BetterWalk
that's exactly how it did it, and this approach worked fine.
However...

> Or are you proposing to map d_type to st_mode?

Yes, that's exactly what I was proposing -- sorry if that wasn't clear.

> Hence I'm +1 on the general idea but -1 on something stat like. IMHO
> os.scandir() should yield four objects:
>
>  * name
>  * inode
>  * file type or DT_UNKNOWN
>  * stat_result or None

This feels quite heavy to me. And I don't like it how for the normal
case (checking whether something was a file or directory) you'd have
to check file_type against DT_UNKNOWN as well as stat_result against
None before doing anything with it:

for item in os.scandir():
if item.file_type == DT_UNKNOWN and item.stat_result is None:
# call os.stat()

I guess that's not *too* bad.

> That's also problematic because st_mode would only have file type
> bits, not permission bits.

You're right. However, given that scandir() is intended as a
low-level, OS-specific function, couldn't we just document this and
move on? Keep the API nice and simple and still cover 95% of the use
cases. How often does anyone actually iterate through a directory
doing stuff with the permission bits.

The nice thing about having it return a stat-like object is that in
almost all cases you don't have to have two different code paths
(d_type and st_mode), you just deal with st_mode. And we already have
the stat module for dealing with st_mode stuff, so we wouldn't need
another bunch of code/constants for dealing with d_type.

The documentation could just say something like:

"The exact information returned in st_mode is OS-specific. In
practice, on Windows it returns all the information that stat() does.
On Linux and OS X, it's either None or it includes the mode bits (but
not the permissions bits)."

Antoine said: "But what if some systems return more than the file type
and less than a
full stat result?"

Again, I just think that debating the very fine points like this to
get that last 5% of use cases will mean we never have this very useful
function in the library.

In all the *practical* examples I've seen (and written myself), I
iterate over a directory and I just need to know whether it's a file
or directory (or maybe a link). Occassionally you need the size as
well, but that would just mean a similar check "if st.st_size is None:
st = os.stat(...)", which on Linux/OS X would call stat(), but it'd
still be free and fast on Windows.

-Ben
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 435 - ref impl disc 2

2013-05-10 Thread Glenn Linderman 
So, thanks everyone for helping me understand the metaclass issues, and 
helping fix my code and the reference implementation, so that I got a 
working workaround for enumerations.


Twiddling some more newly using hg and bitbucket... learned a lot 
today... at 


Premise: For API parameters that are bitfields, it would be nice to have 
an Enum-type class that tracks the calculations done with the named 
values to create bit masks.


Currently available: Enum (or IntEnum) that can group the collection of 
named bit-field values into values of a single and unique type, but 
loses the names during calculations.


Just written: a class IntET (for "int expression tracker" which has an 
expression name as well as a value.  As a side effect, this could be 
used to track effective calculations of integers for debugging, because 
that is the general mechanism needed to track combined flag values, also 
for debug reporting.


So it is quite possible to marry the two, as Ethan helped me figure out 
using an earlier NamedInt class:


class NIE( IntET, Enum ):
x = ('NIE.x', 1)
y = ('NIE.y', 2)
z = ('NIE.z', 4)

and then expressions involving members of NIE (and even associated 
integers) will be tracked... see demo1.py.


But the last few lines of demo1 demonstrate that NIE doesn't like, 
somehow, remember that its values, deep down under the covers, are 
really int.  And doesn't even like them when they are wrapped into IntET 
objects.  This may or may not be a bug in the current Enum implementation.


It is cumbersome to specify redundant names for the enumeration members 
and the underlying IntET separately, however.


Turns out that adding one line to ref435 (which I did in ref435a) will 
allow (nay, require) that base types for these modified Enums must have 
names, which must be supplied as the first parameter to their 
constructor.  This also works around whatever problem "real" Enum has 
with using named items internally, as demonstrated by demo2.py 
(reproduced in part here):


class NIE( IntET, Enum ):
x = 1
y = 2
z = 4

print( repr( NIE.x + NIE.y ))
IntET('(NIE.x + NIE.y)', 3)

So the questions are:
1) Is there a bug in ref435 Enum that makes demo1 report errors instead 
of those lines working?
2) Is something like demo2 interesting to anyone but me? Of course, I 
think it would be great for reporting flag values using names rather 
than a number representing combined bit fields.
3) I don't see a way to subclass the ref435 EnumMeta except by replacing 
the whole __new__ method... does this mechanism warrant a slight 
refactoring of EnumMeta to make this mechanism easier to subclass with 
less code redundancy?
4) Or is it simple enough and useful enough to somehow make it a feature 
of EnumMeta, enabled by a keyword parameter?  Or one _could_ detect the 
existence of a __name__ property on the first base type, and key off of 
that, but that may sometimes be surprising (of course, that is what 
documentation is for: to explain away the surprises people get when they 
don't read it).
5) All this is based on "IntET"... which likely suffices for API flags 
parameters... but when I got to __truediv__ and __rtruediv__, which 
don't return int, then I started wondering how to write a vanilla ET 
class that inherits from "number" instead of "int" or "float"? One 
could, of course, make cooperating classes FloatET and DecimalET  is 
this a language limitation, or is there more documentation I haven't 
read? :)   (I did read footnote [1] of 
, 
and trembled.)


Probably some of these questions should be on stackoverflow or python 
ideas, but it is certainly an outgrowth of the Enum PEP, and personally, 
I'd hate to see flag APIs converted to Enum without the ability to track 
combinations of them... so I hope that justifies parts of this 
discussion continuing here. I'm happy to take pieces to other places, if 
so directed.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 435 - ref impl disc 2

2013-05-10 Thread Ethan Furman 

On 05/10/2013 10:15 PM, Glenn Linderman wrote:


But the last few lines of demo1 demonstrate that NIE doesn't like, somehow, 
remember that its values, deep down under
the covers, are really int.  And doesn't even like them when they are wrapped 
into IntET objects.  This may or may not
be a bug in the current Enum implementation.


You're right, sort of.  ;)

If you do

  print( repr( NIE1.x.value ))

you'll see

  ('NIE1.x', 1)

In other words, the value of NEI.x is `('NEI1.x', 1)` and that is what you would have to pass back into NEI to get the 
enum member.


As an aside, I suspect you are doing this the hard way.  Perhaps writing your own __new__ in NIE will have better 
results (I'd try, but I gotta get some sleep! ;) .  Oh, newest code posted.


--
~Ethan~
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com