Re: [Python-Dev] Non-string keys in type dict
Nick Coghlan wrote: On Thu, Mar 8, 2012 at 11:42 AM, Benjamin Peterson benja...@python.org wrote: 2012/3/7 Victor Stinner victor.stin...@gmail.com: Can't we simply raise an error if the dict contains non-string keys? Sounds okay to me. For 3.3, the most we can do is trigger a deprecation warning, since removing this feature *will* break currently running code. I don't have any objection to us starting down that path, though. I think it would be sad to lose that functionality. If we are going to, though, we may as well check the string to make sure it's a valid identifier: -- class A: -- pass -- setattr(A, '42', 'hrm') -- A.42 File stdin, line 1 A.42 ^ SyntaxError: invalid syntax Doesn't seem very useful. ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Thu, Mar 8, 2012 at 08:46, Ethan Furman et...@stoneleaf.us wrote: I think it would be sad to lose that functionality. If we are going to, though, we may as well check the string to make sure it's a valid identifier: That would break even more code. I have encountered many cases of attributes that aren't valid identifiers, in particular using dots or dashes. Admittedly this is often in cases where the object has both attribute access and key access, so you can make foo['bar-frotz'] instead. But when should we then require that it is a valid identifier and when not? -- class A: -- pass -- setattr(A, '42', 'hrm') -- A.42 File stdin, line 1 A.42 ^ SyntaxError: invalid syntax Doesn't seem very useful. You have to set it with setattr, so you have to get it with getattr. I don't see the problem. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Thu, Mar 8, 2012 at 1:10 AM, Lennart Regebro rege...@gmail.com wrote: On Thu, Mar 8, 2012 at 08:46, Ethan Furman et...@stoneleaf.us wrote: I think it would be sad to lose that functionality. If we are going to, though, we may as well check the string to make sure it's a valid identifier: That would break even more code. I have encountered many cases of attributes that aren't valid identifiers, in particular using dots or dashes. Admittedly this is often in cases where the object has both attribute access and key access, so you can make foo['bar-frotz'] instead. But when should we then require that it is a valid identifier and when not? -- class A: -- pass -- setattr(A, '42', 'hrm') -- A.42 File stdin, line 1 A.42 ^ SyntaxError: invalid syntax Doesn't seem very useful. You have to set it with setattr, so you have to get it with getattr. I don't see the problem. I'm with Lennart. I've spoken out on this particular question several times before; it is a *feature* that you can use any arbitrary string with getattr() and setattr(). However these functions should (and do!) reject non-strings. I'm lukewarm on forbidding non-strings in namespace dicts if you can get them in there by other means. I presume that some people might use this to hide state they don't want accessible using regular attribute notation (x.foo) and maybe somebody is even using some namespace's keys as a dict to store many things with non-string keys, but that feels like abuse of the namespace to me, and there are plenty of other ways to manage such state, so if it gives a significant speed-up to use string-only dicts, so be it. (Although Python's dicts already have some string-only optimizations -- they just dynamically adapt to a more generic and slightly slower approach once the first non-key string shows up.) As Nick says we should introduce a deprecation period if we do this. On Wed, Mar 7, 2012 at 11:43 PM, Ethan Furman et...@stoneleaf.us wrote: Are you able to modify classes after class creation in Python 3? Without using a metaclass? Yes, by assignment to attributes. The __dict__ is a read-only proxy, but attribute assignment is allowed. (This is because the new type system introduced in Python 2.2 needs to *track* changes to the dict; it does this by tracking setattr/delattr calls, because dict doesn't have a way to trigger a hook on changes.) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
Guido van Rossum wrote:
On Wed, Mar 7, 2012 at 11:43 PM, Ethan Furman wrote:
Are you able to modify classes after class creation in Python 3? Without
using a metaclass?
Yes, by assignment to attributes. The __dict__ is a read-only proxy,
but attribute assignment is allowed. (This is because the new type
system introduced in Python 2.2 needs to *track* changes to the dict;
it does this by tracking setattr/delattr calls, because dict doesn't
have a way to trigger a hook on changes.)
Poorly phrased question -- I meant is it possible to add non-string-name
attributes to classes after class creation. During class creation we
can do this:
-- class Test:
... ns = vars()
... ns[42] = 'green eggs'
... del ns
...
-- Test
class '__main__.Test'
-- Test.__dict__
dict_proxy({
'__module__': '__main__',
42: 'green eggs',
'__doc__': None,
'__dict__': attribute '__dict__' of 'Test' objects,
'__weakref__': attribute '__weakref__' of 'Test' objects,
'__locals__': {
42: 'green eggs',
'__module__': '__main__',
'__locals__': {...}}
})
-- Test.__dict__[42]
'green eggs'
A little more experimentation shows that not all is well, however:
-- dir(Test)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unorderable types: int() str()
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Thu, Mar 8, 2012 at 8:22 AM, Ethan Furman et...@stoneleaf.us wrote:
Guido van Rossum wrote:
On Wed, Mar 7, 2012 at 11:43 PM, Ethan Furman wrote:
Are you able to modify classes after class creation in Python 3? Without
using a metaclass?
Yes, by assignment to attributes. The __dict__ is a read-only proxy,
but attribute assignment is allowed. (This is because the new type
system introduced in Python 2.2 needs to *track* changes to the dict;
it does this by tracking setattr/delattr calls, because dict doesn't
have a way to trigger a hook on changes.)
Poorly phrased question -- I meant is it possible to add non-string-name
attributes to classes after class creation. During class creation we can do
this:
-- class Test:
... ns = vars()
... ns[42] = 'green eggs'
... del ns
...
-- Test
class '__main__.Test'
-- Test.__dict__
dict_proxy({
'__module__': '__main__',
42: 'green eggs',
'__doc__': None,
'__dict__': attribute '__dict__' of 'Test' objects,
'__weakref__': attribute '__weakref__' of 'Test' objects,
'__locals__': {
42: 'green eggs',
'__module__': '__main__',
'__locals__': {...}}
})
-- Test.__dict__[42]
'green eggs'
A little more experimentation shows that not all is well, however:
-- dir(Test)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unorderable types: int() str()
So what conclusion do you draw?
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Thu, Mar 8, 2012 at 2:43 AM, Ethan Furman et...@stoneleaf.us wrote: PJ Eby wrote: Short version: AddOns are things you can use to dynamically extend instances -- a bit like the decorator in decorator pattern (not to be confused with Python decorators). Rather than synthesize a unique string as a dictionary key, I just used the AddOn classes themselves as keys. This works fine for object instances, but gets hairy once classes come into play. Are you able to modify classes after class creation in Python 3? Without using a metaclass? For ClassAddOns, it really doesn't matter; you can't remove them from the class they attach to. Addons created after the class is finalized use a weakref dictionary to attach to their classes. Now that I've gone back and looked at the code, the only reason that ClassAddOns even use the class __dict__ in the first place is because it's a convenient place to put them while the class is being built. With only slightly hairier code, I could use an __addons__ dict in the class namespace while it's being built, but there'll then be a performance hit at look up time to do cls.__dict__['__addons__'][key] instead of cls.__dict__[key]. Actually, now that I'm thinking about it, the non-modifiability of class dictionaries is actually a feature for this use case: if I make an __addons__ dict, that dict is mutable. That means I'll have to move to string keys or have some sort of immutable dict type available... ;-) (Either that, or do some other, more complex refactoring.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Wed, Mar 7, 2012 at 5:39 PM, Victor Stinner victor.stin...@gmail.com wrote: Hi, During the Language Summit 2011 (*), it was discussed that PyPy and Jython don't support non-string key in type dict. An issue was open to emit a warning on such dict, but the patch has not been commited yet. It should be noted that Jython started supporting non-string dict keys in version 2.5. IIRC this was done to get Django running, but in general we decided to go with compatibility. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
Guido van Rossum wrote:
On Thu, Mar 8, 2012 at 8:22 AM, Ethan Furman et...@stoneleaf.us wrote:
Guido van Rossum wrote:
On Wed, Mar 7, 2012 at 11:43 PM, Ethan Furman wrote:
Are you able to modify classes after class creation in Python 3? Without
using a metaclass?
Yes, by assignment to attributes. The __dict__ is a read-only proxy,
but attribute assignment is allowed. (This is because the new type
system introduced in Python 2.2 needs to *track* changes to the dict;
it does this by tracking setattr/delattr calls, because dict doesn't
have a way to trigger a hook on changes.)
Poorly phrased question -- I meant is it possible to add non-string-name
attributes to classes after class creation. During class creation we can do
this:
-- class Test:
... ns = vars()
... ns[42] = 'green eggs'
... del ns
...
-- Test
class '__main__.Test'
-- Test.__dict__
dict_proxy({
'__module__': '__main__',
42: 'green eggs',
'__doc__': None,
'__dict__': attribute '__dict__' of 'Test' objects,
'__weakref__': attribute '__weakref__' of 'Test' objects,
'__locals__': {
42: 'green eggs',
'__module__': '__main__',
'__locals__': {...}}
})
-- Test.__dict__[42]
'green eggs'
A little more experimentation shows that not all is well, however:
-- dir(Test)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unorderable types: int() str()
So what conclusion do you draw?
That other changes (that have definitely been for the better) are making
the 'feature' of non-string keys in namespace dicts less and less
friendly. Rather than letting it slowly fall into complete shambles we
should go ahead and deprecate, then remove, that functionality.
Because namespace dicts already have tacit approval to not support
non-string keys, it doesn't make much sense to spend developer resources
on fixing dir and whatever other functions exist that deal with
namespace dicts and assume string-only keys.
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
Hi, 2012/3/8 Ethan Furman et...@stoneleaf.us: A little more experimentation shows that not all is well, however: -- dir(Test) Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: int() str() So what conclusion do you draw? That other changes (that have definitely been for the better) are making the 'feature' of non-string keys in namespace dicts less and less friendly. Rather than letting it slowly fall into complete shambles we should go ahead and deprecate, then remove, that functionality. Not that I disagree with the conclusion, but the obvious thing to do here is to fix dir() and return only string attributes, i.e. those you can access with getattr. Cheers, -- Amaury Forgeot d'Arc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
2012/3/7 Victor Stinner victor.stin...@gmail.com: So my question is: what is the use case of such dict? Why do we still support it? Probably a side-effect of implementation. Can't we simply raise an error if the dict contains non-string keys? Sounds okay to me. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
During the Language Summit 2011 (*), it was discussed that PyPy and
Jython don't support non-string key in type dict. An issue was open to
emit a warning on such dict, but the patch has not been commited yet.
It's the issue #11455. As written in the issue, there are two ways to
create such type:
class A(object):
locals()[42] = abc
or
type(A, (object,), {42: abc})
Both look like an ugly hack.
Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Wed, Mar 7, 2012 at 5:45 PM, Victor Stinner victor.stin...@gmail.comwrote:
During the Language Summit 2011 (*), it was discussed that PyPy and
Jython don't support non-string key in type dict. An issue was open to
emit a warning on such dict, but the patch has not been commited yet.
It's the issue #11455. As written in the issue, there are two ways to
create such type:
class A(object):
locals()[42] = abc
or
type(A, (object,), {42: abc})
Both look like an ugly hack.
Here is a cleaner version, using metaclasses (Python 2.6):
class M(type):
def __new__(mcs, name, bases, dict):
dict[42] = 'abc'
return super(M, mcs).__new__(mcs, name, bases, dict)
class A(object):
__metaclass__ = M
Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ckaynor%40zindagigames.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
I see that I've misunderstood this entirely, nevermind me.
--Brett
On 08/03/12 14:48, Brett Wilkins wrote:
I assume when you say non-string keys this includes numbers.
But in Pypy, I can certainly use numbers:
{'1':1, 1:2}.keys()
['1', 1]
I can even use a lambda (obviously not a string, a number, nor what I
would consider a primitive):
{'1':1, (lambda x: x):2}.keys()
['1', function lambda at 0x7fdb0b837da8]
These are in Pypy 1.8.
--Brett
On Thu 08 Mar 2012 14:39:40 NZDT, Victor Stinner wrote:
Hi,
During the Language Summit 2011 (*), it was discussed that PyPy and
Jython don't support non-string key in type dict. An issue was open to
emit a warning on such dict, but the patch has not been commited yet.
I'm trying to Lib/test/crashers/losing_mro_ref.py: I wrote a patch
fixing the specific issue (keep a strong reference to the MRO during
the lookup, see #14199), but I realized that the real problem is that
we allow custom objects in the type dict.
So my question is: what is the use case of such dict? Why do we still
support it? Can't we simply raise an error if the dict contains
non-string keys?
(*) http://blog.python.org/2011/03/2011-language-summit-report.html
Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/brett%40brett.geek.nz
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
I assume when you say non-string keys this includes numbers.
But in Pypy, I can certainly use numbers:
{'1':1, 1:2}.keys()
['1', 1]
I can even use a lambda (obviously not a string, a number, nor what I
would consider a primitive):
{'1':1, (lambda x: x):2}.keys()
['1', function lambda at 0x7fdb0b837da8]
These are in Pypy 1.8.
--Brett
On Thu 08 Mar 2012 14:39:40 NZDT, Victor Stinner wrote:
Hi,
During the Language Summit 2011 (*), it was discussed that PyPy and
Jython don't support non-string key in type dict. An issue was open to
emit a warning on such dict, but the patch has not been commited yet.
I'm trying to Lib/test/crashers/losing_mro_ref.py: I wrote a patch
fixing the specific issue (keep a strong reference to the MRO during
the lookup, see #14199), but I realized that the real problem is that
we allow custom objects in the type dict.
So my question is: what is the use case of such dict? Why do we still
support it? Can't we simply raise an error if the dict contains
non-string keys?
(*) http://blog.python.org/2011/03/2011-language-summit-report.html
Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/brett%40brett.geek.nz
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Thu, Mar 8, 2012 at 11:42 AM, Benjamin Peterson benja...@python.org wrote: 2012/3/7 Victor Stinner victor.stin...@gmail.com: Can't we simply raise an error if the dict contains non-string keys? Sounds okay to me. For 3.3, the most we can do is trigger a deprecation warning, since removing this feature *will* break currently running code. I don't have any objection to us starting down that path, though. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Thu, Mar 08, 2012 at 12:20:21PM +1000, Nick Coghlan wrote: On Thu, Mar 8, 2012 at 11:42 AM, Benjamin Peterson benja...@python.org wrote: 2012/3/7 Victor Stinner victor.stin...@gmail.com: Can't we simply raise an error if the dict contains non-string keys? Sounds okay to me. For 3.3, the most we can do is trigger a deprecation warning, since removing this feature *will* break currently running code. I don't have any objection to us starting down that path, though. Could we make string-key-only dicts a public type instead of an implementation detail? I've used string-only keys in my code, and it seems silly to have to re-invent the wheel. I don't care if it is a built-in. I don't even care if I have to do something gnarly like StringDict = type(type.__dict__), so long as doing so is officially supported. (But from collections import StringDict would be better from the point of discoverability.) -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Wed, Mar 7, 2012 at 8:39 PM, Victor Stinner victor.stin...@gmail.comwrote: So my question is: what is the use case of such dict? Well, I use them for this: http://pypi.python.org/pypi/AddOns (And I have various other libraries that depend on that library.) Short version: AddOns are things you can use to dynamically extend instances -- a bit like the decorator in decorator pattern (not to be confused with Python decorators). Rather than synthesize a unique string as a dictionary key, I just used the AddOn classes themselves as keys. This works fine for object instances, but gets hairy once classes come into play. ( http://pypi.python.org/pypi/AddOns#class-add-ons - an orthogonal alternative to writing hairy metaclasses with registries for special methods, persisted attributes, and all other sorts of things one would ordinarily use metaclasses for.) In principle, I could refactor AddOns to use synthetic (i.e. made-up) strings as keys, but it honestly seemed unpythonic to me to make up a key when the One Obvious key to use is the AddOn type itself. (Or in some cases, a tuple comprised of an AddOn type plus additional values - which would mean string manipulation for every access.) Another possible solution would be to not store addons directly in a class' dictionary, but instead throw in an __addons__ key with a subdictionary; again this seemed like pointless indirection, wasted memory and access time when there's already a perfectly good dictionary lying about. IOW, it's one of those places where Python's simple orthogonality seems like a feature rather than a bug that needs fixing. I mean, next thing you know, people will be saying that *instance* dictionaries need to have only string keys or something. ;-) Of course, if my library has to change to be able to work on 3.3, then I guess it'll have to change. IIRC, this is *probably* the only place I'm using non-string keys in type or instance dictionaries, so in the big scheme of porting costs, it's not that much. But, since you asked, that's the main use case I know of for non-string keys in type dictionaries, and I wouldn't be terribly surprised if I'm the only person with public code that does this. ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
On Wed, Mar 7, 2012 at 5:42 PM, Benjamin Peterson benja...@python.org wrote: 2012/3/7 Victor Stinner victor.stin...@gmail.com: So my question is: what is the use case of such dict? Why do we still support it? Probably a side-effect of implementation. Can't we simply raise an error if the dict contains non-string keys? Sounds okay to me. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com I think the original reason given was that enforcing the type would make a performance hit, but I might be misremembering. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Non-string keys in type dict
PJ Eby wrote: Short version: AddOns are things you can use to dynamically extend instances -- a bit like the decorator in decorator pattern (not to be confused with Python decorators). Rather than synthesize a unique string as a dictionary key, I just used the AddOn classes themselves as keys. This works fine for object instances, but gets hairy once classes come into play. Are you able to modify classes after class creation in Python 3? Without using a metaclass? ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
