[issue13821] misleading return from isidentifier

[R. David Murray]

R. David Murray added the comment:

See also issue 30772 about the deeper problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[STINNER Victor]

Changes by STINNER Victor :


--
nosy:  -haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[R. David Murray]

R. David Murray added the comment:

IMO allow_non_nfkc=True that just returns False would be a bad idea, since as 
Benjamin points out it *is* a valid identifier, it's just not normalized (yet). 
 Raising might work, that way you could tell the difference, but that would be 
a weird API for such a check function.  Regardless, we should probably keep 
this issue to a doc patch, and open a new issue for any proposed enhancement 
request.  

And you probably want to discuss it on python-ideas first, since the underlying 
issue is a bit complex and the solution non-obvious, with possible knock-on 
effects.  (Or maybe I'm wrong and the consensus will be that returning False 
with that flag would be fine.)

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[Matthias Bussonnier]

Matthias Bussonnier added the comment:

I have been bitten by that as well. I think the doc should mention to verify 
that the given string is normalized, not that it **should** be normalized.

Agreed that If isidentifier could also possibly grow a `allow_non_nfkc=True` 
default parameter that would allow to deactivate internal normalisation and 
return False/Raise on Non NKFC that would be great. 

I'm also interested on having an option on ast.parse or compile to not 
normalize to at least be able to lint wether users are using non NFKC form, but 
that's another issue.

I'll see if I can come up with – at least – a documentation patch.

--
nosy: +mbussonn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python, haypo, serhiy.storchaka
stage:  -> needs patch
type:  -> behavior
versions: +Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[Jim Jewett]

New submission from Jim Jewett jimjjew...@gmail.com:

Python identifiers are in NFKC form; string method .isidentifier() returns true 
on strings that are not in that form.  In some contexts, these non-canonical 
strings will be replaced with their NFKC equivalent, but in other contexts 
(such as the builtins hasattr, getattr, delattr) they will not.


 cha=chr(170)
 cha
'ª'

 cha.isidentifier()
True

 uc.normalize(NFKC, cha)
'a'

 obj.ª = 5
 hasattr(obj, ª)
False
 obj.a
5

--
components: Unicode
messages: 151597
nosy: Jim.Jewett, ezio.melotti
priority: normal
severity: normal
status: open
title: misleading return from isidentifier

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13821
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[Benjamin Peterson]

Benjamin Peterson benja...@python.org added the comment:

I don't see why that's invalid. str.isidentifier() returning True means 
Python will accept it as an identifier.

--
nosy: +benjamin.peterson

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13821
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[Jim Jewett]

Jim Jewett jimjjew...@gmail.com added the comment:

My preference would be for non_NFKC.isidentifier() to return False, but that 
may be a problem for backwards compatibility.

It *may* be worth adding an asidentifier() method that returns either False or 
the canonicalized string that should be used instead.

At a minimum, the documentation (including docstring) should warn that the 
method doesn't check for NFKC form, and that if the input is not ASCII, the 
caller should first ensure this by calling str1=unicodedata.normalize(NFKC, 
str1)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13821
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[Benjamin Peterson]

Benjamin Peterson benja...@python.org added the comment:

2012/1/18 Jim Jewett rep...@bugs.python.org:

 Jim Jewett jimjjew...@gmail.com added the comment:

 My preference would be for non_NFKC.isidentifier() to return False

It *is* an identifier, though. Python will happily accept it.


 It *may* be worth adding an asidentifier() method that returns either False 
 or the canonicalized string that should be used instead.

 At a minimum, the documentation (including docstring) should warn that the 
 method doesn't check for NFKC form, and that if the input is not ASCII, the 
 caller should first ensure this by calling str1=unicodedata.normalize(NFKC, 
 str1)

Sounds fine to me.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13821
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[Jim Jewett]

Jim Jewett jimjjew...@gmail.com added the comment:

@Benjamin -- the catch is, if it isn't already in NFKC form, then python won't 
really accept it as an identifier.  Sometimes it will silently canonicalize it 
for you so that it seems to work, but other times it won't.  And program 
calling isidentifier is likely to be a program that uses the strings directly 
for access, instead of always routing them through the parser.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13821
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13821] misleading return from isidentifier

[Benjamin Peterson]

Benjamin Peterson benja...@python.org added the comment:

2012/1/18 Jim Jewett rep...@bugs.python.org:

 Jim Jewett jimjjew...@gmail.com added the comment:

 @Benjamin -- the catch is, if it isn't already in NFKC form, then python 
 won't really accept it as an identifier.  Sometimes it will silently 
 canonicalize it for you so that it seems to work, but other times it won't.  
 And program calling isidentifier is likely to be a program that uses the 
 strings directly for access, instead of always routing them through the 
 parser.

AFAIK, the only time it will silently canonicalize it for you is
parsing. Even if it wasn't, you can't say it's not an identifier, it's
just not normalized.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13821
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com