New submission from Steve B <>:

Here is an example involving the unicode character MIDDLE DOT · : The line

ab·cd = 7

is valid Python 3 code and is happily accepted by the CPython interpreter. 
However, does not like it. It says that the middle-dot is an error 
token. Here is an example you can run to see that:

    import tokenize
    from io import BytesIO
    test = 'ab·cd = 7'.encode('utf-8')
    x = tokenize.tokenize(BytesIO(test).readline)
    for i in x: print(i)

For reference, the official definition of identifiers is:

and details about MIDDLE DOT are at

MIDDLE DOT has the "Other_ID_Continue" property, so I think the interpreter is 
behaving correctly (i.e. consistent with the documented spec), while is wrong.

components: Library (Lib), Unicode
messages: 313168
nosy: ezio.melotti, steve, vstinner
priority: normal
severity: normal
status: open
title: parses unicode identifiers incorrectly
type: behavior
versions: Python 3.6

Python tracker <>
Python-bugs-list mailing list

Reply via email to