[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-21 Thread Antoine Pitrou
On Fri, 21 Jun 2019 12:22:21 +0900 Inada Naoki wrote: > On Fri, Jun 21, 2019 at 1:28 AM Victor Stinner wrote: > > > > Le jeu. 20 juin 2019 à 11:15, Inada Naoki a écrit > > : > > > Can we change _PyUnicode_FromId to use _PyUnicode_FromASCII? > > > > How would a developer detect a mistake

[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-21 Thread Inada Naoki
OK. I start optimizing PyUnicode_GetString() already. It was 2x slower than _PyUnicode_FromASCII. But it can be only 1.5x slower than _PyUnicode_FromASCII. And as a bonus, `b"foo".decode()` become 10% faster too. -- Inada Naoki ___ Python-Dev

[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-21 Thread Serhiy Storchaka
20.06.19 19:28, Victor Stinner пише: Le jeu. 20 juin 2019 à 11:15, Inada Naoki a écrit : Can we change _PyUnicode_FromId to use _PyUnicode_FromASCII? How would a developer detect a mistake (non-ASCII) character? Does _PyUnicode_FromASCII() raise an exception, even in release mode? The

[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-20 Thread David Mertz
Ok. Good point. I used Unicode identifiers in my own project, not in CPython. Some Greek letters that represent common mathematical concepts. On Thu, Jun 20, 2019, 11:27 PM Inada Naoki wrote: > On Fri, Jun 21, 2019 at 6:55 AM David Mertz wrote: > > > > This change would break two lovely

[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-20 Thread Inada Naoki
On Fri, Jun 21, 2019 at 6:55 AM David Mertz wrote: > > This change would break two lovely functions I wrote this week. > Are your lovely functions in CPython, or in your project? _Py_Identifier is still private and undocumented. But I expect someone start using it outside of CPython. So I

[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-20 Thread Inada Naoki
On Fri, Jun 21, 2019 at 1:28 AM Victor Stinner wrote: > > Le jeu. 20 juin 2019 à 11:15, Inada Naoki a écrit : > > Can we change _PyUnicode_FromId to use _PyUnicode_FromASCII? > > How would a developer detect a mistake (non-ASCII) character? Does > _PyUnicode_FromASCII() raise an exception, even

[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-20 Thread Inada Naoki
On Fri, Jun 21, 2019 at 1:23 AM Steve Dower wrote: > > What benefit would this provide? It is faster, of course. The whole benefit will be not significant for now. But I'm thinking of making _Py_Identifier public (CPython API) in the future. If we make it public, breaking change is hard after

[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-20 Thread David Mertz
This change would break two lovely functions I wrote this week. On Thu, Jun 20, 2019, 12:35 PM Steve Dower wrote: > On 20Jun2019 0205, Inada Naoki wrote: > > Hi, all. > > > > Both of PyUnicdoe_FromString() and _Py_Identifier are > > supporting UTF-8 for now. We can not change

[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-20 Thread Victor Stinner
Le jeu. 20 juin 2019 à 11:15, Inada Naoki a écrit : > Can we change _PyUnicode_FromId to use _PyUnicode_FromASCII? How would a developer detect a mistake (non-ASCII) character? Does _PyUnicode_FromASCII() raise an exception, even in release mode? The function is only called once (that's the

[Python-Dev] Re: _Py_Identifier should support non-ASCII string?

2019-06-20 Thread Steve Dower
On 20Jun2019 0205, Inada Naoki wrote: Hi, all. Both of PyUnicdoe_FromString() and _Py_Identifier are supporting UTF-8 for now. We can not change PyUnicode_FromString() for backward compatibility, so we have _PyUnicode_FromASCII instead. But _Py_Identifier is still private. And non-ASCII