[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-07-03 Thread Victor Stinner
Sorry, ignore my comment: Cython no longer uses PyUnicode_FromString(NULL, 0) in the master branch. The change was backported to the 0.29.x branch, but this stable branch requires a second fix, so I wrote it: https://github.com/cython/cython/pull/3721 Victor Le ven. 3 juil. 2020 à 16:00, Victor

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-07-03 Thread Victor Stinner
Le sam. 13 juin 2020 à 12:39, Inada Naoki a écrit : > Of course, there is an API to create an empty string: PyUnicode_New(0, 0); > But since Cython is using `PyUnicode_FromString(NULL, 0)`, > keep it working for some versions will mitigate the breaking change. > Note that we can remove wchar_t

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-17 Thread Inada Naoki
On Mon, Jun 15, 2020 at 4:25 PM Serhiy Storchaka wrote: > > I have a plan for more graduate removing of this feature. I created a PR > which adds several compile options, so Python can be built in one of > three modes: > > 1. Support wchar_t* cache and use it. It is the current mode. > > 2.

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-16 Thread Inada Naoki
On Wed, Jun 17, 2020 at 4:16 AM Steve Dower wrote: > > On 16Jun2020 1641, Inada Naoki wrote: > > * This change doesn't affect to pure Python packages. > > * Most of the rest uses Cython. Since I already report an issue to Cython, > >regenerating with new Cython release fixes them. > > The

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-16 Thread Steve Dower
On 16Jun2020 1641, Inada Naoki wrote: * This change doesn't affect to pure Python packages. * Most of the rest uses Cython. Since I already report an issue to Cython, regenerating with new Cython release fixes them. The precedent set in our last release with tp_print was that regenerating

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-16 Thread Brett Cannon
Inada Naoki wrote: > On Tue, Jun 16, 2020 at 9:30 PM Victor Stinner vstin...@python.org wrote: > > > > Le mar. 16 juin 2020 à 10:42, Inada Naoki songofaca...@gmail.com a écrit : > > Hmm, Is there any chance to add > > DeprecationWarning in Python 3.9? > > In my experience, more and more projects

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-16 Thread Inada Naoki
On Tue, Jun 16, 2020 at 9:30 PM Victor Stinner wrote: > > Le mar. 16 juin 2020 à 10:42, Inada Naoki a écrit : > > Hmm, Is there any chance to add DeprecationWarning in Python 3.9? > > In my experience, more and more projects are running their test suite > with -Werror, which is a good thing.

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-16 Thread Victor Stinner
Le mar. 16 juin 2020 à 10:42, Inada Naoki a écrit : > Hmm, Is there any chance to add DeprecationWarning in Python 3.9? In my experience, more and more projects are running their test suite with -Werror, which is a good thing. Introducing a new warning is likely to "break" many of these

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-16 Thread Antoine Pitrou
On Mon, 15 Jun 2020 11:22:09 +0100 Mark Shannon wrote: > > I don't like this approach. > Adding compile time options means we need to test more versions, but is > no help to end users as they will end up with the release version anyway. I agree with Mark. This sounds less pointless

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-16 Thread Inada Naoki
On Tue, Jun 16, 2020 at 12:35 AM Victor Stinner wrote: > > Hi INADA-san, > > IMO Python 3.11 is too early because we don't emit a > DeprecationWarning on every single deprecation function. > > 1) Emit a DeprecationWarning at runtime (ex: Python 3.10) > 2) Wait two Python releases: see >

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-15 Thread Victor Stinner
Hi INADA-san, IMO Python 3.11 is too early because we don't emit a DeprecationWarning on every single deprecation function. 1) Emit a DeprecationWarning at runtime (ex: Python 3.10) 2) Wait two Python releases: see https://discuss.python.org/t/pep-387-backwards-compatibilty-policy/4421 3) Remove

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-15 Thread Mark Shannon
Hi Serhiy, On 15/06/2020 8:22 am, Serhiy Storchaka wrote: 12.06.20 11:32, Inada Naoki пише: Hi, all. Py_UNICODE has been deprecated since PEP 393 (Flexible string representation). wchar_t* cache in the string object is used only in deprecated APIs. It waste 1 word (8 bytes on 64bit

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-15 Thread Kyle Stanley
> I'm sorry, I was wrong. Py_DEPRECATED(3.3) is commented out for some APIs. So Python 3.8 doesn't show warning for them. Ah, no problem. Thanks for checking up on that. > I still think 2 years are enough to removal. Hmm, okay. At the least though, it does mean we have to be a bit more

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-15 Thread Serhiy Storchaka
12.06.20 11:32, Inada Naoki пише: Hi, all. Py_UNICODE has been deprecated since PEP 393 (Flexible string representation). wchar_t* cache in the string object is used only in deprecated APIs. It waste 1 word (8 bytes on 64bit machine) per string instance. The deprecated APIs are documented as

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-14 Thread Inada Naoki
On Sat, Jun 13, 2020 at 8:20 PM Inada Naoki wrote: > > 2020年6月13日(土) 20:12 Kyle Stanley : >> >> > Additionally, raise DeprecationWarning runtime when these APIs are used. >> >> So, just to clarify, current usage of these 7 unicode APIs does not emit any >> warnings and would only start doing so

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-13 Thread Kyle Stanley
> They have been deprecated in C already. Compiler emits warning. > This additional proposal is adding runtime warning before removal. Oh, sorry. I misunderstood the previous statement then. In that case, I think scheduling the removal for 3.11 is perfectly reasonable if the compiler warnings

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-13 Thread Inada Naoki
2020年6月13日(土) 20:12 Kyle Stanley : > > Additionally, raise DeprecationWarning runtime when these APIs are used. > > So, just to clarify, current usage of these 7 unicode APIs does not emit > any warnings and would only start doing so in 3.10? > They have been deprecated in C already. Compiler

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-13 Thread Kyle Stanley
> Additionally, raise DeprecationWarning runtime when these APIs are used. So, just to clarify, current usage of these 7 unicode APIs does not emit any warnings and would only start doing so in 3.10? If so, I think we may want to consider giving users until 3.12 until they're removed. Especially

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-13 Thread Inada Naoki
On Fri, Jun 12, 2020 at 5:32 PM Inada Naoki wrote: > > > My proposal is, schedule the removal on Python 3.11. But we will postpone > the removal if we can not remove its usage until it. > Additionally, raise DeprecationWarning runtime when these APIs are used. -- Inada Naoki

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-13 Thread Inada Naoki
On Sat, Jun 13, 2020 at 1:36 AM MRAB wrote: > > > * Most of them are `PyUnicode_FromUnicode(NULL, 0);` > >* We may be able to keep PyUnicode_FromUnicode, but raise error when > > length>0. > > > I think it would be strange to keep PyUnicode_FromUnicode but complain > unless length == 0. If

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-12 Thread MRAB
On 2020-06-12 09:32, Inada Naoki wrote: Hi, all. Py_UNICODE has been deprecated since PEP 393 (Flexible string representation). wchar_t* cache in the string object is used only in deprecated APIs. It waste 1 word (8 bytes on 64bit machine) per string instance. The deprecated APIs are

[Python-Dev] Re: When can we remove wchar_t* cache from string?

2020-06-12 Thread Mark Shannon
Hi, A big +1 to exposing fewer internals of the PyUnicodeObject to C code. Ultimately, making PyUnicodeObject immutable to C code would be a real bonus. It would make the code cleaner, safer and faster. A triple win! I don't think removing Py_UNICODE API be sufficient for that, have you