[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Steve Dower]

Steve Dower  added the comment:


New changeset 3a8fdb28794b2f19f6c8464378fb8b46bce1f5f4 by Alex Gaynor in branch 
'master':
bpo-41784: make PyUnicode_AsUTF8AndSize part of the limited API (GH-22252)
https://github.com/python/cpython/commit/3a8fdb28794b2f19f6c8464378fb8b46bce1f5f4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Steve Dower]

Steve Dower  added the comment:

Thanks, Alex!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Steve Dower]

Steve Dower  added the comment:

Agreed that there's no way we can make Py_buffer part of the limited ABI.

I just looked over the PR and it's missing a What's New entry (e.g. 
https://docs.python.org/3/whatsnew/3.9.html#c-api-changes). Other than that, 
looks fine to me.

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Alex Gaynor]

Alex Gaynor  added the comment:

It's a big project I think :-) Py_Buffer is allocated on the stack, so either 
we'd have to agree to never change it's ABI (size, alignment, etc.) or we'd 
need to completely change the interface.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Oh, would not be worth to add Py_buffer to the limited API?

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Alex Gaynor]

Alex Gaynor  added the comment:

Py_buffer is not part of the limited API at all, so I don't think it's usable 
for this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I agree about PyUnicode_AsUTF8.

But I think it would be worth to ask PyPy team about PyUnicode_AsUTF8AndSize.

An alternate C API is PyUnicode_GetUTF8Buffer (issue39087). It requires 
explicit releasing the buffer after use, so it can be used even on 
implementations with moving garbage collector.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Inada Naoki]

Inada Naoki  added the comment:

PyUnicode_AsUTF8 is useful "API". But it can be implemented as C macro, C 
inline function, or functions/macros in any other languages using 
PyUnicode_AsUTF8AndSize.

PyUnicode_AsUTF8AndSize is more important for "ABI".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PyUnicode_AsUTF8() is used 3 times more than PyUnicode_AsUTF8AndSize().

$ find -type f -name '*.c' -exec egrep 'PyUnicode_AsUTF8AndSize\(' '{}' + | wc 
-l
35
$ find -type f -name '*.c' -exec egrep 'PyUnicode_AsUTF8\(' '{}' + | wc -l
101

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Alex Gaynor]

Alex Gaynor  added the comment:

I think less is more, one API is plenty :-)

It looks to me like the API is already supported on PyPy, so I think it's fine 
from that perspective: 
https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.7/pypy/module/cpyext/unicodeobject.py#L493

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

What about PyUnicode_AsUTF8? Should it be made public too or left for internal 
use only?

What about third-party implementations of Python? How hard to implement this 
API on an implementation without reference counts? It is interesting to hear 
the expert opinion of the core developers of PyPy.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Alex Gaynor]

Change by Alex Gaynor :


--
keywords: +patch
pull_requests: +21307
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22252

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Alex Gaynor]

Change by Alex Gaynor :


--
assignee:  -> alex

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Inada Naoki]

Change by Inada Naoki :


--
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Inada Naoki]

Inada Naoki  added the comment:

+1. It is a very important API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41784] Promote PyUnicode_AsUTF8AndSize to be available with the limited API (PEP 384)

[Alex Gaynor]

New submission from Alex Gaynor :

This function is incredibly useful for efficient interoperability between 
Python and other languages with UTF-8 based strings (e.g. Rust). Right now it's 
not possible to do interop without several copies/allocations if you're trying 
to build an abi3 wheel. This is tactically relevant to me here: 
https://github.com/PyO3/pyo3/issues/1125

This API has been stable since it was introduced in Python 3.1, so I think 
making it stable would be appropriate.

Inada, Benjamin suggested I should ask you for your feedback on this. I'm happy 
to send a pull request.

--
components: C API
keywords: easy (C)
messages: 376896
nosy: alex, benjamin.peterson, inada.naoki
priority: normal
severity: normal
status: open
title: Promote PyUnicode_AsUTF8AndSize to be available with the limited API 
(PEP 384)
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com