[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Eric Snow]

On Fri, Feb 4, 2022, 16:03 Guido van Rossum  wrote:

> I wonder if a better solution than that PR wouldn't be to somehow change
> the implementation of _Py_IDENTIFIER() to do that,
>

Yeah, I had the same realization today.  I'm going to try it out.

-eric

>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A7Q4TBBOCEAXZYOY6GSY3NA2FSVNUMHL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PyPy on PySide6 is there: PyPy with a Gui

[Guido van Rossum]

Congrats Christian! It sounds like this will open new avenues for PyPy (and
maybe also for Qt/PySide6).

--Guido

On Thu, Feb 3, 2022 at 4:25 AM Christian Tismer-Sperling <
tis...@stackless.com> wrote:

> Hi Guido et. al.,
>
> since May 2021 I have been working at running PyPy on PySide6,
> which was a difficult undertaking, since PyPy internally is quite
> a bit different.
>
> I declared the project to be ready-to-use when the Mandelbrot
> example of the PySide examples
>  (examples/corelib/threads/mandelbrot.py)
> is working.
>
> This was finally solved this week on 2022-02-01, so we have the
>
>  first advanced Gui working with PyPy
>
> and with the amazing result of speed:
>
> PyPy 3.8 works
>  10 times faster than the identical code on Python 3.10
> and
>  5.5 times slower than the same example in C++ Qt.
>
> I think to send an official announce when this is available on pip.
>
> This effort marks the completion of my PyPy support, which began
> in 2003 and ended involuntarily in 2006 due to a stroke.
>
> All the best -- Chris
> --
> Christian Tismer-Sperling:^)   tis...@stackless.com
> Software Consulting  : http://www.stackless.com/
> Strandstraße 37  : https://github.com/PySide
> 24217 Schönberg  : GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KNPEW27BRBSLAJ2URIIWWNYLI5VIMHY3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

[Guido van Rossum]

Trying to cut things short, there's one thing I'd like to correct:

On Thu, Feb 3, 2022 at 9:15 AM Victor Stinner  wrote:

> [...]
>
> Another example is that Cython currently calls PyCode_New() to create
> a fake frame object with a filename and line number. IMO it's the
> wrong abstraction level: Python should provide a function to create a
> frame with a filename and line number, so the caller doesn't have to
> bother about the complex PyCode_New() API and frequent PyCodeObject
> changes. (Correct me if this problem has already been solved in
> Python.)
>

That was solved quite a while ago, with the PyCode_NewEmpty() API. Sadly
Cython doesn't call it (or at least not always), because it takes a C
string which is turned into a unicode object, and Cython already has the
unicode object in hand. I don't want to proliferate APIs.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VRC3PHFUOKOFPHPK5THRC4A7JA6GG7ZH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Guido van Rossum]

You *can* allocate unicode objects statically. We do it in deepfreeze, and
Eric's PR under discussion here (
https://github.com/python/cpython/pull/30928) does it. I wonder if a better
solution than that PR wouldn't be to somehow change the implementation of
_Py_IDENTIFIER() to do that, and make the special 'Id' APIs just aliases
for the corresponding unicode-object-based APIs? It wouldn't be ABI
compatible, but none of those APIs are in the stable ABI.

(Is there a requirement that an Id only contain ASCII characters (i.e.,
7-bit)?)

On Fri, Feb 4, 2022 at 12:52 PM Steve Dower  wrote:

> On 2/4/2022 5:37 PM, Eric Snow wrote:
> > On Thu, Feb 3, 2022 at 3:49 PM Eric Snow 
> wrote:
> >> I suppose I'd like to know what the value of _Py_IDENTIFIER() is for
> >> 3rd party modules.
> >
> > Between Guido, Victor, Stefan, and Sebastian, I'm getting the sense
> > that a public replacement for _Py_IDENTIFER() would be worth pursuing.
> > Considering that it would probably help numpy move toward
> > subinterpreter support, I may work on this after all. :)
> >
> > (For core CPython we'll still benefit from the statically initialized
> > strings, AKA gh-30928.)
>
> For me, I'd love to see a statically allocated string type (for a real
> example that's used in Windows, check out [1], specifically when he gets
> to the fast-pass strings).
>
> Essentially, a bare minimum struct around a char* (and/or wchar_t*) that
> acts as a PyUnicodeObject but doesn't ever allocate anything on the
> heap. This would also be helpful for config strings, which are often
> static but need to be copied around a lot, and good for passthrough
> strings that a native extension or host app might insert and receive
> back unmodified.
>
> Because there's nothing to deallocate, it can be "created" and stored
> however the caller likes. As soon as Python code does anything with it
> other than passing it around, a regular PyUnicodeObject is allocated
> (just like the HSTRING example).
>
> I'd expect usage to look very much like the intern examples earlier in
> the thread, but if we actually return a whole struct then we aren't on
> the hook for the allocations.
>
> Cheers,
> Steve
>
> [1]: https://devblogs.microsoft.com/oldnewthing/20160615-00/?p=93675
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/DGX4GBMDJYYFAE7OSVMBGKYAO2HPP3PT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7LGVQRWLQQJNQD5M5BEUAPAFBCQTUWK7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

[Victor Stinner]

On Fri, Feb 4, 2022 at 12:52 AM Eric V. Smith  wrote:
>
> On 2/3/2022 12:15 PM, Victor Stinner wrote:
> >
> > IMO if PyObject* becomes a handle, the migration to the HPy API should
> > be much easier.
>
> It seems to me that moving PyObject* to be a handle leaves you in a
> place very similar to HPy. So why not just focus on making HPy suitable
> for developing C extensions, leave the existing C API alone, and
> eventually abandon the existing C API?

I tried to explain the reasons why HPy doesn't solve all problems in
the PEP 674:
https://www.python.org/dev/peps/pep-0674/#the-c-api-is-here-is-stay-for-a-few-more-years

One problem is to provide a better C API to users: HPy is great for that!

Another problem is the inability to evolve Python because the C API
leaks implementation details: HPy doesn't solve this problem because
Python must continue supporting the C API for a few more years.

My approach is to (slowly) bend the C API towards HPy design/API to
ease the migration to HPy *and* (slowly) allow changing more Python
internals (without affecting the public C API).

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BIAGMMRJP45FR3R5DS772TZZU6AQVO2V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Steve Dower]

On 2/4/2022 5:37 PM, Eric Snow wrote:

On Thu, Feb 3, 2022 at 3:49 PM Eric Snow  wrote:

I suppose I'd like to know what the value of _Py_IDENTIFIER() is for
3rd party modules.


Between Guido, Victor, Stefan, and Sebastian, I'm getting the sense
that a public replacement for _Py_IDENTIFER() would be worth pursuing.
Considering that it would probably help numpy move toward
subinterpreter support, I may work on this after all. :)

(For core CPython we'll still benefit from the statically initialized
strings, AKA gh-30928.)


For me, I'd love to see a statically allocated string type (for a real 
example that's used in Windows, check out [1], specifically when he gets 
to the fast-pass strings).


Essentially, a bare minimum struct around a char* (and/or wchar_t*) that 
acts as a PyUnicodeObject but doesn't ever allocate anything on the 
heap. This would also be helpful for config strings, which are often 
static but need to be copied around a lot, and good for passthrough 
strings that a native extension or host app might insert and receive 
back unmodified.


Because there's nothing to deallocate, it can be "created" and stored 
however the caller likes. As soon as Python code does anything with it 
other than passing it around, a regular PyUnicodeObject is allocated 
(just like the HSTRING example).


I'd expect usage to look very much like the intern examples earlier in 
the thread, but if we actually return a whole struct then we aren't on 
the hook for the allocations.


Cheers,
Steve

[1]: https://devblogs.microsoft.com/oldnewthing/20160615-00/?p=93675
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DGX4GBMDJYYFAE7OSVMBGKYAO2HPP3PT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

[Brett Cannon]

On Thu, Feb 3, 2022 at 3:53 PM Eric V. Smith  wrote:

> On 2/3/2022 12:15 PM, Victor Stinner wrote:
> > I'm working bottom-to-top: prepare PyObject and PyVarObject to become
> > opaque, *and* top-to-bottom: prepare subclasses (structures
> > "inheriting" from PyObject and PyVarObject) to become opaque like
> > PyFrameObject.
> >
> > IMO if PyObject* becomes a handle, the migration to the HPy API should
> > be much easier.
>
> It seems to me that moving PyObject* to be a handle leaves you in a
> place very similar to HPy. So why not just focus on making HPy suitable
> for developing C extensions, leave the existing C API alone, and
> eventually abandon the existing C API?
>

I think that's a possibility. I think it's a question for the team here
whether that's the long-term goal that we want. If so we can make all of
our work head towards that and help out HPy out as best we can.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/US5U3STNFQGWLWVD6PLK6RDHYG5SH3Z3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Stefan Behnel]

Eric Snow schrieb am 04.02.22 um 17:35:

On Fri, Feb 4, 2022 at 8:21 AM Stefan Behnel wrote:

Correct. We (intentionally) have our own way to intern strings and do not
depend on CPython's identifier framework.


You're talking about __Pyx_StringTabEntry (and __Pyx_InitString())?


Yes, that's what we generate. The C code parsing is done here:

https://github.com/cython/cython/blob/79637b23da77732e753b1e1ab5669b3e29978be3/Cython/Compiler/Code.py#L531-L550

The deduplication is a bit complex on our side because it needs to handle 
Python source encodings, and also distinguishes between identifiers (that 
become 'str' in Py2), plain Unicode strings and byte strings. You don't 
need most of that for plain C code. But it's done here:


https://github.com/cython/cython/blob/79637b23da77732e753b1e1ab5669b3e29978be3/Cython/Compiler/Code.py#L1009-L1088

And then there's a whole bunch of code that helps in getting Unicode 
character code points and arbitrary byte values in very long strings pushed 
through C compilers, while keeping it mostly readable for interested users. :)


https://github.com/cython/cython/blob/master/Cython/Compiler/StringEncoding.py

You probably don't need that either, as long as you only deal with ASCII 
strings.


Any way, have fun. Feel free to ask if I can help.

Stefan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QHJBAKIQUKFPIM6GZ7DYNJF3HDMDQQUH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2022-01-28 - 2022-02-04)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7144 (-13)
  closed 51222 (+84)
  total  58366 (+71)

Open issues with patches: 2890 


Issues opened (44)
==

#45711: Simplify the interpreter's (type, val, tb) exception represent
https://bugs.python.org/issue45711  reopened by vstinner

#45773: Compiler hangs during jump elimination
https://bugs.python.org/issue45773  reopened by brandtbucher

#46570: Windows support for OpenSSL 3.0
https://bugs.python.org/issue46570  opened by jay0lee

#46571: Strange `@typing.no_type_check` behavior for class variables
https://bugs.python.org/issue46571  opened by sobolevn

#46575: One-off errors in hashlib.scrypt error messages
https://bugs.python.org/issue46575  opened by ron_kaminsky

#46577: Hostname spoofing via backslashes in URL
https://bugs.python.org/issue46577  opened by meetdash

#46578: cant DEBUG os.spawnv()
https://bugs.python.org/issue46578  opened by michaellongge

#46580: email.utils.unquote strips too many slashes
https://bugs.python.org/issue46580  opened by hbandi

#46581: _typevar_types and _paramspec_tvars are missing from _GenericA
https://bugs.python.org/issue46581  opened by posita

#46585: Should we re-export `PyObj_FromPtr` in `ctypes`?
https://bugs.python.org/issue46585  opened by sobolevn

#46586: In documentation contents enum.property erroneously links to b
https://bugs.python.org/issue46586  opened by Dutcho

#46587: datetime and time tests use non-portal "%4Y" format
https://bugs.python.org/issue46587  opened by christian.heimes

#46589: Improve documentation for typing._GenericAlias
https://bugs.python.org/issue46589  opened by matthew.rahtz

#46592: Undocumented behavior in strptime for ISO week dates
https://bugs.python.org/issue46592  opened by Jonatan Skogsfors

#46593: memoryview lacks support for half floats
https://bugs.python.org/issue46593  opened by pitrou

#46594: Windows "Edit with IDLE >" only has one selection
https://bugs.python.org/issue46594  opened by terry.reedy

#46596: PyLineTable_InitAddressRange isn't exported - causing C Extens
https://bugs.python.org/issue46596  opened by nathan3

#46598: ElementTree: wrong XML prolog for the utf-8-sig encoding
https://bugs.python.org/issue46598  opened by prikryl

#46600: Python built with clang -O0 allocates 10x more stack memory th
https://bugs.python.org/issue46600  opened by vstinner

#46601: macOS installer "Install Certificates.command" fails if pip is
https://bugs.python.org/issue46601  opened by cryptophoto

#46603: `typing._strip_annotations` is not fully covered
https://bugs.python.org/issue46603  opened by sobolevn

#46604: Documentation fix in ssl module
https://bugs.python.org/issue46604  opened by glk0

#46605: Py_XDECREF() module on fail in Py_mod_exec
https://bugs.python.org/issue46605  opened by ov2k

#46606: Large C stack usage of os.getgroups() and os.setgroups()
https://bugs.python.org/issue46606  opened by methane

#46607: Add DeprecationWarning to configparser's LegacyInterpolation
https://bugs.python.org/issue46607  opened by hugovk

#46608: Exclude marshalled-frozen data if deep-freezing to save 300 KB
https://bugs.python.org/issue46608  opened by kumaraditya303

#46609: Generator-based coroutines in Python 3.10 docs
https://bugs.python.org/issue46609  opened by srittau

#46611: Improve coverage of `__instancecheck__` and `__subclasscheck__
https://bugs.python.org/issue46611  opened by sobolevn

#46613: Add PyType_GetModuleByDef to the public & limited API
https://bugs.python.org/issue46613  opened by petr.viktorin

#46614: Add option to output UTC datetimes as "Z" in `.isoformat()`
https://bugs.python.org/issue46614  opened by p-ganssle

#46615: Use-after-free by mutating set during set operations
https://bugs.python.org/issue46615  opened by Dennis Sweeney

#46619: lazy module property not recognized by doctests
https://bugs.python.org/issue46619  opened by jaraco

#46620: Documentation of ipaddress behavior for prefix length with lea
https://bugs.python.org/issue46620  opened by lay20114

#46621: Should map(function, iterable, ...) replace StopIteration with
https://bugs.python.org/issue46621  opened by xavieryao

#46622: Support  decorating a coroutine with functools.cached_property
https://bugs.python.org/issue46622  opened by uranusjr

#46623: test_zlib: test_pair() and test_speech128() fail with s390x ha
https://bugs.python.org/issue46623  opened by vstinner

#46625: timeout option of socket.create_connection is not respected
https://bugs.python.org/issue46625  opened by Nicolas SURRIBAS

#46631: Implement a "strict" mode for getpass.getuser()
https://bugs.python.org/issue46631  opened by eryksun

#46632: test_ssl: 2 tests fail on cstratak-CentOS9-fips-x86_64
https://bugs.python.org/issue46632  opened by vstinner

#46633: AddressSanitizer: Skip tests 

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Eric Snow]

On Thu, Feb 3, 2022 at 3:49 PM Eric Snow  wrote:
> I suppose I'd like to know what the value of _Py_IDENTIFIER() is for
> 3rd party modules.

Between Guido, Victor, Stefan, and Sebastian, I'm getting the sense
that a public replacement for _Py_IDENTIFER() would be worth pursuing.
Considering that it would probably help numpy move toward
subinterpreter support, I may work on this after all. :)

(For core CPython we'll still benefit from the statically initialized
strings, AKA gh-30928.)

-eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/D2LHEZZUQH66Q5ZIOEJTGSCEMQEMKCUQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Eric Snow]

On Fri, Feb 4, 2022 at 8:21 AM Stefan Behnel  wrote:
> Correct. We (intentionally) have our own way to intern strings and do not
> depend on CPython's identifier framework.

You're talking about __Pyx_StringTabEntry (and __Pyx_InitString())?

-eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Q5AL3SLW5BCUA6FLDBUNZTH5Z7ZYAHER/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Sebastian Berg]

On Thu, 2022-02-03 at 15:32 +0100, Victor Stinner wrote:
> By the way, Argument Clinic now produces way faster calling
> conventions than hand-written METH_VARARGS with PyArg_ParseTuple().
> It
> would be make this tool available to 3rd party projects.
> 
> Either extract it and put it on PyPI, but it means that Python will
> need to Python and pip install something to build itself... not good?
> 
> Or we can add it to stdlib. IMO we need to write more tests and more
> documentation. Right now, test_clinic is "light". Another issue is
> that it requires on the currently privte _PyArg_Parser structure. By
> the way, this structure is causing crashes if sub-interpreters are
> run
> in parallel (one GIL per interpreter) because of the
> _PyArg_Parser.kwtuple tuple of keyword parameter names (tuple of
> str).
> 
> If Eric's tool becomes successful inside CPython, we may consider to
> make it available to 3rd party projects as well. Such tool and
> Argument Clinic and not so different than Cython which generates C
> code to ease the development of C extensions modules.


It would be great to have a clear API available to store constants like
this, for NumPy these currently are:
* mainly interned strings
* Some custom singletons which are currently defined in Python
  (including exceptions, but not so important)
* Python functions that are called from C.

I am pretty sure that some of the places that do not have module state
available, but likely many (most?) of them could be refactored (e.g. a
converter function that needs a singleton: it isn't terrible to
manually converter later on).  Many will be in methods.


It would be great to have a "blessed" solution for kwarg parsing.  I
had shunned argument clinic because it felt too internal, probably
CPython specific, and it seemed like you basically need to check in the
generated code for each supported Python version...

The current solution in NumPy [1] is a bit limited and uses those
statics to store the interned strings. And it duplicates Python logic
that would be nice to not duplicate.

Cheers,

Sebastian



[1] For the curious, the API for NumPy looks like this:

/* 
 * Declare a function static struct for string creation/interning
 * and caching other prep (generated on the first call.
 * `npy_parse_arugments` is a macro, but only to insert the cache name)
 */
NPY_PREPARE_ARGPARSER;

if (npy_parse_arguments(funcname, args, len_args, kwnames
"", NULL, _only_obj,
"kw1", , _val,
"|kw2", NULL, _obj,
"$kw_only1", NULL, _only2_obj,
NULL, NULL, NULL) < 0) {
return NULL;
}

So it is very limited to converters (e.g. no "i" or "!O" convenience),
since we don't use that much anyway.
One annoyance is that "i" and "s" don't have a clear "converter" and
that converters can't raise an error message that includes the function
and parameter name.


> 
> Victor
> 
> On Thu, Feb 3, 2022 at 7:50 AM Inada Naoki 
> wrote:
> > 
> > +1 for overall.
> > 
> > On Thu, Feb 3, 2022 at 7:45 AM Eric Snow <
> > ericsnowcurren...@gmail.com> wrote:
> > > 
> > > 
> > > I'd also like to actually get rid of _Py_IDENTIFIER(), along with
> > > other related API including ~14 (private) C-API functions. 
> > > Dropping
> > > all that helps reduce maintenance costs.  However, at least one
> > > PyPI
> > > project (blender) is using _Py_IDENTIFIER().  So, before we could
> > > get
> > > rid of it, we'd first have to deal with that project (and any
> > > others).
> > > 
> > 
> > It would be nice to provide something similar to _PY_IDENTIFIER,
> > but
> > designed (and documented) for 3rd party modules like this.
> > 
> > ```
> > typedef struct {
> >     Py_IDENTIFIER(foo);
> > ...
> > } Modstate;
> > ...
> >     // in some func
> >     Modstate *state = (Modstate*)PyModule_GetState(module);
> >     PyObject_GetAttr(o, PY_IDENTIFIER_STR(state->foo));
> > ...
> > // m_free()
> > static void mod_free(PyObject *module) {
> >     Modstate *state = (Modstate*)PyModule_GetState(module);
> >     Py_IDENTIFIER_CLEAR(state->foo);
> > }
> > ```
> > 
> > 
> > --
> > Inada Naoki  
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at 
> > https://mail.python.org/archives/list/python-dev@python.org/message/ZZ5QOZDOAO734SDRJGMXW6AJGAVEPUHE/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> 
> 
> 


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PBBH2RXKZO2HYWYLNZU5VJ6B7BJSABKZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

[Ethan Furman]

On 2/4/22 6:23 AM, Stefan Behnel wrote:

> One recent example is the new error locations in tracebacks, where PEP 657 
explicitly lists
> the new "co_positions" field in code objects as an implementation detail of 
CPython. If we
> want to implement this in Cython, then there is no other way than to copy 
these implementation
> details pretty verbatimly from CPython and to depend on them.
>
> https://www.python.org/dev/peps/pep-0657/
>
> In this specific case, we're lucky that this can be considered an entirely 
optional feature that we can separately
> disable when users request "public API" mode (let's call it that). Not sure 
if that's what users want, though.

Speaking as a user, I would want Cython to be (in order of preference):

- reliable
- fast
- all the cool Python features

and Python to:

- make public APIs available for Python features

--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LER6WQX565PNDJYWSZI2KCSXDN7MM624/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

[Antonio Cuni]

On Fri, Feb 4, 2022 at 12:55 AM Eric V. Smith  wrote:

> It seems to me that moving PyObject* to be a handle leaves you in a
> place very similar to HPy. So why not just focus on making HPy suitable
> for developing C extensions, leave the existing C API alone, and
> eventually abandon the existing C API?
>

I agree (but I'm biased :)), but I think there is also an important point
which is easy to miss/overlook: it is not enough to declare that now you
have handles instead of refcounting, you also need a way to enforce/check
that the handles are used correctly.

CPython might declare that object references are now handles and that each
handle must be closed individually: this would work formally, but as long
as handles are internally implemented on top of refcounting, things like
closing the same handle twice would just continue to work if by chance the
total refcount is still correct. This means that we will have extensions
which will be formally incorrect but will work well on CPython, and
horribly break as soon as you try to load them on e.g. PyPy.

That's the biggest selling point of the HPy debug mode: in debug mode, HPy
actively check that handles are closed properly, and it warns you if you
close a handle twice or forget to close a handle, even on CPython.

ciao,
Antonio
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EDMW4QYG4N7M6JIMB42RPLFNGJV5IUQ4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Stefan Behnel]

Ronald Oussoren via Python-Dev schrieb am 03.02.22 um 14:46:

On 2 Feb 2022, at 23:41, Eric Snow wrote:
* a little less convenient: adding a global string requires modifying
a separate file from the one where you actually want to use the string
* strings can get "orphaned" (I'm planning on checking in CI)
* some strings may never get used for any given ./python invocation
(not that big a difference though)


The first two cons can probably be fixed by adding some indirection, with some
markers at the place of use and a script that uses those to generate the
C definitions.

Although my gut feeling is that adding a the CI check you mention is good
enough and adding the tooling for generating code isn’t worth the additional
complexity.


It's what we do in Cython, and it works really well there. It's very 
straight forward, you just write something like


PYUNICODE("some text here")
PYIDENT("somename")

in your C code and Cython creates a deduplicated global string table from 
them and replaces the string constants with the corresponding global 
variables. (We have two different names because an identifier in Py2 is 
'str', not 'unicode'.)


Now, the thing with CPython is that the C sources where the replacement 
would take place are VCS controlled. And a script that replaces the 
identifiers would have to somehow make sure that the new references do not 
get renamed, which would lead to non-local changes when strings are added.


What you could try is to number the identifiers, i.e. use a macro like

_Py_STR(123, "some text here")

where you manually add a new identifier as

_Py_STR("some text here")

and the number is filled in automatically by a script that finds all of 
them, deduplicates, and adds new identifiers at the end, adding 1 to the 
maximum number that it finds. That makes sure that identifiers that already 
have an ID number will not be touched, deleted strings disappear 
automatically, and non-local changes are prevented.


Defining the _Py_STR() macro as

   #define _Py_STR(id, text)  (_Py_global_string_table[id])

or

   #define _Py_STR(id, text)  (_Py_global_string_table##id)

would also give you a compile error if someone forgets to run the script.

Stefan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LD3JM2NQ5ZUZDK63RH4IVZPCZ7HC4X3G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

[Stefan Behnel]

Petr Viktorin schrieb am 03.02.22 um 13:47:

On 02. 02. 22 11:50, Stefan Behnel wrote:
Maybe we should advertise the two modes more. And make sure that both 
work. There are certainly issues with the current state of the "limited 
API" implementation, but that just needs work and testing.


I wonder if it can it be renamed? "Limited API" has a specific meaning 
since PEP 384, and using it for the public API is adding to the general 
confusion in this area :(


I was more referring to it as an *existing* compilation mode of Cython that 
avoids the usage of CPython implementation details. The fact that the 
implementation is incomplete just means that we spill over into non-limited 
API code when no limited API is available for a certain feature. That will 
usually be public API code, unless that is really not available either.


One recent example is the new error locations in tracebacks, where PEP 657 
explicitly lists the new "co_positions" field in code objects as an 
implementation detail of CPython. If we want to implement this in Cython, 
then there is no other way than to copy these implementation details pretty 
verbatimly from CPython and to depend on them.


https://www.python.org/dev/peps/pep-0657/

In this specific case, we're lucky that this can be considered an entirely 
optional feature that we can separately disable when users request "public 
API" mode (let's call it that). Not sure if that's what users want, though.


Stefan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A55HYBIFBOTAX5IB4YUYWUHI3IDLRD2F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Stefan Behnel]

Victor Stinner schrieb am 03.02.22 um 22:46:

Oh right, Cython seems to be a false positive.

A code search found 3 references to __Pyx_PyObject_LookupSpecial():

PYPI-2022-01-26-TOP-5000/Cython-0.29.26.tar.gz:
Cython-0.29.26/Cython/Compiler/ExprNodes.py: lookup_func_name =
'__Pyx_PyObject_LookupSpecial'
PYPI-2022-01-26-TOP-5000/Cython-0.29.26.tar.gz:
Cython-0.29.26/Cython/Compiler/Nodes.py: code.putln("%s =
__Pyx_PyObject_LookupSpecial(%s, %s); %s" % (
PYPI-2022-01-26-TOP-5000/Cython-0.29.26.tar.gz:
Cython-0.29.26/Cython/Utility/ObjectHandling.c: static CYTHON_INLINE
PyObject* __Pyx_PyObject_LookupSpecial(PyObject* obj, PyObject*
attr_name) {

Oh, that's not "_PyObject_LookupSpecial()", it doesn't use the
_Py_Identifier type:

static CYTHON_INLINE PyObject*
__Pyx_PyObject_LookupSpecial(PyObject* obj, PyObject* attr_name)
{ ... }


Correct. We (intentionally) have our own way to intern strings and do not 
depend on CPython's identifier framework.


Stefan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4ATP4FSVRNI5CLAJDN43QRDH5IHW7BW2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PyPy on PySide6 is there: PyPy with a Gui

[Christian Tismer-Sperling]

On 04.02.22 08:52, Christopher Barker wrote:

This is very cool Chris -- thanks!

One question:

and with the amazing result of speed:

PyPy 3.8 works
      10 times faster than the identical code on Python 3.10
and
      5.5 times slower than the same example in C++ Qt.


Is this primarily Python-QT interaction? or computing the mandelbrot 
set, for which I would expect to see performance numbers like that.


This is purely the calculation speed of Mandelbrot, run in a Gui,
with the working horse being Python vs. PyPy. Qt only acts as
painter after each generation or when zooming.

Since the numbers for a pure Qt application were interesting too,
this was also given. But the showcase was "look ma, how fast it
is, and it works in a nice Gui".


Anyway, really cool in any case -- a major step for PyPy.



Yes, I think so, too. The existence of a Gui for PyPy
might create much interest for both projects.

Cheers - Chris

--
Christian Tismer-Sperling:^)   tis...@stackless.com
Software Consulting  : http://www.stackless.com/
Strandstraße 37  : https://github.com/PySide
24217 Schönberg  : GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FQHJUDH6HWGWKKSDD224CTLKBQQ2GZ4H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Victor Stinner]

On Thu, Feb 3, 2022 at 11:49 PM Eric Snow  wrote:
> I suppose I'd like to know what the value of _Py_IDENTIFIER() is for
> 3rd party modules.

_Py_IDENTIFIER() API is simple and short, Python creates the Python
string object automatically, *and* clears this object at exit. It's
convenient.

On the other side, functions like PyDict_GetItemString() call
PyUnicode_FromString(): it allocates memory on the heap, it decodes
the bytes string from UTF-8, and then deallocates the memory. It's
inefficient just for a dict lookup.

In the _Py_IDENTIFIER() API, _PyUnicode_FromId() calls
PyUnicode_InternInPlace() which makes dict lookup even more efficient.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZYONQJPBC7IEIJVUTUBHNZXVOV7YOBUF/
Code of Conduct: http://python.org/psf/codeofconduct/