[Python-Dev] Re: Suggestion: a little language for type definitions

[h . vetinari]

> While the SC's decision to keep the syntax uniform is certainly laudable, 
> it's creating the issue of packaging new complexities into a very limited 
> syntactic & semantic space (e.g. no new magic symbols like "->", which I 
> agree with BTW), leaving only very verbose solutions that the typing crowd is 
> chafing against.

Seems that I unintentionally ended up jinxing the callable type syntax PEP - I 
must have misread some of the discussions up until that point, thinking that 
"->" had been already ruled out - sorry! 

> I think accepting that typing has a syntactic cost on the python language as 
> a whole is unavoidable at some point (and I'm not saying that's a bad thing). 
> Having a separate & opt-in mini-language for type declarations seems like a 
> really clean way to delineate that cost resp. extension, and I especially 
> like the t''-string syntax.

In light of the rejection of that PEP, I think this point is worth revisiting 
(in due time).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BKMUZVICTAOZXS6QA2XXO7YMIFBYWB3E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] PEP 677 (Callable Type Syntax): Rejection notice.

[Gregory P. Smith]

Thank you PEP authors for producing a very well written and laid out PEP.
That made it easy to understand the proposal, rationale, and alternatives
considered. We agree that the existing typing.Callable syntax, while
capable, is less than ideal for humans. That said, we deliberated last week
and ultimately decided to reject PEP-677 Callable Type Syntax.

Why? Several considerations led us here:

1. We feel we need to be cautious when introducing new syntax. Our new
parser presents understandably exciting opportunities but we don’t want its
existence to mean we add new syntax easily. A feature for use only in a
fraction of type annotations, not something every Python user uses, did not
feel like a strong enough reason to justify the complexity needed to parse
this new syntax and provide meaningful error messages. Not only code
complexity, humans are also parsers that must look forwards and backwards.

2. While the current Callable[x, y] syntax is not loved, it does work. This
PEP isn’t enabling authors to express anything they cannot already. The PEP
explicitly chose be conservative and not allow more syntax to express
features going beyond what Callable supports. We applaud that decision,
starting simple is good. But we can imagine a future where the syntax would
desire to be expanded upon.

3. In line with past SC guidance, we acknowledge challenges when syntax
desires do not align between typing and Python itself. Each time we add
syntax solely for typing it shifts us further in the direction of typing
being its own mini-language so we aim to tread lightly in what gets added
here. Adopting PEP 677 would lock us into its syntax forever, potentially
preventing other future syntax directions.

4. We did not like the visual and cognitive consequence of multiple `->`
tokens in a def. Especially when code is not formatted nicely. Though we
admit the correlation between Python typing and formatter users is high.

## Recommendation for future syntax enhancements:

When proposing a syntax change, low complexity is better. While not always
possible, it’s ideal if it could still be described using our old <=3.8
parser. It is important to have in mind that adding syntax that only our
modern PEG parser can handle could lead to greater cognitive load and
external tooling implementation costs.

This should not be read as a ban on PEG-only syntax, we just think it
should be used for broadly applicable features or else be relatively
unintrusive.

Thanks,
The 3.11 Python Steering Council
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NHCLHCU2XCWTBGF732WESMN42YYVKOXB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

[Inada Naoki]

On Thu, Feb 10, 2022 at 3:49 AM Brett Cannon  wrote:
> On Wed, Feb 9, 2022 at 4:19 AM Petr Viktorin  wrote:
>> On 09. 02. 22 4:39, h.vetin...@gmx.com wrote:
>>
>> That's an interesting idea -- what's keeping us from C11?
>
> No one asking before, probably because we have been trying to get to C99 for 
> so long. 
>
>> In other words: the main thing keeping us from C99 is MSVC support, and
>> since that compiler apparently skipped C99, should we skip it as well?
>
> If we think "C11 without optional features" is widely supported then I think 
> that's a fine target to have.
>
> For anyone not sure what's optional in C11, I found 
> https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_features 
> . Other than atomics being discussed on Discord for mimalloc, leaving those 
> things out seem reasonable to me.
>

I like it. I want to use anonymous union. It makes complex structure
like PyDictKeysObject simple a little.

I confirmed that XLC supports it.
https://www.ibm.com/docs/en/xl-c-and-cpp-aix/13.1.3?topic=types-structures-unions#strct__anonstruct

Regards,
-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MMTUDLFIB7ET6T3Q73HDLODDGZY74X54/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP-657 and co_positions (was: Please update Cython *before* introcuding C API incompatible changes in Python)

[Stefan Behnel]

Andrew Svetlov schrieb am 09.02.22 um 19:40:

Stefan, do you really need to emulate call stack with positions?
Could the __note__ string with generated Cython part of exception traceback
solve your needs (https://www.python.org/dev/peps/pep-0678/) ?


Thanks for the link, but I think it would be surprising for users if a 
traceback displayed some code positions differently than others, when all 
code lines refer to Python code.


Stefan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BSDVX7MJFDZ6PFB7FG7Z3R4IO56FZ47T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API

[Gregory P. Smith]

On Wed, Feb 9, 2022 at 8:54 AM Victor Stinner  wrote:

> On Wed, Feb 9, 2022 at 1:04 PM Petr Viktorin  wrote:
> > > Right now, a large number of macros cast their argument to a type. A
> > > few examples:
> > >
> > > #define PyObject_TypeCheck(ob, type)
> > > PyObject_TypeCheck(_PyObject_CAST(ob), type)
> > > #define PyTuple_GET_ITEM(op, i) (_PyTuple_CAST(op)->ob_item[i])
> > > #define PyDict_GET_SIZE(mp)  (assert(PyDict_Check(mp)),((PyDictObject
> > > *)mp)->ma_used)
> >
> > When I look at the Rationale points, and for the first three of these
> > macros I didn't find any that sound very convincing:
> > - Functions don't have macro pitfalls, but these simple macros don't
> > fall into the pits.
> > - Fully defining the argument types means getting rid of the cast,
> > breaking some code that uses the macro
> > - Debugger support really isn't that useful for these simple macros
> > - There are no new variables
>
> Using a static inline function, profilers like Linux perf can count
> the CPU time spend in static inline functions (on each CPU instruction
> when using annotated assembly code). For example, you can see how much
> time (accumulated time) is spent in Py_INCREF(), to have an idea of
> the cost of reference counting in Python. It's not possible when using
> macros.
>
> For debuggers, you're right that Py_INCREF() and PyTuple_GET_ITEM()
> macros are very simple and it's not so hard to guess that the debugger
> is executing their code in the C code or the assembly code. But the
> purpose of PEP 670 is to convert way more complex macros. I wrote a PR
> to convert unicodeobject.h macros, IMO there are one of the worst
> macros in Python C API:
> https://github.com/python/cpython/pull/31221
>
> I always wanted to convert them, but some core devs were afraid of
> performance regressions. So I wrote a PEP to prove that there is no
> impact on performance.
>
> IMO the new unicodeobject.h code is way more readable. I added two
> "kind" variables which have a well defined scope. In macros, no macro
> is used currently to avoid macro pitfalls: name conflict if there is
> already a "kind" macro where the macro is used.
>
> The conversion to static inline macros also detected a bug with "const
> PyObject*": using PyUnicode_READY() on a const Unicode string is
> wrong, this function modifies the object if it's not ready (WCHAR
> kind). It also detected bugs on "const void *data" used to *write*
> into string characters (PyUnicode_WRITE).
>

Nice example PR.  For now what we're suggesting is to proceed with changes
where it cannot lead to compilation warnings (failures in the widely used
-Werror mode) being introduced.  Yes, that may still leave a lot of
desirable cleanup to be done.  But we don't want to block everything while
discussing the rest. Incremental improvement is good even if not everything
is being done yet.

Two differing examples from that PR 31221:

Hold off as unsafe for now: Macros that do things like
(PyWhateverObject*)(op) such as PyUnicode_CHECK_INTERNED(op) should not
have the casting part of macros replaced yet. Doing so could result in a
warning or failure at -Werror compile time if someone was not using a
PyObject*.  Code is *supposed* to, but the cast means anyone could've used
PyUnicodeObject or whatnot itself.  Perhaps use a hybrid approach when
feasible similar to:
  #define PyUnicode_CHECK_INTERNED(op)
_PyUnicode_CHECK_INTERNED((PyASCIIObject *)(op))

That should make it safe.  And I believe you do mention this technique as
something to be used in the PEP.

Safe: PyUnicode_WRITE() in that PR. At first glance that is full of casts
on its data and value parameters so it raises suspicion. But further
analysis shows that data becomes a void* so there is no casting issue there
unless someone were passing a non-pointer in which isn't rightfully
something code should *ever* have done. And value would only be an issue if
someone were passing a non-integer type in, that also seems exceedingly
unlikely as there'd never be a point in writing code like that. So that
kind of change is fine to proceed with.

> The "massive change to working code" part is important. Such efforts
> > tend to have unexpected issues, which have an unfortunate tendency to
> > surface before release and contribute to release manager burnout.
>
> Aren't you exaggerating a bit? Would you mind to elaborate? Do you
> have examples of issues caused by converting macros to static inline
> functions?
>

Not quite the same but I've got a related example similar to what macros
casting pointer becoming a function accepting PyObject* without a cast
*could* do:

Between 3.6 and 3.7 we added const to a number of our public Python C APIs.

Migration to 3.7 required updating all sorts of C and C++ extension module
code to be pedantically correct, up through its call chain in some cases
with matching const declarations on types. (including conditional
compilation based on the Python version to support compilation under both
during 

[Python-Dev] Re: Request for feedback: pathlib.AbstractPath prototype

[Barney Gale]

Penny for your thoughts on those questions, Brett? Protocols are new to me.
I see importlib.abc.Traversable is a Protocol, and I'm giving PEP 544 a
read now.

I reckon only stat(), open() and iterdir() would be essential for users to
implement.

On Wed, 9 Feb 2022 at 19:02, Brett Cannon  wrote:

> One thing to discuss (and which has been brought up on the PR), is whether
> this should be an ABC to force people to explicitly raise
> `NotImplementedError`?
>
> The next question is whether any of this should be a (very wide) protocol
> instead of an ABC?
>
> On Wed, Feb 9, 2022 at 7:05 AM Barney Gale  wrote:
>
>> Over the last couple of years I've been tidying up the pathlib internals,
>> with a view towards adding an AbstractPath class to the hierarchy. Users
>> would be able to subclass AbstractPath to implement other kinds of
>> filesystems: s3, google cloud storage, pandas, ftp, git, zip and tar files,
>> etc. By implementing some abstract methods (stat(), iterdir(), open(), etc)
>> they'd benefit from a number of derived methods (is_dir(), glob(),
>> read_text(), etc). There's already a healthy ecosystem of PyPI packages
>> attempting this, but there's presently no officially-supported route.
>>
>> I've now submitted a PR that adds an initial underscore-prefixed
>> implementation of this class:
>> https://github.com/python/cpython/pull/31085. The implementation is
>> simple: wherever Path calls functions in os, io, pwd, etc, AbstractPath
>> instead raises NotImplementedError. The Path class becomes an
>> implementation of AbstractPath.
>>
>> These methods directly raise NotImplementedError: cwd(), stat(),
>> iterdir(), readlink(), resolve(), expanduser(), owner(), group(), open(),
>> touch(), mkdir(), symlink_to(), hardlink_to(), rename(), replace(),
>> chmod(), unlink(), rmdir()
>>
>> These methods call through to the above methods: absolute(), lstat(),
>> exists(), is_dir(), is_file(), is_mount(), is_symlink(), is_block_device(),
>> is_char_device(), is_fifo(), is_socket(), home(), samefile(), scandir(),
>> glob(), rglob(), read_bytes(), read_text(), write_bytes(), write_text(),
>> lchmod(), link_to()
>>
>> Some methods aren't applicable to some kinds of filesystems, e.g. zip
>> files don't support symlinks, working directories or home directories. In
>> these cases I think it's reasonable to raise NotImplementedError. Indeed,
>> pathlib.Path methods already raise NotImplementedError when certain local
>> filesystem features aren't available (readlink(), group(), etc).
>>
>> If any readers of this list have previously tried to extend pathlib to
>> other domains, or are otherwise interested in pathlib development, please
>> could you let me know what you think of the proposed API?
>>
>> Many thanks
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/JB7QGDNI2CNXFX7LQQ2X2WPOZ7DWVNQL/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FMAN7TQA7LU4PYWW7NKA54ZWNWC6ARMU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP-657 and co_positions (was: Please update Cython *before* introcuding C API incompatible changes in Python)

[Stefan Behnel]

Guido van Rossum schrieb am 09.02.22 um 19:36:

On Wed, Feb 9, 2022 at 9:41 AM Pablo Galindo Salgado wrote:

On Wed, 9 Feb 2022 at 17:38, Stefan Behnel wrote:

Pablo Galindo Salgado schrieb am 09.02.22 um 17:40:

Should there be a getter/setter for co_positions?


We consider the representation of co_postions private


Yes, and that's the issue.


I can only say that currently, I am not confident to expose such an API,
at least for co_positions, as the internal implementation is very likely to
heavily change and we want to have the possibility of changing it between
patch versions if required (to address bugs and other things like that).

>
> It might require a detailed API design proposal coming from outside
> CPython
> (e.g. from Cython) to get this to change. I imagine for co_positions in
> particular this would have to use a "builder" pattern.
>
> I am unclear on how this would work though, given that Cython generates C
> code, not CPython bytecode. How would the synthesized co_positions be
> used?
> Would Cython just generate a co_positions fragment at the moment an
> exception is raised, pointing at the .pyx file from which the code was
> generated?

So, what we currently do is to update the line number (which IIRC is really 
the start line number of the current function) on the current frame when an 
exception is raised, and the byte code offset to 0. That's a hack but shows 
the correct code line in the traceback. Probably conflicts with pdb, but 
there are still other issues with that anyway.


I remember looking into the old lnotab mapping at some point and trying to 
implement that with fake byte code offsets but never got it finished.


The idea is pretty simple, though. Instead of byte code offsets, we'd count 
our syntax tree nodes and just store the code position range of each syntax 
node at the "byte code offset" of the node's counter number. That's 
probably fairly easy to do in C code, maybe even with a statically 
allocated data structure. Then, instead of setting the frame function's 
line number, we'd set the frame's byte code instruction counter to the 
number of the failing syntax node, and CPython would retrieve the code 
position from that offset.


That sounds simple enough, probably simpler than any API usage – but 
depends on implementation details.


Especially the idea of storing all this statically in the data segment of 
the shared library sounds very tempting.


Stefan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GAJFB6ABFYXF3RFXFDQ3YUZD23FMXPEY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for feedback: pathlib.AbstractPath prototype

[Brett Cannon]

One thing to discuss (and which has been brought up on the PR), is whether
this should be an ABC to force people to explicitly raise
`NotImplementedError`?

The next question is whether any of this should be a (very wide) protocol
instead of an ABC?

On Wed, Feb 9, 2022 at 7:05 AM Barney Gale  wrote:

> Over the last couple of years I've been tidying up the pathlib internals,
> with a view towards adding an AbstractPath class to the hierarchy. Users
> would be able to subclass AbstractPath to implement other kinds of
> filesystems: s3, google cloud storage, pandas, ftp, git, zip and tar files,
> etc. By implementing some abstract methods (stat(), iterdir(), open(), etc)
> they'd benefit from a number of derived methods (is_dir(), glob(),
> read_text(), etc). There's already a healthy ecosystem of PyPI packages
> attempting this, but there's presently no officially-supported route.
>
> I've now submitted a PR that adds an initial underscore-prefixed
> implementation of this class: https://github.com/python/cpython/pull/31085.
> The implementation is simple: wherever Path calls functions in os, io, pwd,
> etc, AbstractPath instead raises NotImplementedError. The Path class
> becomes an implementation of AbstractPath.
>
> These methods directly raise NotImplementedError: cwd(), stat(),
> iterdir(), readlink(), resolve(), expanduser(), owner(), group(), open(),
> touch(), mkdir(), symlink_to(), hardlink_to(), rename(), replace(),
> chmod(), unlink(), rmdir()
>
> These methods call through to the above methods: absolute(), lstat(),
> exists(), is_dir(), is_file(), is_mount(), is_symlink(), is_block_device(),
> is_char_device(), is_fifo(), is_socket(), home(), samefile(), scandir(),
> glob(), rglob(), read_bytes(), read_text(), write_bytes(), write_text(),
> lchmod(), link_to()
>
> Some methods aren't applicable to some kinds of filesystems, e.g. zip
> files don't support symlinks, working directories or home directories. In
> these cases I think it's reasonable to raise NotImplementedError. Indeed,
> pathlib.Path methods already raise NotImplementedError when certain local
> filesystem features aren't available (readlink(), group(), etc).
>
> If any readers of this list have previously tried to extend pathlib to
> other domains, or are otherwise interested in pathlib development, please
> could you let me know what you think of the proposed API?
>
> Many thanks
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JB7QGDNI2CNXFX7LQQ2X2WPOZ7DWVNQL/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A3P646PI6U27UNXETNR7X76NGKSX6OI4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

[Pablo Galindo Salgado]

That is on pourpose and is the public API for Python. In Python it returns
an iterable of tuples,
which is processed from the actual internal form.

On Wed, 9 Feb 2022 at 18:56, Victor Stinner  wrote:

> On Wed, Feb 9, 2022 at 5:48 PM Pablo Galindo Salgado
>  wrote:
> > We consider the representation of co_postions private, so we don't want
> (for now) to ad
> > getters/setters. If you want to get the position of a instruction, you
> can use PyCode_Addr2Location
>
> The code.co_positions() method is accessible in Python: it's not
> documented, but its name doesn't say that it's private. Was it done on
> purpose? Should it renamed to _co_positions() or even be removed?
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MPQ2XT5HYTLCJ3DEH4EH7QP7BEZ57OCN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should we require IEEE 754 floating-point for CPython?

[Brett Cannon]

On Tue, Feb 8, 2022 at 10:20 PM Steven D'Aprano  wrote:

> On Tue, Feb 08, 2022 at 05:48:46PM -0800, Gregory P. Smith wrote:
> > On Tue, Feb 8, 2022 at 2:41 PM Steven D'Aprano 
> wrote:
>
> > > If the answer to those questions are Yes, that rules out using Unums,
> > > posits, sigmoid numbers etc as the builtin float. (The terminology is a
> > > bit vague, sorry.) Do we want that?
> >
> > It does not rule anything else out should they become viable.  This is
> just
> > a statement that to build cpython we require ieee754 support.  It does
> not
> > say anything about how our Python float type is implemented internally.
>
> Posits do not implement IEEE-754. They aren't merely a different
> internal representation of the IEEE-754 floating point standard, they
> implement a different numeric system altogether.
>
> Things that IEEE-754 require, such as signed zero and signed infinity,
> are not supported by posits. Posits include a single unsigned zero and a
> single unsigned infinity.
>
> Earlier, I made a mistake: I misremembered that posits support a single
> NAN, and so I removed Mark's question about requiring NANs from my
> quoting. I was wrong to do so: posits do not have any NANs.
>
> So if we require NANs, or IEEE-754, that rules out posits as the builtin
> float.
>

As Greg said, all we are discussing is requiring NaN and IEEE-754 in order
to compile CPython. There's nothing saying we can't change how things are
implemented in CPython in the future if something else made sense. And if
someone wants to fork CPython to implement something else they can as well.
And none of this has any bearing on the language based on the trend of this
conversation's answer to Mark's third question. So I am not seeing how any
decision we make from this thread about CPython and IEEE-754 rules anything
out in the future?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6LYTRCA3JGLAX4H4GOOMKCXCYZVPJLEE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

[Victor Stinner]

On Wed, Feb 9, 2022 at 5:48 PM Pablo Galindo Salgado
 wrote:
> We consider the representation of co_postions private, so we don't want (for 
> now) to ad
> getters/setters. If you want to get the position of a instruction, you can 
> use PyCode_Addr2Location

The code.co_positions() method is accessible in Python: it's not
documented, but its name doesn't say that it's private. Was it done on
purpose? Should it renamed to _co_positions() or even be removed?

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TWATPIZVTXWS224A3Z4VITNU5PKWSEHV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

[Victor Stinner]

Hi,

It's already possible to call PyObject_CallMethod(code,
"co_positions", NULL) and then use the iterator in C. Is there an use
case where performance of reading co_positions is critical? If not,
there is no need to add a specialized function.

Victor

On Wed, Feb 9, 2022 at 5:23 PM Petr Viktorin  wrote:
>
> On 04. 02. 22 15:23, Stefan Behnel wrote:
> > Petr Viktorin schrieb am 03.02.22 um 13:47:
> >> On 02. 02. 22 11:50, Stefan Behnel wrote:
> >>> Maybe we should advertise the two modes more. And make sure that both
> >>> work. There are certainly issues with the current state of the
> >>> "limited API" implementation, but that just needs work and testing.
> >>
> >> I wonder if it can it be renamed? "Limited API" has a specific meaning
> >> since PEP 384, and using it for the public API is adding to the
> >> general confusion in this area :(
> >
> > I was more referring to it as an *existing* compilation mode of Cython
> > that avoids the usage of CPython implementation details. The fact that
> > the implementation is incomplete just means that we spill over into
> > non-limited API code when no limited API is available for a certain
> > feature. That will usually be public API code, unless that is really not
> > available either.
> >
> > One recent example is the new error locations in tracebacks, where PEP
> > 657 explicitly lists the new "co_positions" field in code objects as an
> > implementation detail of CPython. If we want to implement this in
> > Cython, then there is no other way than to copy these implementation
> > details pretty verbatimly from CPython and to depend on them.
> >
> > https://www.python.org/dev/peps/pep-0657/
> >
> > In this specific case, we're lucky that this can be considered an
> > entirely optional feature that we can separately disable when users
> > request "public API" mode (let's call it that). Not sure if that's what
> > users want, though.
>
> Should there be a getter/setter for co_positions?
> I'm unfortunately not aware of what Cython needs from code objects, but
> it might be good to extend the API here.
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/XK4DMU7I35TKXYJRYQE4RGMLNNBPDTYN/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/75CJVEZPQA7ARQDMW62WYYHCIGMNMDP4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

[Brett Cannon]

On Wed, Feb 9, 2022 at 4:19 AM Petr Viktorin  wrote:

>
>
> On 09. 02. 22 4:39, h.vetin...@gmx.com wrote:
> >> Maybe a more practical approach would be to use C99 "except of
> > features not supported by MSVC of Visual Studio 2019"?
> >
> > This could be formulated in a more neutral way by saying
> >
> > "C99 without the things that became optional in C11", or perhaps
>
> That sounds like a better wording!
>
> > "C11 without optional features" (at least from the POV of MSVC,
> > haven't checked the other compilers/platforms for C11-compliance).
>
> That's an interesting idea -- what's keeping us from C11?
>

No one asking before, probably because we have been trying to get to C99
for so long. 


> In other words: the main thing keeping us from C99 is MSVC support, and
> since that compiler apparently skipped C99, should we skip it as well?
>

If we think "C11 without optional features" is widely supported then I
think that's a fine target to have.

For anyone not sure what's optional in C11, I found
https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_features
. Other than atomics being discussed on Discord for mimalloc, leaving those
things out seem reasonable to me.

-Brett


>
>
> >> In practice, we can try to support VS 2017, the version currently
> > recommended by the devguide:
> >
> > That is becoming dated quickly, as Microsoft has deprecated, and is
> removing,
> > that version quite rapidly from their CI services (azure/GHA), i.e. mid
> March, see:
> > https://github.com/actions/virtual-environments/issues/4312.
> >
> > It's understandable in the sense that they don't want to support a third
> version
> > in addition to vs2022 and vs2019, but the net effect is that very few
> (open source)
> > projects will keep using vs2017 going forward.
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IIZ6LXK2MANUHZAMYSXDF5KPF3VIRKDJ/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ERQQFXNAGANR2ND3SRDVCIQZKYTT6OBM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SV7M2CXVULHVG4X6SXZV7ZR5Y4TD3IBR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP-657 and co_positions (was: Please update Cython *before* introcuding C API incompatible changes in Python)

[Andrew Svetlov]

Stefan, do you really need to emulate call stack with positions?
Could the __note__ string with generated Cython part of exception traceback
solve your needs (https://www.python.org/dev/peps/pep-0678/) ?

On Wed, Feb 9, 2022 at 7:46 PM Pablo Galindo Salgado 
wrote:

> I can only say that currently, I am not confident to expose such an API,
> at least for co_positions, as the internal implementation is very likely to
> heavily change and we want to have the possibility of changing it between
> patch versions if required (to address bugs and other things like that).
>
> On Wed, 9 Feb 2022 at 17:38, Stefan Behnel  wrote:
>
>> Pablo Galindo Salgado schrieb am 09.02.22 um 17:40:
>> >> Should there be a getter/setter for co_positions?
>> >
>> > We consider the representation of co_postions private
>>
>> Yes, and that's the issue.
>>
>>
>> > so we don't want (for now) to ad
>> > getters/setters. If you want to get the position of a instruction, you
>> can
>> > use PyCode_Addr2Location
>>
>> What Cython needs is the other direction. How can we provide the current
>> source position range for a given piece of code to an exception?
>>
>> As it stands, the way to do this is to copy the implementation details of
>> CPython into Cython in order to let it expose the specific data
>> structures
>> that CPython uses for its internal representation of code positions.
>>
>> I would prefer using an API instead that allows exposing this mapping
>> directly to CPython's traceback handling, rather than having to emulate
>> byte code positions. While that would probably be quite doable, it's far
>> from a nice interface for something that is not based on byte code.
>>
>> And that's not just a Cython issue. The same applies to Domain Specific
>> Languages or other programming languages that integrate with Python and
>> want to show users code positions for their source code.
>>
>> Stefan
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/VQSWX6MFKIA3RYPSX7O6RTVC422LTJH4/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/P7SMK5ZGFAHZMLUKW4WZNNX47CONXIQS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3RVEOQSOJRWVWEVXHM6NDW6WH7DPBGLF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP-657 and co_positions (was: Please update Cython *before* introcuding C API incompatible changes in Python)

[Guido van Rossum]

It might require a detailed API design proposal coming from outside CPython
(e.g. from Cython) to get this to change. I imagine for co_positions in
particular this would have to use a "builder" pattern.

I am unclear on how this would work though, given that Cython generates C
code, not CPython bytecode. How would the synthesized co_positions be used?
Would Cython just generate a co_positions fragment at the moment an
exception is raised, pointing at the .pyx file from which the code was
generated?

On Wed, Feb 9, 2022 at 9:41 AM Pablo Galindo Salgado 
wrote:

> I can only say that currently, I am not confident to expose such an API,
> at least for co_positions, as the internal implementation is very likely to
> heavily change and we want to have the possibility of changing it between
> patch versions if required (to address bugs and other things like that).
>
> On Wed, 9 Feb 2022 at 17:38, Stefan Behnel  wrote:
>
>> Pablo Galindo Salgado schrieb am 09.02.22 um 17:40:
>> >> Should there be a getter/setter for co_positions?
>> >
>> > We consider the representation of co_postions private
>>
>> Yes, and that's the issue.
>>
>>
>> > so we don't want (for now) to ad
>> > getters/setters. If you want to get the position of a instruction, you
>> can
>> > use PyCode_Addr2Location
>>
>> What Cython needs is the other direction. How can we provide the current
>> source position range for a given piece of code to an exception?
>>
>> As it stands, the way to do this is to copy the implementation details of
>> CPython into Cython in order to let it expose the specific data
>> structures
>> that CPython uses for its internal representation of code positions.
>>
>> I would prefer using an API instead that allows exposing this mapping
>> directly to CPython's traceback handling, rather than having to emulate
>> byte code positions. While that would probably be quite doable, it's far
>> from a nice interface for something that is not based on byte code.
>>
>> And that's not just a Cython issue. The same applies to Domain Specific
>> Languages or other programming languages that integrate with Python and
>> want to show users code positions for their source code.
>>
>> Stefan
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/VQSWX6MFKIA3RYPSX7O6RTVC422LTJH4/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/P7SMK5ZGFAHZMLUKW4WZNNX47CONXIQS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YJHFRW4FYIPMYWGX57D4XNV7FIZSBSTY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for feedback: pathlib.AbstractPath prototype

[Barney Gale]

On Wed, 9 Feb 2022 at 15:46, Ethan Furman  wrote:

> On 2/9/22 6:59 AM, Barney Gale wrote:
>
>  > I've now submitted a PR that adds an initial underscore-prefixed
> implementation of this class:
>
> Why the iniitial underscore?  What could change between now and the future?
>

A few things:

Firstly, I'd like to give PyPI package maintainers the opportunity to play
with the underscore-prefixed API and submit feedback, prior to us freezing
the interface. They're already doing naughty things subclassing the
(internal, recently removed) pathlib._Accessor class. I think this feedback
might identify the need for new OSError subclass representing EINVAL for
readlink() implementations, or a new user-friendly os.stat_result
constructor for use in stat(). Or perhaps not :-)

Secondly, there's still a couple of rough edges in the PurePath
implementation, particularly around certain operations being delegated to a
mix-in class (the "flavour"). I'd like to bring those into PurePath before
we unleash this on the world, but it can be worked around for the time
being.

Thirdly, I've yet to address the problem of how we supply things like
sockets or zipfile.ZipFile objects, and persist them to child objects
generated by iterdir() etc. It can be done already by creating a further
subclass with the state assigned as a class attribute, but that's a bit
weird I think!
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QOIDZYXZXQE27NG3J7UFHB6HQQKYIF4V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP-657 and co_positions (was: Please update Cython *before* introcuding C API incompatible changes in Python)

[Pablo Galindo Salgado]

I can only say that currently, I am not confident to expose such an API,
at least for co_positions, as the internal implementation is very likely to
heavily change and we want to have the possibility of changing it between
patch versions if required (to address bugs and other things like that).

On Wed, 9 Feb 2022 at 17:38, Stefan Behnel  wrote:

> Pablo Galindo Salgado schrieb am 09.02.22 um 17:40:
> >> Should there be a getter/setter for co_positions?
> >
> > We consider the representation of co_postions private
>
> Yes, and that's the issue.
>
>
> > so we don't want (for now) to ad
> > getters/setters. If you want to get the position of a instruction, you
> can
> > use PyCode_Addr2Location
>
> What Cython needs is the other direction. How can we provide the current
> source position range for a given piece of code to an exception?
>
> As it stands, the way to do this is to copy the implementation details of
> CPython into Cython in order to let it expose the specific data structures
> that CPython uses for its internal representation of code positions.
>
> I would prefer using an API instead that allows exposing this mapping
> directly to CPython's traceback handling, rather than having to emulate
> byte code positions. While that would probably be quite doable, it's far
> from a nice interface for something that is not based on byte code.
>
> And that's not just a Cython issue. The same applies to Domain Specific
> Languages or other programming languages that integrate with Python and
> want to show users code positions for their source code.
>
> Stefan
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VQSWX6MFKIA3RYPSX7O6RTVC422LTJH4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/P7SMK5ZGFAHZMLUKW4WZNNX47CONXIQS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

[Ethan Furman]

On 2/9/22 8:40 AM, Pablo Galindo Salgado wrote:
> Petr Viktorin wrote:

>> Should there be a getter/setter for co_positions?
>
> We consider the representation of co_postions private, so we don't want (for 
now) to ad
> getters/setters.

Isn't the whole point of getters/setters is to allow public access to information, allowing the internal representation 
of that information to change?  However the exception information is store by CPython, it's going to be needed by other 
frameworks.


--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OX4Q4MROMDNO5MVU35MJ2ARIH4RUB5KZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] PEP-657 and co_positions (was: Please update Cython *before* introcuding C API incompatible changes in Python)

[Stefan Behnel]

Pablo Galindo Salgado schrieb am 09.02.22 um 17:40:

Should there be a getter/setter for co_positions?


We consider the representation of co_postions private


Yes, and that's the issue.



so we don't want (for now) to ad
getters/setters. If you want to get the position of a instruction, you can
use PyCode_Addr2Location


What Cython needs is the other direction. How can we provide the current 
source position range for a given piece of code to an exception?


As it stands, the way to do this is to copy the implementation details of 
CPython into Cython in order to let it expose the specific data structures 
that CPython uses for its internal representation of code positions.


I would prefer using an API instead that allows exposing this mapping 
directly to CPython's traceback handling, rather than having to emulate 
byte code positions. While that would probably be quite doable, it's far 
from a nice interface for something that is not based on byte code.


And that's not just a Cython issue. The same applies to Domain Specific 
Languages or other programming languages that integrate with Python and 
want to show users code positions for their source code.


Stefan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VQSWX6MFKIA3RYPSX7O6RTVC422LTJH4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should we require IEEE 754 floating-point for CPython?

[Jim J. Jewett]

I think you skimmed over "A floating point expert can probably look at this ... 
"

I remember a time when I just assumed more bits was better, and a later time 
when I figured smaller was better, and a still later time when I wanted to 
match the published requirements for bitsize.  So that was several years when I 
didn't really understand the tradeoffs, but could benefit from (or at least 
write better documentation) knowing the size.  During those years, I would have 
recognized the importance of 1024, but would probably not have bothered 
interpreting 2.220446049250313.  

A method (or docstring) with a more friendly interface would be good.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/R4BW2ZL46Y23UYYQCOSWJ2B3KTSRO5LK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API

[Victor Stinner]

On Wed, Feb 9, 2022 at 1:04 PM Petr Viktorin  wrote:
> > Right now, a large number of macros cast their argument to a type. A
> > few examples:
> >
> > #define PyObject_TypeCheck(ob, type)
> > PyObject_TypeCheck(_PyObject_CAST(ob), type)
> > #define PyTuple_GET_ITEM(op, i) (_PyTuple_CAST(op)->ob_item[i])
> > #define PyDict_GET_SIZE(mp)  (assert(PyDict_Check(mp)),((PyDictObject
> > *)mp)->ma_used)
>
> When I look at the Rationale points, and for the first three of these
> macros I didn't find any that sound very convincing:
> - Functions don't have macro pitfalls, but these simple macros don't
> fall into the pits.
> - Fully defining the argument types means getting rid of the cast,
> breaking some code that uses the macro
> - Debugger support really isn't that useful for these simple macros
> - There are no new variables

Using a static inline function, profilers like Linux perf can count
the CPU time spend in static inline functions (on each CPU instruction
when using annotated assembly code). For example, you can see how much
time (accumulated time) is spent in Py_INCREF(), to have an idea of
the cost of reference counting in Python. It's not possible when using
macros.

For debuggers, you're right that Py_INCREF() and PyTuple_GET_ITEM()
macros are very simple and it's not so hard to guess that the debugger
is executing their code in the C code or the assembly code. But the
purpose of PEP 670 is to convert way more complex macros. I wrote a PR
to convert unicodeobject.h macros, IMO there are one of the worst
macros in Python C API:
https://github.com/python/cpython/pull/31221

I always wanted to convert them, but some core devs were afraid of
performance regressions. So I wrote a PEP to prove that there is no
impact on performance.

IMO the new unicodeobject.h code is way more readable. I added two
"kind" variables which have a well defined scope. In macros, no macro
is used currently to avoid macro pitfalls: name conflict if there is
already a "kind" macro where the macro is used.

The conversion to static inline macros also detected a bug with "const
PyObject*": using PyUnicode_READY() on a const Unicode string is
wrong, this function modifies the object if it's not ready (WCHAR
kind). It also detected bugs on "const void *data" used to *write*
into string characters (PyUnicode_WRITE).


> - Macro tricks (parentheses and comma-separated expressions) are needed,
> but they're present and tested.

The PEP rationale starts with:
"The use of macros may have unintended adverse effects that are hard
to avoid, even for experienced C developers. Some issues have been
known for years, while others have been discovered recently in Python.
Working around macro pitfalls makes the macro coder harder to read and
to maintain."

Are you saying that all core devs are well aware of all macro pitfalls
and always avoid them? I'm well aware of these pitfalls, and I fall
into their trap multiple times.

The bpo-30459 issue about PyList_SET_ITEM() is a concrete example of
old bugs that nobody noticed before.


> The "massive change to working code" part is important. Such efforts
> tend to have unexpected issues, which have an unfortunate tendency to
> surface before release and contribute to release manager burnout.

Aren't you exaggerating a bit? Would you mind to elaborate? Do you
have examples of issues caused by converting macros to static inline
functions?

I'm not talking about incompatible C API changes made on purpose, but
about PEP 670 changes.

The PEP lists many macros converted to static inline functions and
static inline functions converted to regular functions:
https://www.python.org/dev/peps/pep-0670/#macros-converted-to-functions-since-python-3-8

Are you aware of release manager burnout caused by these changes?

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LQQDGXEM56J4J7NCSYNJBQPU4JMXHO7B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

[Pablo Galindo Salgado]

> Should there be a getter/setter for co_positions?

We consider the representation of co_postions private, so we don't want
(for now) to ad
getters/setters. If you want to get the position of a instruction, you can
use PyCode_Addr2Location

On Wed, 9 Feb 2022 at 16:22, Petr Viktorin  wrote:

> On 04. 02. 22 15:23, Stefan Behnel wrote:
> > Petr Viktorin schrieb am 03.02.22 um 13:47:
> >> On 02. 02. 22 11:50, Stefan Behnel wrote:
> >>> Maybe we should advertise the two modes more. And make sure that both
> >>> work. There are certainly issues with the current state of the
> >>> "limited API" implementation, but that just needs work and testing.
> >>
> >> I wonder if it can it be renamed? "Limited API" has a specific meaning
> >> since PEP 384, and using it for the public API is adding to the
> >> general confusion in this area :(
> >
> > I was more referring to it as an *existing* compilation mode of Cython
> > that avoids the usage of CPython implementation details. The fact that
> > the implementation is incomplete just means that we spill over into
> > non-limited API code when no limited API is available for a certain
> > feature. That will usually be public API code, unless that is really not
> > available either.
> >
> > One recent example is the new error locations in tracebacks, where PEP
> > 657 explicitly lists the new "co_positions" field in code objects as an
> > implementation detail of CPython. If we want to implement this in
> > Cython, then there is no other way than to copy these implementation
> > details pretty verbatimly from CPython and to depend on them.
> >
> > https://www.python.org/dev/peps/pep-0657/
> >
> > In this specific case, we're lucky that this can be considered an
> > entirely optional feature that we can separately disable when users
> > request "public API" mode (let's call it that). Not sure if that's what
> > users want, though.
>
> Should there be a getter/setter for co_positions?
> I'm unfortunately not aware of what Cython needs from code objects, but
> it might be good to extend the API here.
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/XK4DMU7I35TKXYJRYQE4RGMLNNBPDTYN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N537EIZR4KIR6Z7G3J2PTYC26DPH3YNC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

[Petr Viktorin]

On 04. 02. 22 15:23, Stefan Behnel wrote:

Petr Viktorin schrieb am 03.02.22 um 13:47:

On 02. 02. 22 11:50, Stefan Behnel wrote:
Maybe we should advertise the two modes more. And make sure that both 
work. There are certainly issues with the current state of the 
"limited API" implementation, but that just needs work and testing.


I wonder if it can it be renamed? "Limited API" has a specific meaning 
since PEP 384, and using it for the public API is adding to the 
general confusion in this area :(


I was more referring to it as an *existing* compilation mode of Cython 
that avoids the usage of CPython implementation details. The fact that 
the implementation is incomplete just means that we spill over into 
non-limited API code when no limited API is available for a certain 
feature. That will usually be public API code, unless that is really not 
available either.


One recent example is the new error locations in tracebacks, where PEP 
657 explicitly lists the new "co_positions" field in code objects as an 
implementation detail of CPython. If we want to implement this in 
Cython, then there is no other way than to copy these implementation 
details pretty verbatimly from CPython and to depend on them.


https://www.python.org/dev/peps/pep-0657/

In this specific case, we're lucky that this can be considered an 
entirely optional feature that we can separately disable when users 
request "public API" mode (let's call it that). Not sure if that's what 
users want, though.


Should there be a getter/setter for co_positions?
I'm unfortunately not aware of what Cython needs from code objects, but 
it might be good to extend the API here.



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XK4DMU7I35TKXYJRYQE4RGMLNNBPDTYN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for feedback: pathlib.AbstractPath prototype

[Ethan Furman]

On 2/9/22 6:59 AM, Barney Gale wrote:

> Over the last couple of years I've been tidying up the pathlib internals, 
with a view
> towards adding an AbstractPath class to the hierarchy. Users would be able to 
subclass
> AbstractPath to implement other kinds of filesystems: s3, google cloud 
storage, pandas,
> ftp, git, zip and tar files, etc. By implementing some abstract methods 
(stat(),
> iterdir(), open(), etc) they'd benefit from a number of derived methods 
(is_dir(),
> glob(), read_text(), etc).

Seems like a reasonable idea.

> I've now submitted a PR that adds an initial underscore-prefixed 
implementation of this class:

Why the iniitial underscore?  What could change between now and the future?

--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4KPJJEYPOYYDGS2FOL7ZE46XKX2USEZI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Request for feedback: pathlib.AbstractPath prototype

[Barney Gale]

Over the last couple of years I've been tidying up the pathlib internals,
with a view towards adding an AbstractPath class to the hierarchy. Users
would be able to subclass AbstractPath to implement other kinds of
filesystems: s3, google cloud storage, pandas, ftp, git, zip and tar files,
etc. By implementing some abstract methods (stat(), iterdir(), open(), etc)
they'd benefit from a number of derived methods (is_dir(), glob(),
read_text(), etc). There's already a healthy ecosystem of PyPI packages
attempting this, but there's presently no officially-supported route.

I've now submitted a PR that adds an initial underscore-prefixed
implementation of this class: https://github.com/python/cpython/pull/31085.
The implementation is simple: wherever Path calls functions in os, io, pwd,
etc, AbstractPath instead raises NotImplementedError. The Path class
becomes an implementation of AbstractPath.

These methods directly raise NotImplementedError: cwd(), stat(), iterdir(),
readlink(), resolve(), expanduser(), owner(), group(), open(), touch(),
mkdir(), symlink_to(), hardlink_to(), rename(), replace(), chmod(),
unlink(), rmdir()

These methods call through to the above methods: absolute(), lstat(),
exists(), is_dir(), is_file(), is_mount(), is_symlink(), is_block_device(),
is_char_device(), is_fifo(), is_socket(), home(), samefile(), scandir(),
glob(), rglob(), read_bytes(), read_text(), write_bytes(), write_text(),
lchmod(), link_to()

Some methods aren't applicable to some kinds of filesystems, e.g. zip files
don't support symlinks, working directories or home directories. In these
cases I think it's reasonable to raise NotImplementedError. Indeed,
pathlib.Path methods already raise NotImplementedError when certain local
filesystem features aren't available (readlink(), group(), etc).

If any readers of this list have previously tried to extend pathlib to
other domains, or are otherwise interested in pathlib development, please
could you let me know what you think of the proposed API?

Many thanks
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JB7QGDNI2CNXFX7LQQ2X2WPOZ7DWVNQL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should we require IEEE 754 floating-point for CPython?

[Steve Dower]

On 2/8/2022 9:54 PM, Steven D'Aprano wrote:

Regarding your second point, about domain-specific tasks, I don't
understand. If your domain-specific task doesn't need to use floats,
just don't use floats. There's surely no need to invent a whole new
language for some task because your app only needs to do string
processing, say. Have I misunderstood what you are trying to say?


Only that some people get very concerned about following "standards", 
and we can help them by making our standard easier to follow ("more 
permissive," as you suggest, is fine wording by me).


Some people see things that are written down as rules to be followed, 
while others see them as guidelines. When we write something down wrong 
or loosely, we can cause a surprising amount of angst within the first 
group (for example, I fairly frequently get worried questions about the 
VS version for building CPython due to how the wiki and devguide mention 
it).


I'm just trying to make sure we don't add more of these, by ensuring 
that we properly scope things. In this case *CPython requires IEEE 754 
floats*, but the Python language does not.


Cheers,
Steve

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KANDB5WROSPKE7TANO4A75IOD3IDBQZI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API

[Petr Viktorin]

On 08. 02. 22 12:07, Erlend Aasland wrote:

Petr, SC, thanks for your time and response! Just a quick comment:


On 8 Feb 2022, at 11:32, Petr Viktorin  wrote:

In effect, the SC accepts the first part of the PEP, except cases where:



What status does that put the PEP in?



Still draft. It's still on the SC agenda.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NMFKHTMCICXE6AFSN6SKQV7SJC5DWA6Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

[Petr Viktorin]

On 09. 02. 22 4:39, h.vetin...@gmx.com wrote:

Maybe a more practical approach would be to use C99 "except of

features not supported by MSVC of Visual Studio 2019"?

This could be formulated in a more neutral way by saying

"C99 without the things that became optional in C11", or perhaps


That sounds like a better wording!


"C11 without optional features" (at least from the POV of MSVC,
haven't checked the other compilers/platforms for C11-compliance).


That's an interesting idea -- what's keeping us from C11?
In other words: the main thing keeping us from C99 is MSVC support, and 
since that compiler apparently skipped C99, should we skip it as well?




In practice, we can try to support VS 2017, the version currently

recommended by the devguide:

That is becoming dated quickly, as Microsoft has deprecated, and is removing,
that version quite rapidly from their CI services (azure/GHA), i.e. mid March, 
see:
https://github.com/actions/virtual-environments/issues/4312.

It's understandable in the sense that they don't want to support a third version
in addition to vs2022 and vs2019, but the net effect is that very few (open 
source)
projects will keep using vs2017 going forward.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IIZ6LXK2MANUHZAMYSXDF5KPF3VIRKDJ/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ERQQFXNAGANR2ND3SRDVCIQZKYTT6OBM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API

[Petr Viktorin]

On 08. 02. 22 17:27, Victor Stinner wrote:

Hi Petr,

Thanks for the SC review, it's very helpful! I know that it's a big PEP :-)

On Tue, Feb 8, 2022 at 11:33 AM Petr Viktorin  wrote:

*All other things being equal, static inline functions are better than
macros.*
Specifically, inline static functions should be preferred over
function-like macros in new code. Please add a note about this to PEP 7.


Ok, I created: https://github.com/python/peps/pull/2315



*When there is no backwards compatibility issue, macros can be changed
to static inline functions.*
In effect, the SC accepts the first part of the PEP, except cases where:
a macro is converted to macro and function ("Cast to PyObject*"),


Right now, a large number of macros cast their argument to a type. A
few examples:

#define PyObject_TypeCheck(ob, type)
PyObject_TypeCheck(_PyObject_CAST(ob), type)
#define PyTuple_GET_ITEM(op, i) (_PyTuple_CAST(op)->ob_item[i])
#define PyDict_GET_SIZE(mp)  (assert(PyDict_Check(mp)),((PyDictObject
*)mp)->ma_used)


When I look at the Rationale points, and for the first three of these 
macros I didn't find any that sound very convincing:
- Functions don't have macro pitfalls, but these simple macros don't 
fall into the pits.
- Fully defining the argument types means getting rid of the cast, 
breaking some code that uses the macro

- Debugger support really isn't that useful for these simple macros
- There are no new variables
- Macro tricks (parentheses and comma-separated expressions) are needed, 
but they're present and tested.


Sure, if these were written now, they should be static inline functions. 
But I don't think it's worth a massive change to working code.



The "massive change to working code" part is important. Such efforts 
tend to have unexpected issues, which have an unfortunate tendency to 
surface before release and contribute to release manager burnout.




#define PyWeakref_GET_OBJECT(ref) (... ((PyWeakReference
*)(ref))->wr_object ...)


That one looks more reasonable. How common is it?
How should we identify which macros are dangerous enough to need 
changing now?




Does it mean that these macros must remain macros?


For now, yes.
I hope that changing the macros that are OK will make it easier to 
review the remaining ones, and it might also help future versions of the 
PEP to focus on the problematic cases.


Maybe it might also make sense to go from the other side: are there any 
macros that are clearly dangerous and should be changed ASAP?

(I hope not -- those are hopefully handled one-by-one in individual issues.)



PEP 670 proposes adding a macro to still cast their arguments to the
requested type so the PEP doesn't add any new compiler warnings. The
PEP proposes to consider removing these implicit casts later (to
reduce the scope and limit annoyance caused by the PEP).

If the macro is converted to a static inline function without such
convenient macro, C extensions which don't pass the expected types
will get spammed by warnings. Well, it's easy to fix, and once it is
fixed, the code remains compatible with old Python versions.

I'm not sure that I understood the SC statement here: does it mean
that these specific macros (which cast their arguments) must remain
macros, or that it's ok to convert them to static inline functions
(even if it can emit new warnings)?


You understood correctly. It's not clear that there is enough reason to 
mass-change working code.




If the SC is ok to add new compiler warnings, I'm fine with it. Most
functions are documented with an exact type, they are not documented
to cast implicitly their arguments to the expected types.

For example, Py_INCREF() expects a PyObject* type in the documentation:
https://docs.python.org/dev/c-api/refcounting.html#c.Py_INCREF

--

Or is the problem that having a macro to wrap a static inline function
requires to change the function name? Well, technically, it's not
needed... I wrote a PR so the macro and the function have the same
name:
https://github.com/python/cpython/pull/31217

For example, the Py_INCREF() macro now calls Py_INCREF(), instead of
calling _Py_INCREF().

Static inline functions are not part of the ABI since they are
inlined, but it's maybe better to avoid the "_" prefix to avoid
confusion in debuggers and profilers (especially when comparing old
and new Python versions).



and where the return value is removed.


In this case, I propose to leave these macros unchanged in PEP 670 in
this case, as it already excludes macros which can be used as l-values
(macros changed by PEP 674).

I would prefer to have the whole PEP 670 either accepted or rejected.
I'm not comfortable with a half-accepted status, it's misleading for
readers.


The SC is *not sure yet* if it can be accepted as-is.
It continues to be like any other draft PEP: a proposal, which can still 
be changed.


You now can:
- wait for the SC to discuss the rest of the proposal (which will take 
time, and will probably involve more public