[Python-Dev] Re: What is the pyclbr public API?

[Guido van Rossum]

+1

On Fri, Jan 29, 2021 at 6:28 PM Terry Reedy  wrote:

> On 1/29/2021 8:57 PM, Guido van Rossum wrote:
> > On Fri, Jan 29, 2021 at 5:39 PM Terry Reedy  > > wrote:
> >
> > Guido, thank you for the helpful discussion.  I now think that we
> > should
> > just add 'end_lineno=None' at the end of the Function/Class __init__
> > signatures.  pyclbr makes one call to each to contruct the tree it
> > returns, and the test functions make another call to each.
> >
> > If those are the only two calls to each, it hardly matters what they
> > look like.  If there are non-stdlib calls, it is not worth breaking
> > them. And others might well localize their Function/Class calls
> within
> > wrappers similar to _nest_function() and _nest_class().
> >
> >
> > Okay, I wasn't quite ready to recommend this, but I agree that that's
> > the most compatible solution. (Put a `*` before the optional arg so it
> > must be specified as a keyword.)
>
> I presume you mean immediately before 'end_lineno', as putting it before
> all optional params would break positional passing.
>
> > A more important pyclbr issue, I think, is that readline and
> > readline_ex
> > return a 'half node', a dict of children, instead of a Module node.
> It
> > is a nuisance, such as when constructing IDLE's module browser
> > tree.  It
> > is the same as if ast_parse were to return the body list of
> ast.Module
> > instead of ast.Module itself.  A readmodule() function could return a
> > proper tree with a root Module node, with attributes file, name,
> lineno
> > (1), end_lineno, and children.
> >
> > Sounds good. Thanks for caring about this very old module! (I am not
> > 100% sure of its origins but I think it may have been written in
> > Python's earliest years by one of my office mates, Sjoerd Mullender.)
>
> Adding a new API would be an opportunity to 'do it right'.  The test of
> it being an improvement is if it allows simplification of the module
> browser code.
>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/KAQBTNC5R5IHHXKPUIVMKAPFTNIHE7BK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OC3EQZ6S2R637LPNNO6TKDJCZXSJ5C3M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Comments on PEP 558

[Nick Coghlan]

On Sat, 30 Jan 2021 at 10:30, Nick Coghlan  wrote:
> On Sat, 30 Jan 2021, 12:13 am Mark Shannon,  wrote:
>> With a direct proxy coherency is not an issue.
>
> For things in the frame, it *is* a direct proxy - reads pull from the frame 
> object, and writes go to both the frame object and the mapping: 
> https://github.com/python/cpython/pull/3640/files#diff-7b8cef249e5cca077d30de4e428a6bde6b9b803464e790e9cffa7e052e19efddR1315

Reviewing the code again, I'd misremembered how the draft
implementation works - right now, reads are relying on the snapshot
being up to date.

> Given the resulting C API compatibility break and the loss of code sharing, I 
> don't think it's a net win to drop the extra storage.

For now, I have a PR adding this as an open question:
https://github.com/python/peps/pull/1787/files

Given the performance benefit of being able to more reasonably drop
the implicit call to `PyFrame_LocalsToFast()`, I'm mostly convinced
that switching reads to pull from the frame if possible is the right
thing to do, even if it reduces the amount of code that can be
inherited without modification from MappingProxyType.

The API compatibility concerns would mean the extra mapping store
still needed to stick around, but it would only be used for:

* providing backwards compatibility for the `PyEval_GetLocals()` and
C-level `f_locals` interfaces
* reading and writing names that don't have entries in the `fast_refs` mapping
* writing shadow updates for names in the `fast_refs` mapping

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZKS4NG6HNOSNHJVUYHKHNSWT747FFPFP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improve CPython tracing performance

[Nick Coghlan]

On Wed, 20 Jan 2021 at 19:22, Victor Stinner  wrote:
>
> Reply to an old thread.
>
> On Sat, Oct 31, 2020 at 8:02 AM Nick Coghlan  wrote:
> > > Debuggers and profilers usually only care of specific frames or
> > > function calls (ex: 10% of function calls or even a single function
> > > call in a whole application). The problem is how to make them as
> > > efficient as possible for "no operation" calls, when they don't care
> > > about the current frame. Avoiding PyFrame_FastToLocalsWithError() to
> > > enter the debugger/profile and avoiding PyFrame_LocalsToFast() on exit
> > > sounds a simple and practical solution.
> >
> > Aye, I agree. I just don't think we can remove those implicit calls
> > without preparing a replacement API first.
>
> Again, I don't think that it's incompatible. We can enforce calling
> PyFrame_FastToLocalsWithError() at enter and PyFrame_LocalsToFast() at
> exit for now, and enhance the API later.

PEP 558 makes `PyFrame_LocalsToFast()` raise an exception, so the two
approaches definitely aren't compatible :)

> To be clear: currently, PyFrame_FastToLocalsWithError() is called at
> enter and PyFrame_LocalsToFast() is called at exit. So asking
> debuggers/profilers to call them explicitly doesn't make the situation
> worse (nor better ;-)) for generators/coroutines, it would be exactly
> the same behavior. It's just an optimization.
>
> The PEP 558 is being discussed for 5 years and still a draft. I don't
> think that it should hold bpo-42197 optimization.

No, what should hold up the bpo-42197 PR is the fact that it's an API
compatibility break that shouldn't be done without a PEP.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PDHPZ3DQLT2Z4S5PTUKLB6FUJ3R676NI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: What is the pyclbr public API?

[Terry Reedy]

On 1/29/2021 8:57 PM, Guido van Rossum wrote:
On Fri, Jan 29, 2021 at 5:39 PM Terry Reedy > wrote:


Guido, thank you for the helpful discussion.  I now think that we
should
just add 'end_lineno=None' at the end of the Function/Class __init__
signatures.  pyclbr makes one call to each to contruct the tree it
returns, and the test functions make another call to each.

If those are the only two calls to each, it hardly matters what they
look like.  If there are non-stdlib calls, it is not worth breaking
them. And others might well localize their Function/Class calls within
wrappers similar to _nest_function() and _nest_class().


Okay, I wasn't quite ready to recommend this, but I agree that that's 
the most compatible solution. (Put a `*` before the optional arg so it 
must be specified as a keyword.)


I presume you mean immediately before 'end_lineno', as putting it before 
all optional params would break positional passing.



A more important pyclbr issue, I think, is that readline and
readline_ex
return a 'half node', a dict of children, instead of a Module node.  It
is a nuisance, such as when constructing IDLE's module browser
tree.  It
is the same as if ast_parse were to return the body list of ast.Module
instead of ast.Module itself.  A readmodule() function could return a
proper tree with a root Module node, with attributes file, name, lineno
(1), end_lineno, and children.

Sounds good. Thanks for caring about this very old module! (I am not 
100% sure of its origins but I think it may have been written in 
Python's earliest years by one of my office mates, Sjoerd Mullender.)


Adding a new API would be an opportunity to 'do it right'.  The test of 
it being an improvement is if it allows simplification of the module 
browser code.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KAQBTNC5R5IHHXKPUIVMKAPFTNIHE7BK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: What is the pyclbr public API?

[Guido van Rossum]

On Fri, Jan 29, 2021 at 5:39 PM Terry Reedy  wrote:

> Guido, thank you for the helpful discussion.  I now think that we should
> just add 'end_lineno=None' at the end of the Function/Class __init__
> signatures.  pyclbr makes one call to each to contruct the tree it
> returns, and the test functions make another call to each.
>
> If those are the only two calls to each, it hardly matters what they
> look like.  If there are non-stdlib calls, it is not worth breaking
> them. And others might well localize their Function/Class calls within
> wrappers similar to _nest_function() and _nest_class().
>

Okay, I wasn't quite ready to recommend this, but I agree that that's the
most compatible solution. (Put a `*` before the optional arg so it must be
specified as a keyword.)


> A more important pyclbr issue, I think, is that readline and readline_ex
> return a 'half node', a dict of children, instead of a Module node.  It
> is a nuisance, such as when constructing IDLE's module browser tree.  It
> is the same as if ast_parse were to return the body list of ast.Module
> instead of ast.Module itself.  A readmodule() function could return a
> proper tree with a root Module node, with attributes file, name, lineno
> (1), end_lineno, and children.
>

Sounds good. Thanks for caring about this very old module! (I am not 100%
sure of its origins but I think it may have been written in Python's
earliest years by one of my office mates, Sjoerd Mullender.)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6N4KRSI33BYRJC4QYS3GOXQPRP4MNXFQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: What is the pyclbr public API?

[Terry Reedy]

Guido, thank you for the helpful discussion.  I now think that we should 
just add 'end_lineno=None' at the end of the Function/Class __init__ 
signatures.  pyclbr makes one call to each to contruct the tree it 
returns, and the test functions make another call to each.


If those are the only two calls to each, it hardly matters what they 
look like.  If there are non-stdlib calls, it is not worth breaking 
them. And others might well localize their Function/Class calls within 
wrappers similar to _nest_function() and _nest_class().


A more important pyclbr issue, I think, is that readline and readline_ex 
return a 'half node', a dict of children, instead of a Module node.  It 
is a nuisance, such as when constructing IDLE's module browser tree.  It 
is the same as if ast_parse were to return the body list of ast.Module 
instead of ast.Module itself.  A readmodule() function could return a 
proper tree with a root Module node, with attributes file, name, lineno 
(1), end_lineno, and children.



On 1/28/2021 11:32 PM, Guido van Rossum wrote:
On Thu, Jan 28, 2021 at 8:08 PM Terry Reedy > wrote:


On 1/27/2021 7:01 PM, Guido van Rossum wrote:
 > It's likely that the additions are going to break someone's code;

Since at least 2007, when Georg moved the 3i reST tree into the 3k
branch, the instances have been described as "Class/Function Descriptor
Objects", returned by readmodule(_ex), that "provide the following data
members".  There are no methods (maybe __repr__ should be added for
debugging), so these are pure data classes, identical in purpose to
pure
dataclasses.


Lots of people just read the source code though. In general we've often 
been careful with changing undocumented APIs if we believe they are 
commonly used in ways not endorsed by the official documentation.


The only intended reason to call these would be to create a tree for
testing. 



Really? Couldn't someone have an application where they insert new 
objects (like is documented for the ast module)?


We do this indirectly with a couple of private test-only
pyclbr functions.  Being private, *those* functions can have the new
parameter added anywhere, regardless of the Class/Function signatures.

test_pyclbr uses a constructed tree as the expected output from
readline_ex(test_code).  IDLE's test_browser uses a construction as
test
input to the browser widget. If someone does similar testing, it could
break with the proposed changes.


So the question is whether we care. I don't know how to assess that.

 > the
 > question seems to be do we care, since the constructors are
undocumented?

If any library functions return dataclasses, are the signatures of the
(possibly generated) __init__ functions documented?


I'd say so, because we document how `@dataclass` constructs the 
`__init__()` method.


I care more about breaking tests than possibly non-existent 'off-label'
uses, but Python won't know.

 > But shouldn't we just document the signatures,

The existing or proposed signatures?


I meant the proposed signatures.

 > so that in the future
 > we're not going to make the same mistake?

I am not sure what you mean by the 'mistake' to not be repeated.  I
consider it be had been either not declaring the signature private
or at
least not making optional parameters keyword only.


Yeah, by 'mistake' I meant leaving it ambiguous whether the constructor 
signature is part of the API. And yes, we could also have designed the 
constructor signatures more carefully.


 > I also wonder if some judicious use of keyword-only args might help
 > users who currently call the old constructors catch the breakage
where
 > it's easily diagnosed (i.e. at the call site) instead of once the
object
 > is instantiated (e.g. "why is end_lineno/parent the wrong type")?

With 'end_lineno' added after existing required parameters, just before
the existing optional parameter 'parent', an existing Class call ending
with '..., posint, parent=SomeClass)' would fail with

TypeError: Class.__init__ missing 1 required positional argument:
'end_lineno'


Unless you give end_lineno a default (I don't think we should do this 
unless we find significant rogue usages).


If parent had previously been made 'keyword only', a call ending
instead
with '..., num, SomeClass' would be a failure now.  But since not,


Right. One of the mistakes.

 > Perhaps even add some type checks to catch obvious mistakes
(could be as
 > simple as `assert isinstance(end_lineno, int)").

Do you consider this enough to make the proposed positioning of
'end_lineno' acceptible?  The tradeoff is some immediate annoyance
versus forever annoyance of a misplaced required 'optional' parameter.


Yes, I think that would be sufficient -- we'd break rogue usages (if 

[Python-Dev] Re: Comments on PEP 558

[Nick Coghlan]

On Sat, 30 Jan 2021, 12:13 am Mark Shannon,  wrote:

> Hi
>
> It is a lot more complex, because you need to worry about coherency.
> With a direct proxy coherency is not an issue.
>

For things in the frame, it *is* a direct proxy - reads pull from the frame
object, and writes go to both the frame object and the mapping:
https://github.com/python/cpython/pull/3640/files#diff-7b8cef249e5cca077d30de4e428a6bde6b9b803464e790e9cffa7e052e19efddR1315

The technical pay-off from having the mapping there is that most read-only
dict operations are handled by the existing MappingProxyType.

If you drop the mapping, then you lose any code sharing, and have to
implement the full mapping API from scratch, instead of just the mutation
and single key lookup parts.

You also reduce compatibility with the status quo, since writes to unknown
keys will fail rather than being stored solely in the mapping section, and
there's also no way to preserve the "PyEval_GetLocals()" C API, which is
expected to return a borrowed reference to a plain Python dict stored in
the C level "f_locals" field on the frame struct.

The cache coherency issues on the bulk mapping APIs this perpetuates also
aren't any worse than those on the locals() built-in itself: they'll see
the state as of the last snapshot update rather than the live state on the
frame (which may not exist any more if the proxy outlives the underlying
frame).

Given the resulting C API compatibility break and the loss of code sharing,
I don't think it's a net win to drop the extra storage.

Cheers,
Nick.






>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BZAMC7NXQGYY7WGDAOUT3TYXBDP7OZCT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why aren't we allowing the use of C11?

[Random832]

On Thu, Jan 28, 2021, at 22:57, Emily Bowman wrote:
>  
> On Thu, Jan 28, 2021 at 1:31 PM MRAB  wrote:
> > I have Microsoft Visual Studio Community 2019 Version 16.8.4 (it's free) 
> > and it supports C11 and C17. 
> 
> While an upgrade for Community is free, for Pro/Enterprise without an 
> annual license it's not.

Something that's hard for me to find a clear answer for in a few minutes of 
searching:

Does the current version of the *Windows SDK* still come with a compiler, and 
can it be used without the Visual Studio IDE [the situation is confusing 
because it seems to install through the visual studio installer now] or at 
least without a license for Visual Studio Pro/Enterprise [i.e. by users who do 
not qualify for community edition] or with earlier versions of the Visual 
Studio IDE?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LUOB5H2S6PTHLZEDJVFUZTXLETS54KNX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why aren't we allowing the use of C11?

[Ivan Pozdeev via Python-Dev]

On 29.01.2021 20:15, Victor Stinner wrote:

Hi Mark,

I tried to use C11 _Thread_local (Thread Local Storage, TLS) only with
GCC and clang and I got issues on macOS:
https://github.com/python/cpython/pull/23976

My PR uses __thread if _Thread_local is not available.

I don't think that MSC (Visual Studio) implements C11 _Thread_local,
but it provides __declspec(thread). I tried it and I got issues with
DLL. I didn't dig into this issue. MSC loves C++ but seems stuck at
C99 (I'm not even sure if it's fully implemented).


According to https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=msvc-160#c-standards-support-1 
and https://docs.microsoft.com/en-us/cpp/overview/install-c17-support?view=msvc-160 , they only partially support C99, and C11 and C17 
support is available in the Insider Preview version.
The preview version supports all the required but no optional features (including optional features that were required in C99) and suggest 
to use features test macros.

They also say that the 2 standard modes are functonally equivalent except the 
version macro since C17 is a bugfix release.

AFAICS, this means that global C99 requirement is out of the question; C11/C17 can be considered when that MSVC version is released -- but 
alternative solutions are still needed for code chunks using optional features.





It seems like declaring a TLS in libpython and using it from an
extension module (another dynamic library) is causing issues depending
on the linker. It "should" work on macOS, but it doesn't.

See https://bugs.python.org/issue40522 for the produced machine code
on x86-64. In short, it's usually a single MOV using the FS register
(two MOV in the worst case).

Victor

On Thu, Jan 28, 2021 at 5:29 PM Mark Shannon  wrote:

Hi everyone,

PEP 7 says that C code should conform to C89 with a subset of C99 allowed.
It's 2021 and all the major compilers support C11 (ignoring the optional
parts).

C11 has support for thread locals, static asserts, and anonymous structs
and unions. All useful features.

Is there a good reason not to start using C11 now?

Cheers,
Mark.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PLXETSQE7PRFXBXN2QY6VNPKUTM6I7OD/
Code of Conduct: http://python.org/psf/codeofconduct/




--
Regards,
Ivan
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/54Y55JCNK6PMSVFO4YUX7GZMH4VC53DQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2021-01-22 - 2021-01-29)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7588 (+10)
  closed 47211 (+50)
  total  54799 (+60)

Open issues with patches: 3032 


Issues opened (40)
==

#33387: Simplify bytecodes for try-finally, try-except and with blocks
https://bugs.python.org/issue33387  reopened by iritkatriel

#36675: Doctest directives and comments missing from code samples
https://bugs.python.org/issue36675  reopened by mdk

#43006: Changed behaviour of inspect.signature() in Python 3.10
https://bugs.python.org/issue43006  opened by Zac Hatfield-Dodds

#43007: set_wakeup_fd() only works in main thread
https://bugs.python.org/issue43007  opened by MeneMeneTekel

#43009: Port curses capi pointer array to a struct
https://bugs.python.org/issue43009  opened by shihai1991

#43010: @functools.wraps and abc.abstractmethod interoperability
https://bugs.python.org/issue43010  opened by erezinman

#43012: Remove pathlib accessors
https://bugs.python.org/issue43012  opened by barneygale

#43013: IDLE:  update code, mostly by cleanups of 2.x or 2to3 artifact
https://bugs.python.org/issue43013  opened by terry.reedy

#43015: Add str.replaceall?
https://bugs.python.org/issue43015  opened by Nathaniel Manista

#43016: Improve tests for curses
https://bugs.python.org/issue43016  opened by serhiy.storchaka

#43017: Improve error message in the parser when using un-parenthesise
https://bugs.python.org/issue43017  opened by pablogsal

#43019: wait_for(to_thread)) does not work as expected. Extra document
https://bugs.python.org/issue43019  opened by synchronizing

#43022: Unable to dynamically load functions from python3.dll
https://bugs.python.org/issue43022  opened by paul.moore

#43024: improve signature (in help, etc) for functions taking sentinel
https://bugs.python.org/issue43024  opened by iritkatriel

#43026: Missing words renders meaning unclear in fcntl.html
https://bugs.python.org/issue43026  opened by EzraBC

#43027: Calling _PyBytes_Resize() on 1-byte bytes may raise error
https://bugs.python.org/issue43027  opened by malin

#43029: unittest: Add assertUniqeIn
https://bugs.python.org/issue43029  opened by rousseldenis

#43030: signed/unsigned mismatch in Py_UNICODE_ISSPACE macro
https://bugs.python.org/issue43030  opened by doko

#43034: Python tutorial misleads users about floor division behavior
https://bugs.python.org/issue43034  opened by jessevsilverman

#43035: FileNotFoundError in distutils\file_util.py copy_tree
https://bugs.python.org/issue43035  opened by knzivid

#43036: TOS-behaviour documentation is inconsistent
https://bugs.python.org/issue43036  opened by xmorel

#43039: tempfile.TemporaryDirectory() name string is incorrect
https://bugs.python.org/issue43039  opened by eosborne

#43041: copying WeakValueDictionary is not iteration safe
https://bugs.python.org/issue43041  opened by djromberg

#43042: tutorial ambiguous about creation of local symbol table for re
https://bugs.python.org/issue43042  opened by jessevsilverman

#43043: Python tutorial could make rules for default argument definiti
https://bugs.python.org/issue43043  opened by jessevsilverman

#43045: Instructions for installing pip on Ubuntu/WSL2 is incomplete
https://bugs.python.org/issue43045  opened by lennart.borgman

#43046: argparse: capturing actions
https://bugs.python.org/issue43046  opened by monkeyman79

#43047: logging.config formatters documentation is out of sync with co
https://bugs.python.org/issue43047  opened by iwienand

#43048: Printing RecursionError results in RecursionError
https://bugs.python.org/issue43048  opened by vlad2

#43049: Use io.IncrementalNewlineDecoder for doctest newline conversio
https://bugs.python.org/issue43049  opened by pdonis

#43050: threading timer memory leak
https://bugs.python.org/issue43050  opened by fengjiang

#43052: _dyld_shared_cache_contains_path needs SYSTEM_VERSION_COMPAT=0
https://bugs.python.org/issue43052  opened by isuruf

#43053: Speed up math.isqrt, again
https://bugs.python.org/issue43053  opened by juraj.sukop

#43054: What does the existence of a struct in a header file imply abo
https://bugs.python.org/issue43054  opened by Mark.Shannon

#43056: Use of dicts in sections 4.2 and 4.7 of Python tutorial a bit 
https://bugs.python.org/issue43056  opened by jessevsilverman

#43058: setting a logging Handler name
https://bugs.python.org/issue43058  opened by bcohen

#43059: sqlite3: Externally developed?
https://bugs.python.org/issue43059  opened by user1347091

#43060: Convert _decimal C API from pointer array to struct
https://bugs.python.org/issue43060  opened by erlendaasland

#43062: Non-integer values in collections.Counter
https://bugs.python.org/issue43062  opened by HuangFuSL

#43063: zipfile.Path / importlib.resources raises KeyError if a file w
https://bugs.python.org/issue43063  

[Python-Dev] Re: Why aren't we allowing the use of C11?

[Victor Stinner]

Hi Mark,

I tried to use C11 _Thread_local (Thread Local Storage, TLS) only with
GCC and clang and I got issues on macOS:
https://github.com/python/cpython/pull/23976

My PR uses __thread if _Thread_local is not available.

I don't think that MSC (Visual Studio) implements C11 _Thread_local,
but it provides __declspec(thread). I tried it and I got issues with
DLL. I didn't dig into this issue. MSC loves C++ but seems stuck at
C99 (I'm not even sure if it's fully implemented).

It seems like declaring a TLS in libpython and using it from an
extension module (another dynamic library) is causing issues depending
on the linker. It "should" work on macOS, but it doesn't.

See https://bugs.python.org/issue40522 for the produced machine code
on x86-64. In short, it's usually a single MOV using the FS register
(two MOV in the worst case).

Victor

On Thu, Jan 28, 2021 at 5:29 PM Mark Shannon  wrote:
>
> Hi everyone,
>
> PEP 7 says that C code should conform to C89 with a subset of C99 allowed.
> It's 2021 and all the major compilers support C11 (ignoring the optional
> parts).
>
> C11 has support for thread locals, static asserts, and anonymous structs
> and unions. All useful features.
>
> Is there a good reason not to start using C11 now?
>
> Cheers,
> Mark.
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/PLXETSQE7PRFXBXN2QY6VNPKUTM6I7OD/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OXLEWT7VMBEIU4GW23C2PY2NXHMWCEQX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why aren't we allowing the use of C11?

[Eric Snow]

On Thu, Jan 28, 2021 at 9:28 AM Mark Shannon  wrote:
> Is there a good reason not to start using C11 now?

Would C17 be a better choice?  It sounds like it exists to fix
problems with C11 (and doesn't actually add any new features).

-eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2D25C5KI73LBRVLFHDBGH4OIKSCCEPUO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Comments on PEP 558

[Mark Shannon]

Hi Nick,

On 29/01/2021 1:21 pm, Nick Coghlan wrote:
On Thu, 28 Jan 2021, 11:18 pm Mark Shannon, > wrote:



Hi Nick,

Regarding `f_locals` PEP 558 states:

"""
Instead of being a direct reference to the internal dynamic snapshot
used to populate the independent snapshots returned by locals(),
frame.f_locals will be updated to instead return a dedicated proxy type
(implemented as a private subclass of the existing
types.MappingProxyType) that has two internal attributes not exposed as
part of the Python runtime API:

*    mapping: an implicitly updated snapshot of the function local
variables and closure references, as well as any arbitrary items that
have been set via the mapping API, even if they don't have storage
allocated for them on the underlying frame
*    frame: the underlying frame that the snapshot is for

"""

This seems rather complex, and consequently fragile.
I fear that this is just going to result in different bugs, rather than
fixing the bugs it proposes to fix.

Why not just make `f_local` a direct view on the underlying frame?
It would be simpler to understand, more robust, and should perform
better.


The concern I have with the simplification is that I don't know what 
would break if trace hooks lost the ability to stash additional state 
that the compiler doesn't know anything about in f_locals.


Do you know of any tools do that? It seems highly unlikely to me.
In fact tool authors are asking for less state, not more: 
https://bugs.python.org/issue42197

(for performance, rather than correctness reasons, I should note)



Rather than trying to assess how common such usage is, and whether we 
care about breaking any use cases that people have for it, I instead 
elected to just keep it working.


"keep it working" implies that it works now. It doesn't.
https://bugs.python.org/issue30744



The extra implementation complexity beyond what's already needed to cope 
with closure cells also isn't that much.


It is a lot more complex, because you need to worry about coherency. 
With a direct proxy coherency is not an issue.


https://bugs.python.org/issue30744 shows that the interactions between
stateful local caches, nonlocals, and concurrency is complex and likely 
to be buggy. If you are going to substitute one stateful construct for 
another, then you need to provide evidence that it won't introduce new bugs.


Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Y7MAAIETRGYRJQ3GI6V75BCUCZXXEVGE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Comments on PEP 558

[Nick Coghlan]

On Thu, 28 Jan 2021, 11:18 pm Mark Shannon,  wrote:

>
> Hi Nick,
>
> Regarding `f_locals` PEP 558 states:
>
> """
> Instead of being a direct reference to the internal dynamic snapshot
> used to populate the independent snapshots returned by locals(),
> frame.f_locals will be updated to instead return a dedicated proxy type
> (implemented as a private subclass of the existing
> types.MappingProxyType) that has two internal attributes not exposed as
> part of the Python runtime API:
>
> *mapping: an implicitly updated snapshot of the function local
> variables and closure references, as well as any arbitrary items that
> have been set via the mapping API, even if they don't have storage
> allocated for them on the underlying frame
> *frame: the underlying frame that the snapshot is for
>
> """
>
> This seems rather complex, and consequently fragile.
> I fear that this is just going to result in different bugs, rather than
> fixing the bugs it proposes to fix.
>
> Why not just make `f_local` a direct view on the underlying frame?
> It would be simpler to understand, more robust, and should perform better.
>

The concern I have with the simplification is that I don't know what would
break if trace hooks lost the ability to stash additional state that the
compiler doesn't know anything about in f_locals.

Rather than trying to assess how common such usage is, and whether we care
about breaking any use cases that people have for it, I instead elected to
just keep it working.

The extra implementation complexity beyond what's already needed to cope
with closure cells also isn't that much.

Cheers,
Nick.



> Cheers,
> Mark.
>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MCRXLN5O5NVOY4WGYLTOSQYSGD2255WB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why aren't we allowing the use of C11?

[Charalampos Stratakis]

- Original Message -
> From: "Nathaniel Smith" 
> To: "Gregory P. Smith" 
> Cc: "Charalampos Stratakis" , "Python Dev" 
> 
> Sent: Friday, January 29, 2021 6:29:58 AM
> Subject: Re: [Python-Dev] Re: Why aren't we allowing the use of C11?
> 
> On Thu, Jan 28, 2021 at 9:03 PM Gregory P. Smith  wrote:
> >
> > On Thu, Jan 28, 2021 at 10:52 AM Charalampos Stratakis
> >  wrote:
> >>
> >>
> >>
> >> - Original Message -
> >> > From: "Mark Shannon" 
> >> > To: "Python Dev" 
> >> > Sent: Thursday, January 28, 2021 5:26:37 PM
> >> > Subject: [Python-Dev] Why aren't we allowing the use of C11?
> >> >
> >> > Hi everyone,
> >> >
> >> > PEP 7 says that C code should conform to C89 with a subset of C99
> >> > allowed.
> >> > It's 2021 and all the major compilers support C11 (ignoring the optional
> >> > parts).
> >> >
> >> > C11 has support for thread locals, static asserts, and anonymous structs
> >> > and unions. All useful features.
> >> >
> >> > Is there a good reason not to start using C11 now?
> >> >
> >> > Cheers,
> >> > Mark.
> >> >
> >> >
> >> > ___
> >> > Python-Dev mailing list -- python-dev@python.org
> >> > To unsubscribe send an email to python-dev-le...@python.org
> >> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> >> > Message archived at
> >> > https://mail.python.org/archives/list/python-dev@python.org/message/PLXETSQE7PRFXBXN2QY6VNPKUTM6I7OD/
> >> > Code of Conduct: http://python.org/psf/codeofconduct/
> >> >
> >> >
> >>
> >> Depends what platforms the python core developers are willing to support.
> >>
> >> Currently downstream on e.g. RHEL7 we compile versions of CPython under
> >> gcc 4.8.2 which does not support C11.
> >>
> >> In addition the manylinux2014 base image is also based on CentOS 7, which
> >> wouldn't support C11 as well.
> >
> >
> > I suspect this is the primary technical reason not to adopt C11 left.
> >
> > But aren't things like manylinux2014 defined by the contents of a centrally
> > maintained docker container?
> > If so (I'm not one who knows how wrong my guess likely is...), can we get
> > those updated to include a more modern compiler so we can move on sooner
> > than the deprecation of manylinux2014?
> 
> RedHat maintains builds of gcc 8.2.1 for CentOS/RHEL 7, that have some
> clever hacks to guarantee that the resulting binaries will work on
> CentOS/RHEL 7:
> https://www.softwarecollections.org/en/scls/rhscl/devtoolset-8/
> 
> I'm pretty sure that's what the manylinux2014 image is using.
> 
> -n
> 
> --
> Nathaniel J. Smith -- https://vorpus.org
> 
> 

That is correct, Software Collections allow that (I'm also one of the 
maintainers of the Python collection), not sure though if indeed that's the 
case for manylinux2014.

Apart from that though, in general Python is compatible with a wide variety of 
Platforms and operating systems. Are all those supported platforms compatible 
with C11 or later?

RHEL7 is not (apart from using scl's) for example. If the standard changes that 
will also render useless a number of buildbots for Python's later branches.

-- 
Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EP2NGZY6RZVVU567HUHX5QFVPOST747T/
Code of Conduct: http://python.org/psf/codeofconduct/