[issue44405] add program passed as string to dis module.

2021-07-23 Thread Nick Coghlan


Nick Coghlan  added the comment:

I suspect the only reason the dis CLI isn't documented is because it's been 
around for so long (22 years!) that nobody previously noticed it was missing: 
https://github.com/python/cpython/commit/1fdae12c93246fcf4abbf882ba08df789070dfcc

Folks then either didn't know about it, or already knew how to use it.

Even the migration to use argparse was a code cleanup from a user that happened 
to be reading the module source code and noticed that it didn't already do so: 
https://bugs.python.org/issue18538

I can see value in supporting `-c` and `-m` options to dis to align with the 
main interpreter CLI.

--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue44405>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43838] There is a way to access an underlying mapping in MappingProxyType

2021-07-23 Thread Nick Coghlan


Nick Coghlan  added the comment:

Delegating operations to the underlying mapping is still OK, all we're wanting 
to bypass is the operand coercion dances in abstract.c and object.c that may 
expose the underlying mapping to the *other* operand.

We don't want to implicitly copy though, as the performance hit can be 
non-trivial for large mappings, even if the algorithmic complexity doesn't 
change.

For the `nb_or` slot, it should be correct to retrieve and call `__or__` from 
the underlying mapping when the left operand is a MappingProxy instance, and 
`__ror__` when the right operand is a MappingProxy instance (but the left 
operand is not).

Rich comparison is messier though, since PyObject_RichCompare handles both the 
operand coercion dance *and* the mapping of the specific comparison operations 
to their implementation slots.

I don't see a clean solution to that other than defining a new 
PyObject_DelegateRichCompare helper function for the mapping proxy to use that 
provides the mapping from specific operands to their implementation slots 
*without* doing the coercion dance.

However we resolve it, I think it probably won't be worth the risk of 
backporting the change (this has been the way it currently is for a long time, 
and it's a "reduce the risk of error" feature rather than any kind of security 
boundary).

--

___
Python tracker 
<https://bugs.python.org/issue43838>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44561] Some expired hyperlinks in Python documentation

2021-07-18 Thread Nick Coghlan


Change by Nick Coghlan :


--
stage: patch review -> backport needed

___
Python tracker 
<https://bugs.python.org/issue44561>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44561] Some expired hyperlinks in Python documentation

2021-07-18 Thread Nick Coghlan


Nick Coghlan  added the comment:


New changeset b494685b2548489efcc66993cc6c13b027ce3b26 by Steven Hsu in branch 
'main':
bpo-44561: Update hyperlinks in Doc/distributing/index.rst (#27032)
https://github.com/python/cpython/commit/b494685b2548489efcc66993cc6c13b027ce3b26


--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue44561>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43838] There is a way to access an underlying mapping in MappingProxyType

2021-07-10 Thread Nick Coghlan


Nick Coghlan  added the comment:

I stumbled across this independently in bpo-44596, but missed this existing 
report due to the specific word I was looking for (encapsulation).

In addition to the comparison operand coercion, there's now another access 
option through the `__ror__` method.

The bug in both cases arises from delegating a method/function implementation 
to the underlying mapping type in a way that invokes the full operand coercion 
dance. (PyObject_RichCompare in one case, PyNumber_Or in the other)

Delegating these operations to the underlying mapping does make sense, but it 
needs to be a lower level delegation that bypasses the operand coercion dance, 
so the other operand only ever sees the proxy object, not the underlying 
mapping.

--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue43838>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44596] Operand coercion breaks MappingProxyType encapsulation

2021-07-10 Thread Nick Coghlan


New submission from Nick Coghlan :

The fast locals proxy implementation for PEP 558 used the existing 
MappingProxyType implementation as a starting point.

This meant I noticed the following code in the mapping proxy comparison 
implementation:

==
static PyObject *
mappingproxy_richcompare(mappingproxyobject *v, PyObject *w, int op)
{
return PyObject_RichCompare(v->mapping, w, op);
}
==

Seeing that code lead to trying the following experiment:

==
>>> from types import MappingProxyType
>>> d = {}
>>> proxy = MappingProxyType(d)
>>> class Sneaky:
... def __eq__(self, other):
... if other is d:
... raise RuntimeError("Broke encapsulation")
... 
>>> proxy == Sneaky()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 4, in __eq__
RuntimeError: Broke encapsulation
==

The new `__or__` implementation has a corresponding loophole:


>>> class SneakyOr:
... def __or__(self, other):
... if other is d:
... raise RuntimeError("Broke encapsulation")
... def __ror__(self, other):
... return self.__or__(other)
... 
>>> proxy | SneakyOr()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 6, in __ror__
  File "", line 4, in __or__
RuntimeError: Broke encapsulation


Delegating these operations to the underlying mapping via the abstract 
operation layer means that the underlying mapping gets exposed to the operand 
coercion machinery and can hence be accessed by the other operand.

--
messages: 397242
nosy: ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Operand coercion breaks MappingProxyType encapsulation
type: behavior
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue44596>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42197] Disable automatic update of frame locals during tracing

2021-06-27 Thread Nick Coghlan


Nick Coghlan  added the comment:

I've updated PEP 558 with a reference back to this ticket as additional 
motivation (the update also addresses several of the fragility concerns that 
Mark raised): https://github.com/python/peps/pull/1787

There's still work to be done to bring the reference implementation into line 
with the updated proposal, but I'm making it an explicit goal to get the 
proposal submitted for pronouncement in time for inclusion in 3.11.

--
versions: +Python 3.11 -Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44515] contextlib test incompatibility with non-refcounted GC

2021-06-26 Thread Nick Coghlan


Change by Nick Coghlan :


--
keywords: +patch
pull_requests: +25486
stage: commit review -> patch review
pull_request: https://github.com/python/cpython/pull/26910

___
Python tracker 
<https://bugs.python.org/issue44515>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44515] contextlib test incompatibility with non-refcounted GC

2021-06-26 Thread Nick Coghlan


Change by Nick Coghlan :


--
versions: +Python 3.10, Python 3.11

___
Python tracker 
<https://bugs.python.org/issue44515>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44515] contextlib test incompatibility with non-refcounted GC

2021-06-26 Thread Nick Coghlan


New submission from Nick Coghlan :

Backporting the latest contextlib module and test suite to contextlib2, I ran 
into a couple of CI failures on PyPy3.

Investigation showed that a couple of the new test cases were assuming the use 
of a refcounted GC. One could be fixed by switching to using a synchronous 
context manager instead of a ``__del__`` method, but the other needed a few 
explicit gc.collect() calls.

--
assignee: ncoghlan
keywords: 3.10regression
messages: 396545
nosy: ncoghlan, yselivanov
priority: normal
severity: normal
stage: commit review
status: open
title: contextlib test incompatibility with non-refcounted GC
type: behavior

___
Python tracker 
<https://bugs.python.org/issue44515>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37398] contextlib.ContextDecorator decorating async functions

2021-06-26 Thread Nick Coghlan


Nick Coghlan  added the comment:

bpo-40816 took the much simpler approach of introducing a separate 
contextlib.AsyncContextDecorator class.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Add missed AsyncContextDecorator to contextlib
versions: +Python 3.10 -Python 3.9

___
Python tracker 
<https://bugs.python.org/issue37398>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42972] [C API] Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-05-27 Thread Nick Coghlan


Nick Coghlan  added the comment:


New changeset 59af59c2dfa52dcd5605185263f266a49ced934c by Erlend Egeberg 
Aasland in branch 'main':
bpo-42972: Fully support GC for pyexpat, unicodedata, and dbm/gdbm heap types 
(GH-26376)
https://github.com/python/cpython/commit/59af59c2dfa52dcd5605185263f266a49ced934c


--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue42972>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-29 Thread Nick Coghlan


Nick Coghlan  added the comment:

After the next docs build, the documentation for the explicit AST nodes should 
be available https://docs.python.org/dev/library/ast.html#pattern-matching

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41682] [Windows] test_asyncio: Proactor test_sendfile_close_peer_in_the_middle_of_receiving failure

2021-04-27 Thread Nick Coghlan


Nick Coghlan  added the comment:

Another instance at 
https://github.com/python/cpython/pull/25585/checks?check_run_id=2447701034

I unlinked the previously linked PR, as that looked to be for bpo-43539 rather 
than this issue.

--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue41682>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41682] [Windows] test_asyncio: Proactor test_sendfile_close_peer_in_the_middle_of_receiving failure

2021-04-27 Thread Nick Coghlan


Change by Nick Coghlan :


--
stage: patch review -> needs patch

___
Python tracker 
<https://bugs.python.org/issue41682>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41682] [Windows] test_asyncio: Proactor test_sendfile_close_peer_in_the_middle_of_receiving failure

2021-04-27 Thread Nick Coghlan


Change by Nick Coghlan :


--
pull_requests:  -24157

___
Python tracker 
<https://bugs.python.org/issue41682>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

2021-04-27 Thread Nick Coghlan


Change by Nick Coghlan :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> enhancement

___
Python tracker 
<https://bugs.python.org/issue43312>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-26 Thread Nick Coghlan


Change by Nick Coghlan :


--
assignee: ncoghlan -> brandtbucher

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-26 Thread Nick Coghlan


Change by Nick Coghlan :


--
stage: patch review -> commit review

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-26 Thread Nick Coghlan


Nick Coghlan  added the comment:

PR is green now, and I don't have any further changes I want to make, so 
Brandt, please feel free to merge if you're happy with the proposed AST nodes, 
and the way I've gone about implementing them.

If you'd like some adjustments, it probably makes the most sense to just amend 
the PR directly - otherwise the 24-hour turnaround on comments due to the time 
zone difference will potentially give us trouble with the beta deadline.

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-25 Thread Nick Coghlan


Nick Coghlan  added the comment:

PR against the main repo is now up: https://github.com/python/cpython/pull/25585

A couple of things in the unparser tripped me up (ast_unparse.c only handles 
annotation expressions, the way ast._Unparser.visit() is defined means 
NodeVisitor.generic_visit can't be used), so the PR also includes some 
clarifying comments for future editors.

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-25 Thread Nick Coghlan


Change by Nick Coghlan :


--
pull_requests: +24305
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/25585

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-25 Thread Nick Coghlan


Nick Coghlan  added the comment:

Working on the unparser picked up an inconsistency I had previously missed: 
aside from named expressions, "target" attributes in the AST are generally full 
expression subnodes, rather than identifier attributes. Identifier attributes 
on nodes are typically called "name", including in the originally implemented 
MatchAs node.

Accordingly, I've switched both MatchStar and MatchAs over to having "name" 
attributes instead of "target" attributes.

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-25 Thread Nick Coghlan


Nick Coghlan  added the comment:

Interesting idea - I had missed that you were suggested "identifier*" for the 
cls node. It's simple enough to check for Name-or-Attribute in the AST 
validator though, and sticking with a standard expr_ty node means getting a lot 
of standard handling in the rest of the compiler and the unparser.

My commitments today finished earlier than I expected, so I'm about to tackle 
the unparser now.

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-23 Thread Nick Coghlan


Nick Coghlan  added the comment:

Setting clear timing expectations, given beta is so close: I'm not expecting to 
have time to finish the unparser work over the weekend, but I do expect to have 
time to finish it on Monday evening my time (UTC+10).

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-23 Thread Nick Coghlan


Nick Coghlan  added the comment:

https://github.com/ncoghlan/cpython/pull/9/commits/54940a3817df3046da3f9c51d4d426a73b2ec786
 implements Brandt's suggestion of disallowing NULL keys in MatchMapping and 
instead passing in an explicit "rest" identifier to capture the extra keys.

This did require moving to a multi-branch pattern definition in the grammar, 
similar to class patterns, in order to enforce the separating comma when both 
ordinary key:pattern pairs and a double-star target are present. I think it's a 
nice ergonomic improvement on the AST node itself though, as it makes it 
trivial to check if a mapping patterns captures extra keys or not (vs the 
relatively non-obvious check to see if the last key is NULL/None).

That just leaves updating the unparser to handle the new node types.

Capturing the latest node definitions from the draft PR:

pattern = MatchAlways
 | MatchValue(expr value)
 | MatchSingleton(constant value)
 | MatchSequence(pattern* patterns)
 | MatchMapping(expr* keys, pattern* patterns, identifier? rest)
 | MatchClass(expr cls, pattern* patterns, identifier* kwd_attrs, 
pattern* kwd_patterns)

 | MatchStar(identifier? target)
 -- The optional "rest" MatchMapping parameter handles capturing extra 
mapping keys

 | MatchAs(pattern? pattern, identifier target)
 | MatchOr(pattern* patterns)
  attributes (int lineno, int col_offset, int end_lineno, int 
end_col_offset)

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-23 Thread Nick Coghlan


Nick Coghlan  added the comment:

To get the "0+0" matching case rejected again I had to flesh out a bit more of 
the AST validator. This initial piece of the validator replaces the previously 
pattern-matching-specific AST optimisation code that refused to fold BinOp 
expressions that weren't complex numbers, allowing the compiler to reject them.

I also added two new tests (240b and 241b) to ensure that 0+0 & f"" were also 
rejected as mapping keys. 241b passes with the current PR, as f"" gets rejected 
by both the check in the AST validator *and* by the "constant or attribute 
lookup" restriction in the compiler, so the latter check covers for the leaky 
validator.

240b is disabled for now, as it won't pass until the AST validator checks all 
the subexpressions being used as keys in a mapping pattern (because it gets 
constant folded to "0", the check in the compiler accepts it).

That leaves fixing the unparser tests as the key thing to do before I create 
the main PR.

Before that, though, I'll look at potentially simplifying the MatchMapping code 
by changing the signature as Brandt suggested.

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-23 Thread Nick Coghlan


Nick Coghlan  added the comment:

Segfaults are fixed (I had a couple of cases where I wasn't checking for NULL, 
and another where I was looking up MatchMapping attributes on a MatchClass 
node).

test_patma passes except for test cases 240, 241, and 242, which look like 
genuine regressions - the logic to reject illegal MatchValue nodes is missing 
from the code generator side, and these particular values make it through the 
parser OK. I'll need to add back some of the code I took out so these still get 
rejected.

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-23 Thread Nick Coghlan


Nick Coghlan  added the comment:

My proposed MatchConstant node name was bothering me, as most constants are 
actually matched by equality in MatchValue, the same way variables are.

So I'm switching my proposed name for that node to be MatchSingleton, since 
that better reflects the key difference with MatchValue (i.e. comparing by 
identity rather than by value, and specifically comparing with the builtin 
singletons).

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-23 Thread Nick Coghlan


Nick Coghlan  added the comment:

I've removed the "?" from the end attributes for pattern nodes.

The draft PR branch now compiles, but I've clearly made a mistake somewhere as 
it segfaults when trying to compile test_patma.py.

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43897] Implement support for validation of pattern matching ASTs

2021-04-22 Thread Nick Coghlan


Change by Nick Coghlan :


--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue43897>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43897] Implement support for validation of pattern matching ASTs

2021-04-22 Thread Nick Coghlan


Change by Nick Coghlan :


--
priority: normal -> critical

___
Python tracker 
<https://bugs.python.org/issue43897>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-22 Thread Nick Coghlan


Nick Coghlan  added the comment:

The draft PR moved because I messed up the previous branch name: 
https://github.com/ncoghlan/cpython/pull/9/files

It's to the point where everything except symtable.c compiles.

I put an initial skeleton of a validator in, but only enough to check that the 
parallel sequence lengths in the class and mapping pattern nodes are consistent 
- there's still plenty to be done in Batuhan's PR (e.g. because the AST 
constant folding now just delegates to the expression folding functions for 
MatchValue values and MatchMapping keys, this iteration only enforces the 
restrictions on permitted subexpressions in the surface syntax).


In addition to Brandt's MatchStar suggestion, I've also tweaked the 
MatchClassparameter names to better match PEP 634 (the old names were based on 
PEP 642's attribute pattern concept, so "extra_*" made sense, but "kwd_*" makes 
more sense for PEP 634):

pattern = MatchAlways
 | MatchValue(expr value)
 | MatchConstant(constant value)
 | MatchSequence(pattern* patterns)
 | MatchMapping(expr* keys, pattern* patterns)
 | MatchClass(expr cls, pattern* patterns, identifier* kwd_attrs, 
pattern* kwd_patterns)

 | MatchStar(identifier? target)
 -- A NULL entry in the MatchMapping key list handles capturing extra 
mapping keys

 | MatchAs(pattern? pattern, identifier target)
 | MatchOr(pattern* patterns)
  attributes (int lineno, int col_offset, int? end_lineno, int? 
end_col_offset)

I think the idea of making the MatchMapping node signature "MatchMapping(expr* 
keys, pattern* patterns, identifier? rest)" has merit, but I'd like to get this 
initial iteration reviewed first to minimise the diff in the 
compiler_pattern_mapping implementation (right now it is fairly clear that the 
code generation for mapping patterns hasn't changed, but that will become 
significantly less obvious if/when the "**rest" handling changes away from 
working the same way _PyAST_Dict works)

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-22 Thread Nick Coghlan


Change by Nick Coghlan :


--
stage: patch review -> needs patch

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-22 Thread Nick Coghlan


Change by Nick Coghlan :


--
keywords: +patch
pull_requests: +24241
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/25521

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-20 Thread Nick Coghlan


Nick Coghlan  added the comment:

https://www.python.org/dev/peps/pep-0642/#representing-patterns-explicitly-in-the-abstract-syntax-tree
 covers the rationale for why it is potentially problematic to reuse expression 
nodes for match patterns: it doesn't semantically align with the fact that 
patterns are *not* expressions, so the AST ends up being closer to a concrete 
syntax tree than we would like.

For the specifics of the AST nodes:

* restricting sub expressions directly in the AST itself is tricky without side 
effects on the AST for non-pattern-matching code, so I'm intending to add any 
such restrictions that we want to enforce to the validator (other statements 
and expressions like yield, await, return, break, and continue already have 
separately enforced restrictions like that)
* cls needs to be an expression node to allow for attribute lookups
* the linked draft PR already updates the grammar to emit the new AST. 
MatchClass is able to collect the keyword args and their corresponding patterns 
in roughly the same way that MatchMapping collects its key:pattern pairs

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-20 Thread Nick Coghlan


Nick Coghlan  added the comment:

It's specifically the definition of "match_case" in the AST that is affected:

match_case = (pattern pattern, expr? guard, stmt* body)

pattern = MatchAlways
 | MatchValue(expr value)
 | MatchConstant(constant value)
 | MatchSequence(pattern* patterns)
 | MatchMapping(expr* keys, pattern* patterns)
 | MatchClass(expr cls, pattern* patterns, identifier* extra_attrs, 
pattern* extra_patterns)

 | MatchRestOfSequence(identifier? target)
 -- A NULL entry in the MatchMapping key list handles capturing extra 
mapping keys

 | MatchAs(pattern? pattern, identifier target)
 | MatchOr(pattern* patterns)
  attributes (int lineno, int col_offset, int? end_lineno, int? 
end_col_offset)

Relative to the PEP 642 AST, the notion of a "matchop" is gone - MatchValue 
always compares by equality, and there's a MatchConstant node for the identity 
comparisons against None, True, and False.

The grammar, code generator, AST validator, and unparser are then updated to 
produce or consume the new AST nodes.

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-20 Thread Nick Coghlan


Nick Coghlan  added the comment:

Agreed. I had wanted the AST to be part of the PEPs specifically *because* it's 
a public API, but didn't check until today whether or not the original PEP 634 
implementation had been merged as-is, or with a cleaned up AST definition.

https://github.com/ncoghlan/cpython/pull/8/files is the initial implementation 
with the AST and Grammar file updated.

I'll only create the CPython PR once the tests are passing, but I'll try to 
give Brandt at least a week to review it (I'll also ping him directly via 
email).

--

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43892] Make match patterns explicit in the AST

2021-04-20 Thread Nick Coghlan


New submission from Nick Coghlan :

In the SC submission ticket for PEP 642 
(https://github.com/python/steering-council/issues/43 ), Guido indicated he was 
in favour of the more explicit pattern matching AST changes suggested in that 
PEP.

This ticket covers adapting the pattern matching implementation to explicitly 
separate pattern nodes from expr nodes in the AST, so the code generator 
doesn't need to be context dependent.

--
assignee: ncoghlan
messages: 391430
nosy: brandtbucher, ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Make match patterns explicit in the AST
type: enhancement
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue43892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43133] Accept file mode "rw" to open-or-create a file and seek to the start

2021-02-04 Thread Nick Coghlan


New submission from Nick Coghlan :

After yet again trying to use "rw" to open a file in read/write mode, having it 
fail, and then having to look up the docs to remind myself that the correct 
spelling is "r+", I'm finally filing this to suggest we make the obvious "rw" 
spelling actually work.

Rather than making it an alias for any of the existing modes, I'd suggest that 
it combine the features of "r+" and "a+":

* like "r+" it would start the file pointer at the beginning of the file
* like "a+" it would create the file if it didn't exist, while leaving existing 
files alone

--
messages: 386510
nosy: ncoghlan
priority: low
severity: normal
status: open
title: Accept file mode "rw" to open-or-create a file and seek to the start
type: enhancement

___
Python tracker 
<https://bugs.python.org/issue43133>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42783] asyncio.sleep(0) idiom is not documented

2021-01-06 Thread Nick Coghlan


Nick Coghlan  added the comment:

I merged the update to the 3.10 docs, but the automated backport to 3.9 to 
update the online docs failed.

--
resolution:  -> fixed
stage: patch review -> backport needed
versions: +Python 3.10, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42783>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42783] asyncio.sleep(0) idiom is not documented

2021-01-06 Thread Nick Coghlan


Nick Coghlan  added the comment:


New changeset 5c30145afb6053998e3518befff638d207047f00 by Simon Willison in 
branch 'master':
bpo-42783: Documentation for asyncio.sleep(0) (#24002)
https://github.com/python/cpython/commit/5c30145afb6053998e3518befff638d207047f00


--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue42783>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42455] Add decorator_factory function to functools module

2021-01-05 Thread Nick Coghlan


Nick Coghlan  added the comment:

Sorry, I meant to include this comment as well:

The other filter I applied here was "Could someone figure out the boilerplate 
code themselves in less time than it would take them to find, read, and 
understand the documentation for the new helper function?"

I'm not sure the answer to that question is "Yes" when writing a decorator 
factory for the first time, but I am confident it is "Yes" when reading a 
decorator factory, and for anyone writing their second and subsequent factory 
(especially if they use the "def factory(decorated=None, /, *, keyword="only", 
params=True)" idiom the way dataclass and _tp_cache do).

--

___
Python tracker 
<https://bugs.python.org/issue42455>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42455] Add decorator_factory function to functools module

2021-01-05 Thread Nick Coghlan


Nick Coghlan  added the comment:

I think the idea is a plausible and well-presented suggestion, but I'm afraid 
I'm still going to agree with the view that we shouldn't add this.

>From a maintainability point of view, *generically* detecting the difference 
>between "applied without parentheses as a decorator" and "called with 
>parameters to produce a partial function to be used as a decorator" isn't 
>quite as simple as just calling "callable(args[0])", since it's entirely 
>possible for decorator factories to accept callables as parameters. We could 
>try to document our way around that problem, but we wouldn't be able to 
>eliminate it entirely.

Given the relative rarity of the use case, and the potential for subtle issues 
when the assumptions of the decorator_factory decorator aren't met, I'm happy 
to leave this task to inline mostly-but-not-completely-boilerplate code rather 
than trying to abstract it away.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42455>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33944] Deprecate and remove code execution in pth files

2020-12-31 Thread Nick Coghlan


Nick Coghlan  added the comment:

PEP 648 has been posted with a proposal to migrate sitecustomize.py, 
usersitecustomize.py and arbitrary code execution in pth files to a directory 
based `__sitecustomize__` structure: https://www.python.org/dev/peps/pep-0648/

--

___
Python tracker 
<https://bugs.python.org/issue33944>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42642] logging: add high priority log level for warnings being cleared

2020-12-14 Thread Nick Coghlan


New submission from Nick Coghlan :

When using the logging module for long running services, there's one limitation 
of the predefined logging levels that I semi-regularly run into: the only 
entirely reliable log level for reporting that a WARNING state has been cleared 
is itself WARNING.

Using INFO instead means that it is common to get error logs that show warning 
states being entered without any matching "error cleared" notification.

My idea for resolving this would be to define a new ESSENTIAL log level between 
WARNING and ERROR. The idea of the new log level would be to have a 
logged-by-default level for non-fault-but-essential messages like:

* service startup (including version info)
* service shutdown
* warnings being cleared

(NOTICE would be another possible name for such a level)

--
messages: 383027
nosy: ncoghlan, vinay.sajip
priority: low
severity: normal
stage: needs patch
status: open
title: logging: add high priority log level for warnings being cleared
type: enhancement
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42642>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42260] [C API] Add PyInterpreterState_SetConfig(): reconfigure an interpreter

2020-11-14 Thread Nick Coghlan


Change by Nick Coghlan :


--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue42260>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42282] Constant folding is skipped in named expressions

2020-11-07 Thread Nick Coghlan


Change by Nick Coghlan :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42282] Constant folding is skipped in named expressions

2020-11-07 Thread Nick Coghlan


Nick Coghlan  added the comment:

Since this was only a performance issue, I'm not planning to backport it to 
earlier releases.

--
stage: patch review -> resolved

___
Python tracker 
<https://bugs.python.org/issue42282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42282] Constant folding is skipped in named expressions

2020-11-07 Thread Nick Coghlan


Nick Coghlan  added the comment:


New changeset 8805a4dad201473599416b2c265802b8885f69b8 by Nick Coghlan in 
branch 'master':
bpo-42282: Fold constants inside named expressions (GH-23190)
https://github.com/python/cpython/commit/8805a4dad201473599416b2c265802b8885f69b8


--

___
Python tracker 
<https://bugs.python.org/issue42282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42282] Constant folding is skipped in named expressions

2020-11-07 Thread Nick Coghlan


Change by Nick Coghlan :


--
keywords: +patch
pull_requests: +22093
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/23190

___
Python tracker 
<https://bugs.python.org/issue42282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42282] Constant folding is skipped in named expressions

2020-11-07 Thread Nick Coghlan


Change by Nick Coghlan :


--
assignee:  -> ncoghlan

___
Python tracker 
<https://bugs.python.org/issue42282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42282] Constant folding is skipped in named expressions

2020-11-06 Thread Nick Coghlan


New submission from Nick Coghlan :

While working on the PEP 642 reference implementation I removed the "default:" 
case from the switch statement in astfold_expr as part of making sure the new 
SkippedBinding node was being handled everywhere it needed to be.

This change picked up that NamedExpr_kind wasn't being handled either, and a 
check with the dis module confirmed that using the walrus operator turns off 
constant folding:

```
[ncoghlan@thechalk cpython]$ ./python
Python 3.10.0a2+ (heads/pep-642-constraint-patterns-dirty:4db2fbd609, Nov  7 
2020, 16:42:19) 
[GCC 10.2.1 20201016 (Red Hat 10.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dis
>>> dis.dis("1 + 1")
  1   0 LOAD_CONST   0 (2)
  2 RETURN_VALUE
>>> dis.dis("(x := 1 + 1)")
  1   0 LOAD_CONST   0 (1)
  2 LOAD_CONST   0 (1)
  4 BINARY_ADD
  6 DUP_TOP
  8 STORE_NAME   0 (x)
 10 RETURN_VALUE
>>>
```

The missing switch statement entry is just:

```
case NamedExpr_kind:
CALL(astfold_expr, expr_ty, node_->v.NamedExpr.value);
break;
```

Which gives the expected result:

```
[ncoghlan@thechalk cpython]$ ./python -c "import dis; dis.dis('(x := 1 +1)')"
  1   0 LOAD_CONST   0 (2)
  2 DUP_TOP
  4 STORE_NAME   0 (x)
  6 RETURN_VALUE
```

--
messages: 380494
nosy: ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Constant folding is skipped in named expressions
type: performance
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40952] GCC overflow warnings (format-overflow, stringop-overflow)

2020-11-06 Thread Nick Coghlan


Nick Coghlan  added the comment:

I *think* the lnotab one is the compiler failing to detect that the pointer has 
been updated to point inside the body of a Python object, but I'm also not 100% 
sure that it's a false alarm.

--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue40952>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42186] unittest overrides more serious warnings filter added before unittest.main()

2020-10-30 Thread Nick Coghlan


Nick Coghlan  added the comment:

Closing the old one as partially fixed, and linking here as a superseder makes 
sense to me, so I went ahead and did that.

--

___
Python tracker 
<https://bugs.python.org/issue42186>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15626] unittest.main negates -bb option and programmatic warning configuration

2020-10-30 Thread Nick Coghlan


Nick Coghlan  added the comment:

Issue #42186 now covers the request to have a way to tell unittest to leave the 
warnings filter alone even if it's set programmatically rather than by 
modifying ``sys.warnoptions``.

(Changing version to 3.7 to indicate when the interaction with `-bb` was fixed, 
rather than the version it was reported against)

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions: +Python 3.7 -Python 3.6

___
Python tracker 
<https://bugs.python.org/issue15626>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15626] unittest.main negates -bb option and programmatic warning configuration

2020-10-30 Thread Nick Coghlan


Change by Nick Coghlan :


--
superseder:  -> unittest overrides more serious warnings filter added before 
unittest.main()

___
Python tracker 
<https://bugs.python.org/issue15626>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32002] test_c_locale_coercion fails when the default LC_CTYPE != "C"

2020-06-28 Thread Nick Coghlan


Nick Coghlan  added the comment:

Removing issue assignment, as I'm no longer actively investigating this.

--
assignee: ncoghlan -> 

___
Python tracker 
<https://bugs.python.org/issue32002>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30672] PEP 538: Unexpected locale behaviour on *BSD (including Mac OS X)

2020-06-28 Thread Nick Coghlan


Nick Coghlan  added the comment:

Removing issue assignment, as I'm not actively investigating this.

--
assignee: ncoghlan -> 

___
Python tracker 
<https://bugs.python.org/issue30672>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34206] Move and clarify Py_Main documentation

2020-06-28 Thread Nick Coghlan


Nick Coghlan  added the comment:

Added 3.8 back in to the target versions. However, if the automatic backport 
doesn't work for that branch, I'll probably skip it rather than fixing any 
conflicts.

--
versions: +Python 3.8

___
Python tracker 
<https://bugs.python.org/issue34206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34206] Move and clarify Py_Main documentation

2020-06-28 Thread Nick Coghlan


Nick Coghlan  added the comment:

Adjusted target versions, as I never previously got around to merging this PR.

--
versions: +Python 3.10, Python 3.9 -Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue34206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31898] Add a `recommended-packages.txt` file

2020-06-28 Thread Nick Coghlan


Nick Coghlan  added the comment:

Removing the issue assignment, as I'm not actively working on this (although I 
still think it's a reasonable idea).

--
assignee: ncoghlan -> 

___
Python tracker 
<https://bugs.python.org/issue31898>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31899] Ensure backwards compatibility with recommended packages

2020-06-28 Thread Nick Coghlan


Nick Coghlan  added the comment:

Removing the issue assignment, as I'm not actively working on this (although I 
still think it's a reasonable idea).

--
assignee: ncoghlan -> 

___
Python tracker 
<https://bugs.python.org/issue31899>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17490] Improve ast.literal_eval test suite coverage

2020-06-28 Thread Nick Coghlan


Nick Coghlan  added the comment:

Belatedly removing the issue assignment here, as I'm not actively working on 
this.

I've also marked this as an easy newcomer friendly task, as all that's involved 
is taking the `test_ast.py` changes from 
https://bugs.python.org/file29520/issue17490_ast_literal_eval_converters.diff 
and turning them into a test suite PR on GitHub.

--
assignee: ncoghlan -> 
keywords: +easy, newcomer friendly -patch
stage: patch review -> needs patch

___
Python tracker 
<https://bugs.python.org/issue17490>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29988] with statements are not ensuring that __exit__ is called if __enter__ succeeds

2020-06-28 Thread Nick Coghlan


Nick Coghlan  added the comment:

Belatedly clearing the issue assignment here - while I do still sometimes 
ponder this problem, I haven't been actively working on it since the 2017 core 
sprint where Greg & I made our last serious attempt at trying to improve the 
situation.

Mark's PR at https://github.com/python/cpython/pull/18334 looks very promising 
to me, though - my main request was just to bring over the tests I wrote at the 
2017 core dev sprints, and confirm that the revised eval breaker logic solves 
the issue.

--
assignee: ncoghlan -> 

___
Python tracker 
<https://bugs.python.org/issue29988>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24048] remove_module() needs to save/restore exception state

2020-06-28 Thread Nick Coghlan


Nick Coghlan  added the comment:

Belatedly marking this as resolved.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue24048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34820] binascii.c:1578:1: error: the control flow of function ‘binascii_crc32’ does not match its profile data (counter ‘arcs’)

2020-06-17 Thread Nick Coghlan

Nick Coghlan  added the comment:

I'm seeing this as well when attempting to run an optimised Python 3.8 build on 
an old Debian 9 system (with the curses and socket extension modules).

For example:

cpython/Modules/socketmodule.c: In function ‘PyInit__socket’:
cpython/Modules/socketmodule.c:8304:1: error: the control flow of function 
‘PyInit__socket’ does not match its profile data (counter ‘arcs’) 
[-Werror=coverage-mismatch]
cpython/Modules/socketmodule.c:8304:1: error: the control flow of function 
‘PyInit__socket’ does not match its profile data (counter ‘time_profiler’) 
[-Werror=coverage-mismatch]

That said, I did install missing optional dependendencies and then do a 
reconfigure without doing a "make clean", which is pretty dubious.

So David's assessment here sounds good to me: the bug is that "make clean" 
should be removing the intermediate PGO files, but is currently leaving them 
lying around.

--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue34820>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33944] Deprecate and remove code execution in pth files

2020-05-12 Thread Nick Coghlan


Nick Coghlan  added the comment:

While it's still not entirely accurate, I've tweaked the title on the issue to 
refer to the arbitrary code execution behavior.

Getting "Make pth file sys.path modifications easier to debug" in there as well 
would be rather tricky :)

--

___
Python tracker 
<https://bugs.python.org/issue33944>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33944] Deprecate and remove code execution in pth files

2020-05-12 Thread Nick Coghlan


Change by Nick Coghlan :


--
title: Deprecate and remove pth files -> Deprecate and remove code execution in 
pth files

___
Python tracker 
<https://bugs.python.org/issue33944>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28180] Implementation of the PEP 538: coerce C locale to C.utf-8

2020-03-22 Thread Nick Coghlan


Nick Coghlan  added the comment:

The test cases for locale coercion *not* triggering still assume that 
bpo-19977, using surrogateescape on the standard streams in the POSIX locale, 
has been implemented (since that was implemented in Python 3.5).

Hence the various test cases complaining that they found "ascii:strict" (Py 3.4 
behaviour without bpo-19977) where they expected "ascii:surrogateescape" (the 
Py 3.5+ behaviour *with* bpo-19977).

To get a PEP 538 backport to work as intended on 3.4, you'd need to backport 
that earlier IO stream error handling change as well.

--

___
Python tracker 
<https://bugs.python.org/issue28180>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39976] Add "**other_popen_kwargs" to subprocess API signatures in docs

2020-03-16 Thread Nick Coghlan


New submission from Nick Coghlan :

Two of my colleagues missed the "The arguments shown above are merely the most 
common ones, ..." caveat on the subprocess.run documentation, and assumed that 
Python 3.5 only supported the "cwd" option in the low level Popen API, and not 
any of the higher level APIs.

Something we could potential do is include a "**other_popen_kwargs" placeholder 
in the affected API signatures (run, call, check_call, check_output) that makes 
it clear there are more options beyond the explicitly listed ones.

--
assignee: docs@python
components: Documentation
messages: 364295
nosy: docs@python, ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Add "**other_popen_kwargs" to subprocess API signatures in docs
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39976>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39892] Enable DeprecationWarnings by default when not explicit in unittest.main()

2020-03-15 Thread Nick Coghlan


Nick Coghlan  added the comment:

Issue 10535 says they should already be on by default in unittest.

I seem to recall checking that was still true when implementing the default 
warning filter changes in 3.7.

--

___
Python tracker 
<https://bugs.python.org/issue39892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39824] Multi-phase extension module (PEP 489): don't call m_traverse, m_clear nor m_free before the module state is allocated

2020-03-15 Thread Nick Coghlan


Nick Coghlan  added the comment:

Petr's point that any subclass state should be managed in the subclass cleanup 
functions is a good one, so I withdraw my concern:

* custom module subclasses should clean up like any other class instance
* the module slots are then only needed if the module state actually gets 
populated, so we can safely skip them if it never even gets allocated

--

___
Python tracker 
<https://bugs.python.org/issue39824>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39725] unrelated `from None` exceptions hide prior exception information

2020-03-15 Thread Nick Coghlan


Nick Coghlan  added the comment:

Tweaked title to be "hide" rather "lose" (all the info is theoretically still 
there in various places, it's just hidden by the default traceback printing 
machinery, and hard to extract from the exception tree).

Issue #18861 is the original report of this problem.

My concern with Serhiy's suggestion is that it would take us from merely hiding 
information to actually losing it.

I'm wondering if we're actually missing a piece of lower level infrastructure: 
a query API that reports *all* live exceptions in a thread.

For example, there might be a "sys.active_exceptions()" API that returned a 
list of all active exceptions.

* []: No exception active
* [exc]: Running an except/finally/__exit__ for "exc"
* [exc_inner, exc_outer]: Running an except/finally/__exit__ for "exc_inner" 
inside one for "exc_outer"
* etc

So in Serhiy's example, at the point where the context is suppressed, that new 
API would report [ValueError(), TypeError()] (with the relevant caught 
exception instances).

Given that extra low level API, we would gain more options for populating 
attributes on exceptions.

Alternatively, Serhiy's suggested approach might be sufficient, even without 
that new API, if we added a new __suppressed_context__ attribute to capture the 
original value of __context__ before it was replaced.

--
title: unrelated  `from None` exceptions lose prior exception information -> 
unrelated  `from None` exceptions hide prior exception information

___
Python tracker 
<https://bugs.python.org/issue39725>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34822] Simplify AST for slices

2020-03-07 Thread Nick Coghlan


Nick Coghlan  added the comment:

The one thing in the PR that makes me slightly wary is the point Vedran raised: 
in the old AST _Unparser code, the fact that index tuples containing slices 
should be printed without parentheses was encapsulated in the ExtSlice node 
type, but with Index and ExtSlice removed, that behaviour is now instead 
implemented by duplicating the visit_Tuple logic directly in visit_Subscript, 
purely to omit the surrounding parens.

So I agree that the parse tree simplification is an improvement, but I'm 
wondering if it might make sense to add an "is_subscript" attribute to 
expression nodes.

That way the Tuple vs ExtSlice and Expr vs Index distinctions would be 
preserved, but the representation of those distinctions would change in a way 
that meant that most consuming code didn't need to care that the distinctions 
existed (ExtSlice would become a Tuple with is_subscript set to True, and 
similarly, Index would become Expr instances with is_subscript set to True).

It's been so long since I changed the AST, though, that I'm not sure how 
painful adding that attribute would be.

--

___
Python tracker 
<https://bugs.python.org/issue34822>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39824] Multi-phase extension module (PEP 489): don't call m_traverse, m_clear nor m_free before the module state is allocated

2020-03-07 Thread Nick Coghlan


Nick Coghlan  added the comment:

One of the intended use cases for Py_mod_create is to return instances of 
ModuleType subclasses rather than straight ModuleType instances. And those are 
definitely legal to define:

>>> import __main__
>>> class MyModule(type(__main__)): pass
... 
>>> m = MyModule('example')
>>> m


So it isn't valid to skip calling the cleanup functions just because md_state 
is NULL - we have no idea what Py_mod_create might have done that needs to be 
cleaned up.

It would *probably* be legitimate to skip calling the cleanup functions when 
there's no Py_mod_create slot defined, but then the rules for "Do I need to 
account for md_state potentially being NULL or not?" are getting complicated 
enough that the safest option for a module author is to always assume that 
md_state might be NULL and handle that case appropriately.

--

___
Python tracker 
<https://bugs.python.org/issue39824>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39465] Design a subinterpreter friendly alternative to _Py_IDENTIFIER

2020-02-08 Thread Nick Coghlan


Nick Coghlan  added the comment:

As Petr notes, as long as all subinterpreters share the GIL, and share str 
instances, then the existing _Py_IDENTIFIER mechanism will work fine for both 
single phase and multi-phase initialisation.

However, that constraint also goes the other way: as long as we have modules 
that use the existing _Py_IDENTIFIER mechanism, then subinterpreters *must* 
share str instances, and hence *must* share the GIL.

Hence the "enhancement" classification: there's nothing broken right now, but 
if we're ever going to achieve the design goal of using subinterpreters to 
exploit multiple CPU cores without the overhead of running multiple full 
interpreter processes, we're going to need to design a different way of 
handling this.

Something to keep in mind with `_Py_IDENTIFIER` and any replacement API: the 
baseline for performance comparisons is 
https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_InternFromString

The reason multi-phase initialisation makes this more complicated is that it 
means we can't use the memory addresses of C process globals as unique 
identifiers any more, since more than one module object may be created from the 
same C shared library.

However, if we assume we've moved to per-module state storage (to get unique 
memory addresses back), then we can largely re-use the existing 
`_Py_IDENTIFIER` machinery to make the lookup as fast as possible, while still 
avoiding conflicts between subinterpreters.

--

___
Python tracker 
<https://bugs.python.org/issue39465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39487] Merge duplicated _Py_IDENTIFIER identifiers in C code

2020-01-29 Thread Nick Coghlan


Nick Coghlan  added the comment:

In the subinterpreter context: perhaps it would make sense to move *all* 
Py_IDENTIFIER declarations to file scope?

That would make it much clearer which of our extension modules actually have 
hidden state for caching purposes.

If we did that though, we'd also want to update the usage instructions in 
object.h

--

___
Python tracker 
<https://bugs.python.org/issue39487>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39487] Merge duplicated _Py_IDENTIFIER identifiers in C code

2020-01-29 Thread Nick Coghlan


Nick Coghlan  added the comment:

My apologies, my comment above was based on an outdated understanding of how 
the identifier structs get initialised (it's the usage that initialises them, 
not the declaration).

That means this is a useful refactoring to help identify blockers to full 
subinterpreter support.

--
resolution: rejected -> 
stage: resolved -> patch review
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue39487>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39487] Merge duplicated _Py_IDENTIFIER identifiers in C code

2020-01-29 Thread Nick Coghlan


Nick Coghlan  added the comment:

We can't make this change, as it means the statics get initialised before the 
Python interpreter has been initialised, and won't be reinitialised if the 
interpreter is destroyed and recreated.

--
nosy: +ncoghlan
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39487>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39153] Clarify refcounting semantics of PyDict_SetItem[String]

2020-01-29 Thread Nick Coghlan


Nick Coghlan  added the comment:


New changeset e1e80002e28e1055f399a20918c49d50d093709e by Joannah Nanjekye in 
branch 'master':
bpo-39153: Clarify C API *SetItem refcounting semantics (GH-18220)
https://github.com/python/cpython/commit/e1e80002e28e1055f399a20918c49d50d093709e


--

___
Python tracker 
<https://bugs.python.org/issue39153>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39465] Design a subinterpreter friendly alternative to _Py_IDENTIFIER

2020-01-27 Thread Nick Coghlan


Change by Nick Coghlan :


--
type:  -> enhancement

___
Python tracker 
<https://bugs.python.org/issue39465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39465] Design a subinterpreter friendly alternative to _Py_IDENTIFIER

2020-01-27 Thread Nick Coghlan


New submission from Nick Coghlan :

Both https://github.com/python/cpython/pull/18066 (collections module) and 
https://github.com/python/cpython/pull/18032 (asyncio module) ran into the 
problem where porting them to multi-phase initialisation involves replacing 
their usage of the `_Py_IDENTIFIER` macro with some other mechanism.

When _posixsubprocess was ported, the replacement was a relatively ad hoc 
combination of string interning and the interpreter-managed module-specific 
state: 
https://github.com/python/cpython/commit/5a7d2e11aaea2dd32878dc5c6b1aae8caf56cb44

I'm wondering if we may able to devise a comparable struct-field based system 
that replaces the `_Py_IDENTIFIER` local static variable declaration macro and 
the `Py_Id_` lookup convention with a combination like (using the posix 
subprocess module conversion as an example):

// Identifier usage declaration (replaces _Py_IDENTIFIER)
_Py_USE_CACHED_IDENTIFIER(_posixsubprocessstate(m), disable);

// Identifier usage remains unchanged, but uses a regular local variable
// rather than the static variable declared by _Py_IDENTIFIER
result = _PyObject_CallMethodIdNoArgs(gc_module, _disable);

And then the following additional state management macros would be needed to 
handle the string interning and reference counting:

// Module state struct declaration
typedef struct {
// This would declare an initialised array of _Py_Identifier structs
// under a name like __cached_identifiers__. The end of the array
// would be indicated by a strict with "value" set to NULL.
_Py_START_CACHED_IDENTIFIERS;
_Py_CACHED_IDENTIFIER(disable);
_Py_CACHED_IDENTIFIER(enable);
_Py_CACHED_IDENTIFIER(isenabled);
_Py_END_CACHED_IDENTIFIERS;
);
} _posixsubprocessstate;

// Module tp_traverse implementation
_Py_VISIT_CACHED_IDENTIFIERS(_posixsubprocessstate(m));

// Module tp_clear implementation (also called by tp_free)
_Py_CLEAR_CACHED_IDENTIFIERS(_posixsubprocessstate(m));

With the requirement to declare usage of the cached identifiers, they could be 
lazily initialized the same way the existing static variables are (even 
re-using the same struct declaration).

Note: this is just a draft of one possible design, the intent of this issue is 
to highlight the fact that this issue has now come up multiple times, and it 
would be good to have a standard answer available.

--
messages: 360766
nosy: eric.snow, ncoghlan, petr.viktorin, shihai1991
priority: normal
severity: normal
status: open
title: Design a subinterpreter friendly alternative to _Py_IDENTIFIER

___
Python tracker 
<https://bugs.python.org/issue39465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details

2020-01-20 Thread Nick Coghlan


Nick Coghlan  added the comment:


New changeset 1e420f849d0c094098543d2c27d35eaec69b2784 by Nick Coghlan in 
branch 'master':
bpo-35134: Migrate frameobject.h contents to cpython/frameobject.h (GH-18052)
https://github.com/python/cpython/commit/1e420f849d0c094098543d2c27d35eaec69b2784


--

___
Python tracker 
<https://bugs.python.org/issue35134>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details

2020-01-20 Thread Nick Coghlan


Change by Nick Coghlan :


--
pull_requests: +17475
pull_request: https://github.com/python/cpython/pull/18052

___
Python tracker 
<https://bugs.python.org/issue35134>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

2020-01-18 Thread Nick Coghlan


Change by Nick Coghlan :


--
pull_requests: +17447
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18052

___
Python tracker 
<https://bugs.python.org/issue30744>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39048] Change the lookup order of __aenter__ and __aexit__ for async with

2020-01-14 Thread Nick Coghlan


Change by Nick Coghlan :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type: behavior -> enhancement
versions:  -Python 3.8

___
Python tracker 
<https://bugs.python.org/issue39048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39048] Change the lookup order of __aenter__ and __aexit__ for async with

2020-01-14 Thread Nick Coghlan

Nick Coghlan  added the comment:


New changeset 1d1b97ae643dd8b22d87785ed7bd2599c6c8dc8d by Nick Coghlan (Géry 
Ogam) in branch 'master':
bpo-39048: Look up __aenter__ before __aexit__ in async with (GH-17609)
https://github.com/python/cpython/commit/1d1b97ae643dd8b22d87785ed7bd2599c6c8dc8d


--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue39048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39301] Specification of bitshift on integers should clearly state floor division used

2020-01-11 Thread Nick Coghlan


Nick Coghlan  added the comment:

Aye, adding "floor" to the existing footnote would be the minimal fix. I'm just 
wondering whether it's also worth stating that this means that positive 
integers saturate at zero, while negative integers saturate at -1.

--

___
Python tracker 
<https://bugs.python.org/issue39301>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26515] Update extending/embedding docs to new way to build modules in C

2020-01-11 Thread Nick Coghlan


Nick Coghlan  added the comment:

Changed target version as per Petr's comment (PEP 573 is close to being 
accepted for 3.9 - it just needs some editing to improve clarity in the PEP 
itself, rather than needing any changes to the technical proposal)

--
versions: +Python 3.9 -Python 3.6, Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue26515>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39302] Language reference does not clearly describe modern operand coercion

2020-01-11 Thread Nick Coghlan


New submission from Nick Coghlan :

While reviewing ISO-IECJTC1-SC22-WG23's latest draft of their Python security 
annex, I found a description of operand coercion that was based on the legacy 
coercion model described at https://docs.python.org/2.5/ref/coercion-rules.html

That's still the second highest link if you search for "Python operand 
coercion", while the highest link is this old, very brief, summary from Python 
in a Nutshell: 
https://www.oreilly.com/library/view/python-in-a/0596001886/ch04s05.html (still 
based on the old semantics where the inputs were coerced to a common type 
before calling the slot method, rather than giving the method direct access to 
the original operands).

The third highest link at least goes to PEP 208 
(https://www.python.org/dev/peps/pep-0208/), which correctly describes the 
modern semantics, but it describes them in terms of the CPython C slot API, not 
the Python level special method APIs.

https://docs.python.org/3.7/reference/datamodel.html#emulating-numeric-types 
does technically provide the required information, but it's implicit in the 
description of the numeric arithmetic methods, rather than being clearly 
spelled out as a clear description of "Python operand coercion". (There are 
also some oddities around operand coercion for three-argument pow() that I'm 
going to file as their own issue)

https://docs.python.org/3/library/constants.html#NotImplemented references 
https://docs.python.org/3/library/numbers.html#implementing-the-arithmetic-operations
 which describes defining new numeric ABCs in a coercion-friendly way, but 
still doesn't spell out the operand precedence and coercion dance.

We could likely improve this situation by adding a new "Special method 
invocation" subject at the end of 
https://docs.python.org/3.7/reference/datamodel.html, moving the existing 
"Special method lookup" subsection under it, and then adding a second 
subsection called "Operand precedence and coercion".

That new subsection would then cover the basic principles of:

* for unary operands, there is no ambiguity
* for binary operands of the same type, only the forward operation is tried (it 
is assumed that if the forward operation doesn't work, the reflected one won't 
either)
* for binary operands where the type of the RHS is a subclass of the type of 
the LHS, the reflected operation is tried first (if it exists), followed by the 
forward operation if the reflected call does not exist or returns the 
NotImplemented singleton
* for binary operands of unrelated types, the forward operation is tried first 
(if it exists), followed by the reflected operation if the forward call does 
not exist or returns the NotImplemented singleton
* for ternary operands (i.e. 3 argument pow()), the behaviour is currently 
implementation defined (as the test suite doesn't enforce any particular 
behaviour, and what CPython does isn't obviously useful)

Other specific points to be covered would be:

* any argument coercion that occurs is up to the individual method 
implementations
* logical short-circuiting expressions (and, or, if/else) only call the 
equivalent of bool(expr)

While the corresponding reflected operations for the binary operators are 
covered in the documentation of the forward operations, it would also likely be 
worthwhile including a summary table in this new subsection of exactly which 
special methods support reflection, and what the reflected method names are.

--
messages: 359788
nosy: ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Language reference does not clearly describe modern operand coercion
type: enhancement
versions: Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39301] Specification of bitshift on integers should clearly state floor division used

2020-01-10 Thread Nick Coghlan


New submission from Nick Coghlan :

While reviewing ISO-IECJTC1-SC22-WG23's latest draft of their Python security 
annex, I noticed that 
https://docs.python.org/3.7/library/stdtypes.html#bitwise-operations-on-integer-types
 doesn't explicitly state that *floor* division is used for right shift 
operations, so right-shifting a negative number by more bits than it contains 
gives -1 rather than 0.

This is consistent with the way the language spec defines both binary 
right-shifts (as division by "pow(2, n)" and floor division (as rounding 
towards negative infinity), so this is just a documentation issue to note that 
we should make it clearer that this behaviour is intentional.

--
assignee: docs@python
components: Documentation
messages: 359786
nosy: docs@python, ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Specification of bitshift on integers should clearly state floor 
division used
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39301>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37194] Move new vector private declarations to the internal C API

2020-01-07 Thread Nick Coghlan


Change by Nick Coghlan :


--
pull_requests:  -17306

___
Python tracker 
<https://bugs.python.org/issue37194>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36476] Runtime finalization assumes all other threads have exited.

2019-12-30 Thread Nick Coghlan


Nick Coghlan  added the comment:

Thinking about that idea further, I don't think that change would help much, 
since the relevant operations should already be checking for thread termination 
when they attempt to reacquire the GIL.

That means what we're missing is:

1. When daemon threads still exist after the non-daemon threads terminate, 
deliberately giving them additional time to run (and hence terminate)
2. Explicitly attempting to kick daemon threads out of blocking system calls by 
sending them signals to provoke EINTR (I have no idea if there's a windows 
equivalent for this, but we should be able to use pthread_kill on POSIX 
systems. However, choosing *which* wakeup signal to send could be fraught with 
compatibility problems)

--

___
Python tracker 
<https://bugs.python.org/issue36476>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36476] Runtime finalization assumes all other threads have exited.

2019-12-30 Thread Nick Coghlan


Nick Coghlan  added the comment:

Perhaps we need a threading.throw() API, similar to the one we have for 
generators and coroutines?

If we had that, then Py_FinalizeEx() could gain a few new features:

* throw SystemExit into all daemon threads and then give them a chance to 
terminate before calling any atexit handlers (printing a warning if some of the 
threads don't exit)
* throw SystemExit into all daemon and non-daemon threads after running atexit 
handlers (printing a warning if any such threads exist at all, along with 
another warning if some of the threads don't exit)

Adding that would require an enhancement to the PendingCall machinery, though, 
since most pending calls are only processed in the main thread (there's no way 
to route them to specific child threads).

A simpler alternative would be to have an atomic "terminate_threads" counter in 
the ceval runtime state that was incremented to 1 to request that SystemExit be 
raised in daemon threads, and then to 2 to request that SystemExit be raised in 
all still running threads. When a thread received that request to exit, it 
would set a new flag in the thread state to indicate it was terminating, and 
then raise SystemExit. (The thread running Py_FinalizeEx would set that flag in 
advance so it wasn't affected, and other threads would use it to ensure they 
only raised SystemExit once). The runtime cost of this would just be another 
_Py_atomic_load_relaxed call in the eval_breaker branch. (Probably inside 
`make_pending_calls`, so it gets triggered both by the eval_breaker logic, and 
by explicit calls to `Py_MakePendingCalls`).

--

___
Python tracker 
<https://bugs.python.org/issue36476>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20443] __code__. co_filename should always be an absolute path

2019-12-30 Thread Nick Coghlan


Nick Coghlan  added the comment:

With the sys.argv[0] change reverted, I think this overall issue is fixed now - 
code objects will get absolute paths, while sys.argv[0] will continue to 
reflect how __main__ was identified.

--
priority:  -> normal
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue20443>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39037] Fix the trial order of the __exit__ and __enter__ methods in the with statement documentation

2019-12-29 Thread Nick Coghlan

Nick Coghlan  added the comment:


New changeset 226e6e7d4326cf91ef37e13528eb1f62de1bb832 by Nick Coghlan (Géry 
Ogam) in branch 'master':
bpo-39037: Fix lookup order of magic methods in with statement documentation 
(GH-17608)
https://github.com/python/cpython/commit/226e6e7d4326cf91ef37e13528eb1f62de1bb832


--

___
Python tracker 
<https://bugs.python.org/issue39037>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39161] Py_NewInterpreter docs need updating for multi-phase initialization

2019-12-29 Thread Nick Coghlan


New submission from Nick Coghlan :

The Py_NewInterpreter docs only cover the behaviour of extension modules that 
use single-phase initialization: 
https://docs.python.org/3/c-api/init.html#c.Py_NewInterpreter

Multi-phase initialization allows each subinterpreter to get its own copy of 
extension modules as well, with only C/C++ level static and global variables 
being shared.

--
assignee: docs@python
components: Documentation
messages: 359019
nosy: docs@python, ncoghlan, petr.viktorin
priority: normal
severity: normal
stage: needs patch
status: open
title: Py_NewInterpreter docs need updating for multi-phase initialization
type: enhancement
versions: Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39161>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39153] Clarify refcounting semantics of PyDict_SetItem[String]

2019-12-29 Thread Nick Coghlan


Nick Coghlan  added the comment:

Right, that's why I don't think the other "*SetItem*" operations should get a 
Sphinx note - just a sentence, as was already done for PySequence_SetItem.

If it weren't for PyList_SetItem being different, none of the others would need 
the clarification at all.

--

___
Python tracker 
<https://bugs.python.org/issue39153>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39153] Clarify refcounting semantics of PyDict_SetItem[String]

2019-12-29 Thread Nick Coghlan


Change by Nick Coghlan :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
stage:  -> needs patch
type:  -> enhancement
versions: +Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39153>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   6   7   8   9   10   >