[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-07-21 Thread Eric Snow


Change by Eric Snow :


--
assignee: eric.snow -> 
stage: patch review -> needs patch

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-17 Thread Eric Snow


Eric Snow  added the comment:

FWIW, I've wrapped up the key parts that I wanted to get done here
(co_localplusnames/kinds, MAKE_CELL, eliminate unused fast local
for arg cells).  I'm leaving this open for now as there are a few
things I didn't do that seem part of the original intention of this
issue:

* fully interleave the cells with the locals in their "natural" order
  (rather than only interleaving arg cells)
* update the compiler to track names/kinds rather than computing
  them after the fact during the assembler step
  (this will allow us to remove a decent amount of code)
* track the specific arg kinds in localspluskinds
  (this should allow us to make _PyEval_MakeFrameVector() simpler
  and a bit more efficient)

--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-15 Thread Eric Snow


Eric Snow  added the comment:


New changeset ac38a9f2dfbba95f5d4338eb11a0221d38ef9328 by Eric Snow in branch 
'main':
bpo-43693: Eliminate unused "fast locals". (gh-26587)
https://github.com/python/cpython/commit/ac38a9f2dfbba95f5d4338eb11a0221d38ef9328


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21760] inspect documentation describes module type inaccurately

2021-06-09 Thread Eric Snow


Eric Snow  added the comment:

I've merged the changes for __file__.  Thanks, furkanonder!

The fixes in the types module remain to be done, though now I see 4 of the 
relevant attributes instead of 2.  (missing: __path__, __file__, __cached__)

--

___
Python tracker 
<https://bugs.python.org/issue21760>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42088] types.SimpleNamespace.__repr__ documentation inconsistency

2021-06-09 Thread Eric Snow


Eric Snow  added the comment:

> According to the documentation for types.SimpleNamespace, 
> `repr(SimpleNamespace())`
> should return `"SimpleNamespace()"`, but in actuality returns `"namespace()"`.

Note that I purposefully wrote "roughly" in the docs ("The type is roughly 
equivalent to the following code:").  That code was meant to illustrate the 
functionality rather than be proscriptive of the implementation.  That said, it 
certainly would be good for the documentation to match. :)

> but also the (perhaps less interesting issue) of 
> `eval(repr(SimpleNamespace))` resulting in a NameError.

This is true of many of the types exposed by the types module (e.g. 
types.CodeType).

> I propose that `_PyNamespaceObject`'s __repr__ method be changed to return 
> `"SimpleNamespace()"`.

My preference would be as outlined in my previous comment: make it a builtin.  
However, I'm not in a position to make that happen at the moment.

--

___
Python tracker 
<https://bugs.python.org/issue42088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42088] types.SimpleNamespace.__repr__ documentation inconsistency

2021-06-09 Thread Eric Snow


Eric Snow  added the comment:

I'm not sure what to think about this.

The type is explicitly exposed to Python code "SimpleNamespace" via the types 
module.  However, that's the same as how other builtin types are exposed in 
that module.  For example, the builtin `PyCode_Type` (for `PyCodeObject`) is 
exposed as `types.CodeType`.  I called it "SimpleNamespace" because "Namespace" 
(or "NamespaceType") was a little too unclear and I wanted it to say what it 
was.  In retrospect, "SimpleNamespace" was probably a good choice.

FYI, `tp_name` was originally set to "namespace", rather than 
"types.SimpleNamespace".  I changed it to "types.SimpleNamespace" to add pickle 
support (in b5c8f927829a1679c6748df84a6289fb68343e51, for bpo-15022).  IIRC, 
changing the name was the easiest was to allow pickle to find it.  That said, 
it would be better to do it properly and set `tp_name` to the correct value.

It would probably help to have a little more context on the name and my 
intentions. 
 When I added it (for use in the PEP 421 implementation), I thought of the type 
as one that others would find very useful.  So I imagined we would eventually 
expose it as one of the builtins.  Basically all the other builtin types have 
lowercase names, hence I named it "namespace".  I just didn't get around to 
proposing adding it to the builtins and it fell off my radar.  Note that 
changing the name to "types.SimpleNamespace", while primarly to benefit 
pickling, also made the type more discoverable, which becomes less relevant if 
it is a builtin.

FWIW, I still think it is a good candidate for the builtins, though I'd 
probably name it "simplenamespace" at this point.  Cementing the name as 
"types.SimpleNamespace" would probably be a disappointment for me.  Instead it 
would be better to fix the pickle support, so tp_name could go back to the 
correct name, and make it a builtin.  This isn't a priority for me, though, and 
I don't have a huge sense of ownership here, so I don't feel like I am in much 
of a position to champion my preferences on this.  (I'd be glad to mentor 
someone on this though.)

Thus I'm not sure what to think about this. :)

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue42088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44363] Address sanitizer (gcc version) is generating false positives

2021-06-09 Thread Eric Snow


Eric Snow  added the comment:

This was my bad.  I only partially fixed original problem when un-reverting 
3fa63e in gh-26609.  I've merged the rest of the fix in gh-26626.  Sorry about 
that.

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue44363>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-09 Thread Eric Snow


Eric Snow  added the comment:


New changeset e6e34e45222b9c7a63ba92386612acf768082ba0 by Eric Snow in branch 
'main':
bpo-43693: Do not check co_cell2arg if a non-cell offset. (gh-26626)
https://github.com/python/cpython/commit/e6e34e45222b9c7a63ba92386612acf768082ba0


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-09 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +25212
pull_request: https://github.com/python/cpython/pull/26626

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-08 Thread Eric Snow


Eric Snow  added the comment:


New changeset 3e1c7167d86a2a928cdcb659094aa10bb5550c4c by Eric Snow in branch 
'main':
 bpo-43693: Un-revert commit f3fa63e. (#26609)
https://github.com/python/cpython/commit/3e1c7167d86a2a928cdcb659094aa10bb5550c4c


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-08 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +25192
pull_request: https://github.com/python/cpython/pull/26609

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-08 Thread Eric Snow


Eric Snow  added the comment:

Thanks, Pablo!

--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-07 Thread Eric Snow


Eric Snow  added the comment:


New changeset 165c884154901deae46b5e328a6414d130e6bfff by Eric Snow in branch 
'main':
bpo-43693: Silence some compiler warnings. (gh-26588)
https://github.com/python/cpython/commit/165c884154901deae46b5e328a6414d130e6bfff


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-07 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +25173
pull_request: https://github.com/python/cpython/pull/26588

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-07 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +25172
pull_request: https://github.com/python/cpython/pull/26587

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-07 Thread Eric Snow


Eric Snow  added the comment:


New changeset 631f9938b1604d4f893417ec339b9e0fa9196fb1 by Eric Snow in branch 
'main':
bpo-43693: Add the MAKE_CELL opcode and interleave fast locals offsets. 
(gh-26396)
https://github.com/python/cpython/commit/631f9938b1604d4f893417ec339b9e0fa9196fb1


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-07 Thread Eric Snow


Eric Snow  added the comment:


New changeset 2ab27c4af4ddf7528e1375e77c787c7fbb09b5e6 by Eric Snow in branch 
'main':
bpo-43693: Un-revert commits 2c1e258 and b2bf2bc. (gh-26577)
https://github.com/python/cpython/commit/2ab27c4af4ddf7528e1375e77c787c7fbb09b5e6


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-07 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +25165
pull_request: https://github.com/python/cpython/pull/26577

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-04 Thread Eric Snow


Eric Snow  added the comment:

Thanks Pablo.  I'll get this sorted out.  Sorry for the pain.

--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-03 Thread Eric Snow


Eric Snow  added the comment:


New changeset b2bf2bc1ece673d387341e06c8d3c2bc6e259747 by Mark Shannon in 
branch 'main':
bpo-43693: Compute deref offsets in compiler (gh-25152)
https://github.com/python/cpython/commit/b2bf2bc1ece673d387341e06c8d3c2bc6e259747


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-06-03 Thread Eric Snow


Eric Snow  added the comment:


New changeset 2c1e2583fdc4db6b43d163239ea42b0e8394171f by Eric Snow in branch 
'main':
bpo-43693: Add new internal code objects fields: co_fastlocalnames and 
co_fastlocalkinds. (gh-26388)
https://github.com/python/cpython/commit/2c1e2583fdc4db6b43d163239ea42b0e8394171f


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-05-27 Thread Eric Snow


Eric Snow  added the comment:


New changeset 9f494d492944c3a6a7a7471b4ad3a025dc7de289 by Eric Snow in branch 
'main':
bpo-43693: Add _PyCode_New(). (gh-26375)
https://github.com/python/cpython/commit/9f494d492944c3a6a7a7471b4ad3a025dc7de289


--

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-05-26 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +24989
pull_request: https://github.com/python/cpython/pull/26396

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-05-26 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +24981
pull_request: https://github.com/python/cpython/pull/26388

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-05-25 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +24967
pull_request: https://github.com/python/cpython/pull/26375

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-05-25 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +24955
pull_request: https://github.com/python/cpython/pull/26364

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-05-19 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +24865
pull_request: https://github.com/python/cpython/pull/26258

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43693] Logically merge cell and locals array. They are already contiguous in memory

2021-05-18 Thread Eric Snow


Change by Eric Snow :


--
nosy: +eric.snow
nosy_count: 4.0 -> 5.0
pull_requests: +24833
pull_request: https://github.com/python/cpython/pull/26216

___
Python tracker 
<https://bugs.python.org/issue43693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-17 Thread Eric Snow


Eric Snow  added the comment:

FYI, I'm going to focus discussion on the capi-sig thread.

--

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-17 Thread Eric Snow


Eric Snow  added the comment:

> PEP 489 is *very much* part of the limited API.

Gah, I missed that.  That said, I don't think it matters; I just lose an easy 
point in the rationale. :)

--

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-17 Thread Eric Snow


Eric Snow  added the comment:

> I am confused. How can widening the usable number of functions (i.e. using
> the whole C-API rather than the limited API) help c-extension modules be
> usable in subinterpreters? Aren't the same singletons, exception types, and
> other types exposed in the full C-API?

If Py_LIMITED_API is defined then things would stay the same.  Otherwise we 
would replace the names with macros to do the appropriate lookup.  (That isn't 
the whole story since the Py*_Type names are PyTypeObject and not PyObject*.)

--

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-16 Thread Eric Snow


Eric Snow  added the comment:

FYI, I posted to capi-sig about this:

https://mail.python.org/archives/list/capi-...@python.org/thread/INLCGPMTYFLRTWQL7RB4MUQZ37JAFRAU/

--

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-16 Thread Eric Snow


Eric Snow  added the comment:

One simple solution is to explicitly state that the limited API does not 
support subinterpreters.  This is already implied by the fact that the 
multi-phase init API (PEP 489) requires subinterpreter support but is not part 
of the limited API.

If we establish this constraint then the problem I originally described here 
goes away (and we can close this issue).

(Note: I'm pretty sure this is something someone suggested to me at some point, 
rather than my own idea.)

--

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39511] [subinterpreters] Per-interpreter singletons (None, True, False, etc.)

2021-03-15 Thread Eric Snow


Change by Eric Snow :


--
nosy: +eric.snow
nosy_count: 2.0 -> 3.0
pull_requests: +23645
pull_request: https://github.com/python/cpython/pull/24828

___
Python tracker 
<https://bugs.python.org/issue39511>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-15 Thread Eric Snow


Change by Eric Snow :


--
components: +Extension Modules, Interpreter Core, Subinterpreters

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-15 Thread Eric Snow


Change by Eric Snow :


--
keywords: +patch
pull_requests: +23642
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/24828

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-15 Thread Eric Snow


Change by Eric Snow :


--
nosy: +Mark.Shannon, nascheme, vstinner

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-15 Thread Eric Snow


Eric Snow  added the comment:

If the stable ABI weren't an issue then we would probably:

* deprecate using the objects directly
* do something like (2a) in the meantime

It may make sense to do that for "#ifndef Py_LIMITED_API", regardless of how we 
handle the limited API.

--

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-15 Thread Eric Snow


Eric Snow  added the comment:

Here are some solutions that I've considered:

1. immutable objects
   a. make the objects truly immutable/const
  * not trivial, if possible
   b. make the objects effectively immutable
  * (see GH-24828) use a really high refcount to make races irrelevant
2. per-interpreter objects
   a. replace them with macros that do a per-interpreter lookup
   b. replace them with simple placeholders and do a per-interpreter lookup 
internally
   c. replace them with PyObject placeholders and do a per-interpreter lookup 
internally

As far as I'm aware, only (1b) and (2c) are realistic and won't break the 
stable ABI (i.e. preserve layout).

(FWIW, I think that even with (1b) we would still have per-interpreter objects.)

-- Regarding (1a) --

See see GH-24828 for an example implementation.  This includes storing some 
state for the objects in PyInterpreterState and doing a lookup internally.

pros:
* relatively straightforward to implement
* overlaps with other interests (see bpo-40255)
* makes the objects shareable between interpreters (making them more efficient)

cons:
* we have to ensure the objects stay immutable (a tractable problem if the 
solution is constrained to only the limited API objects)

-- Regarding (2c) --

This involves adding an API to get the per-interpreter object for a given 
identity value (flag, global, etc.) and then mapping the limited API objects to 
the corresponding per-interpreter ones.

pros:
* avoids a one-off solution
* extensions can stop using the variables directly (in favor of the new lookup 
API)

cons:
* effectively couples the C-API to the design (as long as the objects are in 
the limited API)
* many touches all over the C-API
* any future changes/additions in the C-API must deal with the objects

--

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40255] Fixing Copy on Writes from reference counting

2021-03-15 Thread Eric Snow


Eric Snow  added the comment:

While the eventual solution may overlap, my interests are divergent enough from 
those here that I've created a separate issue: bpo-43503.

--

___
Python tracker 
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43503] [subinterpreters] PyObject statics exposed in the limited API break isolation.

2021-03-15 Thread Eric Snow


New submission from Eric Snow :

In the limited C-API we expose the following static PyObject variables:

* 5 singletons
* ~70 exception types
* ~70 other types

Since they are part of the limited API, they have a direct effect on the stable 
ABI.

The problem is that these objects should not be shared between interpreters.  
There are a number of possible solutions for isolating the objects, but the big 
constraint is that the solution cannot break the stable ABI.

--
components: C API
messages: 388759
nosy: eric.snow
priority: normal
severity: normal
stage: needs patch
status: open
title: [subinterpreters] PyObject statics exposed in the limited API break 
isolation.
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue43503>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40255] Fixing Copy on Writes from reference counting

2021-03-11 Thread Eric Snow


Eric Snow  added the comment:

Note that my PR does make the builtin types immortal (at least those 
initialized in _PyTypes_Init()).  However, I did not make the singletons or 
other candidate objects immortal in the PR.  That can be done separately.

--

___
Python tracker 
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40255] Fixing Copy on Writes from reference counting

2021-03-11 Thread Eric Snow


Eric Snow  added the comment:

FYI, I've just put up a similar PR [1] to Eddie's, with a focus on object 
immortality rather than immutability.  However, if Py_IMMORTAL_CONST_REFCOUNTS 
is defined then it should have the behavior Eddie is after.

[1] https://github.com/python/cpython/pull/24828

--

___
Python tracker 
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40255] Fixing Copy on Writes from reference counting

2021-03-11 Thread Eric Snow


Change by Eric Snow :


--
nosy: +eric.snow
nosy_count: 15.0 -> 16.0
pull_requests: +23592
pull_request: https://github.com/python/cpython/pull/24828

___
Python tracker 
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-12-25 Thread Eric Snow


Eric Snow  added the comment:


New changeset 5ae9be68d9f1a628fdc920b647257f94afb77887 by Eric Snow in branch 
'master':
bpo-36876: [c-analyzer tool] Additional CLI updates for "capi" command. 
(gh-23929)
https://github.com/python/cpython/commit/5ae9be68d9f1a628fdc920b647257f94afb77887


--

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-12-24 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +22779
pull_request: https://github.com/python/cpython/pull/23929

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-12-24 Thread Eric Snow


Eric Snow  added the comment:


New changeset 7ec59d8861ef1104c3028678b2cacde4c5693e19 by Eric Snow in branch 
'master':
bpo-36876: [c-analyzer tool] Add a "capi" subcommand to the c-analyzer tool. 
(gh-23918)
https://github.com/python/cpython/commit/7ec59d8861ef1104c3028678b2cacde4c5693e19


--

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-12-23 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +22769
pull_request: https://github.com/python/cpython/pull/23918

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-11-20 Thread Eric Snow


Eric Snow  added the comment:


New changeset 9f02b479e6b6b48d0c2aad621978cff82e530b15 by Eric Snow in branch 
'master':
bpo-36876: [c-analyzer tool] Tighten up the results and output. (GH-23431)
https://github.com/python/cpython/commit/9f02b479e6b6b48d0c2aad621978cff82e530b15


--

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-11-20 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +22322
pull_request: https://github.com/python/cpython/pull/23431

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-10-30 Thread Eric Snow


Eric Snow  added the comment:


New changeset 4fe72090deb7fb7bc09bfa56c92f6b3b0967d395 by Eric Snow in branch 
'master':
bpo-36876: Small adjustments to the C-analyzer tool. (GH-23045)
https://github.com/python/cpython/commit/4fe72090deb7fb7bc09bfa56c92f6b3b0967d395


--

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-10-30 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +21964
pull_request: https://github.com/python/cpython/pull/23045

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-10-22 Thread Eric Snow


Eric Snow  added the comment:


New changeset 345cd37abe324ad4f60f80e2c3133b8849e54e9b by Eric Snow in branch 
'master':
bpo-36876: Fix the C analyzer tool. (GH-22841)
https://github.com/python/cpython/commit/345cd37abe324ad4f60f80e2c3133b8849e54e9b


--

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36225] [subinterpreters] Lingering subinterpreters should be implicitly cleared on shutdown

2020-10-22 Thread Eric Snow


Eric Snow  added the comment:

> I see multiple non trivial problems:
>
> * To finalize the subinterpreter A, Py_Finalize() may switch the current 
> Python thread state from the main interpreter to a Python thread in the 
> subintrepeter A. It can lead to new funny issues.

I'm not sure what you mean with this.  Are you talking about how another thread 
might be running that threadstate already?  That doesn't sounds like a problem 
specific to this issue but rather a problem with Py_EndInterpreter() in general.

> * A subinterpreter can be stuck for whatever reason and refuse to stop. For 
> example, the subinterpreter A may wait for an even from subinterpreter B. If 
> we don't run two interpreters "in parallel" (in threads), it may be stuck 
> forever.

The wait_for_thread_shutdown() call in Py_FinalizeEx() already has a similar 
behavior.  In the case of one interpreter blocking another like that, how would 
that be possible where it isn't already a problem?

> * Python 3.10 still has weird code to allow daemon threads to continue to run 
> after Py_Finalize() complete. Maybe we need to add such hacks for 
> subinterpreter which fail to be finalized? For example, kill them (exit) as 
> soon as they attempt to acquire the GIL. Search for tstate_must_exit() in 
> Python/ceval_gil.h.

I consider the problem of daemon threads to be a separate problem.  The 
solution proposed here for finalization doesn't change any behavior regarding 
daemon threads.

> By the way, currently Py_Finalize() calls PyInterpreterState_Clear() which 
> call object finalizers in the main thread of the main interpreter, whereas 
> these finalizers might expect to be called from the thread which created them.

Do you have any examples?  I'm having trouble imagining such a case.

> I don't know if we must change anything else.
>
> Again, all these problems are very complex :-(

I agree.  However, automatically finalizing all other interpreters at the 
beginning of Py_FinalizeEx() doesn't introduce any new problems.  At the same 
time, it makes sure that any resources in use in other interpreters have a 
chance to be released and that global resources used there don't cause crashes 
during runtime finalization.

> The simple option, which is sadly a backward incompatible change, is to raise 
> a fatal error in Py_Finalize() if a subinterpreter is still running.

As long as we require Py_FinalizeEx() to be called from the main interpreter, I 
don't see how the proposed change (in PR #17575) would be backward incompatible.

> Maybe we can start by logging a warning into stderr for now, before 
> introducing the backward incompatible change. Maybe even only emit it when -X 
> dev is used? https://docs.python.org/dev/library/devmode.html

I'd like to see a ResourceWarning if any other interpreters were still around.

--

___
Python tracker 
<https://bugs.python.org/issue36225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36876] [subinterpreters] Global C variables are a problem

2020-10-20 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +21795
pull_request: https://github.com/python/cpython/pull/22841

___
Python tracker 
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32604] [subinterpreters] PEP 554 implementation: add interpreters module

2020-06-16 Thread Eric Snow


Eric Snow  added the comment:

@Pablo, yeah, I'll try to take a look this week.  Are these new failures?

Keep in mind that these tests can sometimes expose existing bugs in our runtime 
(as we fix runtime issues elsewhere) rather than regressions, though that isn't 
necessarily the case here. :)

--

___
Python tracker 
<https://bugs.python.org/issue32604>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32604] [subinterpreters] PEP 554 implementation: add interpreters module

2020-06-16 Thread Eric Snow


Eric Snow  added the comment:


New changeset 818f5b597ae93411cc44e404544247d436026a00 by Eric Snow in branch 
'master':
bpo-32604: Clean up test.support.interpreters. (gh-20926)
https://github.com/python/cpython/commit/818f5b597ae93411cc44e404544247d436026a00


--

___
Python tracker 
<https://bugs.python.org/issue32604>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32604] [subinterpreters] PEP 554 implementation: add interpreters module

2020-06-16 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +20105
pull_request: https://github.com/python/cpython/pull/20926

___
Python tracker 
<https://bugs.python.org/issue32604>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39075] types.SimpleNamespace should preserve attribute ordering (?)

2020-05-15 Thread Eric Snow


Change by Eric Snow :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39075>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40572] Support basic asynchronous cross-interpreter operations.

2020-05-08 Thread Eric Snow


Change by Eric Snow :


--
keywords: +patch
pull_requests: +19323
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/20012

___
Python tracker 
<https://bugs.python.org/issue40572>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40572] Support basic asynchronous cross-interpreter operations.

2020-05-08 Thread Eric Snow


New submission from Eric Snow :

(This is a continuation of the work from bpo-33608.  That issue ended up with a 
lot of baggage and clutter (due to problems that have since been resolved), so 
we closed it.  This issue is where we're picking it up fresh.)

When two interpreters are cooperating, there are sometimes operations that one 
of them needs the other to perform.  Thus far this is limited to mutation or 
release of data/objects owned by that "other" interpreter.  We need safe, 
reliable public C-API to facilitate such operations.

All the necessary machinery already exists ("pending calls"), exposed in the 
internal C-API: _Py_AddPendingCall().  (There is a public Py_AddPendingCall(), 
but it should be removed.)  That API adds a function to a per-interpreter 
queue, from which it is executed later (asynchronously) by the ceval loop of 
the interpreter's main thread.  The challenge is that the repercussions of such 
async operations within the eval loop are not fully explored.  We have some 
confidence because this is the same machinery used to handle signals.  However, 
it's better avoid as much complexity in those async operations as possible.  
That is why we don't just expose `_Py_AddPendingCall()` in the public C-API.

Instead the plan is to add a small number of public C-API functions that are 
each specific to a focused use case, ideally with with little/no chance of 
errors or other side effects.  Initially we will target `Py_DECREF()`, 
`PyMem_Free()`, and `PyBuffer_Release()`.  If we need more then we can assess 
those needs later.

--
assignee: eric.snow
components: C API
messages: 368476
nosy: eric.snow, vstinner
priority: normal
severity: normal
stage: needs patch
status: open
title: Support basic asynchronous cross-interpreter operations.
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40572>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32604] Expose the subinterpreters C-API in Python for testing use.

2020-05-07 Thread Eric Snow


Eric Snow  added the comment:


New changeset a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57 by Eric Snow in branch 
'master':
bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)
https://github.com/python/cpython/commit/a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57


--

___
Python tracker 
<https://bugs.python.org/issue32604>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40533] Subinterpreters: don't share Python objects between interpreters

2020-05-06 Thread Eric Snow


Eric Snow  added the comment:

Yep, before per-interpreter GIL is official we must get to the point where *no* 
PyObject objects are shared.

Making PyObject.ob_refcnt atomic until then (only as part of the experiment) 
should be fine.

--

___
Python tracker 
<https://bugs.python.org/issue40533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40058] Running test_datetime twice fails with: module 'datetime' has no attribute '_divide_and_round'

2020-05-06 Thread Eric Snow


Eric Snow  added the comment:

FYI, with the following additions in Lib/test/test_datetime.py...

   before = set(sys.modules)
   try:
   pure_tests = import_fresh_module(TESTS, fresh=['datetime', '_strptime'],
 blocked=['_datetime'])
   _pure = set(sys.modules)
   fast_tests = import_fresh_module(TESTS, fresh=['datetime',
  '_datetime', '_strptime'])
   _fast = set(sys.modules)
   print(f'added (pure): {sorted(_pure-before)}')
   print(f'added (fast): {sorted(_fast-before)}')
   print('---')
   finally:
  ...

I get the following output running "./python -m test test_datetime 
test_datetime":

   0:00:00 load avg: 0.52 Run tests sequentially
   0:00:00 load avg: 0.52 [1/2] test_datetime
   added (pure): ['_decimal', '_strptime', '_testcapi', 'decimal', 'numbers', 
'test.datetimetester']
   added (fast): ['_decimal', '_strptime', '_testcapi', 'decimal', 'numbers', 
'test.datetimetester']
   ---
   0:00:05 load avg: 0.52 [2/2] test_datetime
   added (pure): ['_datetime', '_strptime', 'datetime']
   added (fast): ['_datetime', '_strptime', 'datetime']
   ---
   [snipped]

That definitely tells a story. :)  Unfortunately, for now I don't have any more 
time to investigate.  Sorry.

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue40058>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40514] Add --experimental-isolated-subinterpreters build option

2020-05-05 Thread Eric Snow


Eric Snow  added the comment:

It would probably make sense to remove the build option in the 3.9 release.  We 
can leave it in master, but remove it in the 3.9 branch once it has been 
created.

--

___
Python tracker 
<https://bugs.python.org/issue40514>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40010] Inefficient signal handling in multithreaded applications

2020-05-05 Thread Eric Snow


Eric Snow  added the comment:

Good catch on this, Victor.  Thanks for doing it.

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue40010>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

2020-05-05 Thread Eric Snow


Eric Snow  added the comment:

If this issue covers the GIL (which it seems to) then I'd expect 
_PyRuntimeState.gilstate to be handled here too.

--

___
Python tracker 
<https://bugs.python.org/issue40513>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

2020-05-05 Thread Eric Snow


Eric Snow  added the comment:

>From a user perspective, does it make sense to have a different 
>recursion_limit per interpreter?  I don't see a problem with it.  However, 
>does it make sense to also keep a global value that we default to when a 
>per-interpreter value is not set?  I think it might.

I suppose a bigger question is what users will expect the recursion limit (AKA 
"sys.getrecursionlimit()") to be for a newly created subinterpreter.  Will it 
be some global default?  Will it be the value from the parent interpreter?  I'd 
go with a global default, which would imply that the default value should be 
stored under _PyRuntimeState like we had it (but still keep the actual 
per-interpreter field for the actual value).

FWIW, the existing docs don't really block either approach.

--

___
Python tracker 
<https://bugs.python.org/issue40513>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

2020-05-05 Thread Eric Snow


Eric Snow  added the comment:

FWIW, I think it would make sense to keep "signals_pending" under 
_PyRuntimeState rather than moving it to PyInterpreterState.

Signals are only handled by the main interpreter (in its main thread).  Even 
though "signals_pending" is useful to only one interpreter in the runtime, 
making it per-interpreter sends the wrong message that it is significant at 
that level.  This may be confusing to readers of the code.

At the very least there should be a clear comment with the field in 
Include/internal/pycore_interp.h explaining how it is only used by the main 
interpreter and why we made it per-interpreter anyway.

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue40513>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38880] Subinterpreters: List interpreters associated with a channel end

2020-05-01 Thread Eric Snow


Eric Snow  added the comment:

Thanks again, Lewis!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38880>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40390] Implement _xxsubinterpreters.channel_send_wait().

2020-05-01 Thread Eric Snow


Eric Snow  added the comment:

Thanks for working on this, Ben!  FWIW, I've put up a separate PR to 
demonstrate how I was thinking we would solve this:  
https://github.com/python/cpython/pull/19829.

--

___
Python tracker 
<https://bugs.python.org/issue40390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40350] modulefinder chokes on numpy - dereferencing None in spec.loader

2020-05-01 Thread Eric Snow


Eric Snow  added the comment:

Ah, namespace packages. :)  Yeah, the code is not taking the "spec.loader is 
None" case into account.  I expect the fix would be to add handling of that 
case a few lines up in the code, right after handling BuiltinImporter and 
FrozenImporter.  Offhand I'm not sure if the "type" should be _PKG_DIRECTORY or 
some new one just for namespace packages.  How does imp.find_module() (on which 
modulefinder._find_module() is based) respond to namespace packages?

[1] 
https://docs.python.org/3/library/importlib.html#importlib.machinery.ModuleSpec.loader

--
nosy: +barry, brett.cannon, eric.smith, eric.snow

___
Python tracker 
<https://bugs.python.org/issue40350>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40350] modulefinder chokes on numpy - dereferencing None in spec.loader

2020-05-01 Thread Eric Snow


Change by Eric Snow :


--
stage:  -> test needed
versions: +Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40350>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40417] PyImport_ReloadModule emits deprecation warning

2020-05-01 Thread Eric Snow


Eric Snow  added the comment:

Did you have a need for this to be fixed in 3.8 or earlier?  This seems 
reasonably and simple enough to backport.  I suppose someone could be relying 
on an implicit import of the "imp" module, but that seems highly unlikely and 
suspect anyway. :)

--
nosy: +brett.cannon, ncoghlan

___
Python tracker 
<https://bugs.python.org/issue40417>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40417] PyImport_ReloadModule emits deprecation warning

2020-05-01 Thread Eric Snow


Eric Snow  added the comment:

Yeah, that looks like an oversight.  I've approved your PR.  Thanks!

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue40417>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40453] Add PyConfig._isolated_interpreter: isolated subinterpreters

2020-05-01 Thread Eric Snow


Change by Eric Snow :


--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue40453>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32604] Expose the subinterpreters C-API in Python for testing use.

2020-04-30 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +19149
pull_request: https://github.com/python/cpython/pull/19829

___
Python tracker 
<https://bugs.python.org/issue32604>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32604] Expose the subinterpreters C-API in Python for testing use.

2020-04-28 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +19090
pull_request: https://github.com/python/cpython/pull/19770

___
Python tracker 
<https://bugs.python.org/issue32604>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40390] Implement _xxsubinterpreters.channel_send_wait().

2020-04-28 Thread Eric Snow


Eric Snow  added the comment:

Thanks, Ben.  I'll take a look.

--
components: +Library (Lib) -C API
title: Implement a C API for channel_send_wait for subinterpreters. -> 
Implement _xxsubinterpreters.channel_send_wait().

___
Python tracker 
<https://bugs.python.org/issue40390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32604] Expose the subinterpreters C-API in Python for testing use.

2020-04-28 Thread Eric Snow


Change by Eric Snow :


--
pull_requests: +19088
pull_request: https://github.com/python/cpython/pull/19768

___
Python tracker 
<https://bugs.python.org/issue32604>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40360] Deprecate lib2to3 (and 2to3) for future removal

2020-04-22 Thread Eric Snow


Change by Eric Snow :


--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue40360>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37266] Daemon threads must be forbidden in subinterpreters

2020-04-08 Thread Eric Snow


Eric Snow  added the comment:

I've opened bpo-40234 to address backward incompatibility from this
change (e.g. affecting mod-wsgi).

--

___
Python tracker 
<https://bugs.python.org/issue37266>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40234] Disallow daemon threads in subinterpreters optionally.

2020-04-08 Thread Eric Snow


New submission from Eric Snow :

In bpo-37266 we strictly disallowed creation of daemon threads in 
subinterpreters.  However, this is backward-incompatible for existing users of 
the subinterpreter C-API (such as mod-wsgi).

Rather than reverting that change I suggest that we make it opt-in through the 
interpreter config.  That would preserve backward-compatibility.  It would also 
make it so we can disallow daemon threads in subinterpreters created through 
PEP 554.  We could also deprecate use of daemon threads in *all* 
subinterpreters, with the goal of dropping support after a while.

--
components: Interpreter Core
messages: 366029
nosy: eric.snow, grahamd, vstinner
priority: normal
severity: normal
stage: needs patch
status: open
title: Disallow daemon threads in subinterpreters optionally.
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40234>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33608] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

2020-04-08 Thread Eric Snow


Eric Snow  added the comment:

> I close this issue with a complex history.
>
> If someone wants to continue to work on this topic, please open an issue with 
> a very clear description of what should be done and how it is supposed to be 
> used.

Yeah, there is more to do.  I'll create a new issue.

--

___
Python tracker 
<https://bugs.python.org/issue33608>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40077] Convert static types to PyType_FromSpec()

2020-04-08 Thread Eric Snow


Eric Snow  added the comment:

> Wouldn't having less static types slow down startup time?

FWIW, I've been considering an approach where the main interpreter
keeps using static types but subinterpreters use heap types.  If it
isn't too much effort (or too hacky) then it might be a sufficient
solution for now.

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue40077>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33608] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

2020-03-27 Thread Eric Snow


Eric Snow  added the comment:

FYI, in bpo-39984 Victor moved pending calls to PyInterpreterState, which was 
part of my reverted change.  However, there are a few other pieces of that 
change that need to be applied before this issue is resolved.  I'm not sure 
when I'll get to it, but hopefully soon.

--

___
Python tracker 
<https://bugs.python.org/issue33608>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39984] Move pending calls from _PyRuntime to PyInterpreterState

2020-03-27 Thread Eric Snow


Eric Snow  added the comment:

Awesome!  Thanks for doing this, Victor.  I'll take a look when I can and 
adjust the changes for bpo-33608.  If you'll recall, I made a similar change as 
part of the solution for that issue, which we later reverted due to problems we 
discovered with daemon threads during runtime finalization.

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue39984>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39829] __len__ called twice in the list() constructor

2020-03-09 Thread Eric Snow


Eric Snow  added the comment:

I'm not opposed. :)  I just don't want to impose on your time.

--
assignee:  -> pablogsal
resolution: not a bug -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue39829>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39829] __len__ called twice in the list() constructor

2020-03-09 Thread Eric Snow


Eric Snow  added the comment:

FWIW, I encouraged Kim to file this.  Thanks Kim!

While it isn't part of any specification, it is an unexpected change in 
behavior that led to some test failures.  So I figured it would be worth 
bringing up. :)  I did find it surprising that we were not caching the result, 
but don't think that's necessarily a problem.

All that said, the change did not actually break anything other than some tests 
(not the code they were testing).  So I don't have a problem with closing this.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39829>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37497] Add inspect.Signature.from_text().

2020-03-06 Thread Eric Snow


Eric Snow  added the comment:

Honestly, I don't recall exactly the concrete use case for which I opened this. 
:)  I *think* it was related to applying a more restrictive signature onto an 
existing function (e.g. with a decorator).

--

___
Python tracker 
<https://bugs.python.org/issue37497>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17422] language reference should specify restrictions on class namespace

2020-03-06 Thread Eric Snow


Eric Snow  added the comment:

Thanks for fixing that, Caleb!

FWIW, I've opened a separate issue (#39879) for adding a note in the language 
reference about dict ordering.  Sorry for the confusion.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue17422>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39879] Update language reference to specify that dict is insertion-ordered.

2020-03-06 Thread Eric Snow


New submission from Eric Snow :

As of 3.7 [1], dict is guaranteed to preserve insertion order:

  the insertion-order preservation nature of dict
  objects has been declared to be an official part
  of the Python language spec.

However, at least one key part of the language reference [2] was not updated to 
reflect this:  "3.2. The standard type hierarchy" > "Mappings" > "Dictionaries".

Note that the library docs [3] *were* updated.


[1] https://docs.python.org/3/whatsnew/3.7.html#summary-release-highlights
[2] https://docs.python.org/3/reference/datamodel.html#index-30
[3] https://docs.python.org/3/library/stdtypes.html#typesmapping

--
assignee: docs@python
components: Documentation
keywords: easy
messages: 363533
nosy: docs@python, eric.snow
priority: normal
severity: normal
stage: needs patch
status: open
title: Update language reference to specify that dict is insertion-ordered.
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39879>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33234] Improve list() pre-sizing for inputs with known lengths

2020-03-02 Thread Eric Snow


Eric Snow  added the comment:

Possible backward incompatibility caused by this issue: issue39829

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue33234>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17422] language reference should specify restrictions on class namespace

2020-02-27 Thread Eric Snow


Eric Snow  added the comment:

Thanks for working on this.  Sorry I didn't get a chance to see your PR sooner. 
 There was one small thing that needs to be changed back, as I implied in my 
comment on the PR [1].  Please undo the change in the text from "ordered 
mapping" to "dict".

The original "is initialised as an empty ordered mapping" text line should be 
restored.  "dict" is currently still not correct (the language spec does not 
dictate that dict be ordered).  However, PEP 520 specifies that it must be 
ordered.

Thanks!


[1] https://github.com/python/cpython/pull/18559#pullrequestreview-366098695

--
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
versions: +Python 3.7, Python 3.8, Python 3.9 -Python 3.4, Python 3.5

___
Python tracker 
<https://bugs.python.org/issue17422>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

2020-02-07 Thread Eric Snow


Change by Eric Snow :


--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue1635741>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39511] [subinterpreters] Per-interpreter singletons (None, True, False, etc.)

2020-02-04 Thread Eric Snow


Eric Snow  added the comment:

> This is pretty much one of the two approaches I have been considering.

The other approach is to leave the current static singletons alone and
only use them for the main interpreter.  Each subinterpreter would get
its own copy, created when that interpreter is initialized.

--

___
Python tracker 
<https://bugs.python.org/issue39511>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39511] [subinterpreters] Per-interpreter singletons (None, True, False, etc.)

2020-02-04 Thread Eric Snow


Eric Snow  added the comment:

On Sun, Feb 2, 2020 at 3:32 PM Raymond Hettinger  wrote:
> Random idea (not carefully thought-out):  Would it be simpler to have these 
> objects just
> ignore their refcount by having dealloc() be a null operation or having it 
> set the refcount
> back to a positive number).  That would let sub-interpreters share the 
> objects without
> worrying about race-conditions on incref/decref operations.

This is pretty much one of the two approaches I have been considering. :)

Just to be clear, singletons normally won't end up with a refcount of
0, by design.  However, if there's a incref race between two
interpreters (that don't share a GIL) then you *can* end up triggering
dealloc (via Py_DECREF) and even get negative refcounts (which cause
Py_FatalError() on debug builds).  Here are some mitigations:

* (as noted above) make dealloc() for singletons a noop
* (as noted above) in dealloc() set the refcount back to some positive value
* make the initial refcount sufficiently large such that it is
unlikely to reach 0 even with races
* incref each singleton once during each interpreter initiialization
(and decref once during interp. finalization)

We would have to special-case refleak checks for singletons, to avoid
false positives.

Also note that currently extension authors (and CPython contributors)
can rely on the runtime to identify when then accidentally
incref/decref a singleton too much (or forget to do so).  So it may
make sense to do something in the "singleton dealloc()" in debug
builds to provide similar checks.

>  To make this work, the objects can register themselves as permanent, shared, 
> objects;
> then, during shutdown, we could explicitly call a hard dealloc on those 
> objects.

great point

--

___
Python tracker 
<https://bugs.python.org/issue39511>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39511] [subinterpreters] Per-interpreter singletons (None, True, False, etc.)

2020-02-04 Thread Eric Snow


Eric Snow  added the comment:

On Sun, Feb 2, 2020 at 2:53 PM Raymond Hettinger  wrote:
> Is the sub-interpreter PEP approved?

PEP 554 is not approved yet (and certainly is not guaranteed, though
I'm hopeful).  However, that PEP is exclusively about exposing
subinterpreters in the stdlib.

There is no PEP for the effort to isolate subinterpreters and stop
sharing the GIL between them.  We are not planning on having such a
PEP since the actual proposal is so basic ("make the GIL
per-interpreter") and there has been no controversy.

I need to do a better job about communicating the difference, as folks
keep conflating PEP 554 with the effort to stop sharing the GIL
between subinterpreters.  Note that both are part of my "multi-core
Python" project. [1]

> If not, I had thought the plan was to only
> implement PRs that made clean-ups that would have been necessary anyway.

Right.  In this case the "cleanup" is how singletons are finalized
when the runtime shuts down.  This has a material impact on projects
that embed the CPython runtime.  Currently runtime finalization is a
mess.

That said, making the singletons per-interpreter isn't a prerequisite
of that cleanup.  For specific case of the per-interpreter GIL world,
we just need an approach that addresses their life cycle in a
thread-safe way.

[1] https://github.com/ericsnowcurrently/multi-core-python

--

___
Python tracker 
<https://bugs.python.org/issue39511>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37224] test__xxsubinterpreters fails randomly

2020-02-04 Thread Eric Snow


Eric Snow  added the comment:

Thanks, Kyle.  That helps at least a little. :)

--

___
Python tracker 
<https://bugs.python.org/issue37224>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39516] ++ does not throw a SyntaxError

2020-02-04 Thread Eric Snow


Eric Snow  added the comment:

FWIW, this is the sort of thing that is usually best suited to be reported by 
linters, not the Python runtime.

--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue39516>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   6   7   8   9   10   >