[Python-Dev] Re: PEP 684: A Per-Interpreter GIL

[Guido van Rossum]

That sounds like a horrible idea. The GIL should never be held during an
I/O operation.

On Fri, Mar 11, 2022 at 19:00 Jim J. Jewett  wrote:

> > Is ``allow_all_extensions`` the best name for the context manager?
>
> Nope.  I'm pretty sure that "parallel processing via multiple simultaneous
> interpreters" won't be the only reason people ever want to exclude certain
> extensions.
>
> It might be easier to express that through package or module name, but
> importlib and util aren't specific enough.
>
> For an example of an extension that works with multiple interpreters but
> only if they share a single GIL ... why wouldn't that apply to any
> extension designed to work with a Singleton external resource?  For
> example, the interpreters could all share a single database connection, and
> repurpose the GIL to ensure that there isn't a thread (or interpreter)
> switch mid-transaction.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RUDVIEDDCNFDRBIQVQU334GMPW77ZNOK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VKFNIFQRQZ5EVZ4G4TQIR2NGMO7BAMWY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 684: A Per-Interpreter GIL

[Jim J. Jewett]

> Is ``allow_all_extensions`` the best name for the context manager?

Nope.  I'm pretty sure that "parallel processing via multiple simultaneous 
interpreters" won't be the only reason people ever want to exclude certain 
extensions.

It might be easier to express that through package or module name, but 
importlib and util aren't specific enough. 

For an example of an extension that works with multiple interpreters but only 
if they share a single GIL ... why wouldn't that apply to any extension 
designed to work with a Singleton external resource?  For example, the 
interpreters could all share a single database connection, and repurpose the 
GIL to ensure that there isn't a thread (or interpreter) switch mid-transaction.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RUDVIEDDCNFDRBIQVQU334GMPW77ZNOK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Defining tiered platform support

[Jeremy Kloth]

On Fri, Mar 11, 2022 at 6:21 PM Victor Stinner  wrote:
> But I don't know how to get this info from the Microsoft
> documentation. I usually dig into Wikipedia articles to check which
> Windows version is still supported or not, but I'm confused between
> "mainstream support" and "extended support".

As PEP 11 states, it is the extended support time-frame that is of interest.

> For example, which Python version still support Windows 7? Wikipedia
> says that Windows 7 mainstream support ended in 2015, and extended
> support ended in 2020. But Python still has a Windows 7 SP1 buildbot
> for Python 3.8: https://buildbot.python.org/all/#/builders/60

Python 3.8.0 was release when Windows 7 SP1 was within the extended
support time-frame.  Therefore, Win7 is supported (and tested) until
3.8 reaches EOL.

> What is the minimum Windows supported by Python 3.10?

>From the link listed in PEP 11, Windows 8.1 does not leave extended
support until Jan 10, 2023.  So it seems that 3.10 and 3.11 will need
to support Windows 8.1.

-- 
Jeremy Kloth
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NFPSZD44DLFJS7CFGT2SVX3CSK7H5VRJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount" (round 3)

[Eric Snow]

responses inline

-eric

On Wed, Mar 9, 2022 at 8:23 AM Petr Viktorin  wrote:
> "periodically reset the refcount for immortal objects (only enable this
> if a stable ABI extension is imported?)" -- that sounds quite expensive,
> both at runtime and maintenance-wise.

Are you talking just about "(only enable this if a stable ABI
extension is imported?)"?  Such a check could certainly be expensive
but it doesn't have to be.  However, I'm guessing that you are
actually talking about the mechanism to periodically reset the
refcount.

The actual periodic reset doesn't seem like it needs to be all that
expensive overall.  It would just need to be in a place that gets
triggered often enough, but not too often such that the extra cost of
resetting the refcount would be a problem.

One important factor is whether we need to worry about potential
de-immortalization for all immortal objects or only for a specific
subset, like the most commonly used objects (at least most commonly
used by the problematic older stable ABI extensions),  Mostly, we only
need to be concerned with the objects that are likely to trigger
de-immortalization in those extensions.  Realistically, there aren't
many potential immortal objects that would be exposed to the
de-immortalization problem (e.g. None, True, False), so we could limit
this workaround to them.

A variety of options come to mind.  In each case we would reset the
refcount of a given object if it is immortal.  (We would also only do
so if the refcount actually changed--to avoid cache invalidation and
copy-on-write.)

If we need to worry about *all* immortal objects then I see several options:

1. in a single place where stable ABI extensions are likely to pass
all objects often enough
2. in a single place where all objects pass through often enough

On the other hand, if we only need to worry about a fixed set of
objects, the following options come to mind:

1. in a single place that is likely to be called by older stable ABI extensions
2. in a place that runs often enough, targeting a hard-coded group of
immortal objects (common static globals like None)
   * perhaps in the eval breaker code, in exception handling, etc.
3. like (2) but rotate through subsets of the hard-coded group (to
reduce the overall cost)
4. like (2), but in spread out in type-specific code (e.g. static
types could be reset in type_dealloc())

Again, none of those should be in code that runs often enough that the
overhead would add up.

> "provide a runtime flag for disabling immortality" also doesn't sound
> workable to me. We'd essentially need to run all tests twice every time
> to make sure it stays working.

Yeah, that makes it not worth it.

> "Special-casing immortal objects in tp_dealloc() for the relevant types
> (but not int, due to frequency?)" sounds promising.
>
> The "relevant types" are those for which we skip calling incref/decref
> entirely, like in Py_RETURN_NONE. This skipping is one of the optional
> optimizations, so we're entirely in control of if/when to apply it.

We would definitely do it for those types.  NoneType and bool already
have a tp_dealloc that calls Py_FatalError() if triggered.  The
tp_dealloc for str & tuple have special casing for some singletons
that do likewise.  In PyType_Type.tp_dealloc we have a similar assert
for static types.  In each case we would instead reset the refcount to
the initial immortal value.  Regardless, in practice we may only need
to worry (as noted above) about the problem for the most commonly used
global objects, so perhaps we could stop there.

However, it depends on what the level of risk is, such that it would
warrant incurring additional potential performance/maintenance costs.
What is the likelihood of actual crashes due to pathological
de-immortalization in older stable ABI extensions?  I don't have a
clear answer to offer on that but I'd only expect it to be a problem
if such extensions are used heavily in (very) long-running processes.

> How much would it slow things back down if it wasn't done for ints at all?

I'll look into that.  We're talking about the ~260 small ints, so it
depends on how much they are used relative to all the other int
objects that are used in a program.

> Some more reasoning for not worrying about de-immortalizing in types
> without this optimization:
> These objects will be de-immortalized with refcount around 2^29, and
> then incref/decref go back to being paired properly. If 2^29 is much
> higher than the true reference count at de-immortalization, this'll just
> cause a memory leak at shutdown.
> And it's probably OK to assume that the true reference count of an
> object can't be anywhere near 2^29: most of the time, to hold a
> reference you also need to have a pointer to the referenced object, and
> there ain't enough memory for that many pointers. This isn't a formally
> sound assumption, of course -- you can incref a million times with a
> single pointer if you pair the decrefs correctly. But it 

[Python-Dev] Re: Defining tiered platform support

[Victor Stinner]

It would be great to have the list of supported platforms per Python version!

Maybe supporting new platforms and dropping support for a platform
should be document in What's New in Python x.y. GCC does that for
example. It also *deprecates* support for some platforms. Example:
https://gcc.gnu.org/gcc-9/changes.html

--

It's always hard for me to know what is the minimum supported Windows
version. PEP 11 refers to Windows support:
https://peps.python.org/pep-0011/#microsoft-windows

But I don't know how to get this info from the Microsoft
documentation. I usually dig into Wikipedia articles to check which
Windows version is still supported or not, but I'm confused between
"mainstream support" and "extended support".

For example, which Python version still support Windows 7? Wikipedia
says that Windows 7 mainstream support ended in 2015, and extended
support ended in 2020. But Python still has a Windows 7 SP1 buildbot
for Python 3.8: https://buildbot.python.org/all/#/builders/60

What is the minimum Windows supported by Python 3.10?

Victor

On Mon, Mar 7, 2022 at 8:06 PM Christian Heimes  wrote:
>
> On 07/03/2022 18.02, Petr Viktorin wrote:
> >> Why the devguide? I view the list of platforms as important for public
> >> consumption as for the core dev team to know what to (not) accept PRs
> >> for.
> >
> > So, let's put it in the main docs?
> > Yes, I guess the devguide is a weird place to check for this kind of
> > info. But a Python enhancement proposal is even weirder.
>
>
> +1 for our main docs (cpython/Doc/)
>
> Platform support is Python versions specific. Python 3.10 may support
> different version than 3.11 or 3.12. It makes sense to keep the support
> information with the code.
>
> Christian
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/LJID7Y7RFSCRUYLJS3E56WBGJU2R44E4/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JJRNJWNRYC4R47YB64D6ZGO4325WG32R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2022-03-04 - 2022-03-11)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7233 ( +9)
  closed 51481 (+56)
  total  58714 (+65)

Open issues with patches: 2940 


Issues opened (43)
==

#45138: [sqlite3] expand bound values in traced statements when possib
https://bugs.python.org/issue45138  reopened by erlendaasland

#46924: make install hangs when installing zoneinfo/_zoneinfo.py
https://bugs.python.org/issue46924  opened by AmericanEnglish

#46925: Document dict behavior when setting equal but not identical ke
https://bugs.python.org/issue46925  opened by malthe

#46926: runpy.run_path didn't set __package__ to None as describe in d
https://bugs.python.org/issue46926  opened by yanhao.charles

#46930: Iterating over cls.__dict__ in classmethod causes RuntimeError
https://bugs.python.org/issue46930  opened by PythonF

#46931: zipfile library will raise uncaught oserror when reading lengt
https://bugs.python.org/issue46931  opened by ultimalium

#46934: Started multiprocessing.Process instances are unserialisable
https://bugs.python.org/issue46934  opened by maggyero

#46938: dataclass __post_init__ recursion
https://bugs.python.org/issue46938  opened by bar.harel

#46939: Specialize calls for Python classes
https://bugs.python.org/issue46939  opened by kj

#46942: Convert Object/classobject.c to Argument Clinic
https://bugs.python.org/issue46942  opened by arhadthedev

#46943: fix[imaplib]: call Exception with string instance
https://bugs.python.org/issue46943  opened by spaceone

#46944: Use FASTCALL calling convention in generator.throw
https://bugs.python.org/issue46944  opened by kumaraditya303

#46945: Quantifier and Expanded Regex Expression Gives Different Resul
https://bugs.python.org/issue46945  opened by vmd3.14

#46946: Port core types to Argument Clinic
https://bugs.python.org/issue46946  opened by arhadthedev

#46949: Print an indication if traceback exceeds sys.tracebacklimit
https://bugs.python.org/issue46949  opened by JelleZijlstra

#46950: Windows 11 venv
https://bugs.python.org/issue46950  opened by darrel.opry

#46951: Zipapp contents are in filesystem-dependent order
https://bugs.python.org/issue46951  opened by h.finucane

#46953: use FASTCALL for __import__ builtin
https://bugs.python.org/issue46953  opened by kumaraditya303

#46956: TextIOWrapper.seek silently wraps around on very large offsets
https://bugs.python.org/issue46956  opened by vlaci

#46957: Logger with a custom class breaks on copy
https://bugs.python.org/issue46957  opened by govinda18

#46958: json dump/dumps prints each array element on a new line (bad f
https://bugs.python.org/issue46958  opened by Entirity

#46959: ctypes.util.find_library can delete /dev/null
https://bugs.python.org/issue46959  opened by barry.c.davis

#46960: Docs: Link from settrace to frame
https://bugs.python.org/issue46960  opened by guettli

#46961: Caching/interning of small ints sometimes fails
https://bugs.python.org/issue46961  opened by steven.daprano

#46962: Fix docstrings that do not honor --without-doc-strings
https://bugs.python.org/issue46962  opened by arhadthedev

#46963: Deep Lazy Imports - Interpreter-level deferred module loading
https://bugs.python.org/issue46963  opened by Kronuz

#46964: The global config should not be stored on each interpreter
https://bugs.python.org/issue46964  opened by eric.snow

#46965: Enable informing callee it's awaited via vector call flag
https://bugs.python.org/issue46965  opened by dino.viehland

#46966: c_void_p array is a footgun on I32LP64 systems
https://bugs.python.org/issue46966  opened by taralx

#46967: Type union for except
https://bugs.python.org/issue46967  opened by Henry Schreiner

#46968: Insufficient sigaltstack size used by CPython prevents extensi
https://bugs.python.org/issue46968  opened by oleksandr-pavlyk

#46970: dataclass(slots=True) incompatible with __init_subclass__
https://bugs.python.org/issue46970  opened by crusaderky

#46973: Provide make target to regenerate autoconf files with containe
https://bugs.python.org/issue46973  opened by christian.heimes

#46975: clang: error: linker command failed with exit code 1 (use -v t
https://bugs.python.org/issue46975  opened by Battant

#46976: Update macOS installer builds to use ncurses 6.3
https://bugs.python.org/issue46976  opened by ned.deily

#46977: tempfile.TemporaryDirectory fails removing dir in some edge ca
https://bugs.python.org/issue46977  opened by afeblot

#46978: Doc strings for built-in, in-place operators are misleading
https://bugs.python.org/issue46978  opened by nickovs

#46981: Empty typing.Tuple
https://bugs.python.org/issue46981  opened by serhiy.storchaka

#46983: test_sqlite3: test_trace_too_much_expanded_sql() failed on AMD
https://bugs.python.org/issue46983  opened by vstinner

#46984: test_concurrent_futures logs many 

[Python-Dev] Python Language Summit at PyCon 2022 in Salt Lake City

[Mariatta]

We're excited to announce that the signups for Python Language Summit at
PyCon 2022 are now open.

Full details at: https://us.pycon.org/2022/events/language-summit/

After two years of virtual/online summit, we will be returning to in-person
format. We will be following the health and safety guidelines at PyCon US:
https://us.pycon.org/2022/attend/health-safety-guidelines/

*TL;DR*

When: Wednesday, April 27, 2022
Where: Salt Palace Convention Center, room TBD

Sign up to attend:  https://forms.gle/CS8B6wJdcaN3rtWV8
 (closes March 25 th, 2022 AoE)
Sign up to discuss a topic: https://forms.gle/LAFE6TTYi15jL5RaA (closes
March 25th, 2022 AoE)

*Who can attend*

We welcome Python core developers and triage team members, active core
contributors to Python and alternative Python implementations, and anyone
else who has a topic to discuss with core developers.

*Who can propose a discussion topic*

If you have discussion items; seeking consensus; awaiting decision on a
PEP; needing help with your core dev work; or have specific questions that
need answers from core developers, please submit a proposal. According to
feedback, our audience prefers more discussions and shorter talks.

In your proposal, please include:
- why is this topic relevant to the core developers
- what is needed from core developers out of this topic

This year's event will be covered by Alex Waygood. Detailed summary of the
event will be published at The PSF's Blog.

*Is this event recorded? Can I watch the livestream?*

No, there will be no recording and no livestream available. If you'd like
to participate in discussions, please sign up to attend. If you'd like to
listen in, please wait for Alex's blog posts after the summit.

Thank you,

Mariatta, Łukasz, & Senthil
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OQ2UJCXV42FERRQAAS7IVPERJBAZIMHC/
Code of Conduct: http://python.org/psf/codeofconduct/