[Python-Dev] Re: Deadlock when interrupting interpreter initialisation with ptrace?

2022-06-06 Thread Gabriele
> Don't you have the backtrace from libunwind that you could save from austinp 
> itself?

Unfortunately no as the "deadlock" happens before any samples have a
chance to be collected. Upon further investigation, it seems that
trying to resume a thread over and over when ptrace fails takes quite
"some" time (in fact, more than I'd have hoped). Playing with a larger
wait timeout (100 ms, but the largest number I've seen so far on my
machine is 4 ms, which is still an eternity compared to a sensible
sampling interval of 10 ms) seems to "cure" the problem, which I've
only seen during interpreter initialisation. So perhaps Python itself
is off the hook!

On Mon, 6 Jun 2022 at 19:20, Barry Scott  wrote:
>
>
>
> On 6 Jun 2022, at 17:52, Gabriele  wrote:
>
> I've found it hard to give an answer to this question. Because austinp
> is already tracing the interpreter, I cannot use, e.g., gdb to dump a
> backtrace.
>
>
> Don't you have the backtrace from libunwind that you could save from austinp 
> itself?
>
> Barry
>


-- 
"Egli è scritto in lingua matematica, e i caratteri son triangoli,
cerchi, ed altre figure
geometriche, senza i quali mezzi è impossibile a intenderne umanamente parola;
senza questi è un aggirarsi vanamente per un oscuro laberinto."

-- G. Galilei, Il saggiatore.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Y2GHXXQGJ6CNHHIO6DVERKWKDRUCOQIR/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Accepting PEP 681 – Data Class Transforms

2022-06-06 Thread Petr Viktorin



On 06. 06. 22 20:05, Steve Dower wrote:

+1. Glad it's not just me

On 6/6/2022 2:36 PM, Victor Stinner wrote:

First, I understood that "Arbitrary Literal String Type" was adding a
new built-in types for "literal strings" :-) Nope. It's just about
type annotations ;-)


I was also excited about the new built-in type :-)

Pablo is already separating the PEPs in new release announcements (or 
has at least agreed to, not sure any releases have happened since I asked).


But with the amount of work going on in separate venues these days, 
separating "CPython implementation" from "Language specification" from 
"Typing specification" would be helpful. (Packaging is already separate, 
but doesn't appear in release announcements anyway.)


There's an issue about adding the category metadata: 
https://github.com/python/peps/issues/2572
Rather than putting it in the title itself, it might be be better to 
only show the category more prominently in lists & announcements.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KY63VEYYMBETD2OKXHKI4BT7DRGPUOVE/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Accepting PEP 681 – Data Class Transforms

2022-06-06 Thread Pablo Galindo Salgado
> or has at least agreed to, not sure any releases have happened since I
asked).
I did:

https://www.python.org/downloads/release/python-3110b3/#:~:text=3.11.0b3%20is%20the%20second,support%20the%20new%20feature%20release
.

On Mon, 6 Jun 2022 at 19:13, Steve Dower  wrote:

> +1. Glad it's not just me
>
> On 6/6/2022 2:36 PM, Victor Stinner wrote:
> > First, I understood that "Arbitrary Literal String Type" was adding a
> > new built-in types for "literal strings" :-) Nope. It's just about
> > type annotations ;-)
>
> I was also excited about the new built-in type :-)
>
> Pablo is already separating the PEPs in new release announcements (or
> has at least agreed to, not sure any releases have happened since I asked).
>
> But with the amount of work going on in separate venues these days,
> separating "CPython implementation" from "Language specification" from
> "Typing specification" would be helpful. (Packaging is already separate,
> but doesn't appear in release announcements anyway.)
>
> Cheers,
> Steve
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/3VJEOP2VYQ66G4PS665QV52P5TGRYP5O/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/R3Z4V7NZ2MOCUYDZZKNJLPZIGFA6Z2GN/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Deadlock when interrupting interpreter initialisation with ptrace?

2022-06-06 Thread Barry Scott


> On 6 Jun 2022, at 17:52, Gabriele  wrote:
> 
> I've found it hard to give an answer to this question. Because austinp
> is already tracing the interpreter, I cannot use, e.g., gdb to dump a
> backtrace.


Don't you have the backtrace from libunwind that you could save from austinp 
itself?

Barry

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BJHDFD7GHL7NZK3X5YSW4G74663LCEIP/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Accepting PEP 681 – Data Class Transforms

2022-06-06 Thread Steve Dower

+1. Glad it's not just me

On 6/6/2022 2:36 PM, Victor Stinner wrote:

First, I understood that "Arbitrary Literal String Type" was adding a
new built-in types for "literal strings" :-) Nope. It's just about
type annotations ;-)


I was also excited about the new built-in type :-)

Pablo is already separating the PEPs in new release announcements (or 
has at least agreed to, not sure any releases have happened since I asked).


But with the amount of work going on in separate venues these days, 
separating "CPython implementation" from "Language specification" from 
"Typing specification" would be helpful. (Packaging is already separate, 
but doesn't appear in release announcements anyway.)


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3VJEOP2VYQ66G4PS665QV52P5TGRYP5O/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] [RELEASE] Python 3.10.5 is available

2022-06-06 Thread Pablo Galindo Salgado
The latest bugfix drop for Python 3.10 is here: Python 3.10.5. This release
packs more than 230 bugfixes and docs changes, so you surely want to update
:) You can get it here:

https://www.python.org/downloads/release/python-3105/

## This is the fourth maintenance release of Python 3.10

Python 3.10.5 is the newest major release of the Python programming
language, and it contains many new features and optimizations.

# Major new features of the 3.10 series, compared to 3.9

Among the new major new features and changes so far:

* [PEP 623](https://www.python.org/dev/peps/pep-0623/) -- Deprecate and
prepare for the removal of the wstr member in PyUnicodeObject.
* [PEP 604](https://www.python.org/dev/peps/pep-0604/) -- Allow writing
union types as X | Y
* [PEP 612](https://www.python.org/dev/peps/pep-0612/) -- Parameter
Specification Variables
* [PEP 626](https://www.python.org/dev/peps/pep-0626/) -- Precise line
numbers for debugging and other tools.
* [PEP 618 ](https://www.python.org/dev/peps/pep-0618/) -- Add Optional
Length-Checking To zip.
* [bpo-12782](https://bugs.python.org/issue12782): Parenthesized context
managers are now officially allowed.
* [PEP 632 ](https://www.python.org/dev/peps/pep-0632/) -- Deprecate
distutils module.
* [PEP 613 ](https://www.python.org/dev/peps/pep-0613/) -- Explicit Type
Aliases
* [PEP 634 ](https://www.python.org/dev/peps/pep-0634/) -- Structural
Pattern Matching: Specification
* [PEP 635 ](https://www.python.org/dev/peps/pep-0635/) -- Structural
Pattern Matching: Motivation and Rationale
* [PEP 636 ](https://www.python.org/dev/peps/pep-0636/) -- Structural
Pattern Matching: Tutorial
* [PEP 644 ](https://www.python.org/dev/peps/pep-0644/) -- Require OpenSSL
1.1.1 or newer
* [PEP 624 ](https://www.python.org/dev/peps/pep-0624/) -- Remove
Py_UNICODE encoder APIs
* [PEP 597 ](https://www.python.org/dev/peps/pep-0597/) -- Add optional
EncodingWarning

[bpo-38605](https://bugs.python.org/issue38605): `from __future__ import
annotations` ([PEP 563](https://www.python.org/dev/peps/pep-0563/)) used to
be on this list
in previous pre-releases but it has been postponed to Python 3.11 due to
some compatibility concerns. You can read the Steering Council
communication about it [here](
https://mail.python.org/archives/list/python-dev@python.org/thread/CLVXXPQ2T2LQ5MP2Y53VVQFCXYWQJHKZ/)
to learn more.

# More resources

* [Changelog](https://docs.python.org/3.10/whatsnew/changelog.html#changelog
)
* [Online Documentation](https://docs.python.org/3.10/)
* [PEP 619](https://www.python.org/dev/peps/pep-0619/), 3.10 Release
Schedule
* Report bugs at [https://bugs.python.org](https://bugs.python.org).
* [Help fund Python and its community](/psf/donations/).

# And now for something completely different
Strange quarks are the third lightest quarks, which are subatomic particles
that are so small,  they are believed to be the fundamental particles, and
not further divisible. Like down quarks, strange quarks have a charge of
-1/3. Like all fermions (which are particles that can not exist in the same
place at the same time), strange quarks have a spin of 1/2. What makes
strange quarks different from down quarks–apart from having 25 times the
mass of down quarks–is that they have something that scientists call
"strangeness." Strangeness is basically a resistance to decay against
strong force and electromagnetism. This means that any particle that
contains a strange quark can not decay due to strong force (or
electromagnetism), but instead with the much slower weak force. It was
believed that this was a 'strange' method of decay, which is why the
scientists gave the particles that name.

# We hope you enjoy the new releases!

Thanks to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python
Software Foundation.

https://www.python.org/psf/

Your friendly release team,

Ned Deily @nad https://discuss.python.org/u/nad
Steve Dower @steve.dower https://discuss.python.org/u/steve.dower
Pablo Galindo Salgado @pablogsal https://discuss.python.org/u/pablogsal
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/V4AUICXW2EDGFK2EIDSFA6RXMNWLCC54/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Deadlock when interrupting interpreter initialisation with ptrace?

2022-06-06 Thread Gabriele
> Do you know what is involved in the deadlock (as in, what the threads are 
> waiting on)?

I've found it hard to give an answer to this question. Because austinp
is already tracing the interpreter, I cannot use, e.g., gdb to dump a
backtrace. The event is also quite rare and it seems to happen before
austinp has the chance to capture any samples. With the new support
for 3.11 I might be able to see if I come across the same issue with
the latest beta. I was hoping that the description of the issue could
have rung a bell for anybody more familiar than me with all the
locking going on during imports. The logs from austinp seem to suggest
that the thread fails to resume after being interrupted, so something
for me to explore is whether attempting to resume the thread more
times before giving up is the actual solution in this case.

> How do you spawn a new process?

I should have clarified that this is just a plain fork/exec from C:
https://github.com/P403n1x87/austin/blob/e3d79ddc9f9737a791362e6962b5cac25a4e3dc2/src/py_proc.c#L972-L1010

Cheers,
Gabriele

On Mon, 6 Jun 2022 at 16:30, Victor Stinner  wrote:
>
> On Mon, Jun 6, 2022 at 4:35 PM Gabriele  wrote:
> > The austinp variant is a variant of Austin
> > (https://github.com/P403n1x87/austin) for Linux that uses ptrace to
> > seize and interrupt/continue threads to capture native stack traces
> > using libunwind. During testing, I have discovered that there are good
> > chances of causing what looks like a deadlock in Python if the seizing
> > and interrupting of threads happen very early when spawning a Python
> > subprocess from austinp.
>
> Do you have a backtrace of the Python main thread when the hang
> happens? How do you spawn a new process? With the Python subprocess
> module?
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.



-- 
"Egli è scritto in lingua matematica, e i caratteri son triangoli,
cerchi, ed altre figure
geometriche, senza i quali mezzi è impossibile a intenderne umanamente parola;
senza questi è un aggirarsi vanamente per un oscuro laberinto."

-- G. Galilei, Il saggiatore.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DHRBK2WNXQJNKKQDHTGNSKRQ32MO365H/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Deadlock when interrupting interpreter initialisation with ptrace?

2022-06-06 Thread Victor Stinner
On Mon, Jun 6, 2022 at 4:35 PM Gabriele  wrote:
> The austinp variant is a variant of Austin
> (https://github.com/P403n1x87/austin) for Linux that uses ptrace to
> seize and interrupt/continue threads to capture native stack traces
> using libunwind. During testing, I have discovered that there are good
> chances of causing what looks like a deadlock in Python if the seizing
> and interrupting of threads happen very early when spawning a Python
> subprocess from austinp.

Do you have a backtrace of the Python main thread when the hang
happens? How do you spawn a new process? With the Python subprocess
module?

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3OUS6BAV3B7A4BQBCLHLKUQMJM3PF646/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Deadlock when interrupting interpreter initialisation with ptrace?

2022-06-06 Thread Pablo Galindo Salgado
Hi Gabriele,

If everything you are doing is pausing and restarting, there should be no
reason
why this would interfere with anything more than if you are doing this at
any other time
other than the interpreter initialization. The only thing I can think of is
that at this stage
locking is much more common.

The other thing that could be at play is that ptrace sends SIGSTOP
on PTRACE_ATTACH but
the signal cannot be captured by the interpreter (or any other process) so
no signal handler
should be at play either.

Do you know what is involved in the deadlock (as in, what the threads are
waiting on)?

Answering your questions directly:

> However, I think this poses one question: is this behaviour from
Python to be expected or is it perhaps an indication of a potential
bug?

Is not expected or unexpected because is not something we support. Is not
also something we
explicitly forbid either, is just that there is nothing in the design or
the test suite that ensures that
this will work.

> is it acceptable that the pausing and resuming of the execution of a
thread
lead to a potential deadlock?

It depends if this is something that we can control in a reasonable way or
if this is outside our control.
It may be a bug in our code in which case we can try to fix it, but without
a more concrete pointer is
going to be complicated, especially given that is more likely that this is
outside our control.

We probably will reject any proposal to add complexity to support this use
case but we likely will
be happy to do small changes if there is something small that we do that is
preventing the use case.

Cheers from cloudy London,
Pablo Galindo Salgado


On Mon, 6 Jun 2022 at 15:38, Gabriele  wrote:

> Hi there
>
> I hope you don't mind me sharing my experience with testing the
> austinp variant of Austin with Python >=2.7,<3.11.
>
> The austinp variant is a variant of Austin
> (https://github.com/P403n1x87/austin) for Linux that uses ptrace to
> seize and interrupt/continue threads to capture native stack traces
> using libunwind. During testing, I have discovered that there are good
> chances of causing what looks like a deadlock in Python if the seizing
> and interrupting of threads happen very early when spawning a Python
> subprocess from austinp. This seems to coincide with the
> initialisation of the interpreter when modules are being loaded. To
> avoid interfering so destructively with Python, I have added a sleep
> of about 0.5s on fork to prevent sampling during this initialisation
> phase, which has helped significantly.
>
> However, I think this poses one question: is this behaviour from
> Python to be expected or is it perhaps an indication of a potential
> bug? Whilst I find it conceivable that something like this could
> happen, given the locking that happens around imports, is it
> acceptable that the pausing and resuming of the execution of a thread
> lead to a potential deadlock?
>
> Cheers,
> Gabriele
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EWE5IK53IAME7ODZOGCQGSSP4YBE37YX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KH2CRDMPSRA4YHXQNQ4Z2WLSG3Y5IPMR/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Deadlock when interrupting interpreter initialisation with ptrace?

2022-06-06 Thread Gabriele
Hi there

I hope you don't mind me sharing my experience with testing the
austinp variant of Austin with Python >=2.7,<3.11.

The austinp variant is a variant of Austin
(https://github.com/P403n1x87/austin) for Linux that uses ptrace to
seize and interrupt/continue threads to capture native stack traces
using libunwind. During testing, I have discovered that there are good
chances of causing what looks like a deadlock in Python if the seizing
and interrupting of threads happen very early when spawning a Python
subprocess from austinp. This seems to coincide with the
initialisation of the interpreter when modules are being loaded. To
avoid interfering so destructively with Python, I have added a sleep
of about 0.5s on fork to prevent sampling during this initialisation
phase, which has helped significantly.

However, I think this poses one question: is this behaviour from
Python to be expected or is it perhaps an indication of a potential
bug? Whilst I find it conceivable that something like this could
happen, given the locking that happens around imports, is it
acceptable that the pausing and resuming of the execution of a thread
lead to a potential deadlock?

Cheers,
Gabriele
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EWE5IK53IAME7ODZOGCQGSSP4YBE37YX/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Accepting PEP 681 – Data Class Transforms

2022-06-06 Thread Victor Stinner
Hi,

Side note: it would be nice to add "typing: " prefix or mention "type
annotation" or "type check" in the title of PEPs which are about that.

Just from the PEP title, it's hard *for me* to guess that it's about
type annotations.

Examples of other PEP titles which confused me:

PEP 612 – Parameter Specification Variables
PEP 645 – Allow writing optional types as x?
PEP 646 – Variadic Generics
PEP 647 – User-Defined Type Guards
PEP 673 – Self Type
PEP 675 - Arbitrary Literal String Type
PEP 677 – Callable Type Syntax

First, I understood that "Arbitrary Literal String Type" was adding a
new built-in types for "literal strings" :-) Nope. It's just about
type annotations ;-)

From what I understood, the purpose of these PEPs outside type
annotations is limited or non existent :-)

Victor

On Mon, Jun 6, 2022 at 2:51 PM Petr Viktorin  wrote:
>
> Hello,
> With the latest wording changes, PEP 681 – Data Class Transforms is now
> fully accepted. Feel free to mark it as such at your convenience.
>
> Happy typing,
> — Petr, on behalf of the Steering Council
>
>
> On 23. 04. 22 13:26, Petr Viktorin wrote:
> > Hello,
> > As an initial implementation that will be improved in the future, the
> > specification in PEP 681 is fine. Feel free to add the decorator to
> > Python 3.11 at your convenience.
> >
> > However, the PEP includes several worrying recommendations like:
> >
> > - we recommend that the maintainers of attrs move away from the legacy
> > semantics and adopt auto_attribs behaviors by default.
> > - We chose not to support this feature and recommend that attrs users
> > avoid converters.
> > - Attrs users should use the dataclass-standard eq and order parameter
> > names instead.
> >
> > These are probably meant as recommendations from typing-sig, but an
> > accepted PEP represents consensus of the entire Python community. A
> > typing PEP is not an appropriate place to make recommendations like
> > this, especially without reaching out to the maintainer of attrs.
> > As far as I know,the attrs and pydantic libraries are using the
> > reference implementation, but their authors weren't consulted on the PEP
> > itself.
> >
> > Could you either change the wording (e.g. say that the unsupported
> > features need bespoke type-checker functionality for proper type
> > checking), or work with attrs to make the same recommendations in its
> > documentation?
> >
> >
> > Happy typing,
> > — Petr, on behalf of the Steering Council
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/R4A2IYLGFHKFDYJPSDA5NFJ6N7KRPJ6D/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LVNXFRLEMJXTEPC3N4M3NKQ4YRYQQZTA/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Accepting PEP 681 – Data Class Transforms

2022-06-06 Thread Petr Viktorin

Hello,
With the latest wording changes, PEP 681 – Data Class Transforms is now 
fully accepted. Feel free to mark it as such at your convenience.


Happy typing,
— Petr, on behalf of the Steering Council


On 23. 04. 22 13:26, Petr Viktorin wrote:

Hello,
As an initial implementation that will be improved in the future, the 
specification in PEP 681 is fine. Feel free to add the decorator to 
Python 3.11 at your convenience.


However, the PEP includes several worrying recommendations like:

- we recommend that the maintainers of attrs move away from the legacy 
semantics and adopt auto_attribs behaviors by default.
- We chose not to support this feature and recommend that attrs users 
avoid converters.
- Attrs users should use the dataclass-standard eq and order parameter 
names instead.


These are probably meant as recommendations from typing-sig, but an 
accepted PEP represents consensus of the entire Python community. A 
typing PEP is not an appropriate place to make recommendations like 
this, especially without reaching out to the maintainer of attrs.
As far as I know,the attrs and pydantic libraries are using the 
reference implementation, but their authors weren't consulted on the PEP 
itself.


Could you either change the wording (e.g. say that the unsupported 
features need bespoke type-checker functionality for proper type 
checking), or work with attrs to make the same recommendations in its 
documentation?



Happy typing,
— Petr, on behalf of the Steering Council

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/R4A2IYLGFHKFDYJPSDA5NFJ6N7KRPJ6D/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Looking for volunteers to maintain the multiprocessing and concurrent.futures modules

2022-06-06 Thread Victor Stinner
Hi,

On buildbots, it's common that we get at least one multiprocessing
test failure per week. While I just reported a new one, I wanted to
add a "expert-multiprocessing" label, but it didn't exist. I just
created the label :-)

   https://github.com/python/cpython/labels/expert-multiprocessing

I added the label manually to 240 issues! Oh, they are many open
issues about multiprocessing and concurrent.futures modules.
Documentation issues, feature requests, but also a long list of real
bugs ("hang" is a common term in these reports).

Handling multiple processes is hard to get it right, especially when
supporting multiple operating systems matter. In Python 3.8,
multiprocessing got shared memory support which opened another can of
worms :-)

Sadly, current multiprocessing maintainers are no longer available and
the number of open issues and open pull requests is increasing.

I'm not an expert in multiprocessing, but I'm volunteer to mentor
someone who is expert and is interested to go through these lists. I
cannot dedicate a lot of time, but I will try to be my best to be
available :-)

See also the GitHub Projects:

Multiprocessing issues:
https://github.com/orgs/python/projects/14

Pickle and copy issues:
https://github.com/orgs/python/projects/9

If you are interested, contact me in private. Sorry, I am not
available to teach you how multiprocessing works. I can only assist
people who already know it.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6FZECDUVJVEYY7DCCK2OQO66NIEQUE5M/
Code of Conduct: http://python.org/psf/codeofconduct/