[Python-Dev] Re: Python multithreading without the GIL

2021-10-09 Thread Dan Stromberg
On Thu, Oct 7, 2021 at 9:10 PM Chris Angelico  wrote:

> Concurrency is *hard*. There's no getting around it, there's no
> sugar-coating it. There are concepts that simply have to be learned,
> and the failures can be extremely hard to track down. Instantiating an
> object on the wrong thread can crash GTK, but maybe not immediately.
> Failing to sleep in one thread results in other threads stalling. I
> don't think any of this is changed by different modes (with the
> exception of process-based parallelism, which fixes a lot of
> concurrency at the cost of explicit IPC), and the more work
> programmers want their code to do, the more likely that they'll run
> into this.
>

I'd like to encourage folks not to give up on looking for new, simpler
parallelism/concurrency formalisms.

They're out there - consider how well bash does with its parallelism in
pipelines.

The truly general ones may end up looking like Java, but I don't think they
have to be fully general to be useful.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AVDCKO3OH2SPU54VWAY5AY4HEQWDMPRC/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Python multithreading without the GIL

2021-10-09 Thread Chris Angelico
On Sun, Oct 10, 2021 at 2:31 PM Dan Stromberg  wrote:
>
>
> On Thu, Oct 7, 2021 at 9:10 PM Chris Angelico  wrote:
>>
>> Concurrency is *hard*. There's no getting around it, there's no
>> sugar-coating it. There are concepts that simply have to be learned,
>> and the failures can be extremely hard to track down. Instantiating an
>> object on the wrong thread can crash GTK, but maybe not immediately.
>> Failing to sleep in one thread results in other threads stalling. I
>> don't think any of this is changed by different modes (with the
>> exception of process-based parallelism, which fixes a lot of
>> concurrency at the cost of explicit IPC), and the more work
>> programmers want their code to do, the more likely that they'll run
>> into this.
>
>
> I'd like to encourage folks not to give up on looking for new, simpler 
> parallelism/concurrency formalisms.
>
> They're out there - consider how well bash does with its parallelism in 
> pipelines.

That's process-based parallelism with unidirectional byte-stream IPC.
It's incredibly specific, but also incredibly useful in its place :)

Simpler parallelism techniques are always possible if you don't need
much interaction between the processes. The challenge isn't making the
simple cases work, but making the harder ones efficient.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UPR7EIZTVKESA2ND4ISFXEYBZYLNSTCE/
Code of Conduct: http://python.org/psf/codeofconduct/