[Python-Dev] Re: Starting a new thread

[Gregory P. Smith]

On Tue, May 10, 2022 at 10:34 AM Barney Stratford <
barney_stratf...@fastmail.fm> wrote:

> > 1. Does t = target(...) start the thread? I think it does.
> I think it does too. In the commonest use case, immediately after creating
> a thread, you start it. And if you want to delay the thread but still use
> the decorator, then you can do that explicitly with some locks. In fact,
> it’s probably better to explicitly delay execution than have hidden
> requirements concerning the timing of thread creation and startup.
>
> > 2. Is it possible to create daemon threads?
> Not at the moment. I did think about this, but felt that simpler is
> better. Like you say, it’d be easy to add. In fact, I might just go ahead
> and add it to the PR in a bit. The simplest way to do it is probably to
> define a second decorator for daemonic threads.
>

If this is even to be added (i personally lean -1 on it), I suggest
intentionally not supporting daemon threads. We should not encourage them
to be used, they were a misfeature that in hindsight we should never have
created. Daemon threads can lead to very bad surprises upon interpreter
finalization - an unfixable problem given how daemon threads are defined to
behave.

> 3. Can you create multiple threads for the same function? I assume t1,
> > t2, t3 = target(arg1), target(arg2), target(arg3) would work.
> That’s exactly what I had in mind. Make it so that thread creation and
> function call look exactly alike. You can call a function as many times as
> you want with whatever args you want, and you can create threads as often
> as you want with whatever args you want.
>
> There isn’t a single use case where the decorator is particularly
> compelling; rather, it’s syntactic sugar to hide the mechanism of thread
> creation so that code reads better.
>

This is my take as well. I don't like calling code to hide the fact that a
thread is being spawned. Use this decorator and if you fail to give the
callable a name communicating that it spawns and returns a thread, you will
have surprised readers of the calling code.

A nicer design pattern is to explicitly manage threads. Use
concurrent.futures.ThreadPoolExecutor. Or use the async stuff that Joao
mentioned or similar libraries. I think we already provide decent batteries
with the threading APIs.

-gps
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3I7GBDGMHU4BHHR57EFOL6B4JMURFSGE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Victor Stinner]

On Tue, May 10, 2022 at 7:18 PM Guido van Rossum  wrote:
> Shouldn't we wean our internal tools off this obsolete version of distutils 
> too, rather than just move it around?

Here is a first PR to build the test C++ extension with setuptools in
test_cppext:
https://github.com/python/cpython/pull/92639

It implements what I proposed in the issue. It runs the test in a
venv. Python stdlib (ensurepip) includes a setuptools wheel package
and so is able to install it without adding any new external
dependency.

If this approach works, we can use the same for test_peg_generator.

Victor
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S6UKSWFTJGPSTK2MZ4RIPVQJ7CKAJCQG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Steve Dower]

On 5/10/2022 9:38 PM, Christopher Barker wrote:
So if you can't entirely drop distutils in the stdlib, I don't think we 
can count on the rest of the community being able to drop it as well.


The only reason distutils is used to build Python is for 
self-bootstrapping. setuptools would have to avoid any module that may 
potentially be built as an extension module if it were to be used at 
that stage of build, which is unfair (and this use is also likely to 
become Makefile for Linux and macOS, just as it uses static build 
scripts on Windows already).


The other uses are tests or tools. There's nothing wrong with 
setuptools's functionality, merely the fact that it isn't part of the 
CPython repository and we don't like relying on third-party repositories 
for a CPython build/test.


Cheers,
Steve

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Y2LVG23ZMH3JOBBZFQ2D2P554DZWRI36/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Steve Dower]

If we're not shipping it, we should just move it out of the way.

Otherwise, there are a few places in the various Windows packages that 
will need it to be explicitly excluded. We should already have similar 
exclusions for the test suite and a few other modules, but they're a bit 
scattered around PC, PCbuild and Tools/msi. Easier to just remove it 
from Lib.


Cheers,
Steve

On 5/10/2022 9:24 PM, Victor Stinner wrote:

On Tue, May 10, 2022 at 6:16 PM Steve Dower  wrote:

I agree. The internal tools that use it are all in our Tools directory,
so we can move distutils there and explicitly add an import path to
locate it. No need to keep it in the stdlib (Lib/) at all.

Migrating to Makefile builds is probably better long-term, but not as
important as moving distutils out from the stdlib so that setuptools can
rely on their copy being the "main" one.


With my current PR, Lib/_distutils/ is not installed by "make
install". Moving it to Tools/ would also work.

Victor

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JG2BVL54C3RYXFFIX2TL7C53YPIWQHBQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Christopher Barker]

Just a note:

The huge amount of work/progress that has gone into modernizing the
building/packaging tools is great, but to be honest, C extensions have been
somewhat neglected. e.g. I still need to use old fashioned setup.py to
build some of my projects (C and Cython extensions)

This may not be relevant to this discussion, as I think setuptools does
still support C extensions as well as it has for years -- All of my recent
(last ten years) projects use it.

But it makes me nervous when I see suggestions to keep distutils in the
stdlib: if the stdlib finds it still needs distutils, then that means that
there may be others in the community that need it too.

So if you can't entirely drop distutils in the stdlib, I don't think we can
count on the rest of the community being able to drop it as well.

Thanks,

-CHB




On Tue, May 10, 2022 at 1:29 PM Victor Stinner  wrote:

> On Tue, May 10, 2022 at 6:16 PM Steve Dower 
> wrote:
> > I agree. The internal tools that use it are all in our Tools directory,
> > so we can move distutils there and explicitly add an import path to
> > locate it. No need to keep it in the stdlib (Lib/) at all.
> >
> > Migrating to Makefile builds is probably better long-term, but not as
> > important as moving distutils out from the stdlib so that setuptools can
> > rely on their copy being the "main" one.
>
> With my current PR, Lib/_distutils/ is not installed by "make
> install". Moving it to Tools/ would also work.
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/T43OZ4LWUT7QBENWPKIA3LFT52GWJV3Q/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HRUMPQICS5WT6M67WH7QVY6CHQFGMF6C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Victor Stinner]

On Tue, May 10, 2022 at 6:16 PM Steve Dower  wrote:
> I agree. The internal tools that use it are all in our Tools directory,
> so we can move distutils there and explicitly add an import path to
> locate it. No need to keep it in the stdlib (Lib/) at all.
>
> Migrating to Makefile builds is probably better long-term, but not as
> important as moving distutils out from the stdlib so that setuptools can
> rely on their copy being the "main" one.

With my current PR, Lib/_distutils/ is not installed by "make
install". Moving it to Tools/ would also work.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/T43OZ4LWUT7QBENWPKIA3LFT52GWJV3Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Brett Cannon]

On Tue, May 10, 2022 at 10:23 AM Guido van Rossum  wrote:

> Shouldn't we wean our internal tools off this obsolete version of
> distutils too, rather than just move it around?
>

I think so. We technically have a year to get this done, so if we can't do
that then I'm afraid we have much bigger issues with the project's
maintenance.

-Brett


>
> On Tue, May 10, 2022 at 9:26 AM Steve Dower 
> wrote:
>
>> On 5/10/2022 4:33 PM, Christian Heimes wrote:
>> > On 10/05/2022 13.18, Victor Stinner wrote:
>> >> test_peg_generator and test_cppext build C extensions with distutils.
>> >> Until these tests are modified to use something else, we still need to
>> >> keep distutils. So I propose to rename it to _distutils to remove it
>> >> from the stdlib. Maybe these tests can use ensurepip to install
>> >> setuptools which provides distutils.
>> >>
>> >> There is also the c-analyzer tool which uses distutils to run a C
>> >> preprocessor.
>> >
>> > We can easily take care of test_cppext and add the build step to
>> > Makefile. How does test_peg_generator depend on distutils? I don't see
>> > an import of distutils in the directory.
>> >
>> > I would prefer to fix and remove all imports of distutils before we
>> > resort to renaming the package. Please give us time until alpha 1.
>> There
>> > is no need to rush it *now*.
>>
>> I agree. The internal tools that use it are all in our Tools directory,
>> so we can move distutils there and explicitly add an import path to
>> locate it. No need to keep it in the stdlib (Lib/) at all.
>>
>> Migrating to Makefile builds is probably better long-term, but not as
>> important as moving distutils out from the stdlib so that setuptools can
>> rely on their copy being the "main" one.
>>
>> Cheers,
>> Steve
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/LGF4BJMN3H7L6QFTZTDBMOA2GPZQFHC6/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/L3JRME4CC5SXJJ2MSYSXBU6OBYLWVKKS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/45EXS6S7PMTNWYMVU6EWLFFIIYAM3WR4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Brett Cannon]

On Tue, May 10, 2022 at 12:59 AM Victor Stinner  wrote:

> Hi,
>
> Two years ago, PEP 632 "Deprecate distutils module" was accepted: the
> distutils package was deprecated in Python 3.10 and scheduled for
> removal in Python 3.12. Questions.
>
> * Is the Python ecosystem ready for the distutils removal? How many
> projects are expected to be broken by this removal?
>
> * Is setuptools a drop-in replacement of distutils for most cases? Are
> there tools and documentation explaining how to replace distutils with
> setuptools?
>

Change `import distutils` with `import setuptools`.  But more thorough
guidance can be found at
https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html?highlight=distutils
.


>
> * Is there a tool to migrate from setup.py (distutils) to
> pyproject.toml (setuptools)? The dephell project can convert a
> setup.py script to pyproject.toml using poetry as the build system.
>

You don't have to migrate a setuptools project to pyproject.toml to keep
functioning. I would strongly encourage people at least specify a
build-system table in pyproject.toml as it's easy to do, but pip doesn't
require even this for setuptools projects:

```toml

[build-system]requires = ["setuptools"]build-backend = "setuptools.build_meta"

```

But the rest of the project's metadata can stay in setup.py or setup.cfg if
they want. But there is experimental support to fully support PEP 621 and
specifying metadata in pyproject.toml:
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html .


>
> * Can we simply suggest installing setuptools to users who still use
> "import distutils"? setuptools now provides the a "distutils" package.
>

If they are unwilling to update, yes.


>
> * Should we keep distutils in Python stdlib a few more Python releases
> if the removal causes too many practical issues?
>

I don't think so.

-Brett


>
>
> A code search in top 5000 PyPI projects (at 2022-01-26) using the
> regex '(import|from) distutils' matchs 5,567 lines in 1,229 projects.
> Some statistics:
>
> * 851 projects (1,372 lines) import distutils in setup.py
> * 233 projects (700 lines) use distutils.version
> * 92 projects (205 lines) use distutils.util
> * 1,018 lines are type annotations (.pyi files)
>
> I failed to group the remaining 2,272 lines.
>
> Converting 851 projects from setup.py with distutils to pyproject.toml
> is going to take a few months if not years. Python 3.12 will likely be
> released in October 2023 (in year and a half).
>
>
> Since setuptools 60 (December 2021), setuptools installs a
> distutils-precedence.pth file to override the stdlib module with its
> local distutils copy by default. The SETUPTOOLS_USE_DISTUTILS=stdlib
> environment variable can be used to explicitly request the stdlib
> flavor. If I understood correct, distutils is now maintained in
> setuptools, rather than in the Python stdlib.
>
> Python "make install" installs an up to date setuptools and so
> indirectly its distutils-precedence.pth file which makes the distutils
> local copy of setuptools available. But many Linux distributions
> splits setuptools and pip from the "python" package.
>
>
> Right now, Python still uses distutils internally for multiple use
> cases. I propose to start with renaming the distutils package to
> _distutils in the stdlib:
>
> * https://github.com/python/cpython/issues/92584
> * https://github.com/python/cpython/pull/92585
>
> Right now, the change is blocked by pip which still imports distutils
> (even if it prefers sysconfig in practice on Python 3.10 and newer). I
> reported the issue to pip, it should be easy to fix.
>
> Once Python will no longer depend on _distutils, it will be possible
> to fully remove it.
>
>
> By the way, Fedora 35 now longer provides automatically setuptools
> when building a Python package. A dependency to setuptools must now be
> explicit, since they are projects which don't use setuptools nor
> distutils, but other build systems.
>
>
> https://fedoraproject.org/wiki/Changes/Reduce_dependencies_on_python3-setuptools
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/XFZJ43VSIWCYNQ5ZYT64XF7PMH4YDUWC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7YOD2KW3JND5DDHOBS22XVCUYQWWTEP7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Starting a new thread

[Barney Stratford]

> 1. Does t = target(...) start the thread? I think it does.
I think it does too. In the commonest use case, immediately after creating a 
thread, you start it. And if you want to delay the thread but still use the 
decorator, then you can do that explicitly with some locks. In fact, it’s 
probably better to explicitly delay execution than have hidden requirements 
concerning the timing of thread creation and startup.

> 2. Is it possible to create daemon threads?
Not at the moment. I did think about this, but felt that simpler is better. 
Like you say, it’d be easy to add. In fact, I might just go ahead and add it to 
the PR in a bit. The simplest way to do it is probably to define a second 
decorator for daemonic threads.

> 3. Can you create multiple threads for the same function? I assume t1,
> t2, t3 = target(arg1), target(arg2), target(arg3) would work.
That’s exactly what I had in mind. Make it so that thread creation and function 
call look exactly alike. You can call a function as many times as you want with 
whatever args you want, and you can create threads as often as you want with 
whatever args you want.

There isn’t a single use case where the decorator is particularly compelling; 
rather, it’s syntactic sugar to hide the mechanism of thread creation so that 
code reads better. I could give some examples of where I used it recently if 
you want, but I don’t think it would be terribly illuminating. More useful 
might be to look at the problem from the opposite perspective: would anyone 
like to write

result = call_function(target = foo, args = (42,), kwargs = {“bar”: 60})

in preference to

result = foo(42, bar = 60)

Cheers,
Barney.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YESYSRUCR666HASBXYHO2SI6AI6T5MEW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Guido van Rossum]

Shouldn't we wean our internal tools off this obsolete version of distutils
too, rather than just move it around?

On Tue, May 10, 2022 at 9:26 AM Steve Dower  wrote:

> On 5/10/2022 4:33 PM, Christian Heimes wrote:
> > On 10/05/2022 13.18, Victor Stinner wrote:
> >> test_peg_generator and test_cppext build C extensions with distutils.
> >> Until these tests are modified to use something else, we still need to
> >> keep distutils. So I propose to rename it to _distutils to remove it
> >> from the stdlib. Maybe these tests can use ensurepip to install
> >> setuptools which provides distutils.
> >>
> >> There is also the c-analyzer tool which uses distutils to run a C
> >> preprocessor.
> >
> > We can easily take care of test_cppext and add the build step to
> > Makefile. How does test_peg_generator depend on distutils? I don't see
> > an import of distutils in the directory.
> >
> > I would prefer to fix and remove all imports of distutils before we
> > resort to renaming the package. Please give us time until alpha 1. There
> > is no need to rush it *now*.
>
> I agree. The internal tools that use it are all in our Tools directory,
> so we can move distutils there and explicitly add an import path to
> locate it. No need to keep it in the stdlib (Lib/) at all.
>
> Migrating to Makefile builds is probably better long-term, but not as
> important as moving distutils out from the stdlib so that setuptools can
> rely on their copy being the "main" one.
>
> Cheers,
> Steve
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LGF4BJMN3H7L6QFTZTDBMOA2GPZQFHC6/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L3JRME4CC5SXJJ2MSYSXBU6OBYLWVKKS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Starting a new thread

[Joao S. O. Bueno]

On Tue, May 10, 2022 at 1:20 PM Paul Moore  wrote:

> On Tue, 10 May 2022 at 16:31, Barney Stratford
>  wrote:
> >
> > Hello all.
> >
> > It occurs to me that creating threads in Python is more awkward than it
> needs to be. Every time I want to start a new thread, I end up writing the
> same thing over and over again:
> >
> > def target(*args, **kwds):
> > ...
> > t = threading.Thread(target = target, args = , kwargs=
> )
> > t.start()
> >
> > This becomes especially repetitive when calling a target function that
> only makes sense when run in a new thread, such as a timer.
> >
> > In my own code, I’ve taken to decorating functions that always run in
> new threads. Then I can call the function using the usual function call
> syntax, and have the new thread returned to me. With the decorator, the
> code reads instead:
> >
> > @threaded
> > def target(*args, **kwds):
> > …
> > t = target(*args, **kwds)
> >
> > This has a couple of advantages. I don’t have to import the threading
> module all over my code. I can use the neater syntax of function calls. The
> function’s definition makes it clear it’s returning a new thread since it’s
> decorated. It gets the plumbing out of the way so I can concentrate more on
> what my code does and less in how it does it.
> >
> > It feels like the right place for this decorator is the standard
> library, so I’ve created PR #91878 for it. @rhettinger suggests that this
> is a bit premature, and that we should discuss it here first. Thoughts?
>
> I like this. I'd probably use it if it existed. Your example isn't
> particularly compelling, though - calling the function "target" makes
> sense when it's the target of the thread, but not so much when it's
> the thread itself. Could you give a motivating example based on real
> code, rather than just the "this is what the syntax would look like"
> as you did above? Ideally showing what it would look like with and
> without the decorator?
>
> Things I *think* i get, but I'd like to see clearly in an example, include:
>
> 1. Does t = target(...) start the thread? I think it does.
> 2. Is it possible to create daemon threads? @threaded(daemon=True)
> seems plausible. I suspect your PR doesn't have this, but it wouldn't
> be difficult to add. But I don't actually know if it's worth adding,
> TBH.
> 3. Can you create multiple threads for the same function? I assume t1,
> t2, t3 = target(arg1), target(arg2), target(arg3) would work.
>
> I'm probably being dense here - I'm pretty sure your proposal actually
> is as simple as it looks (which is a good thing!) and I'm just
> over-complicating things in my head.
>
> Things that come to mind that should be considered here:
1) not every two liner function should be added to the stdlib
and, on the other hand,
2) asyncio features `loop.run_in_executor`
which does essentially the same, with more care for what happens
in "real world" code, but is only usable in async code.

all in all it can be useful for quick and dirty code - and maybe
something more elaborate allowing control on whether to create
daemon threads, or maybe something that will deal with
concurrent.future.executors
as well, and allow access to the function call results

All in all, I find it a nice snippet, but one that is not aligned with
the trend Python has been taking over the last years,
facilitating the writing of "quick and dirty" scripts in contrast
with application code that will take care of all possible corner-cases.

As a good example of what I am talking about, we've seen the contrary
 movement take place, long ago, when
"os.popen" was superseded by "subprocess.*": I think up to today people
had not catched up with the far more complicated API of the latter.

All in all, I'd eventually use this "@threaded" if it was there.



> Paul
>
>js
  -><-
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FWSH6ZXIQQ3S5JOTUEXDZULBRDFIJSEB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Steve Dower]

On 5/10/2022 4:33 PM, Christian Heimes wrote:

On 10/05/2022 13.18, Victor Stinner wrote:

test_peg_generator and test_cppext build C extensions with distutils.
Until these tests are modified to use something else, we still need to
keep distutils. So I propose to rename it to _distutils to remove it
from the stdlib. Maybe these tests can use ensurepip to install
setuptools which provides distutils.

There is also the c-analyzer tool which uses distutils to run a C 
preprocessor.


We can easily take care of test_cppext and add the build step to 
Makefile. How does test_peg_generator depend on distutils? I don't see 
an import of distutils in the directory.


I would prefer to fix and remove all imports of distutils before we 
resort to renaming the package. Please give us time until alpha 1. There 
is no need to rush it *now*.


I agree. The internal tools that use it are all in our Tools directory, 
so we can move distutils there and explicitly add an import path to 
locate it. No need to keep it in the stdlib (Lib/) at all.


Migrating to Makefile builds is probably better long-term, but not as 
important as moving distutils out from the stdlib so that setuptools can 
rely on their copy being the "main" one.


Cheers,
Steve

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LGF4BJMN3H7L6QFTZTDBMOA2GPZQFHC6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python grammar test cases

[Pablo Galindo Salgado]

That's used as a literal '/' to make positional-only arguments work. For
example:

lambda x, y, /, z: x+y+z

On Tue, 10 May 2022 at 17:11, Venkat Ramakrishnan <
venkat.archit...@gmail.com> wrote:

> I am looking at:
> https://docs.python.org/3/reference/grammar.html
>
> in which the following definition is found:
>
> lambda_slash_no_default:
> | lambda_param_no_default+ '/' ','
> | lambda_param_no_default+ '/' &':'
>
> Can someone tell me how '/' is being used here in the syntax?
>
> Thanks,
> Venkat.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/WNQVENTVABM5VIVUMEPR4WTOV7GMHFXS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OXIMIVB5AME7IBP6MBIGFVGH3Y5JMZKB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Starting a new thread

[Paul Moore]

On Tue, 10 May 2022 at 16:31, Barney Stratford
 wrote:
>
> Hello all.
>
> It occurs to me that creating threads in Python is more awkward than it needs 
> to be. Every time I want to start a new thread, I end up writing the same 
> thing over and over again:
>
> def target(*args, **kwds):
> ...
> t = threading.Thread(target = target, args = , kwargs= )
> t.start()
>
> This becomes especially repetitive when calling a target function that only 
> makes sense when run in a new thread, such as a timer.
>
> In my own code, I’ve taken to decorating functions that always run in new 
> threads. Then I can call the function using the usual function call syntax, 
> and have the new thread returned to me. With the decorator, the code reads 
> instead:
>
> @threaded
> def target(*args, **kwds):
> …
> t = target(*args, **kwds)
>
> This has a couple of advantages. I don’t have to import the threading module 
> all over my code. I can use the neater syntax of function calls. The 
> function’s definition makes it clear it’s returning a new thread since it’s 
> decorated. It gets the plumbing out of the way so I can concentrate more on 
> what my code does and less in how it does it.
>
> It feels like the right place for this decorator is the standard library, so 
> I’ve created PR #91878 for it. @rhettinger suggests that this is a bit 
> premature, and that we should discuss it here first. Thoughts?

I like this. I'd probably use it if it existed. Your example isn't
particularly compelling, though - calling the function "target" makes
sense when it's the target of the thread, but not so much when it's
the thread itself. Could you give a motivating example based on real
code, rather than just the "this is what the syntax would look like"
as you did above? Ideally showing what it would look like with and
without the decorator?

Things I *think* i get, but I'd like to see clearly in an example, include:

1. Does t = target(...) start the thread? I think it does.
2. Is it possible to create daemon threads? @threaded(daemon=True)
seems plausible. I suspect your PR doesn't have this, but it wouldn't
be difficult to add. But I don't actually know if it's worth adding,
TBH.
3. Can you create multiple threads for the same function? I assume t1,
t2, t3 = target(arg1), target(arg2), target(arg3) would work.

I'm probably being dense here - I'm pretty sure your proposal actually
is as simple as it looks (which is a good thing!) and I'm just
over-complicating things in my head.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FVETRDHASCD4ZYJ5YQG77LTG7R6TLIZO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python grammar test cases

[Jelle Zijlstra]

El mar, 10 may 2022 a las 9:06, Venkat Ramakrishnan (<
venkat.archit...@gmail.com>) escribió:

> I am looking at:
> https://docs.python.org/3/reference/grammar.html
>
> in which the following definition is found:
>
> lambda_slash_no_default:
> | lambda_param_no_default+ '/' ','
> | lambda_param_no_default+ '/' &':'
>
> Can someone tell me how '/' is being used here in the syntax?
>

It just means the literal / character. It's the marker for positional-only
parameters from PEP 570.


>
> Thanks,
> Venkat.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/WNQVENTVABM5VIVUMEPR4WTOV7GMHFXS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZBE2YFAOB3D6TY5VL5IEP6QGBE5P65S7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python grammar test cases

[Venkat Ramakrishnan]

I am looking at:
https://docs.python.org/3/reference/grammar.html

in which the following definition is found:

lambda_slash_no_default:
| lambda_param_no_default+ '/' ',' 
| lambda_param_no_default+ '/' &':' 

Can someone tell me how '/' is being used here in the syntax?

Thanks,
Venkat.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WNQVENTVABM5VIVUMEPR4WTOV7GMHFXS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Victor Stinner]

On Tue, May 10, 2022 at 5:33 PM Christian Heimes  wrote:
> We can easily take care of test_cppext and add the build step to
> Makefile.

What matters in test_cppext is that using the Python C API in C++ does
not emit compiler warnings, so it uses -Werror. Since this test is
very new, I would prefer to not block the Python build just because of
that. Moreover, I'm not sure that I want to require a C++ compiler to
build CPython (written in C).

But yeah, something can be done in Makefile :-)

> How does test_peg_generator depend on distutils? I don't see
> an import of distutils in the directory.

See Tools/peg_generator/pegen/build.py file. The test builds C
extensions and then execute them.

> I would prefer to fix and remove all imports of distutils before we
> resort to renaming the package. Please give us time until alpha 1. There
> is no need to rush it *now*.

Sure, there is no need to rush it now. But I prefer to discuss the
change as soon as possible (in the 3.12 dev cycle), to make sure that
we are aware of all issues, and have more time to fix them.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L6ECTYWEHYITDBFVMY4HTYCVFXUD6W3S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Jeremy Kloth]

On Tue, May 10, 2022 at 9:34 AM Christian Heimes  wrote:
> We can easily take care of test_cppext and add the build step to
> Makefile. How does test_peg_generator depend on distutils? I don't see
> an import of distutils in the directory.

You need to look at the files in the Tools directory.  That is where
c-analyzer and peg_generator live.

The peg_generator tests depend on Tools/peg_generator/pegen/build.py
which uses distutils to build
multiple extensions to test the grammar generator.

-- 
Jeremy Kloth
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7WMU6N6XGHLSO2FFSTXEFZC6UDPMNOB3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Christian Heimes]

On 10/05/2022 13.18, Victor Stinner wrote:

On Tue, May 10, 2022 at 11:28 AM Christian Heimes  wrote:

Right now, Python still uses distutils internally for multiple use
cases. I propose to start with renaming the distutils package to
_distutils in the stdlib:

* https://github.com/python/cpython/issues/92584
* https://github.com/python/cpython/pull/92585


Erlend and I got most extension modules ported to autoconf and
Modules/Setup.stdlib. The remaining modules should be done in a couple
of weeks. I recommend that we do not rename distutils and instead remove
it entirely.


test_peg_generator and test_cppext build C extensions with distutils.
Until these tests are modified to use something else, we still need to
keep distutils. So I propose to rename it to _distutils to remove it
from the stdlib. Maybe these tests can use ensurepip to install
setuptools which provides distutils.

There is also the c-analyzer tool which uses distutils to run a C preprocessor.


We can easily take care of test_cppext and add the build step to 
Makefile. How does test_peg_generator depend on distutils? I don't see 
an import of distutils in the directory.


I would prefer to fix and remove all imports of distutils before we 
resort to renaming the package. Please give us time until alpha 1. There 
is no need to rush it *now*.


Christian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5FOZXCSYPFJGWVFI7S6QNUKKICNPU5AQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Starting a new thread

[Barney Stratford]

Hello all.

It occurs to me that creating threads in Python is more awkward than it needs 
to be. Every time I want to start a new thread, I end up writing the same thing 
over and over again:

def target(*args, **kwds):
...
t = threading.Thread(target = target, args = , kwargs= )
t.start()

This becomes especially repetitive when calling a target function that only 
makes sense when run in a new thread, such as a timer.

In my own code, I’ve taken to decorating functions that always run in new 
threads. Then I can call the function using the usual function call syntax, and 
have the new thread returned to me. With the decorator, the code reads instead:

@threaded
def target(*args, **kwds):
…
t = target(*args, **kwds)

This has a couple of advantages. I don’t have to import the threading module 
all over my code. I can use the neater syntax of function calls. The function’s 
definition makes it clear it’s returning a new thread since it’s decorated. It 
gets the plumbing out of the way so I can concentrate more on what my code does 
and less in how it does it.

It feels like the right place for this decorator is the standard library, so 
I’ve created PR #91878 for it. @rhettinger suggests that this is a bit 
premature, and that we should discuss it here first. Thoughts?

Cheers,
Barney.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z6W56AQHBAA7PJ3HAUK6YQNMOJO27V6Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python grammar test cases

[Pablo Galindo Salgado]

Could you share with us some of the context of what are you trying to
achieve? That way we can offer more specialized help :)

On Tue, 10 May 2022 at 15:50, Venkat Ramakrishnan <
venkat.archit...@gmail.com> wrote:

> Presuming I am looking at the right link?:
> https://github.com/python/cpython/tree/main/Lib/test
>
> I am currently focusing on Lambda function. I have a
> few test cases written. I have some questions too. Would
> be great if I could talk/chat to someone about these
> questions.
>
> Thanks.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LPIBWDUTRQZWQBU4RUEM6GTMA2XQIZMX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GBROHP7GTMWANA7UHDXTIVKI7M4MJEZI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python grammar test cases

[Venkat Ramakrishnan]

Presuming I am looking at the right link?:
https://github.com/python/cpython/tree/main/Lib/test

I am currently focusing on Lambda function. I have a
few test cases written. I have some questions too. Would
be great if I could talk/chat to someone about these
questions.

Thanks.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LPIBWDUTRQZWQBU4RUEM6GTMA2XQIZMX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python grammar test cases

[Pablo Galindo Salgado]

In addition, test_exceptions test a bunch of syntax errors and related
locations.

On Tue, 10 May 2022 at 06:41, Serhiy Storchaka  wrote:

> 10.05.22 08:10, Venkat Ramakrishnan пише:
> > I'm wondering if there's a repository of test cases that
> > test the Python grammar. Any help would be appreciated.
>
> See test_grammar and test_syntax. And there are some test files for
> specific grammar features, like test_string_literals, test_unpack_ex,
> test_genexps, test_fstring, etc.
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/NW2D3WMQOOFVF5WHJDEG5B4TCIY7UR5U/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NWOU4BMUJYYGEDBGO4AWEAAVGUJKIJP7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Victor Stinner]

On Tue, May 10, 2022 at 11:28 AM Christian Heimes  wrote:
> > Right now, Python still uses distutils internally for multiple use
> > cases. I propose to start with renaming the distutils package to
> > _distutils in the stdlib:
> >
> > * https://github.com/python/cpython/issues/92584
> > * https://github.com/python/cpython/pull/92585
>
> Erlend and I got most extension modules ported to autoconf and
> Modules/Setup.stdlib. The remaining modules should be done in a couple
> of weeks. I recommend that we do not rename distutils and instead remove
> it entirely.

test_peg_generator and test_cppext build C extensions with distutils.
Until these tests are modified to use something else, we still need to
keep distutils. So I propose to rename it to _distutils to remove it
from the stdlib. Maybe these tests can use ensurepip to install
setuptools which provides distutils.

There is also the c-analyzer tool which uses distutils to run a C preprocessor.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TFBMHY4JV3QJL5I7VB65G7VHVUMJD4GX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Christian Heimes]

On 10/05/2022 09.53, Victor Stinner wrote:

Hi,

Two years ago, PEP 632 "Deprecate distutils module" was accepted: the
distutils package was deprecated in Python 3.10 and scheduled for
removal in Python 3.12. Questions.

* Is the Python ecosystem ready for the distutils removal? How many
projects are expected to be broken by this removal?

* Is setuptools a drop-in replacement of distutils for most cases? Are
there tools and documentation explaining how to replace distutils with
setuptools?

* Is there a tool to migrate from setup.py (distutils) to
pyproject.toml (setuptools)? The dephell project can convert a
setup.py script to pyproject.toml using poetry as the build system.

* Can we simply suggest installing setuptools to users who still use
"import distutils"? setuptools now provides the a "distutils" package.

* Should we keep distutils in Python stdlib a few more Python releases
if the removal causes too many practical issues?



Setuptools now provides _distutils_hack meta importer that injects a 
facade distutils package. "from distutils import setup" will keep 
working with modern setuptools.




Right now, Python still uses distutils internally for multiple use
cases. I propose to start with renaming the distutils package to
_distutils in the stdlib:

* https://github.com/python/cpython/issues/92584
* https://github.com/python/cpython/pull/92585


Erlend and I got most extension modules ported to autoconf and 
Modules/Setup.stdlib. The remaining modules should be done in a couple 
of weeks. I recommend that we do not rename distutils and instead remove 
it entirely.


Christian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FYRFMWETDJXU3SRAOTZJIQRN6QDLAMXT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Michał Górny]

On Tue, 2022-05-10 at 09:53 +0200, Victor Stinner wrote:
> Hi,
> 
> Two years ago, PEP 632 "Deprecate distutils module" was accepted: the
> distutils package was deprecated in Python 3.10 and scheduled for
> removal in Python 3.12. Questions.
> 
> * Is the Python ecosystem ready for the distutils removal? How many
> projects are expected to be broken by this removal?

In Gentoo, I have decided to handle migration from `setup.py install` to
PEP517 backends and distutils replacement simultaneously.  That is,
existing packages that still use the legacy codepath have =stdlib
distutils forced, and =local distutils are used when they are updated to
use PEP517 build.  So far we're nearing updating 40% of Python packages
in Gentoo.

From my porting experience, I can't think of any major issue caused by
replacing stdlib distutils with the setuptools version.  IIRC the only
problem I recall was that the "local" version installs .egg-info
as a directory (like setuptools does) while "stdlib" version installs it
as a file.  This could mean trouble when the same version of the package
is rebuilt with the other distutils version but I don't know how that
impacts other package managers.

So far, much more trouble is caused by PEP517 backends and wheels,
though admittedly this is primarily a problem with setup.py allowing
very deep customization and people using it to do pretty much anything.

Therefore, from Gentoo's perspective, there should be no major problem
with removing distutils from stdlib in Python 3.12.  Admittedly,
distributions are in the more convenient position here since we can
easily patch packages should we find any breakage, while users of pypi
are generally stuck with the published versions.

-- 
Best regards,
Michał Górny

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VRPUICOUP56LFIN4UYXENRCCBYNCLJKU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migration plan for the distutils removal in Python 3.12

[Christian Heimes]

On 10/05/2022 09.53, Victor Stinner wrote:

Hi,

Two years ago, PEP 632 "Deprecate distutils module" was accepted: the
distutils package was deprecated in Python 3.10 and scheduled for
removal in Python 3.12. Questions.

* Is the Python ecosystem ready for the distutils removal? How many
projects are expected to be broken by this removal?

* Is setuptools a drop-in replacement of distutils for most cases? Are
there tools and documentation explaining how to replace distutils with
setuptools?

* Is there a tool to migrate from setup.py (distutils) to
pyproject.toml (setuptools)? The dephell project can convert a
setup.py script to pyproject.toml using poetry as the build system.

* Can we simply suggest installing setuptools to users who still use
"import distutils"? setuptools now provides the a "distutils" package.

* Should we keep distutils in Python stdlib a few more Python releases
if the removal causes too many practical issues?



Setuptools now provides _distutils_hack meta importer that injects a 
facade distutils package. "from distutils import setup" will keep 
working with modern setuptools.




Right now, Python still uses distutils internally for multiple use
cases. I propose to start with renaming the distutils package to
_distutils in the stdlib:

* https://github.com/python/cpython/issues/92584
* https://github.com/python/cpython/pull/92585


Erlend and I got most extension modules ported to autoconf and 
Modules/Setup.stdlib. The remaining modules should be done in a couple 
of weeks. I recommend that we do not rename distutils and instead remove 
it entirely.


Christian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7R5DKP5NZI5MVYQ7H44LFIGDHHHO3FOM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Migration plan for the distutils removal in Python 3.12

[Victor Stinner]

Hi,

Two years ago, PEP 632 "Deprecate distutils module" was accepted: the
distutils package was deprecated in Python 3.10 and scheduled for
removal in Python 3.12. Questions.

* Is the Python ecosystem ready for the distutils removal? How many
projects are expected to be broken by this removal?

* Is setuptools a drop-in replacement of distutils for most cases? Are
there tools and documentation explaining how to replace distutils with
setuptools?

* Is there a tool to migrate from setup.py (distutils) to
pyproject.toml (setuptools)? The dephell project can convert a
setup.py script to pyproject.toml using poetry as the build system.

* Can we simply suggest installing setuptools to users who still use
"import distutils"? setuptools now provides the a "distutils" package.

* Should we keep distutils in Python stdlib a few more Python releases
if the removal causes too many practical issues?


A code search in top 5000 PyPI projects (at 2022-01-26) using the
regex '(import|from) distutils' matchs 5,567 lines in 1,229 projects.
Some statistics:

* 851 projects (1,372 lines) import distutils in setup.py
* 233 projects (700 lines) use distutils.version
* 92 projects (205 lines) use distutils.util
* 1,018 lines are type annotations (.pyi files)

I failed to group the remaining 2,272 lines.

Converting 851 projects from setup.py with distutils to pyproject.toml
is going to take a few months if not years. Python 3.12 will likely be
released in October 2023 (in year and a half).


Since setuptools 60 (December 2021), setuptools installs a
distutils-precedence.pth file to override the stdlib module with its
local distutils copy by default. The SETUPTOOLS_USE_DISTUTILS=stdlib
environment variable can be used to explicitly request the stdlib
flavor. If I understood correct, distutils is now maintained in
setuptools, rather than in the Python stdlib.

Python "make install" installs an up to date setuptools and so
indirectly its distutils-precedence.pth file which makes the distutils
local copy of setuptools available. But many Linux distributions
splits setuptools and pip from the "python" package.


Right now, Python still uses distutils internally for multiple use
cases. I propose to start with renaming the distutils package to
_distutils in the stdlib:

* https://github.com/python/cpython/issues/92584
* https://github.com/python/cpython/pull/92585

Right now, the change is blocked by pip which still imports distutils
(even if it prefers sysconfig in practice on Python 3.10 and newer). I
reported the issue to pip, it should be easy to fix.

Once Python will no longer depend on _distutils, it will be possible
to fully remove it.


By the way, Fedora 35 now longer provides automatically setuptools
when building a Python package. A dependency to setuptools must now be
explicit, since they are projects which don't use setuptools nor
distutils, but other build systems.

https://fedoraproject.org/wiki/Changes/Reduce_dependencies_on_python3-setuptools

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XFZJ43VSIWCYNQ5ZYT64XF7PMH4YDUWC/
Code of Conduct: http://python.org/psf/codeofconduct/