Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Michael Kesper
Hi all,

Am 12.02.21 um 20:52 schrieb Stefano Rivera:
>  This package is a dependency package, which depends on the full
>  standard library of Python for Python developers. Including modules
>  used only at build-time, such as venv and distutils, and modules with
>  complex dependencies, such as tk and IDLE.

Tk and Idle would belong into a hypothetical python3-extra, I guess.
This is the first time they're mentioned here, at least as far as my 
searching skills show.

Bye
Michael



OpenPGP_signature
Description: OpenPGP digital signature


Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Stefano Rivera
Hi Thomas (2021.02.12_09:16:28_+)

I should have combined this reply with my previous one, but it didn't
thread there cleanly:

> I mostly agree to add a metapackage. I just don't agree with the choice
> of package name. It makes our user believe that Python isn't "full"
> without it, and they then may install it when they don't need it to
> consume whatever is packaged in Debian. Reality is different.

The reality is that you're conflating two statements there:
1. Python isn't "full" without it.
2. Users may believe that they need this to run things that are
   packaged in Debian.

I'd say 1 is true, but users believing 2 would be incorrect. As I said
in my other mail, I hope this can be dealt with in the description. And
there's certainly no harm in users making this mistake, beyond
installing unnecessary junk.

We split the Python standard library up into multiple binary packages,
for a variety of reasons, and when you install python3, you don't get it
all. Developers outside of Debian's sphere of influence won't appreciate
those splits and software they write won't be expecting them. We can
deal with this within Debian, but not when users install 3rd party
software directly.

While we may have the goal of packaging all useful software in Debian,
the reality is that this is only ever a goal, constantly unachievable.
We serve our users better when we make it easy to extend beyond Debian.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  +1 415 683 3272



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Stefano Rivera
Hi Thomas (2021.02.12_08:10:51_+)
> > From your arguments above, it doesn't sound like the python3-full solves
> > a problem you experience. So, I'm not sure why you'd be using it.
> 
> I don't think I would. And to me, Python is already "full"(y supported)
> without these. Though at least, adding "dev" in its name shows it's not
> for our users.

Yeah, I can see the argument for calling this python3-full-dev.

On the flip side, in debian packaging, -dev implies C headers, and this
is more than that.

I think this would also be dealt with in the package description, I'd
imagine something like:

 Python, the high-level, interactive object oriented language,
 includes an extensive class library with lots of goodies for
 network programming, system administration, sounds and graphics.
 .
 This package is a dependency package, which depends on the full
 standard library of Python for Python developers. Including modules
 used only at build-time, such as venv and distutils, and modules with
 complex dependencies, such as tk and IDLE.
 .
 This package depends on Debian's default python 3 version's full
 standard library (currently v3.9).

BTW, I noticed some other vague descriptions while I was writing this
example description. Fixed:
https://salsa.debian.org/cpython-team/python3-defaults/-/commit/d1430d009688b9affcfa3b95ffcba4dcdcaf33ff

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  +1 415 683 3272



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Jeremy Stanley
On 2021-02-12 01:11:07 +0100 (+0100), Thomas Goirand wrote:
[...]
> Please do not add distutils, venv and lib2to3 in this python3-full
> metapackage. IMO that's falling into a design that isn't Debian. This
> would probably be best in a "python3-dev-full" or something similar, as
> from the distribution perspective, we see them as developer use only.
> Don't confuse our users so that they install something they don't need.
[...]

I'm failing to see the distinction here. Who are the direct "users"
of the current python3 package if not developers (i.e. those who
would explicitly `apt install python3` on their systems if it
weren't already present)? Any "python3 users" who aren't developers
are getting the python3 package as a dependency of some Python-based
software they're using, they're not going out and installing the
python3 package on their own.

The proposal already indicated that no other packages should declare
dependencies on python3-full anyway, so its only consumers will be
people manually installing it... that is, developers of Python-based
software, or people wanting to run software which isn't packaged in
Debian (which you seem to consider synonymous with being a software
developer for some reason, but I'll go along with it for the sake of
argument).

So it seems like you're saying the people who manually install
python3 will be confused by the presence of python3-full and install
it instead, and accidentally get "software developer tools" when
they do so. But who else is specifically choosing to install a
Python interpreter if not people writing and running non-packaged
Python source?
-- 
Jeremy Stanley


signature.asc
Description: PGP signature


Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Matthias Klose
On 2/12/21 2:08 PM, Julien Palard wrote:
> Hi,
> 
> As far as I understand, the divergence between "Python upstream" and 
> Debian is:
> 
> - It looks like Debian target users consuming software, users just 
> install a package and it works, no venv needed obviously, it always just 
> work, it's fantastic. Users may not even care if the program is written 
> in Python or not at this point.
> 
> - Upstream Python is obviously composed by people writing in Python and 
> know many people who write some Python too: all in need of venv to work.

Yes, I think that's a good summary for the user's view. But keep in mind, that
when "upstream" is speaking about the "user" experience of Python on Debian,
they usually mean the developer experience.

> So while I do agree that a *normal user* does not need a venv as I 
> myself appreciate, as a user, how fantastic it is to be able to install 
> a software using Debian packages, I still think venv is a very usefull 
> tool that I'm very happy to have in the Python stdlib and use daily.

Debian itself uses venv in build tests and in autopkg tests. So we also need a
working venv, and we ship it.  If something is broken with setting up a venv, we
should fix it.

What is less seen by "Python upstream" is that recommended upstream practices
like CPython defaulting to site-packages by default until recently (sudo python3
setup.py install), or sudo pip install|upgrade do break the user's system.
Compare that with safe approaches like autoconf defaulting to /usr/local
installs.  Let me say that "Python upstream" is not always enthusiastic to
acknowledge these issues.

There also seems to be some level of frustration of "Python upstream" supporting
users using Python on Debian on upstream communication channels like #python on
Freenode.  Yes, I have heard the complaints, but usually things are not
forwarded to Debian.  Stefano and myself have recently joined #python, so let's
see if that can help.

Matthias



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Valentin Vidic
On Fri, Feb 12, 2021 at 05:59:12PM +0200, Andrius Merkys wrote:
> On 2021-02-12 16:18, Simon McVittie wrote:
> > python3-minimal ≤ python3-core ≤ python3 ≤ python3-full
> 
> +1. Exactly how I would understand these names.

I see your point too. I guess python3-full is the way to go then,
but maybe also leave it at that if we don't want to bloat python3
too much (python3 becomes python3-core and python3-full becomes
python3).

-- 
Valentin



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Andrius Merkys
On 2021-02-12 16:18, Simon McVittie wrote:
> We have a python3 package already. If I saw a python3 package and a
> python3-core package, I would expect that either they're the same thing,
> or python3-core is a smaller and less fully-featured version of python3.
> 
> Conversely, we already have a python3-minimal package, and I would expect
> python3-core to be larger and more fully-featured than python3-minimal
> (or maybe the same), because by definition if it's minimal then it's
> the least Python you can have. So:
> 
> python3-minimal ≤ python3-core ≤ python3 ≤ python3-full

+1. Exactly how I would understand these names.

Andrius



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Simon McVittie
On Fri, 12 Feb 2021 at 10:40:48 +0100, Valentin Vidic wrote:
> Perhaps python3-core would be more appropriate, and python3-full can be
> left for something even bigger.

We have a python3 package already. If I saw a python3 package and a
python3-core package, I would expect that either they're the same thing,
or python3-core is a smaller and less fully-featured version of python3.

Conversely, we already have a python3-minimal package, and I would expect
python3-core to be larger and more fully-featured than python3-minimal
(or maybe the same), because by definition if it's minimal then it's
the least Python you can have. So:

python3-minimal ≤ python3-core ≤ python3 ≤ python3-full

Changing the meaning of the python3 name is not an option right now,
because that would be a disruptive change, and we're already in the
Debian 11 freeze.

If we want to have a metapackage that is "larger" than our current python3,
then the only option that's really feasible for Debian 11 is for that
larger metapackage to have a new name that is chosen to imply "this is
larger than python3", like python3-full.

> collectd-core - statistics collection and monitoring daemon (core system)
> gnome-core - GNOME Desktop Environment -- essential components

collectd-core is smaller than collectd, gnome-core is smaller than gnome,
and so on.

smcv



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Julien Palard
Hi,

As far as I understand, the divergence between "Python upstream" and 
Debian is:

- It looks like Debian target users consuming software, users just 
install a package and it works, no venv needed obviously, it always just 
work, it's fantastic. Users may not even care if the program is written 
in Python or not at this point.

- Upstream Python is obviously composed by people writing in Python and 
know many people who write some Python too: all in need of venv to work.

> Also, it's a disservice to push our users into the direction of using
> venv which is very ugly way to use Python in a Debian system
I do agree, we could even replace Python with software in this sentence.

And I don't think we're pushing our users to always install things in 
venvs. Providing venv is not a big signal for Debian users to ask them 
to use it to install packages (if a signal at all).

With my "I do write things in Python that may run on non-Debian systems" 
hat, and "I teach Python" (most of them not using Debian) hat, a venv is 
helping me in many many ways, it's literally part of my daily routine:

- It allows me to pin a set of dependencies and sub-dependencies to an 
exact version (I do use pip-compile, from pip-tools), per project, that 
I can use in automatic tests (ensuring if tests passes today, they'll 
pass tomorrow), I can share this set with coworkers and future me ("if 
it works for me, it works for you"), note my coworkers may not use 
Debian at all.

- It allows me to easily replace a dependency with a modified one to 
test it, or make anyone else test it (for example [1]).

- It allows me to work on my Debian testing laptop on code aimed to work 
on Debian stable, or a completly different OS.

- It allows me to work on various projects with a different set of 
incompatible dependencies.

So while I do agree that a *normal user* does not need a venv as I 
myself appreciate, as a user, how fantastic it is to be able to install 
a software using Debian packages, I still think venv is a very usefull 
tool that I'm very happy to have in the Python stdlib and use daily.

(My current use of Python on Debian is: compile it myself, I need 
multiple versions of Python for test purposes anyway.)

[1]: https://github.com/PyCQA/pylint/pull/3890#issuecomment-727591305

P.S.: If you still feel I'm completly wrong to use venv and pip in my 
workflow, I'll be very happy to learn better ways.
-- 
[Julien Palard](https://mdk.fr)



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Christian Kastner
On 12.02.21 10:16, Thomas Goirand wrote:
> I mostly agree to add a metapackage. I just don't agree with the choice
> of package name. It makes our user believe that Python isn't "full"
> without it

I think you are reading waaay too much into just this name. The package
will also have a synopsis and a description. There's no reason for our
users to assume or believe anything at all; the facts will be right
there in front of them.

And there's no way past the synopsis with some generic name like
python3-full, or python3-minimal, or python3-dev, etc.

> Also, it's a disservice to push our users into the direction of using
> venv which is very ugly way to use Python in a Debian system, outside of
> just testing something.

How would merely having these packages installed push a user to do
anything with them?

Furthermore, I think "just testing something" is a major use case we
absolutely should support. Personally, it's the first step in my process
for packaging something for Debian, because some things are better kept
out, as I'm sure we've all made the experience.

And finally, let's face it: for the vast amount of users and even
upstreams(!), we wouldn't be pushing them towards venv, they are already
there. pip (and conda) are already the standard tools for getting Python
packages. I don't see how standing in the way of this will win us any
favors. On the contrary, I do see how this casts a unfavorable light on us.

Best,
Christian

PS: To be clear, personally, I vastly prefer Debian-packaged Python
software, and will package anything I use if it is not yet in the
archive, provided its quality is sufficiently high and there is
continued upstream support.



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Valentin Vidic
On Fri, Feb 12, 2021 at 12:37:39PM +0200, Jonathan Carter wrote:
> I saw some discussion about this before, and it does sound nice, but it
> would require change to a few thousand packages to handle such a
> transition, where adding python3-full doesn't really add work for anyone
> except maintaining another meta-package.

Not sure what you mean, AFAICT python3-core and python3-full don't exist
in Debian at the moment. I'm just suggesting that python3-core might be
a better name for this meta-package than python3-full.

-- 
Valentin



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Jonathan Carter
On 2021/02/12 11:40, Valentin Vidic wrote:
> Perhaps python3-core would be more appropriate, and python3-full can be
> left for something even bigger.

I saw some discussion about this before, and it does sound nice, but it
would require change to a few thousand packages to handle such a
transition, where adding python3-full doesn't really add work for anyone
except maintaining another meta-package.

-Jonathan



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Martin
On 2021-02-12 10:16, Thomas Goirand wrote:
> I mostly agree to add a metapackage. I just don't agree with the choice
> of package name. It makes our user believe that Python isn't "full"
> without it, and they then may install it when they don't need it to
> consume whatever is packaged in Debian. Reality is different.

I agree with your technical point of view. Friends don't let
friends use pip and I banned its use at work.

I don't care about having the package or not and how it were
named, as long as it remains easy to not have it.

Cheers



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Michael Kesper
Hi all,

Am 12.02.21 um 01:11 schrieb Thomas Goirand:
> Hi Elana,
> 
> Thanks for bringing this topic in the channel, and speaking with the
> Python Steering Council, plus Mathias and Stefano. That's very much
> appreciated.
> 
> On 2/11/21 7:12 PM, Elana Hashman wrote:
>> - When users install Python, it should be easy to install all required
>>   modules for what upstream considers core, including venv, distutils,
>>   and lib2to3.
> 
> I understand that upstream python guys probably think the way to consume
> python stuff is through venv, pip, and setuptools. I have a very
> different view on this, and probably I'm not alone.
> 
> We (Debian people) indeed prefer if our user can enjoy a packaged
> versions of things if they are available (and that's not specific to
> Python). In such a packaged environment, venv and distutils are useless,
> as the distribution is taking care of all what these tools would do
> without apt or dpkg. I do prefer my system to *not* have venv support,
> for example.

This seems to be in contrast to how Python gets used by many many people.
I think it would be nice if up to date packages of all Python software
would exist in Debian but that's clearly not feasible.
Using virtualenvs is expected virtually everywhere where Python is mentioned.

My 2 cents
Michael



OpenPGP_signature
Description: OpenPGP digital signature


Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Valentin Vidic
On Fri, Feb 12, 2021 at 11:08:14AM +0200, Tristan Seligmann wrote:
> I wanted to point out that this follows the pattern set by the many other
> *-full packages in Debian, such as ruby-full.

Perhaps python3-core would be more appropriate, and python3-full can be
left for something even bigger.

Some examples:

collectd-core - statistics collection and monitoring daemon (core system)
gnome-core - GNOME Desktop Environment -- essential components
nginx-core - nginx web/proxy server (standard version)
ros-core - Robot OS core metapackage
syslog-ng-core - Enhanced system logging daemon (core)
xserver-xorg-core - Xorg X server - core server

-- 
Valentin



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Tristan Seligmann
On Fri, 12 Feb 2021 at 02:43,  wrote:

> From your arguments above, it doesn't sound like the python3-full solves
> a problem you experience. So, I'm not sure why you'd be using it.
>
> If it doesn't include distutils, venv, lib2to3, etc. then it doesn't
> solve any problem we currently have, and we don't need it. The purpose
> is to provide a package that gives you the entire stdlib.
>

I wanted to point out that this follows the pattern set by the many other
*-full packages in Debian, such as ruby-full.


Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Thomas Goirand
Hi,

Looks like once more I've been not able to express myself clearly enough
in the first message. Hopefully, what's bellow contain *all* of my
thoughts, and that it brings value to this thread.

On 2/12/21 9:30 AM, Raphael Hertzog wrote:
> On Fri, 12 Feb 2021, Thomas Goirand wrote:
>> What I read from Elana, is that *upstream* think we have a problem. But
>> do we really have one? Or are we just being influenced by upstream who
>> is trying to impose a view we don't necessary share?
> 
> Or is it you that is trying to impose your view on those users?
> 
> Let's be clear, I understand what you are saying and I mostly agree
> with your view but Debian is about inclusiveness and about meeting
> the needs of as many people as possible and I believe that implementing
> this python3-full meta-package can only help towards this.

I mostly agree to add a metapackage. I just don't agree with the choice
of package name. It makes our user believe that Python isn't "full"
without it, and they then may install it when they don't need it to
consume whatever is packaged in Debian. Reality is different.

Also, it's a disservice to push our users into the direction of using
venv which is very ugly way to use Python in a Debian system, outside of
just testing something. You then end up with a variety of versions of
things pulled by pip, which are quickly unmanageable. That's why we do
package Python modules, and make sure they work well together
(sometimes, patching them to achieve this goal).

So, by all means, let's create a metapackage, and call it
"python3-addons-that-upstream-thinks-make-python-full" or anything you
like, but not just "python3-without-this-package-python-is-not-full",
misguiding our users to install venv and distutils which they don't need
if they are consuming only Python for things that we package already.

I'm also concerned that this is useful at all. If someone wants to use
venv, 2to3 or setuptools, I believe they will know how to fetch them...

Cheers,

Thomas Goirand (zigo)



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Raphael Hertzog
On Fri, 12 Feb 2021, Thomas Goirand wrote:
> What I read from Elana, is that *upstream* think we have a problem. But
> do we really have one? Or are we just being influenced by upstream who
> is trying to impose a view we don't necessary share?

Or is it you that is trying to impose your view on those users?

Let's be clear, I understand what you are saying and I mostly agree
with your view but Debian is about inclusiveness and about meeting
the needs of as many people as possible and I believe that implementing
this python3-full meta-package can only help towards this.

Cheers,
-- 
  ⢀⣴⠾⠻⢶⣦⠀   Raphaël Hertzog 
  ⣾⠁⢠⠒⠀⣿⡁
  ⢿⡄⠘⠷⠚⠋The Debian Handbook: https://debian-handbook.info/get/
  ⠈⠳⣄   Debian Long Term Support: https://deb.li/LTS



Re: upstream python concerns, python3-full package for bullseye

2021-02-12 Thread Thomas Goirand
On 2/12/21 1:42 AM, stefa...@debian.org wrote:
> Hi Thomas (2021.02.12_00:11:07_+)
>> So indeed, it's a good thing to *not* include distutils and venv by
>> default when someone installs python.
> 
> ...
> 
>>> I propose that we add a python3-full* metapackage for
>>>   bullseye. (*We can use a different name, but it must be a name not
>>>   currently in use.)
>>
>> Please do not add distutils, venv and lib2to3 in this python3-full
>> metapackage. IMO that's falling into a design that isn't Debian. This
>> would probably be best in a "python3-dev-full" or something similar, as
>> from the distribution perspective, we see them as developer use only.
>> Don't confuse our users so that they install something they don't need.
> 
> From your arguments above, it doesn't sound like the python3-full solves
> a problem you experience. So, I'm not sure why you'd be using it.

I don't think I would. And to me, Python is already "full"(y supported)
without these. Though at least, adding "dev" in its name shows it's not
for our users.

> If it doesn't include distutils, venv, lib2to3, etc. then it doesn't
> solve any problem we currently have, and we don't need it. The purpose
> is to provide a package that gives you the entire stdlib.
> 
> SR

What I read from Elana, is that *upstream* think we have a problem. But
do we really have one? Or are we just being influenced by upstream who
is trying to impose a view we don't necessary share?

Cheers,

Thomas Goirand (zigo)