[Distutils] Re: PEP-582 concerns

[Alex Walters]

> -Original Message-
> From: Chad Smith 
> Sent: Friday, March 1, 2019 1:33 PM
> To: Alex Walters 
> Subject: Re: [Distutils] Re: PEP-582 concerns
> 
> I was not aware that discuss.python.org <http://discuss.python.org>  was
> being used now. I started a thread there on PEP-582:
> https://discuss.python.org/t/pep-582-python-local-packages-directory/963.
> Would you mind reposting your response there?
> 

Yes, I actually would greatly mind posting on discuss.python.org, that site is 
itself contentious.  Discussion mediums have been discussed to death, I won't 
rehash it here (nor do I want to open that issue back up), but I posted on the 
list for a reason.

> On Fri, Mar 1, 2019 at 2:30 AM Alex Walters  <mailto:tritium-l...@sdamon.com> > wrote:
> 
> 
>   I kind of feel that "third party tool can/will use this feature" is
>   orthogonal to "how the interpreter behaves out of the box" - unless I
>   misunderstand and you are suggesting python grow support for
> launching
>   entrypoints from the python executable.
> 
>   > -Original Message-
>   > From: chadsmit...@gmail.com <mailto:chadsmit...@gmail.com>
> mailto:chadsmit...@gmail.com> >
>   > Sent: Thursday, February 28, 2019 12:42 PM
>   > To: distutils-sig@python.org <mailto:distutils-sig@python.org>
>   > Subject: [Distutils] Re: PEP-582 concerns
>   >
>   > Running entrypoints/executables in the bin directory has been
> solved by
>   the
>   > node community with npx
> (https://www.npmjs.com/package/npx):
>   >
>   > > Executes  either from a local node_modules/.bin, or
> from a
>   > central cache, installing any packages needed in order for
>  to
>   > run.
>   >
>   > I built similuar support into pipx as well, so `pipx run ENTRYPOINT`
> will
>   search
>   > the appropriate `__pypackages__` path for the bin dir and the
> entrypoint.
>   > This is not the only way to solve it, but it seems to be working for
> the
>   node.
>   >
>   > Indeed, massively popular projects like `create-react-app`
>   > (https://github.com/facebook/create-react-app#quick-overview)
> include
>   > npx as their sole installation instructions.
>   > --
>   > Distutils-SIG mailing list -- distutils-sig@python.org 
> <mailto:distutils-
> s...@python.org>
>   > To unsubscribe send an email to distutils-sig-le...@python.org
> <mailto:distutils-sig-le...@python.org>
>   > https://mail.python.org/mailman3/lists/distutils-sig.python.org/
>   > Message archived at https://mail.python.org/archives/list/distutils-
>   >
> s...@python.org/message/NAXGS7PZ7TELPEW2ZNNHW7N3RQDCPZSN/
> <http://s...@python.org/message/NAXGS7PZ7TELPEW2ZNNHW7N3RQDCPZ
> SN/>
> 
> 

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/4BPUW4QTGA322L44LJXYU7MTZB4OL3ZK/

[Distutils] Re: PEP-582 concerns

[Alex Walters]

I kind of feel that "third party tool can/will use this feature" is
orthogonal to "how the interpreter behaves out of the box" - unless I
misunderstand and you are suggesting python grow support for launching
entrypoints from the python executable.

> -Original Message-
> From: chadsmit...@gmail.com 
> Sent: Thursday, February 28, 2019 12:42 PM
> To: distutils-sig@python.org
> Subject: [Distutils] Re: PEP-582 concerns
> 
> Running entrypoints/executables in the bin directory has been solved by
the
> node community with npx (https://www.npmjs.com/package/npx):
> 
> > Executes  either from a local node_modules/.bin, or from a
> central cache, installing any packages needed in order for  to
> run.
> 
> I built similuar support into pipx as well, so `pipx run ENTRYPOINT` will
search
> the appropriate `__pypackages__` path for the bin dir and the entrypoint.
> This is not the only way to solve it, but it seems to be working for the
node.
> 
> Indeed, massively popular projects like `create-react-app`
> (https://github.com/facebook/create-react-app#quick-overview) include
> npx as their sole installation instructions.
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at https://mail.python.org/archives/list/distutils-
> s...@python.org/message/NAXGS7PZ7TELPEW2ZNNHW7N3RQDCPZSN/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/GXNH6WWD5U5GQZKK2ZPKHJUXKN3XLAQD/

[Distutils] PEP-582 concerns

[Alex Walters]

I have 2 main concerns about PEP 582 that might just be me misunderstanding
the pep.

My first concern is the use of CWD, and prepending ./_pypackages_ for
scripts.  For example, if you were in a directory with a _pypackages_
subdirectory, and had installed the module "super.important.module".  My
understanding is that any scripts you run will have "super.important.module"
available to it before the system site-packages directory.  Say you also run
"/usr/bin/an_apt-ly_named_python_script" that uses "super.important.module"
(and there is no _pypackages_ subdirectory in /usr/bin).  You would be
shadowing "super.important.module".

In this case, this adds no more version isolation than "pip install --user",
and adds to the confoundment factor for a new user.  If this is a
misunderstanding of the pep (which it very well might be!), then ignore that
concern.  If it's not a misunderstanding, I think that should be emphasized
in the docs, and perhaps the pep.

My second concern is a little more... political.

This pep does not attempt to cover all the use-cases of virtualenvs - which
is understandable.  However, this also means that we have to teach new users
*both* right away in order to get them up and running, and teach them the
complexities of both, and when to use one over the other.  Instead of making
it easier for the new user, this pep makes it harder.  This also couldn't
have come at a worse time with the growing use of pipenv which provides a
fully third way of thinking about application dependencies (yes, pipenv uses
virtualenvs under the hood, but it is a functionally different theory of
operation from a user standpoint compared to traditional pip/virtualenv or
this pep).

Is it really a good idea to do this pep at this time?

In a vacuum, I like this pep.  Aside from the (possible) issue of unexpected
shadowing, it's clean and straight forward.  It's easy to teach.  But it
doesn't exist in a vacuum, and we have to teach the methods it is intended
to simplify anyways, and it exists in competition with other solutions.

I am not a professional teacher; I don't run python training courses.  I do,
however, volunteer quite a bit of time on the freenode channel.  I get that
the audience there is self-selecting to those who want to donate their time,
and those who are having a problem (sometimes, those are the same people).
This is the kind of thing that generates a lot of confusion and frustration
to the new users I interact with there.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/SFMFKTQVKTONCYNN7UEKLFAQ2VRKXEHK/

[Distutils] Re: unifying package formats

[Alex Walters]

> -Original Message-
> From: Antonio Cavallo 
> Sent: Sunday, January 27, 2019 5:05 PM
> To: Alex Walters 
> Subject: Re: [Distutils] unifying package formats
> 
> I don't think they serve a much different scope after al:l both uncompress
> files under a filesystem. There are difference in the way .so/.dll are
> maintained/built and some ancillary metadata (and you can see commonality
> in between conda/pip as well) but that's about it.
> 

Then, that puts 7-zip and the PYPA in the same scope.  There is more to project 
and infrastructure scope than the technical scope of what the code is doing.

No, pip's scope is strictly managing python packages, conda's scope is much 
wider than that.

> 
> On Sat, 26 Jan 2019 at 18:21, Alex Walters  <mailto:tritium-l...@sdamon.com> > wrote:
> 
> 
>   > -Original Message-
>   > From: Antonio Cavallo  <mailto:antonio.cavallo...@gmail.com> >
>   > Sent: Saturday, January 26, 2019 6:01 PM
>   > To: distutils-sig@python.org <mailto:distutils-sig@python.org>
>   > Subject: [Distutils] unifying package formats
>   >
>   > Hi,
>   > is there any initiative to share a common package format between
> conda and
>   > pip? They look very similar.
>   >
>   > Thanks
> 
>   They look similar, but they have very different scopes.  Pip is for the
> python environment only.  Conda acts more like a python focused, system-
> isolated APT.Unifying packaging for the general python ecosystem (wheels
> hosted on pypi installed with pip) with conda (full stack dependency
> management, including large quantities of foreign code) is likely to be
> impossible, and even if possible, impractical.
> 
> 

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/6KVBS34UHFN7LRYGI5GS2VRPXKJ3DXU5/

[Distutils] Re: unifying package formats

[Alex Walters]

> -Original Message-
> From: Antonio Cavallo 
> Sent: Saturday, January 26, 2019 6:01 PM
> To: distutils-sig@python.org
> Subject: [Distutils] unifying package formats
> 
> Hi,
> is there any initiative to share a common package format between conda and
> pip? They look very similar.
> 
> Thanks

They look similar, but they have very different scopes.  Pip is for the python 
environment only.  Conda acts more like a python focused, system-isolated APT.  
  Unifying packaging for the general python ecosystem (wheels hosted on pypi 
installed with pip) with conda (full stack dependency management, including 
large quantities of foreign code) is likely to be impossible, and even if 
possible, impractical.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/KMKL7KYUEAPU5CFFP6G7NHQFLWTPQQXA/

[Distutils] Re: manylinux1 guidelines for zlib?

[Alex Walters]

Since zlib is a dependency of python, the assumption has to be that it is
already present.

I think you can look at the python docs for which versions of zlib you
should be dynamically linking against.  The docs state "There are known
incompatibilities between the Python module and versions of the zlib library
earlier than 1.1.3; 1.1.3 has a security vulnerability, so we recommend
using 1.1.4 or later."  So if I were linking against zlib, I would link
against 1.1.4.  However, I don't know if zlib has a stable abi... so take
that with a pinch of salt.

> -Original Message-
> From: Antoine Pitrou 
> Sent: Monday, September 3, 2018 6:51 AM
> To: distutils-sig@python.org
> Subject: [Distutils] manylinux1 guidelines for zlib?
> 
> Hello,
> 
> Surprisingly, the manylinux1 spec doesn't seem to include the zlib in the
list
> of known-to-be-available libraries (are there GNU/Linux systems out there
> without a zlib installed?).
> 
> Since I'm assuming several packages already had a need for that, is there
a
> recommended way to link in the zlib as part of a manylinux1 wheel? Would
> you recommend static linking with a private version, or dynamic linking?
> 
> Regards
> 
> Antoine.
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at https://mail.python.org/mm3/archives/list/distutils-
> s...@python.org/message/ZZG6GL3XTBLBJXSITYHEXMFKN43EREB7/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/HLX65TU6U7N3JTUU4YRLBQNSFKZTG52X/

[Distutils] Re: Packaging Advice for EFF's Certbot

[Alex Walters]

> -Original Message-
> From: Nathaniel Smith 
> Sent: Monday, July 23, 2018 10:31 PM
> To: Brad Warren 
> Cc: distutils-sig 
> Subject: [Distutils] Re: Packaging Advice for EFF's Certbot
> 
> 
> Reading the problem description at the top of your document, my first
> thought was that this seemed like exactly what conda is designed for:
> a "real" package manager designed to be portable across platforms and
> work in isolation from the system package manager.
> 
> You should also look at Nix and Guix, which are the other systems I
> see people mention in this space.
> 
> I'm not an expert in conda at all -- if you want to go down this path
> you should probably have a chat with Anaconda and also conda-forge
> (which is a very impressive community-run packaging and build effort).
> I have some idea about some of the questions you raised though :-):
> 

As a user of certbot, docker, conda, nix, and guix are non-starters.  I'm not 
depending on those tools for my production server (and while docker may be a 
dependency for some people, that is hardly universal).  Adding heavyweight 
technical dependencies are problematic if your goal is to get everyone using 
your software.  You're better off with cx_freeze or pyinstaller binaries 
downloaded from a website or a PPA-like-system to add to system package 
managers, which are not perfect solutions either.

> > How will separately distributed plugins work?
> 
> Conda has a system they call "channels" to let third-parties
> distribute extra conda packages, and existing systems for
> using/hosting/maintaining them. (Sort of similar to Ubuntu PPAs, if
> you know those.)
> 
> > How should the user invoke Certbot (and maybe conda) if we don’t want
> to put another Python in the user’s PATH to avoid breaking other Python
> code on their system?
> 
> A little shell script to set the PATH and then exec the right binary
> should work. Or just setting up the #! line in your main script
> properly.
> 
> > What should we do for systems not using 32-bit or 64-bit x86?
> 
> I know the conda folks have some stuff for ARM, though I don't know the
> details.
> 
> > If we didn’t want to trust any binaries built by someone else or proprietary
> code, how much work would that be?
> 
> This is where you want to talk to conda-forge – one of the original
> motivations was to make a community alternative to Anaconda Inc's
> official packages (which were not originally open-source, and do still
> contain proprietary code). Nowadays everyone's on better terms, but
> having once rebuilt the whole distro from the ground up means they can
> probably share some experience with you here.
> 
> -n
> 
> --
> Nathaniel J. Smith -- https://vorpus.org
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at https://mail.python.org/mm3/archives/list/distutils-
> s...@python.org/message/SFKA346UB3UQHZWNKONC63CT5VSKUTHB/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/F2MIDHEX2BX4TTQLWPIADPSHMKYVRQ6T/

[Distutils] Re: sudo pip install: install pip files into /usr/local on Linux?

[Alex Walters]

 

 

From: Wes Turner <wes.tur...@gmail.com> 
Sent: Wednesday, May 23, 2018 4:37 PM
To: Alex Walters <tritium-l...@sdamon.com>
Cc: Victor Stinner <vstin...@redhat.com>; distutils-sig@python.org
Subject: Re: [Distutils] sudo pip install: install pip files into /usr/local on 
Linux?

 



On Wednesday, May 23, 2018, Alex Walters <tritium-l...@sdamon.com 
<mailto:tritium-l...@sdamon.com> > wrote:

I think the obvious, if socially hard solution, is to make pip panic when it
sees its being run as root (without, perhaps, a flag to tell pip "No, I
really mean it, run as root"), and default to --user. 

 

Maybe default to --user if not in a VIRTUAL_ENV (or a conda env).

 

 

Yes, of course.  The assumption I made is that the behavior being discussed is 
when pip is not being run in a virtualenv.  Pip should default to the most 
isolated environment available to it (venv, followed by user)

 

 

 It is not a good idea
to install packages system wide with pip for reasons more than just
clobbering apt/dnf installed packages.  I still think the best idea for
getting a python program to run system wide is either A: symlink from a
inside a venv  into something on $PATH, B: just set a shebang to the python
in a venv, or C: bundle your application into a .deb or .rpm and use the
system package manager to install it.

 

Pip shouldn't be run as root.

That being said, installing files as user:group root is less insecure than 
making them user-writeable; so long as nothing is setuid is root.

 

 

Until there is something stopping me from uploading an sdist with malicious 
code in setup.py to pypi, running pip as a root is a bad idea even without 
user-writeable or setuid.

 

 

In most cases, it's ill-advised to allow apps to overwrite themselves and/or 
their dependencies.

 


> -Original Message-
> From: Victor Stinner <vstin...@redhat.com <mailto:vstin...@redhat.com> >
> Sent: Wednesday, May 23, 2018 11:22 AM
> To: distutils-sig@python.org <mailto:distutils-sig@python.org> 
> Subject: [Distutils] sudo pip install: install pip files into /usr/local
on Linux?
> 
> Hi,
> 
> pip is currently not well integrated on Linux: it conflicts  with the
> system package manager like apt or rpm. When pip writes files  into
> /usr, it can replace files written by the system package manager  and
> so create different kind of issues. For example, if you check the
> system integry, you will likely see that some Python files have been
> modified.
> 
> I would like to open a discussion to see how each Linux vendor handles
> the issue, and see if a common solution can be designed.
> 
> Debian uses /usr for apt-get install and /usr/local for distutils and
> "sudo pip".
> 
> Fedora  decided to change pip to install files into /usr/local by
> default,  instead of /usr, so "sudo pip install" doesn't replace files
> installed  by dnf (Fedora package manager):
> https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe
> 
> It  gives you 3 main places to install Python code: /usr (managed by
> dnf),  /usr/local (managed by sudo pip), $HOME/.local (managed by pip
> --user).
> 
> Would it make sense to make the Fedora/Debian change upstream? At
> least, give an opt-in option for Linux vendors to use /usr/local?
> 
> I  propose to make the change upstream because there are still issues,
> and  I don't want to be alone to have to fix them :-) It should be
> easier if  we agree on a filesystem layout and an implementation, so
> we can  collaborate on issues!
> 
> 
> Issues with the current Fedora implementation:
> 
> (1)  When Python is embedded in an application, there is an issue with
> the  current heuristic to decide if /usr/local should be added to
> sys.path:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1532287
> 
> (2)  On Fedora, "sudo pip install -U" currently removes old code from
> /usr  and install the new one in /usr/local. We should leave /usr
> unchanged,  since only dnf should touch /usr.
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1550368#c24
> 
> The implementation is made of a single patch on the Python site module:
> 
> https://src.fedoraproject.org/rpms/python3/blob/master/f/00251-change-
> user-install-location.patch
> 
> --
> 
> There are two issues related to the "sudo pip" change, but they
> already exist when pip is installed in $HOME/.local:
> 
> (3) Priority issue between PATH and PYTHONPATH directories.
> 
> When  the user runs "pip", the pip binary may come from /usr,
> /usr/local or  $HOME/.local/bin, but the Python pip module ("import
> pip") may come from  a different path. Which binary and which module
> should be used?
> 
> Obvisouly, users can replace these tw

[Distutils] Re: sudo pip install: install pip files into /usr/local on Linux?

[Alex Walters]

I think the obvious, if socially hard solution, is to make pip panic when it
sees its being run as root (without, perhaps, a flag to tell pip "No, I
really mean it, run as root"), and default to --user.  It is not a good idea
to install packages system wide with pip for reasons more than just
clobbering apt/dnf installed packages.  I still think the best idea for
getting a python program to run system wide is either A: symlink from a
inside a venv  into something on $PATH, B: just set a shebang to the python
in a venv, or C: bundle your application into a .deb or .rpm and use the
system package manager to install it.

> -Original Message-
> From: Victor Stinner 
> Sent: Wednesday, May 23, 2018 11:22 AM
> To: distutils-sig@python.org
> Subject: [Distutils] sudo pip install: install pip files into /usr/local
on Linux?
> 
> Hi,
> 
> pip is currently not well integrated on Linux: it conflicts  with the
> system package manager like apt or rpm. When pip writes files  into
> /usr, it can replace files written by the system package manager  and
> so create different kind of issues. For example, if you check the
> system integry, you will likely see that some Python files have been
> modified.
> 
> I would like to open a discussion to see how each Linux vendor handles
> the issue, and see if a common solution can be designed.
> 
> Debian uses /usr for apt-get install and /usr/local for distutils and
> "sudo pip".
> 
> Fedora  decided to change pip to install files into /usr/local by
> default,  instead of /usr, so "sudo pip install" doesn't replace files
> installed  by dnf (Fedora package manager):
> https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe
> 
> It  gives you 3 main places to install Python code: /usr (managed by
> dnf),  /usr/local (managed by sudo pip), $HOME/.local (managed by pip
> --user).
> 
> Would it make sense to make the Fedora/Debian change upstream? At
> least, give an opt-in option for Linux vendors to use /usr/local?
> 
> I  propose to make the change upstream because there are still issues,
> and  I don't want to be alone to have to fix them :-) It should be
> easier if  we agree on a filesystem layout and an implementation, so
> we can  collaborate on issues!
> 
> 
> Issues with the current Fedora implementation:
> 
> (1)  When Python is embedded in an application, there is an issue with
> the  current heuristic to decide if /usr/local should be added to
> sys.path:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1532287
> 
> (2)  On Fedora, "sudo pip install -U" currently removes old code from
> /usr  and install the new one in /usr/local. We should leave /usr
> unchanged,  since only dnf should touch /usr.
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1550368#c24
> 
> The implementation is made of a single patch on the Python site module:
> 
> https://src.fedoraproject.org/rpms/python3/blob/master/f/00251-change-
> user-install-location.patch
> 
> --
> 
> There are two issues related to the "sudo pip" change, but they
> already exist when pip is installed in $HOME/.local:
> 
> (3) Priority issue between PATH and PYTHONPATH directories.
> 
> When  the user runs "pip", the pip binary may come from /usr,
> /usr/local or  $HOME/.local/bin, but the Python pip module ("import
> pip") may come from  a different path. Which binary and which module
> should be used?
> 
> Obvisouly, users can replace these two environment variables...
> 
> (4)  Related to (3). Running "pip" may run pip binary of one pip
> version,  but pick the "pip" Python module of another pip version.
> 
> For example, pip9 binary from /usr/bin/pip, but pip10 module from
> /usr/local.
> 
> 
> Fedora works around issue (4) with a downstream patch on pip:
> 
> https://src.fedoraproject.org/rpms/python-pip/blob/master/f/pip9-allow-
> pip10-import.patch
> 
> --
> 
> I  don't well well how Linux distributions handle the issue with "sudo
>  pip". So don't hesitate to correct me if I'm wrong :-) My goal is
> just  to start a discussion about a common "upstream" solution.
> 
> Victor
> --
> Distutils-SIG mailing list
> distutils-sig@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at https://mail.python.org/mm3/archives/list/distutils-
> s...@python.org/message/OLGLHTSHLEPLHUTTVNU6L5QFTMNFIB6Z/
--
Distutils-SIG mailing list
distutils-sig@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/I2F6SW6MVFTJFQNTQCB2HTZC7QOT7GBE/

[Distutils] Re: org-mode README file formats

[Alex Walters]

If warehouse punted to a tool like pandoc already, I would see no problem with 
adding support for everything that pandoc supported (can you imagine – 
reademe.docx?), but just skimming the requirements file, they are pulling in 
docutils and mistune, which means they have written warehouse to support rst 
and markdown, and it is not non-trivial (or less non-trivial) to support other 
markup languages.

 

Markdown won in the wider world.  Supporting that is obvious.

 

RestructuredText has inertia in the python community.  Supporting that is 
obvious.

 

Org-mode?  Asciidoc?  Txt2tags? Textile? Mediawiki?  BB Code?  Where do you 
draw the line?  I think the two languages the tooling currently support are 
sufficient for now.

 

From: Brett Cannon  
Sent: Tuesday, May 8, 2018 2:04 PM
To: Diane Trout 
Cc: distutils-sig@python.org
Subject: [Distutils] Re: org-mode README file formats

 

 

On Tue, 8 May 2018 at 11:00 Diane Trout  
> wrote:

On Tue, 2018-05-08 at 16:44 +, Brett Cannon wrote:

 

On Mon, 7 May 2018 at 16:54 Diane Trout  
> wrote:

Hi,

I was building a package where I had a README.org file, which
setuptools couldn't find.

It listed .md as a valid format, so I was wondering if org-mode was
sufficiently plain text to be added to the list of accepted README file
formats?

 

What's org-mode? Sounds like an Emacs thing based on what I have heard Emacs 
people say. :)

If it is an Emacs thing then I would vote "no" since that's very 
editor-specific and I suspect trying to support every plaintext file format is 
never-ending.

 

Well yes, it's certainly most powerful, flexible, and feature complete with 
emacs, but there is work on org-mode support in vim and sublime. Additionally 
org-mode syntax is supported by GitHub, GitLab, and pandoc.

https://en.wikipedia.org/wiki/Org-mode#Integration

 

There's also an argument that org-mode is a good lightweight markup language in 
and of itself.

http://karl-voit.at/2017/09/23/orgmode-as-markup-only/

 

Since it is breaking out of its source community I thought it at least worth 
asking about. Out of curiosity, do you have a feeling for how popular a 
lightweight markup language needs to be before it gets added to the list of 
setuptools recognized formats?

 

When you can say "has broken out" instead of "is breaking out". ;)

Basically if the majority of people at a Python conference won't know the 
format then the burden on the volunteers to support it isn't probably worth it 
IMO. But since I'm not maintaining Warehouse it isn't my call, just my opinion.

--
Distutils-SIG mailing list
distutils-sig@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/D46BMY5L22WIYJXKUYBCTYGJV37RHF2O/

Re: [Distutils] [Python-ideas] Pypi private repo's

[Alex Walters]

I am fairly sure if you give the PyPA that suggestion, they will just deflate 
at the thought of the workload.  Besides, we already offer private repos for 
free, several ways ranging from devpi to python -m SimpleHTTPServer in a 
specially created directory.

 

 

From: Python-ideas  On 
Behalf Of Nick Humrich
Sent: Wednesday, April 4, 2018 12:26 PM
To: python-id...@python.org
Subject: [Python-ideas] Pypi private repo's

 

I am sure this has been discussed before, and this might not even be the best 
place for this discussion, but I just wanted to make sure this has been thought 
about.

What if pypi.org   supported private repos at a cost, similar 
to npm?

This would be able to help support the cost of pypi, and hopefully make it 
better/more reliable, thus in turn improving the python community.

If this discussion should happen somewhere else, let me know.

Nick

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Renaming this list to python-packaging

[Alex Walters]

There is a technical limitation that we would not be renaming the list as much 
as closing this one, and opening a new one.  That creates any number of social 
problems, for little to no benefit – the name is slightly more relevant at the 
cost of breaking everyone’s subscriptions, and the mail filters of anyone who 
filters.  There are any number of glitches and headaches for what end?

 

It sounds nice but, it causes more real pain than the theoretical pain the 
current name causes.

 

From: Distutils-SIG  
On Behalf Of Pradyun Gedam
Sent: Monday, March 19, 2018 1:30 PM
To: DistUtils Mailing List"" 
Subject: [Distutils] Renaming this list to python-packaging

 

Hey all!

 

Given the direction we're moving with PEP 517/518, I propose that we rename 
this list to something like python-packaging or something similar, to make it a 
more relavantly named place for having discussions related to Python Packaging 
in general.

 

I'm curious what others think about this. I'm definitely open to thoughts and 
other ideas. :)

 

Cheers,

Pradyun

 

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] draft PEP: manylinux2

[Alex Walters]

Just out of curiosity, I did a little experiment.  I explained this thread
to my mother.  My mother is a wonderful woman, but she wouldn't know a byte
from a bite.  I explained it as follows:

"There is a tool that can make software run on a lot of different computers,
but only if you build it for an ancient computer.  The tool is a little
complicated - you have to learn how to get it and to use it with any
success.  The people who make it are considering changing the way they name
it.  The new naming scheme is the bare minimum year the computer running the
code can be from.  The old naming scheme is just a sequence - 1, 2, 3.
Would you be confused by the new naming scheme? Do you think people using it
would be confused?"

Her response, which I will paraphrase because my lovely mother likes four
letter words, "If it's complicated to use already, then changing the name
isn't any more confusing."

Not exactly scientific, but based on that, I don't think CalVer will be that
confusing to library developers.

> -Original Message-
> From: Distutils-SIG [mailto:distutils-sig-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Nick Coghlan
> Sent: Sunday, February 11, 2018 7:15 AM
> To: Mark Williams 
> Cc: Geoffrey Thomas ; DistUtils mailing list
> ; Mark Williams 
> Subject: Re: [Distutils] draft PEP: manylinux2
> 
> On 10 February 2018 at 16:03, Mark Williams 
> wrote:
> > On Tue, Feb 06, 2018 at 05:55:36PM +1000, Nick Coghlan wrote:
> >> By contrast, year-based CalVer maintains distro-neutrality, while also
> >> giving a good sense of the maximum age of compatible target platforms.
> >> (e.g. given "manylinux2010", it's a pretty safe guess that Ubuntu
> >> 12.04, 14.04 and 16.04 are all expected to be compatible, while that
> >> isn't as clear given "manylinux2" or "manylinux6")
> >
> > I'm convinced we should use CalVer.
> >
> > I'm still skeptical of the utility of CalVer here.  Debian 6.0
> > (squeeze), for example, was released in 2011 but is incompatible with
> > `manylinux2010` wheels because it uses glibc 2.11.  I'm concerned that
> > the sooner `manylinux2015` is defined, the more likely it is to
> > describe too fuzzy an ABI era for CalVer to convey meaningful
> > information to the LTS audience.
> 
> Yeah, I'd agree with that - there's a fuzzy multi-year period from
> when libraries are available to when they become ubiquitous, so given
> a "manylinux2010", it would be surprising if a 2012 release like
> Ubuntu 12.04 didn't support it, but for distros released in 2010 or
> 2011 you'd still need to check the details. And even after that
> adoption period, there are always going to be distros that make other
> choices (like Alpine deciding glibc was too large).
> 
> > What makes it worth it is the ability to skip and backfill versions.
> > As you you pointed out, it would be a strange version scheme that had
> > an architecture that gained wide support in 2015 become `manylinux3`
> > and one that gained wide support in 2014 `manylinux4`.
> >
> > In particular, Geoffrey Thomas pointed out that it should be possible
> > to produce nearly-`manylinux1` compliant wheels with a much newer
> > toolchain:
> >
> > https://mail.python.org/pipermail/wheel-builders/2017-July/000283.html
> >
> > We may decide that an update to `manylinux1` is worthwhile, and by
> > switching to CalVer, backfilling that version as `manylinux2008` would
> > be straight forward.
> 
> Indeed, that concrete pragmatic benefit provides a more compelling
> rationale for switching the numbering scheme.
> 
> Cheers,
> Nick.
> 
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] draft PEP: manylinux2

[Alex Walters]

-Original Message-
From: Alex Walters [mailto:tritium-l...@sdamon.com] 
Sent: Wednesday, February 7, 2018 12:21 AM
To: 'Janzert' <janz...@janzert.com>
Subject: RE: [Distutils] draft PEP: manylinux2

> -Original Message-
> From: Distutils-SIG [mailto:distutils-sig-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Janzert
> Sent: Tuesday, February 6, 2018 3:33 PM
> To: Distutils-Sig@Python.Org
> Subject: Re: [Distutils] draft PEP: manylinux2
> 
> On 2/5/2018 16:01, Nick Coghlan wrote:
> > Compare:
> >
> > - manylinux1 vs manylinux2 vs manylinux3
> > - manylinux2007 vs manylinux2010 vs manylinux2014
> >
> 
> I'll leave this just as a data point (anecdote) from someone that isn't
> heavily involved with linux sysadmin or python packaging. Feel free to
> make of it what you like. I generally run debian stable and occasionally
> ubuntu lts on servers and the latest ubuntu for my workstation.
> 
> If I were looking to install a package and one of the binaries available
> is manylinux2010 I probably completely pass over that option and don't
> even attempt using it. My assumption would be anything that old probably
> isn't going to work on my 2016 or newer OS. Whereas manylinux(1, 2, 3) I
> would think has a good chance of working on any reasonably modern linux.
> 
> If not for having read the discussion here I would have interpreted a
> date, especially a date that's the better part of a decade in the past,
> completely the wrong way.
> 
> Having said that, I'm pretty sure that pip should in general be handling
> this decision for me and doing the right thing anyway? So it probably
> doesn't matter too much.
> 
> Janzert
> 

This is a really good point.  Since pip is the main interface to packages
for end users anyways, we can call it manylinux8675309 and it wouldn't
really matter to users - the name only really matters to package
maintainers, not users.  And because of that, manylinux2010, manylinux2014,
etc makes more sense.  A package maintainer is expected to be more educated
about these matters, and that naming scheme is more useful to them.  "Whats
the oldest linux system my code will run on?" is a very likely question a
maintainer would have when building binary packages, and the year-based
naming scheme is the logical answer.

+1 to manylinux2010, -0 manylinux2

> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] print module

[Alex Walters]

This is not the correct list to ask about problems with your code (It is
very likely that your program redefined the name print.  Python-list would
be the list to ask).  There is no module to import to get the print
function.  There is a special import to turn the print keyword to the print
function in python 2.7 (from __future__ import print_function).  There is no
third party module for this.  Even if there where, this list is for the
packaging infrastructure of python, not module support.

 

From: Distutils-SIG
[mailto:distutils-sig-bounces+tritium-list=sdamon@python.org] On Behalf
Of Kevin Urban
Sent: Monday, January 29, 2018 12:47 PM
To: distutils-sig@python.org
Subject: [Distutils] print module

 

 

Just starting out unable to use print function on a mac. Which module do I
import to use print?

 

 

Kevin Urban

 

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Disabling non HTTPS access to APIs on PyPI

[Alex Walters]

> -Original Message-
> From: Distutils-SIG [mailto:distutils-sig-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Erik Bray
> Sent: Friday, October 27, 2017 11:16 AM
> To: Distutils 
> Subject: Re: [Distutils] Disabling non HTTPS access to APIs on PyPI
> 
> On Thu, Oct 26, 2017 at 5:11 PM, Donald Stufft  wrote:
> > Historically PyPI was only available over either HTTP or unvalidated HTTPS,
> > and over time we’ve been pushing more and more traffic onto HTTPS. In
> > Warehouse the decision was made to *not* redirect “API” URLs from HTTP
> to
> > HTTPS, but to rather return an error accessing them from HTTP. This is
> > because while logged in views have HSTS to ensure HTTPS in the browser
> (and
> > with humans manually entering them into the URL bar regularly they are
> more
> > error prone) APIs which are typically accessed by automated clients with an
> > URL configured or hardcoded typically do not respect HSTS, so if you had a
> > script that did ``curl http://pypi.python.org/simple/``, it would silently
> > get redirects to https and appear to “work”, but you wouldn’t get any of
> the
> > security properties of TLS because an attacker would just intercept the
> > request prior to the redirect happening.
> >
> > Today I’ve backported this changed to the current production deployment
> of
> > PyPI, which means that you can no longer access /simple/ and /packages/
> over
> > HTTP and you will have to directly go to HTTPS. For most people this should
> > have no effect, because most tooling should be defaulting to HTTPS
> anyways,
> > however if you’re using a significantly old version of tooling, it may still
> > be defaulting to the HTTP url and will now stop functioning.
> >
> > The recommended remediation is to upgrade your tooling to versions that
> > support verified TLS connections and which default to the proper HTTPS
> URLs.
> 
> +1
> 
> This will probably (unfortunately) break some things for some people,
> which is worrying.  But it is the right thing to do and good advice in
> general.

Might want to post a message on the front page of pypi.python.org when this 
becomes effective (if it's not there already.)

> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] new PyPI: a rant from a package maintainer

[Alex Walters]

> Also -- I know that the various tools involved are maintained by different 
> people, but it sure would be nice if, for instance, the latest versions of 
> setuptools and twine would not try to upload non longer supported file types!

The tools should have better reporting on the errors pypi gives, but hard 
coding into the tools what pypi will and won’t accept makes policy decisions on 
pypi more painful to change.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Manual upload of RPM files no longer works

[Alex Walters]

If I remember correctly, RPM, DMG, and DPKG (if there was a DPKG distutils 
option) were disabled from uploading to pypi.  I believe the recommendation is 
to use manylinux1 wheels or to set up private distro specific repositories.

 

From: Distutils-SIG 
[mailto:distutils-sig-bounces+tritium-list=sdamon@python.org] On Behalf Of 
Anthony Tuininga
Sent: Monday, July 24, 2017 4:17 PM
To: distutils-sig@python.org
Subject: [Distutils] Manual upload of RPM files no longer works

 

All,

 

I just tried to upload some RPM files using the "Files" section in PyPI. I get 
the following response:

Gone (This API has been deprecated and removed from legacy PyPI in favor of 
using the APIs available in the new PyPI.org implementation of PyPI (located at 
https://pypi.org/). For more information about migrating your use of this API 
to PyPI.org, please see 
https://packaging.python.org/guides/migrating-to-pypi-org/#uploading. For more 
information about the sunsetting of this API, please see 
https://mail.python.org/pipermail/distutils-sig/2017-June/030766.html)
I have used twine for wheels and source distributions without any difficulty. 
But it refuses to support RPM files:
ValueError: Unknown distribution format: 'cx_Oracle-6.0rc2-py26-1.x86_64.rpm'
Is there any way to get these uploaded now? I can't use bdist_rpm upload and 
I'd like to verify the files *before* they are uploaded, not afterwards! 
Thoughts?
Anthony
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Upload documentation to be hosted at ‘pythonhosted.org’

[Alex Walters]

While the topic of shutting down pythonhosted.org has come up, on and off, for 
a few years, there is one certain thing: The recommended documentation hosting 
platform of the python community is readthedocs.org.

> -Original Message-
> From: Distutils-SIG [mailto:distutils-sig-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Ben Finney
> Sent: Tuesday, July 4, 2017 5:52 AM
> To: Distutils-Sig@Python.Org
> Subject: [Distutils] Upload documentation to be hosted at
> ‘pythonhosted.org’
> 
> Howdy all,
> 
> Is ‘pythonhosted.org’ still a recommended service for hosting
> documentation for Python distributions? It certainly seems so, from the
> page at :
> 
>  This site hosts documentation uploaded by authors of packages in
>  the Python Package Index.
> 
>  To upload documentation, go to your package edit page
>  (http://pypi.python.org/pypi?%3Aaction=pkg_edit=yourpackage),
>  and fill out the form at the bottom of the page.
> 
> Should that change for the fact that ‘pypi.python.org’ is in the process
> of deprecation for uploads?
> 
> --
>  \ “If nature has made any one thing less susceptible than all |
>   `\others of exclusive property, it is the action of the thinking |
> _o__)  power called an idea” —Thomas Jefferson, 1813-08-13 |
> Ben Finney
> 
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Switch PyPA from IRC to Gitter or similar

[Alex Walters]

Other than the fact that we now would need to redirect the 2000 (at any given 
time) users of #python, and 200 (roughly) users of #pypa to an inherently 
commercial, and frankly, hard to discover service?  It's tantamount to 
discontinuing chat support, which if that’s what you want to do, so be it, but 
that’s what it is.

(I have no argument about moving -dev)

> -Original Message-
> From: Distutils-SIG [mailto:distutils-sig-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Jason R. Coombs
> Sent: Friday, June 10, 2016 9:22 AM
> To: distutils-sig@python.org
> Subject: [Distutils] Switch PyPA from IRC to Gitter or similar
> 
> In #pypa-dev, I raised the possibility of moving our PyPA support channels
> from IRC to another hosted solution that enables persistence. Although IRC
> has served us well, there are systems now with clear feature advantages,
> which are crucial to my continuous participation:
> 
> - always-on experience; even if one’s device is suspended or otherwise
> offline.
> - mobile support — the in-cloud experience is essential for low power and
> intermittently connected devices.
> - push notifications allow a project leader to remain largely inactive in a
> channel, but attention raised promptly when users make a relevant mention.
> - continuous, integrated logging for catching up on the conversation.
> 
> Both Gitter and Slack offer the experience I’m after, with Gitter feeling 
> like a
> better fit for open-source projects (or groups of them).
> 
> I’ve tried using IRCCloud, and it provides a similar, suitable experience on 
> the
> same IRC infrastructure, with one big difference. While Gitter and Slack offer
> the above features for free, IRCCloud requires a $5/user/month subscription
> (otherwise, connections are dropped after two hours). I did reach out to
> them to see if they could offer some professional consideration for
> contributors, but I haven’t heard from them. Furthermore, IRCCloud requires
> an additional account on top of the account required for Freenode.
> 
> In addition to the critical features above, Gitter and Slack offer other
> advantages:
> 
> - For Gitter, single-sign on using the same Github account for authentication
> and authorization means no extra accounts. Slack requires one new account.
> - An elegant web-based interface as a first-class feature, a lower barrier of
> entry for users.
> - Zero-install or config.
> - Integration with source code and other systems.
> 
> It’s because of the limitations of these systems that I find myself rarely in
> IRC, only joining when I have a specific issue, even though I’d like to be
> permanently present.
> 
> Donald has offered to run an IRC bouncer for me, but such a bouncer is only a
> half-solution, not providing the push notifications, mobile apps (IRC apps
> exist, but just get disconnected, and often fail to connect on mobile provider
> networks), or integrated logging.
> 
> I note that both Gitter and Slack offer IRC interfaces, so those users who
> prefer their IRC workflow can continue to use that if they so choose.
> 
> I know there are other alternatives, like self-hosted solutions, but I’d like 
> to
> avoid adding the burden of administering such a system. If someone wanted
> to take on that role, I’d be open to that alternative.
> 
> I’d like to propose we move #pypa-dev to /pypa/dev and #pypa to
> /pypa/support in gitter.
> 
> Personally, the downsides to moving to Gitter (other than enacting the move
> itself) seem negligible. What do you think? What downsides am I missing?
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig