[Distutils] Re: Setuptools adopts distutils

[Alex Grönholm]

Indeed – this is major step forward!

On 3.7.2020 18.59, Paul Moore wrote:

Awesome! Congratulations on achieving this :-)
Paul

On Fri, 3 Jul 2020 at 16:26, Jason R. Coombs  wrote:

I’m pleased to announce the release of Setuptools 48, which adopts the 
distutils codebase from CPython per pypa/setuptools#417 and 
pypa/packaging-problems#127.

Given the amount of change this effort involved, it’s likely unstable and thus 
the major version bump. Please report issues at the Setuptools issue tracker. 
I’ll be around today (IRC, Gitter, Slack) to either disable the functionality 
or add an escape hatch if needed.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/74WZ7D3ARF7B3N6MAV2JBV3DW6TRHFIV/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/EWWVJYBUBXGUL46Z7524AAQUCWVWVHQG/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/PPF7QT7FYXWVVOCGPSPE74P2AOGOIWCI/

[Distutils] Re: Setuptools use pip over easy_install when installing through setup.py

[Alex Grönholm]

You should not depend on running setup.py to install dependencies. Use 
pip instead, always.


Pokestar Fan kirjoitti 25.12.2019 klo 6.52:

I was running a test program through setup.py, and it threw an error whenever 
it tried to install a package on a Windows 10 Python 3.8 installation (It 
worked on mac/ubuntu all versions and Python 3.5-3.7 on Windows 10). However, 
if I installed the packages through pip instead of easy install the tests ran 
as excepted. Can installing packages through setup.py go through pip and not 
easy install, as easy install is deprecated?
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/KD4FHDVE7ADQBHSV7ARSH3NGJ7VWF5PS/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/6HGJ75CMGCNMU53XMWPDAGKIVXLEVBPM/

[Distutils] Re: Namespaces and setup.cfg

[Alex Grönholm]

Your setup.cfg seems to be missing the [options.packages.find] section, 
as documented here: 
https://setuptools.readthedocs.io/en/latest/setuptools.html#using-a-src-layout


You need to add this to your setup.cfg:

[options.packages.find]
where = src

After adding this and installing the resulting sdist, I was able to 
import the svg package.


Lennart Regebro kirjoitti 3.11.2019 klo 21.03:

Sure.

$ virtualenv /tmp/svgpathtest
$ /tmp/svgpathtest/bin/python setup.py sdist
$ /tmp/svgpathtest/bin/pip install dist/svg.path-4.0.tar.gz
$ /tmp/svgpathtest/bin/python

import svg

Traceback (most recent call last):
   File "", line 1, in 
ImportError: No module named svg

I attached a full log, plus the failing package file.


On Sun, Nov 3, 2019 at 7:02 PM Alex Grönholm  wrote:

I was unable to reproduce the issues you're seeing. Could you give
detailed repro instructions?

Lennart Regebro kirjoitti 3.11.2019 klo 17.46:

Heya!

I recently moved all the setup.py configuration into setup.cfg for
Pyroma. That worked very nicely. So I started doing it for my other
packages, but with svn.path I failed. Everything *looked* fine, and I
uploaded a 4.0 to PyPI, but if I install that in a virtualenv,
importing svg or svg.path fails. The Python installation can't find
them. Declaring them as a namespace means you can import them, but the
imports are empty, there is no attributes to the modules, so you still
can't use the package.

Is this possible? And if so, how?

Here's a link to the 4.0 release with all the info in the setup.cfg.
https://github.com/regebro/svg.path/tree/5f941b729f0b7e3991f238776c9391d57917e0a9
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/XQJAO65QWJOKJDDNTNX3OETCDFIF2G7G/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/SMEYWUGUETBJ3TWXZIFVXCYQQVOQI57X/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/W4VGGANOWXMHZWYUJXPJPOCO62LQURLR/

[Distutils] Re: Namespaces and setup.cfg

[Alex Grönholm]

I was unable to reproduce the issues you're seeing. Could you give 
detailed repro instructions?


Lennart Regebro kirjoitti 3.11.2019 klo 17.46:

Heya!

I recently moved all the setup.py configuration into setup.cfg for
Pyroma. That worked very nicely. So I started doing it for my other
packages, but with svn.path I failed. Everything *looked* fine, and I
uploaded a 4.0 to PyPI, but if I install that in a virtualenv,
importing svg or svg.path fails. The Python installation can't find
them. Declaring them as a namespace means you can import them, but the
imports are empty, there is no attributes to the modules, so you still
can't use the package.

Is this possible? And if so, how?

Here's a link to the 4.0 release with all the info in the setup.cfg.
https://github.com/regebro/svg.path/tree/5f941b729f0b7e3991f238776c9391d57917e0a9
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/XQJAO65QWJOKJDDNTNX3OETCDFIF2G7G/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/SMEYWUGUETBJ3TWXZIFVXCYQQVOQI57X/

[Distutils] Re: failing to build a windows python38 wheel

[Alex Grönholm]

This was fixed in wheel v0.33.5.

Marius Gedminas kirjoitti 30.10.2019 klo 11.58:

On Tue, Oct 29, 2019 at 03:09:08PM +, Robin Becker wrote:

On 29/10/2019 12:53, Robin Becker wrote:

Hi, I am trying to use the technique of pillow to pre-emptively
build windows 38 wheel for x_64 and also i686.

I used
https://github.com/matthew-brett/multibuild/blob/devel/install_python.ps1
as a starting point, but made the version stuff explicit so my install
ps1 looks like

I am slightly mystified can anyone suggest what I am doing wrong?

I think the problem was that the installed python needed an upgraded
pip & wheel. I seem to be building

reportlab-3.5.32-cp38-cp38-win32.whl

ie without the 'm' in the filename.

That is correct: Python 3.8 no longer uses the 'm' flag to distinguish
pymalloc builds because they're now ABI-compatible:

- https://docs.python.org/3/whatsnew/3.8.html#build-and-c-api-changes
- 
https://docs.python.org/3/whatsnew/3.8.html#debug-build-uses-the-same-abi-as-release-build

Regards,
Marius Gedminas

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/JMRYJAZA5PB5PREETLRLKGQFMD6HZQJL/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/HDYTZOF4TFK2R2FYUO6QIV5UPZRSNQCH/

[Distutils] Review help: Automatic detection of macOS platform tag

[Alex Grönholm]

There's a pull request against wheel that I feel out of my depth 
reviewing: https://github.com/pypa/wheel/pull/314


I'm looking for anyone with experience on macOS binary compatibility to 
help validate the solution. Help is really appreciated!

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/CMKEETMQBJ3EOS37NADX2TW5TROIDH7R/

[Distutils] Re: travis problem

[Alex Grönholm]

This is not related to Python packaging per se, but I'll bite.

I'm not seeing difficulties but then again I am using the Xenial images 
in all my projects. Could you try upgrading from Trusty to Xenial in 
your Travis configuration?


Robin Becker kirjoitti 16.9.2019 klo 19.08:
Channelling the experts for advice on travis. After being absent for a 
couple of weeks my travis builds have started failing


The problem is seems to be related to this message in the failing builds


3.5 is not installed; attempting download


this is later followed by

Downloading archive: 
https://storage.googleapis.com/travis-ci-language-archives/python/binaries/ubuntu/14.04/x86_64/python-3.5.tar.bz2

0.48s$ curl -sSf -o python-3.5.tar.bz2 ${archive_url}
13.65s$ sudo tar xjf python-3.5.tar.bz2 --directory /



my base python in .travis.yml is


python:  3.5


and in the matrix I have this


matrix:
  exclude:
  # Exclude the default Python 3.5 build
  - python: 3.5
  include:
    - os: linux
  env:
    - UNICODE_WIDTH=16
    - MB_PYTHON_VERSION=2.7
    - os: linux
  env: 

...


I see no pattern in which are failing and which succeed


--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/CLIAMZJDUW7YRA7FOTZS42TZKD3NKEM5/

[Distutils] Wheel 0.32 released

[Alex Grönholm]

This release contains the following significant changes:

* Wheel signature generation and verification commands have been
removed
* The wheel installation features have been removed
* The documentation has been rewritten to conform to the PyPA
guidelines
* The ability to add multiple license files to the wheel has been added
(the most common ones are included by default)

There were also a few bug fixes as well.
If you are left missing the removed features, let me know on the issue
tracker.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/6OBPGD6DDW6PYIPUIDU3OOI3OR6ZEV6Q/

[Distutils] Re: disable building wheel for a package

[Alex Grönholm]

I'm curious: what data does it attempt to install and where? Have you
created a ticket for this somewhere?
ke, 2018-09-12 kello 15:56 -0400, sashk kirjoitti:
>  
>  
> > > I agree with Nathaniel - the correct solution is to fix your
> > > package
> > > so that can be built as a wheel which correctly installs as you
> > > wish.
> > > If that's not possible due to limitations in the wheel format or
> > > the
> > > wheel building process, please file issues explaining your use
> > > case
> > > and the problems you're having, so we can address them.
>  
> Package need to install several non-python files into specific path
> and we are  using data_files. Because wheel doesn't support absolute
> path[1], these files are installed into wrong location under site-
> packages. I need some kind of solution for wheel to properly install
> them, or come up with some kind of a way to move them into proper
> path on a first run.
>  
>  
> [1]: https://github.com/pypa/wheel/issues/92
> --Distutils-SIG mailing list -- distutils-sig@python.orgTo
> unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/Message
>  archived at 
> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/RURPBWEP5LJKB2X7MREJDDE7H2V3MOMF/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/DJXEMQLICOHPBCDQDAXQ6DJE2SGVCW6Y/

Re: [Distutils] Cannot install pyopencl

[Alex Grönholm]

This mailing list is for packaging development discussion. Questions on 
how to install packages should be directed to either #pypa or #python on 
Freenode IRC, or sites like StackOverflow.



Tim Graham kirjoitti 16.03.2018 klo 17:12:


Hello

I wonder if you can help…. I am trying to install btcrecover which is 
no simple task. When I try and install pyopencl it keeps saying that 
it does not recognise it as a file name…


Do you have any ideas?

Thanks for your help.

Best regards

*Tim Graham *

T: +61 8 9467 7664

M: +61 457 414 935

W: www.conveyorcn.com

E: tgra...@minsep.com.au 



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PyPI support for linux_ppc64le

[Alex Grönholm]

The manylinux1 platform only supports x86-64 and x86-32 (i686) 
architectures. A quote from PEP 513:


Because CentOS 5 is only available for x86_64 and i686 architectures, 
these are the only architectures currently supported by the manylinux1 
policy.


If support is to be extended to other architectures, it requires a new 
standard (which has recently been discussed on this ML).



Paul Moore kirjoitti 12.03.2018 klo 16:31:

On 12 March 2018 at 14:26, Matt McCormick  wrote:

Hi,

I created a wheel for Linux / ppc64le (built on CentOS 7). When I try
to upload to PyPI via twine, the error message results:

   unsupported platform tag 'linux_ppc64le'


What is the status of the ppc64le wheel support? Is pip / PyPI support
expected soon?

PyPI supports Linux wheels in the form of "manylinux" builds. I don't
think there's any plan to support platform or distribution specific
Linux wheels beyond that. Having said that, I'm sure the Lunux
specialists will be able to give more detail (maybe one of the
manylinux versions handles ppc64le?)

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Building a Python package build service for warehouse

[Alex Grönholm]

I'd be all for it if I wasn't buried under a ton of other things to do. 
Happy hacking and good luck!



Jannis Gebauer kirjoitti 06.02.2018 klo 10:33:

Hi!

I’m currently working on a package build server. My goal is to produce 
useful additional meta data for all packages available on PyPi.


This includes:

- Transitive dependencies
- Is the package installable under Python 3?
- Various automated “code quality” tests like pylint, pyflakes, pep8, 
mccabe etc.

- Automated security tests
- (possibly changelogs, commit logs)
- Licenses!

The main idea is to run the build process in a restricted “sandbox” 
docker container that pulls the package from PyPi, installs it and 
runs a couple of tools on it. Code is still pretty rough, nothing to 
look at at the moment I’m afraid.


Is there any interest in working on this together? Maybe even with the 
goal to make it an open API that can be consumed by warehouse et al.?


Interested in any thoughts on this!

Cheers,

Jannis

P.S: I’m currently crunching trough the data on a 96 CPU cluster. 
There’s an API available, but it’s sitting behind HTTP Basic Auth as 
it is basically an endpoint for remote code execution (and throws lots 
of 500s :D). Send me a mail to j...@pyup.io  if you 
want to play around with it.





___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 508, environment markers & the possibility of Python 3.10

[Alex Grönholm]

Hasn't Guido said publicly that 4.0 would be the next version from 3.9 
(since he hates double digits)?



Nick Coghlan kirjoitti 28.01.2018 klo 07:44:

Hi folks,

In https://github.com/python/peps/issues/560, a user pointed out that
the current definition of python_version in PEP 508 assumes
single-digit major and minor version numbers:

platform.python_version()[:3]

There's a reasonable chance we'll see 3.10 rather than 4.0 in a few
years time, at which point that definition would break.

The suggested fix is to amend that definition to be:

 ".".join(platform.python_version_tuple()[:2])

This seems like a good suggestion to me, so my inclination is to
handle this in a similar way to
https://www.python.org/dev/peps/pep-0440/#summary-of-changes-to-pep-440:
fix it in place, and add a section at the end of the PEP listing the
post-publication changes.

Cheers,
Nick.



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Installed Extras Metadata

[Alex Grönholm]

Pradyun Gedam kirjoitti 26.01.2018 klo 17:11:

Hello! I hope everyone's had a great start to 2018! :)

A few months back, while working on pip, I had noticed an oddity about 
extras.


Installing a package with extras would not store information about the 
fact that the extras were requested. This means, later, it is not 
possible to know which extra-based optional dependencies of a package 
have to be considered when verifying that the packages are compatible 
with each other. This information is relavant for 
resolution/validation since without it, it is not possible to know 
which the extra-requirements to care about.


As an example, installing ``requests[security]`` and then uninstalling 
``PyOpenSSL`` leaves you in a state where you don't really satisfy 
what was asked for but there's no way to detect that either.
What here is specific to extras really? "pip uninstall" does not 
consider dependencies anyway and will happily let you uninstall whatever 
you want, even if it's a dependency of some still installed distribution.


Thus, obviously, I'm interested in making pip to be able to store this 
information. As I understand, this is done needs to be specified in a 
PEP and/or on PyPUG's specification page.


To that end, here's seeding proposal for the discussion: a new 
`extras-requested.txt` file in the .dist-info directory, storing the 
extra names in a one-per-line format.


Cheers!
Pradyun



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 566 - metadata 1.3 changes

[Alex Grönholm]

+1 from me. While I dislike the fact that "2.0" was put to use 
prematurely, using "2.1" is still less confusing than going from 2.0 to 1.3.



Nick Coghlan kirjoitti 20.01.2018 klo 05:07:

On 19 January 2018 at 00:14, Joni Orponen  wrote:

On Thu, Jan 18, 2018 at 5:58 AM, Nick Coghlan  wrote:

Given that, I think it would be reasonable to finally Withdraw PEP 426
(rather than continuing to defer it), and have PEP 566 define metadata
version 2.1, so that it's unambiguously the latest metadata version.

Jump straight to 3.0 to clear out any confusion and/or ambiguity on the next
backwards-incompatible one?

While we could do that, wheel's use of "2.0" actually stems from early
drafts of PEP 426, and PEP 566 *is* backwards compatible with that.

So I like 2.1 - higher than everything previously used, but an
incremental update to the early versions of 426 before we/I started
imagining a ground up redesign of the metadata definition.

Cheers,
Nick.



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 566 - metadata 1.3 changes

[Alex Grönholm]

Whichever we choose, the metadata version should match the PEP version, 
which it currently does not.



Nathaniel Smith kirjoitti 16.01.2018 klo 18:58:
On Jan 12, 2018 8:00 AM, "Alex Grönholm" <alex.gronh...@nextday.fi 
<mailto:alex.gronh...@nextday.fi>> wrote:


On the same note, wheel currently writes "2.0" as its metadata
version. Shouldn't this be changed into 1.3 (along with ditching
metadata.json)?


Should wheel change to emit 1.3, or should the PEP change to become 
2.0? I know there were great hopes for "metadata 2.0", but given that 
there are bazillions of packages out there with a metadata version of 
2.0, we're never going to be able to meaningfully use that version 
number for anything else, and it's confusing if when reading package 
metadata the ordering is 1.2 < 2.0 == 1.3 < 1.4. So maybe we should 
declare that this update is 2.0 or 2.1, the next one will be 2.1 or 
2.2, etc., and if anyone asks why the major version bump, well, it's 
hardly the strangest thing we've done for compatibility :-). (And in 
the unlikely event that PEP 426 lurches back to life, we can make it 3.0.)


-n


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 566 - metadata 1.3 changes

[Alex Grönholm]

On the same note, wheel currently writes "2.0" as its metadata version. 
Shouldn't this be changed into 1.3 (along with ditching metadata.json)?



Thomas Kluyver kirjoitti 12.01.2018 klo 17:26:

On Wed, Jan 10, 2018, at 11:42 PM, Nick Coghlan wrote:

On 11 January 2018 at 00:54, Daniel Holth  wrote:

AFAICT the only missing feature from old-Metadata-2.0 is "description as
message body", which places readable description text after the key/value
pairs.

Do pip/PyPI/et al currently support that?

It looks like twine supports it, at least for wheels:
https://github.com/pypa/twine/blob/f74eae5506300387572c65c9dbfe240d927788c2/twine/wheel.py#L99

I don't think pip needs to support it (does pip do anything with 
descriptions?). I haven't looked at PyPI's code, but I'd guess it uses the 
metadata sent with the upload by tools like twine and flit.

Thomas
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Immutability of Release Metadata in Warehouse

[Alex Grönholm]

Couldn't Warehouse validate the description, and reject the upload (with 
an appropriate message) if it doesn't pass? This at least would 
eliminate those ugly project pages that failed to render...there are a 
lot of them on PyPI.



Donald Stufft kirjoitti 19.12.2017 klo 21:43:
For those who are not aware, legacy PyPI would allow you to run 
``twine register`` on a release that had already been created in order 
to modify the metadata that PyPI had recorded for that release (keyed 
by version number). This wasn’t a super widely used feature, but it’s 
primary use case was when folks would mistakenly release a project 
that had a broken description that wouldn’t correctly render on PyPI. 
With the move from the legacy PyPI code base to Warehouse for handling 
uploads, this feature had been somewhat inadvertently lost.


This issue was raised in 
https://github.com/pypa/warehouse/issues/2170. This email is provide 
notice that after thinking about this issue for awhile, we’re *not* 
going to restore the ability to modify the metadata associated with a 
release after it has been uploaded.The eventual goal is that it should 
be ideally possible to treat the files that are on PyPI as the source 
of truth for all metadata, and that the data stored in the database is 
simply an optimization for accessing and presenting that data on PyPI. 
Obviously if we allow modifications to the metadata as stored in the 
PyPI database, that would allow this metadata to “drift” from what is 
actually stored in the files, which would prevent that goal from being 
realized.


It is true that even with this change, there is not a guarantee that 
the metadata in the database does not match what is in the file(s) 
that have been uploaded to PyPI, even going into the future. Thus the 
decision to not restore this feature is not the only step on the way 
to being able to assert this guarantee, but it is *one* of them.


The most common reason for wanting to modify any metadata after the 
fact is to fix typos etc that made it into the description prior to 
publication. It is our opinion that the best way to handle these is to 
either cut a new release (it can be a post release if that’s all that 
has changed) or to validate the description field prior to publication 
(which can currently be done using readme_renderer or restview). 
Longer term we also plan to introduce the ability to “stage” releases 
so that releases and files can be uploaded to a temporary location 
within Warehouse, and visible with a special URL to allow people to 
manually validate the actual bits that were uploaded before pushing a 
“finalize” or “publish” button that would flip it from being a 
mutable, hidden release to an immutable, publicly viewable release.


If folks have other use cases where they’ve used the ability to modify 
release metadata after it had been released that they feel is an 
important enough use case that it needs to be supported in some way, 
please let us know either in this thread or by opening an issue on 
https://github.com/pypa/warehouse so we can figure out if it’s 
something we want to support, if one of the other mechanisms we’re 
planning on adding will support it, or if there is some new mechanism 
we can add that can support it better.



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Wheel 1.0 roadmap

[Alex Grönholm]

I am planning for a 1.0.0 release of the "wheel" library. I would like 
to start using semver from this point onwards, which in the case of 
wheel means that its command line interface should be well defined and 
remain backwards compatible. As part of this effort, I've rewritten the 
documentation (currently in the "docs-update" branch on Github) to 
conform to the PyPA guidelines. Wheel also had some generated API 
documentation on ReadTheDocs, but as discussed privately with Daniel 
Holth and Nick Coghlan, wheel should not have a public API going forward 
so I've deleted that documentation.


I've also taken a hard look at wheel's features and would like to remove 
those which I consider to be either useless or harmful. I've added these 
tasks as issues on Github.


All the issues that I'd like to get resolved by 1.0.0 have been tagged 
with the proper milestone marker here: 
https://github.com/pypa/wheel/milestone/1


Feedback is very welcome!

ps. Daniel, if you're reading this, would you mind giving the new docs a 
once-over? Also, if you can suggest where to put the "story" page, I'll 
link it back to the main index file.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How do native namespaces work? (And do they play well with setuptools?)

[Alex Grönholm]

Ok, I get you now. So, what you should be doing is using pip to install 
them ("pip install ."). Directly doing "setup.py install" will install 
.egg files which are a thing of the past. Do this and the verify script 
will work fine.


I hope this answers your question.


Thomas Nyberg kirjoitti 25.10.2017 klo 22:05:

On 10/25/2017 04:54 PM, Alex Grönholm wrote:

What do you mean by "using"?

Are you referring to where I say "I'm currently using python 3.5 as
packaged in debian 9"? (I can't find the string "using" anywhere else.)
I just mean that that is the version of python I have installed.


  You mean installing them (for which you
should never use easy_install anyway, btw), or packaging them? The only
limitation regarding packaging them is that you currently need to
explicitly list them in your packaging configuration, as find_packages()
does not yet support namespace packages.

At this point I mean installing them using the `setup.py` script that is
in that repo (i.e. executing `python3 setup.py install`). I mean this
file:
https://github.com/pypa/sample-namespace-packages/blob/master/native/pkg_a/setup.py)
That file is using setuptools.

Maybe the best way to formulate my question is: Should the steps that I
put in the previous email result in error or not? I.e. is that a bug or
expected behavior.

Thanks for the help.

Cheers,
Thomas
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How do native namespaces work? (And do they play well with setuptools?)

[Alex Grönholm]

What do you mean by "using"? You mean installing them (for which you 
should never use easy_install anyway, btw), or packaging them? The only 
limitation regarding packaging them is that you currently need to 
explicitly list them in your packaging configuration, as find_packages() 
does not yet support namespace packages.



Thomas Nyberg kirjoitti 24.10.2017 klo 16:01:

Hello,

I previously posted this to python-list as well as reddit:

https://mail.python.org/pipermail/python-list/2017-October/726935.html

https://www.reddit.com/r/learnpython/comments/78emwr/how_to_native_namespaces_work_in_python_35/

So far (as pointed out in the reddit link I posted) it seems like maybe
I can't use native namespaces in conjunction with setuptools and must
instead use pip. Is this correct?

Here is my original message which contains all the details:

I'm trying to understand native namespaces. I'm currently using python
3.5 as packaged in debian 9. I've been following the instructions here:


https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages

Those instructions link to the following example:

https://github.com/pypa/sample-namespace-packages/tree/master/native

I presume I'm doing something silly, but I can't get that to work. I've
tried creating a virtual environment as:

--
$ python3 -m venv venv
$ source venv/bin/activate
--

_Note_: The virtual environment is active, but I'm stripping the (venv)
off the prompt below to shorten the lines.

And then I've gone into those repo folders and run their setup.py files
(e.g. for namespace a):

--
$ cd sample-namespace-packages/native/pkg_a/
$ python3 setup.py install
--

Then if I try to run their sample verification file, things fail:

--
$ cat sample-namespace-packages/verify_packages.py
from example_pkg import a
from example_pkg import b

print(a.name)
print(a.__path__)
print(b.name)
print(b.__path__)
$ python3 sample-namespace-packages/verify_packages.py
Traceback (most recent call last):
   File "sample-namespace-packages/verify_packages.py", line 1, in 
 from example_pkg import a
ImportError: No module named 'example_pkg'
--

Things seem to be installing:

--
$ ls venv/lib/python3.5/site-packages/
easy-install.pth   pkg_resources
easy_install.pypkg_resources-0.0.0.dist-info
example_pkg_a-1-py3.5.egg  __pycache__
example_pkg_b-1-py3.5.egg  setuptools
pipsetuptools-32.3.1.dist-info
pip-9.0.1.dist-info
--

Am I missing something totally obvious here? Does anyone here see what
I'm doing wrong? Thanks for any help!

Cheers,
Thomas
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

[Alex Grönholm]

Paul Moore kirjoitti 21.10.2017 klo 13:03:

On 20 October 2017 at 23:53, Richard Jones  wrote:

Hiya Paul,

There's a bunch of tooling out there using pip's internals to extending
pip's functionality. Could you please provide a some reasoning as to why
they're all going to be broken at pip 10, and possibly some guidance on how
to get that functionality back?

Hi Richard,
There was a change to the pip docs that clarified the status of pip's
internal code. The PR for that is at
https://github.com/pypa/pip/pull/4743 but unfortunately it appears
that the docs build has been failing so it hasn't yet made it to the
formal pip docs site.

To summarise, pip has *never* supported the use of its internal APIs
by external code. Over time, we've had a steady trickle of people
raising issues when their code broke because of doing so, and it
usually turned out to be because they violated assumptions made by the
pip code - such as that it's running in a single-threaded application,
or it has sole control over the logging subsystem, or even just that
you can run your own code after calling a pip API. We've explained
this position regularly on those issues, but as is typical, people
don't manage to find similar issues when raising new ones, so we spent
a lot of time repeating ourselves.

Coming up to pip 10, there's been a *lot* of internal work going on,
and it's fairly likely that a decent chunk of code using pip's
internal APIs will break anyway. We don't document internals changes,
so we faced the possibility of an extended period of people raising
issues saying "you broke my code" and us having no better response
than "you shouldn't do that", which would likely hinder adoption of
pip 10, and cause problems for the ecosystem as a whole. Rather than
do this, we decided to make a clean compatibility break, where we
could send out a clear message - "everything's moved, if that matters
to you, then you were using unsupported functionality and you should
find a better way". The breakage is still there (and certainly we made
it affect more people, as there are no doubt some people who would
have survived the pip 10 release unscathed if we hadn't done this) but
at least it's clearly defined and contained.

As to alternatives, we don't have all the answers here but I can offer
the following suggestions:

1. There are a number of external packages that provide functionality
equivalent to what pip does - packaging, wheel, distlib, pkg_resources
are the ones I'm aware of. These are designed as libraries and so *do*
provide supported APIs.

I need to correct you here: wheel does *NOT* have a public API!

It did previously have some documented functions but it was not really 
well thought out and we recently decided to remove all traces of API 
documentation so as to not cause any confusion about it.

2. We're making a strong push to standardise *all* of the external
interfaces that pip uses, so you should be far more able to write your
own code if that's necessary, without worrying that it'll work
differently than pip does, or that things will suddenly change and
break your code.
3. You can call pip as a subprocess - that's always been supported and
will remain so. We've added some automation-friendly features there
(such as json output format for "pip list") and we'd be happy to add
more if people want to submit PRs.

Likely the biggest problems will be for people who call into the pip
resolver and build APIs, as I don't know of any alternatives out
there. But they were *definitely* breaking anyway, as we've made major
changes to that code (and will be making more).

Also, I should note that we didn't take this decision lightly. We
don't have any particular objection in principle to having a supported
stable pip API, it's just that we don't have anything even close to
the resources needed to define a supported API, maintain it with
acceptable backward compatibility guarantees, and support users who
will inevitably be using it in unexpected and creative ways (we don't
even have the resources to support the limited use of pip's internals
that we see today). Also, pip was never designed originally as a
library, so we *would* have to design that API from scratch. As I
alluded to above, the existing internals code makes some strong
assumptions about how it's called - assumptions that would be
unacceptable in library code, but are fine in an application's
internal code.

Paul

PS People who want to, can of course hunt out the new equivalents of
the code they were using, and just switch. It's not like we can stop
them. But the new names make it clear that they shouldn't be doing
this, so there's an obvious warning there.
PPS Please disregard xoviat's response. This is something we've been
considering for a while, and most definitely not a spur of the moment
decision. It's unfortunate that he was the one most immediately
affected by the change when we made it, but that was just bad timing
(we didn't suddenly do 

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

[Alex Grönholm]

Yeah, +1 from me too. Pip is one project where people will highly likely 
try out the pre-release versions.



xoviat kirjoitti 20.10.2017 klo 20:34:

+1 on pre-release wheels. I've seen the process in action
with SciPy, and it helped to catch at least one significant bug.

2017-10-20 8:30 GMT-05:00 Paul Moore >:


On 20 October 2017 at 14:26, Matthew Brett
> wrote:
> Thanks for the heads-up.
>
> Will y'all be doing a PyPI pre-release so we can test with `pip
> install --pre -U pip`?

We've not yet decided on that. Traditionally I don't think we have
done so, but I'm inclined to think it's a good idea. It might not be
until noticeably closer to the release, though...

Paul
___
Distutils-SIG maillist  - Distutils-SIG@python.org

https://mail.python.org/mailman/listinfo/distutils-sig





___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

[Alex Grönholm]

Perhaps pkg_resources.find_distributions()?

http://setuptools.readthedocs.io/en/latest/pkg_resources.html#getting-or-creating-distributions


Jannis Gebauer kirjoitti 20.10.2017 klo 16:55:

Thanks for the heads-up, Paul.

I’m currently using `pip.get_installed_distributions` and as far as I 
can see that has moved into `_internal`, too: 
https://github.com/pypa/pip/blob/master/src/pip/_internal/utils/misc.py#L333


Any recommendations?

Thanks,

Jannis

On 20. Oct 2017, at 15:22, Paul Moore > wrote:


We're in the process of starting to plan for a release of pip (the
long-awaited pip 10). We're likely still a month or two away from a
release, but now is the time for people to start ensuring that
everything works for them. One key change in the new version will be
that all of the internal APIs of pip will no longer be available, so
any code that currently calls functions in the "pip" namespace will
break. Calling pip's internal APIs has never been supported, and
always carried a risk of such breakage, so projects doing so should,
in theory, be prepared for such things. However, reality is not always
that simple, and we are aware that people will need time to deal with
the implications.

Just in case it's not clear, simply finding where the internal APIs
have moved to and calling them under the new names is *not* what
people should do. We can't stop people calling the internal APIs,
obviously, but the idea of this change is to give people the incentive
to find a supported approach, not just to annoy people who are doing
things we don't want them to ;-)

So please - if you're calling pip's internals in your code, take the
opportunity *now* to check out the in-development version of pip, and
ensure your project will still work when pip 10 is released.

And many thanks to anyone else who helps by testing out the new
version, as well :-)

Thanks,
Paul
___
Distutils-SIG maillist  - Distutils-SIG@python.org 


https://mail.python.org/mailman/listinfo/distutils-sig




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Entry points: specifying and caching

[Alex Grönholm]

Daniel Holth kirjoitti 18.10.2017 klo 21:06:
http://setuptools.readthedocs.io/en/latest/formats.html?highlight=entry_points.txt#entry-points-txt-entry-point-plugin-metadata 



http://setuptools.readthedocs.io/en/latest/pkg_resources.html?highlight=pkg_resources#creating-and-parsing

It is not very complicated. It looks like the characters are mostly 
'python identifier' rules with a little bit of 'package name' rules.


I am also concerned about the amount of parsing on startup. A hard 
problem for certain, since no one likes outdated cache problems 
either. It is also unpleasant to have too much code with a runtime 
dependency on 'packaging'.
Wasn't someone working on implementing pkg_resources in the standard 
library at some point?


On Wed, Oct 18, 2017 at 1:00 PM Paul Moore > wrote:


On 18 October 2017 at 17:48, Doug Hellmann > wrote:
> Excerpts from Thomas Kluyver's message of 2017-10-18 15:52:00 +0100:
>> We're increasingly using entry points in Jupyter to help integrate
>> third-party components. This brings up a couple of things that
I'd like
>> to do:
>>
>> 1. Specification
>>
>> As far as I know, there's no document describing the details of
entry
>> points; it's a de-facto standard established by setuptools. It
seems to
>> work quite well, but it's worth writing down what is unofficially
>> standardised. I would like to see a document on
>> https://packaging.python.org/specifications/ saying:
>>
>> - Where build tools should put entry points in wheels
>> - Where entry points live in installed distributions
>> - The file format (including allowed characters, case
sensitivity...)
>>
>> I guess I'm volunteering to write this, although if someone
else wants
>> to, don't let me stop you. ;-)
>>
>> I'd also be happy to hear that I'm wrong, that this specification
>> already exists somewhere. If it does, can we add a link from
>> https://packaging.python.org/specifications/ ?
>
> I've always used the setuptools documentation as a reference.
Are you
> suggesting moving that information to a different location to
> allow/encourage other tools to implement it as a standard?

I've never used entry points myself (other than the console script
entry points supported by packaging) but a quick Google search found

http://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins
as the only obvious candidate for documentation (and a bit later I
thought of looking under pkg_resources and found
http://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points).
This doesn't really say how the entry point data is stored in the
project metadata, so it's not clear how I'd read that data in my own
code (the answer is of course to use pkg_resources, but the point of
documenting it as a standard is to allow alternative implementations).
Also, it's not clear how a tool like flit might implement entry points
- again, because the specifications don't describe how the metadata is
stored.

+1 from me on moving the entry point specification to
https://packaging.python.org/specifications/

Paul
___
Distutils-SIG maillist  - Distutils-SIG@python.org

https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] New wheel release (v0.30.0)

[Alex Grönholm]

I have now made my first wheel release as the new maintainer.

This release contains bug fixes and documentation updates. I decided to 
make a release now to get a fix out for some long standing issues. A 
proper overhaul of the documentation is coming later.


The project has also been migrated from Bitbucket to Github 
.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] (no subject)

[Alex Grönholm]

Python 2.1 is very, very old and has not been supported for many years. 
It is doubtful that any currently available Python package will work 
with it.


For future reference, when you ask for help with an error, always 
include the error message in your request. It would also be prudent to 
include a subject in your email.



Natasha Kelkar kirjoitti 05.09.2017 klo 15:13:

Respected sir/madam
I am trying to install jpredapi in python 2.1 using command python -m 
pip install jpreadapi but getting an error


Can you please help me in this?

--
Natasha Kelkar

Institute of Bioinformatics and Biotechnology(iBB),
Savitribai Phule Pune University.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Migrating interoperability specs to packaging.python.org

[Alex Grönholm]

Yes, I see the inclusion of a metadata file which conforms to an 
unaccepted PEP as potentially dangerous.


Perhaps I should disable it in the next release?


Daniel Holth kirjoitti 04.09.2017 klo 17:03:


Some people enjoy using metadata.json which tracked pep 426 but I have 
been meaning to take it out, and perhaps keep the key/value to json 
converter as a command.



On Mon, Sep 4, 2017, 09:33 Nick Coghlan > wrote:


Some time ago, I started the process [1] of adjusting how
distutils-sig uses the PEP process so that the reference
specifications will live on packaging.python.org
, and we use the PEP
process to manage *changes* to those specifications, rather than
serving as the specifications themselves (that is, adopting a process
closer to the URL-centric way the Python language reference is
managed, rather than using the RFCstyle PEP-number-centric model the
way we do now).

I never actually finished that work, and as a result, it's currently
thoroughly unclear [2] that Description-Content-Type and
Provides-Extra are defined at
https://packaging.python.org/specifications/#core-metadata rather than
in a PEP.

I'm currently at the CPython core development sprint in San Francisco,
and I'm thinking that finalising that migration [3] and updating the
affected PEPs accordingly (most notably, PEP 345) is likely to be a
good use of my time.

However, I'm also wondering if it may still be worthwhile writing a
metadata 1.3 PEP that does the following things:

1. Explicitly notes the addition of the two new fields
2. Describes the process change for packaging interoperability
specifications
3. Defines a canonical transformation between the human-readable
key:value format and a more automation friendly JSON format

That PEP would then essentially be the first one to use the new
process: it would supersede PEP 345 as the latest metadata
specification, but it would *also* redirect readers to the relevant
URL on packaging.python.org  as the
canonical source of the
specification, rather than being the reference documentation in its
own right.

Cheers,
Nick.

[1] https://github.com/pypa/pypa.io/issues/11#issuecomment-173833061
[2] https://github.com/python/peps/issues/388

P.S. Daniel, if you're currently thinking "I proposed defining an
incremental metadata 1.3 tweak years ago!", aye, you did. My
subsequent additions to PEP 426 were a classic case of second-system
syndrome: https://en.wikipedia.org/wiki/Second-system_effect (which we
suspected long ago, hence that PEP's original deferral)

Fortunately, the disciplining effect of working with a primarily
volunteer contributor base has prevented my over-engineered
change-for-change's-sake-rather-than-to-solve-actual-user-problems
version from becoming reality ;)

--
Nick Coghlan   | ncogh...@gmail.com 
 |   Brisbane, Australia
___
Distutils-SIG maillist  - Distutils-SIG@python.org

https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] why do we still put version numbers into dist-info folders

[Alex Grönholm]

The question is, if we get rid of version numbers in the folder names, 
will it break anything we care about?



Daniel Holth kirjoitti 01.09.2017 klo 23:38:
A long time ago pkg_resources was able to sometimes get the version 
number only from the .egg-info folder name, and skip opening the 
metadata at all when that data was not needed. We might always do 
listdir() for path in sys.path and parse every METADATA on the first 
pkg_resources import.


On Fri, Sep 1, 2017 at 4:32 PM Alex Grönholm <alex.gronh...@nextday.fi 
<mailto:alex.gronh...@nextday.fi>> wrote:


+1 for getting rid of version numbers in the dist-info folders.


RonnyPfannschmidt kirjoitti 01.09.2017 klo 20:32:
> Hi everyone,
>
> a while now i have wondered - why put version numbers into the
dist info
> folders
> not only makes it lookup more expensive (need to search for a
> distribution->folder map)
> its also not serving a purpose, as the rest of the package is
> unversioned at the root of site-packages.
>
> in addition all those versions in folder names, that serve no
purpose,
> and create duplication/conflicts now creep into other peps as well,
>
> please lets take a note at setuptools - for editable installs it
wont
> put a version in to the egg info folder
>
> and imho thats the correct way to deal with data thats not actually
> versioned at that level where its being stored.
>
>
> -- Ronny
>
> ___
> Distutils-SIG maillist  - Distutils-SIG@python.org
<mailto:Distutils-SIG@python.org>
> https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  - Distutils-SIG@python.org
<mailto:Distutils-SIG@python.org>
https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] why do we still put version numbers into dist-info folders

[Alex Grönholm]

+1 for getting rid of version numbers in the dist-info folders.


RonnyPfannschmidt kirjoitti 01.09.2017 klo 20:32:

Hi everyone,

a while now i have wondered - why put version numbers into the dist info
folders
not only makes it lookup more expensive (need to search for a
distribution->folder map)
its also not serving a purpose, as the rest of the package is
unversioned at the root of site-packages.

in addition all those versions in folder names, that serve no purpose,
and create duplication/conflicts now creep into other peps as well,

please lets take a note at setuptools - for editable installs it wont
put a version in to the egg info folder

and imho thats the correct way to deal with data thats not actually
versioned at that level where its being stored.


-- Ronny

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] status check on PEP 517

[Alex Grönholm]

Daniel Holth kirjoitti 30.07.2017 klo 00:48:


I think the proposal is that flit depends on click depends on flit and 
neither one has a wheel and must be built from sdists. Then you have a 
circular build problem. So don't do that. I put this in the same 
category as accidentally conflicting with a stdlib module; it is 
confusing when it happens but it's also fairly avoidable.


Sure but vendorizing the dependencies would work around the problem, 
yes? Like how setuptools does?


On Sat, Jul 29, 2017, 17:38 Alex Grönholm <alex.gronh...@nextday.fi 
<mailto:alex.gronh...@nextday.fi>> wrote:


Donald Stufft kirjoitti 29.07.2017 klo 23:47:



On Jul 29, 2017, at 12:46 PM, Nathaniel Smith <n...@pobox.com
<mailto:n...@pobox.com>> wrote:

I guess the most obvious example of when this would occur is:
suppose click switches to using flit for builds, and then flit
switches to using click for command line parsing. Now there's a
bit of a chicken and egg problem where 'pip install click' will
end up importing flit with the click source tree on the path,
and this tree of course contains a directory named 'click', so
unless special measures are taken flit will end up importing the
code it's trying to build.

But of course this can happen for lots of reasons; most packages
have names that you wouldn't expect to randomly encounter at the
root of a source tree very often, but with 100,000+ packages on
pypi I expect it will happen eventually.

This doesn't happen with setuptools because setuptools
traditionally has few or no dependencies, but obviously we're
changing that; the whole idea here is that now your build system
has full access to pypi.



This is something to be discouraged anyways, because it creates a
sort of broken situation (the same situation that setuptools
itself had). The problem is that if you’re starting from only
sdists, you have a circular dependency that cannot be broken. You
can’t build click, because  click requires flit, but you can’t
install flit, because flit requires click. The only way to fix
this is to either have an already built wheel that you can use
(which obviously was either built with a flit that didn’t require
click, or a click that didn’t require flit, or it’s provenance
can be traced back to that) or do some hacks that will hopefully
resolve the situation good enough to get your first wheel built.

Setuptools tried to depend on things, and it broke shit for a lot
of people because of this. You basically can’t depend on anything
as a build system that uses you as a build system. You can only
depend on things that use other, different build systems in the
entire dependency tree. Likely the best thing for build systems
to do is either have no dependencies, or to have minimal
dependencies that promise to only use setuptools (or another
build tool, which one doesn’t matter, just as long as it has no
dependencies) forever (and have setuptools or this other build
tool promise to never take a dependency).

Or vendorize their dependencies? To me it seems unrealistic for a
build system to have no dependencies at all. Or perhaps this is
exactly what you meant :)



—
Donald Stufft





___
Distutils-SIG maillist  -Distutils-SIG@python.org 
<mailto:Distutils-SIG@python.org>
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  - Distutils-SIG@python.org
<mailto:Distutils-SIG@python.org>
https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PyPi’s predictable download url

[Alex Grönholm]

Wes Turner kirjoitti 27.07.2017 klo 02:36:



On Wednesday, July 26, 2017, Paul Moore > wrote:


On 26 July 2017 at 15:52, Alexander Belopolsky
 wrote:
> On Wed, Jul 26, 2017 at 10:21 AM, Nick Coghlan
 wrote:
>> .. the most robust option is to instead let a dedicated client
tool like pip
>> handle the problem:
>>
>> * pip install: retrieve *and* install a component
>> * pip download: just retrieve the files without installing them
locally
>
> This is a sound advise in general, but the specific issue here
is how
> to bootstrap virtualenv for python 2.7 on the systems with
> missing/broken pip?  Note that "ensurepip" may not be an option
for a
> user without root privileges.
>
> The bootstrap options offered at
> > don't work.
> There is an open issue about this on Github:
> >. A comment on that
> issues suggests "curl
> https://pypi.python.org/simple/virtualenv-15.0.1.tar.gz
", but that
> does not work either.

I didn't get the impression that this was the problem the OP was
hitting, and it's certainly not what I'd recommend for him, but you
should be able to just clone the virtualenv repo and run ```python
virtualenv.py``` to get started.


A shallow clone would minimize the bandwidth cost:

$ git clone --depth=1 https://github.com/pypa/virtualenv

Doesn't work for projects using setuptools_scm, as they require the 
latest tag to get the version info.
If all you need to do is bootstrap a latest pip and virtualenv, 
there's always get-pip.py?


- Docs: 
https://pip.pypa.io/en/stable/installing/#installing-with-get-pip-py

- Src: https://bootstrap.pypa.io/get-pip.py

$ python get-pip.py --user
$ ~/.local/bin/pip install --user virtualenv virtualenvwrapper



Paul
___
Distutils-SIG maillist  - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Maintenance of the "wheel" package

[Alex Grönholm]

Okay. Will you give me the appropriate access privileges to do that?


Daniel Holth kirjoitti 25.07.2017 klo 16:22:
Fine by me, probably inevitable, obviously I'm not a fan of git. 
Please take care of the useful pending pull requests on bitbucket first.


On Tue, Jul 25, 2017 at 9:05 AM Alex Grönholm 
<alex.gronh...@nextday.fi <mailto:alex.gronh...@nextday.fi>> wrote:


Looking at the commit history, the maintenance of "wheel" has more
or less stalled since 2016. I would like to volunteer myself as a
co-maintainer of this project, if allowed. Here's my list of
things I would like to do with it:

  * Move the project from Bitbucket to the PyPA organization on Github
  * Add a setup() keyword (and documentation for it) for
license_file (currently only usable via the deprecated
[metadata] section in setup.cfg)
  * Refactor the test suite to make better use of py.test
  * Clean up code style for PEP 8 compliance

___
Distutils-SIG maillist  - Distutils-SIG@python.org
<mailto:Distutils-SIG@python.org>
https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Maintenance of the "wheel" package

[Alex Grönholm]

Looking at the commit history, the maintenance of "wheel" has more or 
less stalled since 2016. I would like to volunteer myself as a 
co-maintainer of this project, if allowed. Here's my list of things I 
would like to do with it:


 * Move the project from Bitbucket to the PyPA organization on Github
 * Add a setup() keyword (and documentation for it) for license_file
   (currently only usable via the deprecated [metadata] section in
   setup.cfg)
 * Refactor the test suite to make better use of py.test
 * Clean up code style for PEP 8 compliance

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] continuous integration options (was Re: Travis-CI is not open source, except in fact it *is* open source)

[Alex Grönholm]

I don't know if it has been mentioned before, but Travis already 
provides a way to automatically package and upload sdists and wheels to 
PyPI: https://docs.travis-ci.com/user/deployment/pypi/


I've been using it myself in many projects and it has worked quite well. 
Granted, I haven't had to deal with binary wheels yet, but I think 
Travis should be able to handle creation of macOS and manylinux1 wheels.



03.11.2016, 22:07, Nathaniel Smith kirjoitti:


I think we're drifting pretty far off topic here... IIRC the original 
discussion was about whether the travis-ci infrastructure could be 
suborned to provide an sdist->wheel autobuilding service for pypi. 
(Answer: maybe, though it would be pretty awkward, and no one seems to 
be jumping up to make it happen.)



On Nov 3, 2016 11:28 AM, "Barry Warsaw" > wrote:


On Nov 03, 2016, at 11:08 AM, Glyph Lefkowitz wrote:

>I think phrasing this in terms of "perfect" and "good enough"
presents a
>highly misleading framing.  Examined in this fashion, of course
we may
>reluctantly use the "good enough" option, but don't we want the
best option?

What are the criteria for "best"?

I'm not saying don't use Travis, I'm just trying to express that
there are
technical limitations, which is almost definitely true of any CI
infrastructure.  If those limitations don't affect your project,
great, go for
it!

Cheers,
-Barry

___
Distutils-SIG maillist  - Distutils-SIG@python.org

https://mail.python.org/mailman/listinfo/distutils-sig




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] installing bagit-python from the Library of Congress GitHub

[Alex Grönholm]

Pip is a command line tool. Are you trying to use it from the 
interactive Python interpreter instead?



30.10.2016, 18:01, Benjamin Houtman kirjoitti:

Hello,
I'm trying to install bagit-python from the LoC Github 
https://github.com/LibraryOfCongress/bagit-python 

I have Windows 10 and I downloaded Python 2.7.12 and the most recent 
iteration of bag-it. According to the README file I should be able to 
type in "pip install bagit" to install bagit globally, but instead I 
get "SyntaxEror: invalid syntax." It's my understanding that pip is 
automatically part of 2.7.12.
There are no instructions as to what to do if this happens. I should 
be able to get to something called bagit.py, but it's unclear what 
that means as I can't get any further. If you could let me know what I 
can do it would be much appreciated. I'm pretty new to this.

Best,
Ben


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Compatibility Tags and the Stable API

[Alex Grönholm]

Am I understanding this correctly? Even though the limited API is 
supposed to work on all CPython versions supporting the numbered API, I 
cannot do the compiling using a newer version (say 3.5) or it won't work 
with older ones (3.3, 3.4)?



09.09.2016, 14:00, Daniel Holth kirjoitti:


Correct. I have a pull request on wheel that does just that, and the 
next version of pip will be ready for it.



On Fri, Sep 9, 2016, 04:14 Phil Thompson > wrote:


I'm not sure how to interpret PEP 425 when it comes to packages
containing C extension modules that use the limited Python API
(PEP 384).

The Python v3.4 version of the limited API is used so should the
compatibility tag be...

cp34-abi3-*

That would mean that the 'cp34' Python tag would have to be
interpreted as a minimum (rather than exact) requirement.

Grateful for any clarification.

Phil
___
Distutils-SIG maillist  - Distutils-SIG@python.org

https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] What role to eggs still play?

[Alex Grönholm]

20.08.2016, 01:41, Chris Barker - NOAA Federal kirjoitti:

Thanks, I think I'm getting it.

About the toml file... the *-info metadata is a compiled artifact, 
according to all the existing Python packages. Most sdists even have 
a *.egg-info directory.


If it's a compiled artifact, then shouldn't it NOT be in a source dist?

It is inconvenient if you want to know the true dependencies without 
running setup.py.


Isn't that what the toml file is for?
Isn't the toml file for specifying build dependencies, rather then 
runtime dependencies?


I think we are stuck with it, and it's not all bad - if there is some 
useful metadata that doesn't affect the dependency resolver, and it 
is "too static" or too cumbersome to write out by hand, there's a 
place to do that.


I'm trying to imagine what that info would be,  but I suppose there 
could be meta data about a package that is generated at build time -- 
maybe info about how it was built, for instance.


Hmm -- maybe you could put info in there about non-Python shared libs 
it's linked to, for instance.



Eggs are the only way to add a zipped distribution to PYTHONPATH
and have setuptools find the metadata



Can pip find it in a zipped package? Remember,  I don't care to 
support setuptools only features anyway :-)



. Eggs are used by buildout, especially in the unzipped into a
directory form

And they could still be used for their originally designed use as
a plugin format.



If pkg_resources gets spun off, would it support that?


 *.egg-info or EGG-INFO is the predecessor of the *.dist-info
format designed in PEP-376. You get *.egg-info whenever you
install something with setuptools without going through bdist_wheel.



So setuptools_lite would write a  dist_info.

I take it pip looks for both?




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] What role to eggs still play?

[Alex Grönholm]

19.08.2016, 20:25, Daniel Holth kirjoitti:
Eggs are the only way to add a zipped distribution to PYTHONPATH and 
have setuptools find the metadata (the Python code can be found with 
or without the metadata; setuptools does not discover *.dist-info 
inside zip). Eggs are used by buildout, especially in the unzipped 
into a directory form. And they could still be used for their 
originally designed use as a plugin format.
What is not clear to me is who needs this. Is this not what virtualenvs 
are for? Or could wheels not be used for the same purpose? What am I 
missing?


One of the smaller problems with eggs is that everything had the same 
name. *.egg-info or EGG-INFO is the predecessor of the *.dist-info 
format designed in PEP-376. You get *.egg-info whenever you install 
something with setuptools without going through bdist_wheel.


On Fri, Aug 19, 2016 at 12:54 PM Chris Barker > wrote:


Hi all,

starting a new thread, but this is related to the setuptols-_lite
discussion, and the legacy formats discussion. In another thread
Donald had a footnote:


[1] We can tackle egg at a later point, when setuptools either
has support for Wheels
or is less needed.


So I'm wondering -- does anything else (other than setuptools)
depend on eggs in any way? I know why I pip install stuff, I
(always?) get egg-ish things installed:

.egg-info

directories and all that. Honestly, I'm confused -- is that making
an actual egg? or is that name simply a legacy name for package
meta data?

In any case, does pip, or anything else, require it?

For my part, I find it annoying, name aside -- it seems that all
the package meta-data should be there in the package source
already (pypacakge.toml?)

-Chris







-- 


Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov 
___
Distutils-SIG maillist  - Distutils-SIG@python.org

https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Deprecating little used file types/extensions on PyPI?

[Alex Grönholm]

15.08.2016, 22:28, Donald Stufft kirjoitti:

On Aug 15, 2016, at 3:22 PM, Ian Cordasco  wrote:

My only thought is how we convey this message to users. I wonder if it
would be beneficial to have Twine cut a release that warns users when
they are uploading something that will be unsupported, then have
Warehouse/PyPI start returning a 415 (Unsupported media type)
approximately a few weeks/month later.

I wouldn’t be opposed to something like this, though I’m not entirely sure
it’s going to be super useful. I’m not sure if a 415 is the correct response
code since these are being sent as binary blobs as far as HTTP is concerned,
but the Content-Type of the HTTP request will still be correct, but just the
data inside it will be wrong, so I think that’s solidly a 400 error? Not that
it’s super important, that’s an implementation detail :)
I think it'll be more important to let the users know exactly *why* 
their uploads failed (and what to do about it) when the change finally 
takes place.



I'm +1 for restricting the kinds of things people can upload though.


—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Deprecating little used file types/extensions on PyPI?

[Alex Grönholm]

+1 for disallowing uploads of anything else than sdists and wheels.


15.08.2016, 22:09, Donald Stufft kirjoitti:

Hello!

I'd like to restrict what folks can upload to PyPI in an effort to help narrow
the scope down and to enable more a more consistent experience for everyone.

First off, we currently allow people to upload sdist, bdist_wheel, bdist_egg,
bdist_dmg, bdist_dumb, bdist_msi, bdist_rpm, and bdist_wininst. However I think
that we should try to get rid of support for most of these. Just for reference
currently the number of files uploaded for each type of file looks like:

* sdist: 506,585
* bdist_wheel: 81,207
* bdist_egg: 48,282
* bdist_wininst: 14,002
* bdist_dumb: 5,502
* bdist_msi: 497
* bdist_rpm: 464
* bdist_dmg: 45

Out of all of these, I think that we can easily remove bdist_dmg, bdist_rpm,
and bdist_dumb. I also believe that there is a strong case for removing
bdist_msi and bdist_wininst. I also think we should consider removing
bdist_egg.

First of all, when I say "remove", I mean disallow new uploads, but do not
delete the existing files.

Looking at each file type:

I think that bdist_dumb is a pretty easy one to remove. It's format is such
that it's basically not a very useful format to begin with. It takes the full
path to the files and stores them in the repository. So If I install something
to /opt/mycoolproject/lib/python3.5/site-packages/froblib/ then it will have
paths that look like opt/mycoolproject/lib/python3.5/site-packages/froblib/...
I think this is obviously not very useful and not many people have uploaded any
bdist_dumb files at all. They are also problematic because they have the same
file extension as sdists, so pip doesn't really have a great way to
differentiate between bdist_dumbs and sdists except by trying to guess if it
contains one of distutils's adhoc platform tags.

Looking at bdist_rpm, practically nobody has ever used it with a total of 45
files ever uploaded for it. It's not super useful to be able to upload rpms to
PyPI since it's not an RPM repository so people have to manually download them
and then install them manually. It's also a bit weird to have support for RPMs
but not for all of the other package formats that people might want.

Next we have bdist_dmg, bdist_msi, and bdist_winist. I'm lumping these together
because they're all OS specific installers for OSs that don't already have some
sort of repository. This lack of a repository format for them means that random
downloads are already the norm for people using these systems. For these, I
think the usage numbers for bdist_dmg and bdist_msi easily suggest that they
are not very important to continue to support, but it would be weird to
eliminate them without also elminating bdist_wininst. The wininst format has
the most usage out of all of the seldom used formats, however when we look at
the downloads for the last 30 days only 0.42% of the downloads were for wininst
files, so I think that it's pretty safe to remove them. I think in the past,
these were more important due to the lack of real binary packages on Windows,
but I think in 2016 we have wheel, and Wheel is a better solution. If however
we want to keep them, then I think it's pretty safe to remove them from our
/simple/ pages and any future repository pages and modify our mirroring tooling
to stop mirroring them. IOW, to treat them as some sort of "additional upload"
rather than release uploads to PyPI.

Finally, bdist_egg is quite possibly the trickiest one to justify. A fair
number of people still upload eggs, even though we have the wheel format.
However, I think that we should (and generally do) consider eggs to be
deprecated and while we don't want to break existing packages by removing them,
we should block further uploads for them. Looking again at the download numbers
eggs represented only 1.8% of total downloads in the last 30 days while wheels
represented 41% and sdists represented 56%. Further more, I think we should do
this with the expectation that any new repository API will no longer include
egg files in them, and will just be sdists and wheels.

Doing all of this would leave us with:

* The /simple/ repository only having sdists, wheels, and eggs and disallowing
   new eggs to be uploaded.
* The hypothetical repository 2.0 api only having sdists and wheels.
* *MAYBE* Having "related files" that people could upload things like
   Windows/OSX Installers.

On a related but different note, I'd also like to restrict the acceptable
extensions for sdists. Currently distutils allows people to generate .tar,
.tar.gz, .tgz, .tar.bz2, .tbz, .zip, .tar.xz, .tar.Z and possibly others. This
is a bit problematic because each of those types requires a different set of
optional libraries (which may or may not exist depending on Python version) so
you can't really use a lot of them (for example, while .tar.xz might give you
better compression, it's also not usable before Python 3).

Looking at numbers again, the current number of 

Re: [Distutils] Proposal: "Install and save"

[Alex Grönholm]

pip doesn't yet support pyproject.toml does it?

23.07.2016, 17:43, Daniel Holth kirjoitti:
Here is my attempt. The SConstruct (like a Makefile) builds the 
extension. The .toml file gives the static metadata. No need to put 
the two in the same file.


https://bitbucket.org/dholth/cryptacular/src/tip/SConstruct

https://bitbucket.org/dholth/cryptacular/src/tip/pyproject.toml

On Sat, Jul 23, 2016 at 10:11 AM Alex Grönholm 
<alex.gronh...@nextday.fi <mailto:alex.gronh...@nextday.fi>> wrote:


23.07.2016, 17:04, Thomas Kluyver kirjoitti:
> On Sat, Jul 23, 2016, at 02:32 PM, Alex Grönholm wrote:
>> I'm -1 on this because requirements.txt is not really the
standard way
>> to list dependencies.
>> In the Python world, setup.py is the equivalent of Node's
package.json.
>> But as it is
>> Python code, it cannot so easily be programmatically modified.
> Packaging based on declarative metadata:
> http://flit.readthedocs.io/en/latest/
> 
>
> We have a bit of a divide. Specifying dependencies in setup.py (or
> flit.ini, or upcoming pyproject.toml) is the standard for
library and
> tool packages that are intended to be published on PyPI and
installed
> with pip. requirements.txt is generally used for applications
which will
> be distributed or deployed by other means.
>
> As I understand it, in the Javascript world package.json is used
in both
> cases. Is that something Python should try to emulate? Is it hard to
> achieve given the limitations of setup.py that you pointed out?
This topic has been beaten to death. There is no way to cram the
complexities of C extension compilation setup into purely declarative
metadata. Distutils2 tried and failed. Just look at the setup.py files
of some popular projects and imagine all that logic expressed in
declarative metadata.
> Thomas
> ___
> Distutils-SIG maillist  - Distutils-SIG@python.org
<mailto:Distutils-SIG@python.org>
> https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  - Distutils-SIG@python.org
<mailto:Distutils-SIG@python.org>
https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposal: "Install and save"

[Alex Grönholm]

23.07.2016, 17:04, Thomas Kluyver kirjoitti:

On Sat, Jul 23, 2016, at 02:32 PM, Alex Grönholm wrote:

I'm -1 on this because requirements.txt is not really the standard way
to list dependencies.
In the Python world, setup.py is the equivalent of Node's package.json.
But as it is
Python code, it cannot so easily be programmatically modified.

Packaging based on declarative metadata:
http://flit.readthedocs.io/en/latest/


We have a bit of a divide. Specifying dependencies in setup.py (or
flit.ini, or upcoming pyproject.toml) is the standard for library and
tool packages that are intended to be published on PyPI and installed
with pip. requirements.txt is generally used for applications which will
be distributed or deployed by other means.

As I understand it, in the Javascript world package.json is used in both
cases. Is that something Python should try to emulate? Is it hard to
achieve given the limitations of setup.py that you pointed out?
This topic has been beaten to death. There is no way to cram the 
complexities of C extension compilation setup into purely declarative 
metadata. Distutils2 tried and failed. Just look at the setup.py files 
of some popular projects and imagine all that logic expressed in 
declarative metadata.

Thomas
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposal: "Install and save"

[Alex Grönholm]

I'm -1 on this because requirements.txt is not really the standard way 
to list dependencies.
In the Python world, setup.py is the equivalent of Node's package.json. 
But as it is

Python code, it cannot so easily be programmatically modified.

22.07.2016, 20:48, Chris Angelico kirjoitti:

In teaching my students how to use third-party Python modules, I
generally recommend starting every project with "python3 -m venv env",
and then install dependencies into the virtual environment. They then
need a requirements.txt to keep track of them. There are two workflows
for that:

$ pip install flask
$ pip freeze >requirements.txt

or:

$ echo flask >>requirements.txt
$ pip install -r requirements.txt

Over in the JavaScript world, npm has a much tidier way to do things.
I propose adding an equivalent to pip:

$ pip install --save flask

which will do the same as the second option above (or possibly write
it out with a version number as per 'pip freeze' - bikeshed away). As
well as cutting two commands down to one, it's heaps easier in the
multiple installation case - "pip install --save flask sqlalchemy
gunicorn" is much clearer than messing around with creating a
multi-line file; and the 'pip freeze' option always ends up listing
unnecessary dependencies.

Thoughts?

ChrisA
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Missing IPv6 support on pypi.python.org

[Alex Grönholm]

The legacy software might have issues with IPv6 so I doubt this will 
happen before Warehouse replaces Cheeseshop as the new PyPI.


13.07.2016, 15:42, Baptiste Jonglez kirjoitti:

As a follow-up, Fastly now provides an option to enable IPv6 (but this is
not enabled by default).

See: https://github.com/pypa/pypi-legacy/issues/90#issuecomment-231240046

Does pypi plan to participate in this program?  It would be nice!

Thanks,
Baptiste

On Sun, Nov 08, 2015 at 09:13:49PM -0500, Donald Stufft wrote:

I’m pretty sure that PyPI will get IPv6 support as soon as Fastly supports it 
and not any sooner. I know they’re working on making it happen but I don’t 
think they have a public timeline for it yet.

On November 8, 2015 at 4:34:32 PM, Baptiste Jonglez 
(bapti...@bitsofnetworks.org) wrote:

Hi,
  
pypi.python.org is currently not reachable over IPv6.
  
I know this issue was brought up before [1,2]. This is a real issue for

us, because our backend servers are IPv6-only (clients never need to talk
to backend servers, they go through IPv4-enabled HTTP frontends).
  
So, deploying packages from pypi on the IPv6-only servers is currently a

pain. What is the roadmap to add IPv6 support? It seems that Fastly has
already deployed IPv6 [3].
  
Thanks,

Baptiste
  
  
[1] https://mail.python.org/pipermail/distutils-sig/2014-June/024465.html

[2] https://bitbucket.org/pypa/pypi/issues/90/missing-ipv6-connectivity
[3] http://bgp.he.net/AS54113#_prefixes6
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
  

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] matrix: python_versions x supported_plattforms, cross-compiling vs VM

[Alex Grönholm]

This reminds me of something I wanted to ask – how come the "virtualenv" 
tool installs wheel but the built-in "venv" tool does not? The latter is 
the currently recommended method of creating a virtualenv, isn't it?


27.05.2016, 19:40, Donald Stufft kirjoitti:

On May 27, 2016, at 12:37 PM, Paul Moore  wrote:

On 27 May 2016 at 17:28, Chris Barker  wrote:

So I kind of like the idea of making wheels the default for distributing on
PyPi always -- even for pure python modules. And wheels are trivial to build
from pure python packages -- so why not?

It would be *really* nice to have some sort of metadata/flag that said
"this project is pure Python". Normally, what I want is not *quite*
--only-binary, but rather "only binary except for pure Python where
I'm happy to take a source distribution". But AFAIK, there's no way
for pip to know that :-(


The flip side is it should be trivial for pure Python projects to release 
wheels, often requiring them to do nothing different except ensuring `wheel` is 
installed and running ``setup.py sdist bdist_wheel` instead of just `setup.py 
sdist`.

—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] If you want wheel to be successful, provide a build server.

[Alex Grönholm]

Amen to that, but who will pay for it? I imagine a great deal of 
processing power would be required for this.

How do implementors of other languages handle this?

25.05.2016, 10:13, Thomas Güttler kirjoitti:

If you want wheel to be successful, **provide a build server**.

Quoting the author of psutil:

https://github.com/giampaolo/psutil/issues/824#issuecomment-221359292

{{{
On Linux / Unix the only way you have to install psutil right now is 
via source / tarball. I don't want to provide wheels for Linux (or 
other UNIX platforms). I would have to cover all supported python 
versions (7) both 32 and 64 bits, meaning 14 extra packages to compile 
and upload on PYPI on every release. I do that for Windows because 
installing VS is an order of magnitude more difficult than installing 
gcc on Linux/UNIX but again: not willing to do extra work on that 
front (sorry).
What you could do is create a wheel yourself with python setup.py 
build bdist_wheel by using the same python/arch version you have on 
the server, upload it on the server and install it with pip.

}}}

What do you think?

Regards,
  Thomas Güttler



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

10.05.2016, 19:35, Ethan Furman kirjoitti:

On 05/10/2016 08:41 AM, Alex Grönholm wrote:

10.05.2016, 18:26, Ethan Furman kirjoitti:



Please no.  I'd rather do xml than yaml.


Why do you hate it so much? I strongly prefer YAML to anything else I've
seen here.


It's too complicated and error-prone.  If we want buy-in from casual 
packagers then our configuration language needs to be simple to 
understand and simple to get right.  (The amount of leading whitespace 
on a single line changes your data type? Really??  0644 and 0x644 both 
map to 420? and 644 maps to 644?)
Do you actually expect to use these in your project's configuration? No? 
Then why is this a problem for you in this case?


https://docs.saltstack.com/en/latest/topics/troubleshooting/yaml_idiosyncrasies.html 



https://ciaranm.wordpress.com/2009/03/01/yaml-sucks-gems-sucks-syck-sucks/ 



While YAML is more easily readable than XML, with XML you already know 
you're in hell so you tread more carefully.  ;)


If we want to take the good ideas of YAML and make our own thing I'm 
okay with that -- but not YAML itself.


--
~Ethan~
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

10.05.2016, 18:26, Ethan Furman kirjoitti:

On 05/10/2016 08:14 AM, Donald Stufft wrote:

On May 10, 2016, at 11:00 AM, Antoine Pitrou wrote:



(as an aside, if there's the question of forking an existing parser
implementation for better vendorability, forking a YAML parser may be
more useful to third-party folks than forking a TOML parser :-))


I’m seeing what I can come up with (https://github.com/dstufft/yaml) 
though
I don’t know that I feel like actually maintaining whatever it is I 
end up

figuring out there.


Please no.  I'd rather do xml than yaml.

Why do you hate it so much? I strongly prefer YAML to anything else I've 
seen here.

--
~Ethan~

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

10.05.2016, 18:14, Donald Stufft kirjoitti:

On May 10, 2016, at 11:00 AM, Antoine Pitrou  wrote:

(as an aside, if there's the question of forking an existing parser
implementation for better vendorability, forking a YAML parser may be
more useful to third-party folks than forking a TOML parser :-))


I’m seeing what I can come up with (https://github.com/dstufft/yaml) though
I don’t know that I feel like actually maintaining whatever it is I end up
figuring out there.

What exactly are your goals here?


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

10.05.2016, 18:00, Antoine Pitrou kirjoitti:

On Tue, 10 May 2016 10:55:38 -0400
Donald Stufft  wrote:

I think TOML is more usable than ConfigParser and in particular I think that
the adhoc post processing step makes ConfigParser inherently less usable
because it forces a special syntax that is specific to this one file. It also
means that there's no "right" answer for when you have two different
implementations that interpret the same file differently.

That's true. OTOH, the question is how much better it is for users
that it's worthwhile bothering them with a syntax change that will
require (at one point or another) migrating existing files. TOML doesn't
seem that compelling to me in that regard (quite less than YAML, and I'm
not a YAML fan).

(as an aside, if there's the question of forking an existing parser
implementation for better vendorability, forking a YAML parser may be
more useful to third-party folks than forking a TOML parser :-))

Amen to that, and that's exactly what I'd like to do.
What should the parser be capable of to be accepted for this task? What 
are the requirements?

Regards

Antoine.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

This looks very close to what I'd like to have, but then we'd have the 
situation of an uncommon format with no tooling support, won't we? 
Assuming the actual config file is in xaml format.


10.05.2016, 02:56, Ethan Furman kirjoitti:

On 05/06/2016 07:59 PM, Nathaniel Smith wrote:


Here's that one-stop writeup/comparison of all the major configuration
languages that I mentioned:

https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f


Very nice work-up, thanks!


However, you didn't include XML -- which, while absolutely horrid, can 
be quite readable with the appropriate preprocessor, such as xaml [1] 
:


--- 8< whatever.xaml ---
!!! xml1.0
~base

~schema
// optional
~version: 1

~bootstrap
~requirements
// Temporarily commented out 2016-01-10
// magic-build-helper
~setuptools
~version: >= 27
// for the new frobnicate feature
~numpy
~version: >= 1.10
//Pinned until we get a fix for
// @https://github.com/cyberdyne/the-versionator/issues/123
~the-versionator
~version: 0.13

// The owner of pypi name "flit" decides what goes under the
// extension: flit:
// key
~extensions
~flit
~whatever: true
--- 8< -

which ends up as:

--- 8< whatever.xml 





1






= 27



= 1.10



0.13









true


--- 8< -

--
~Ethan~

[1] https://pypi.python.org/pypi/xaml
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

10.05.2016, 12:43, Ionel Cristian Mărieș kirjoitti:


On Tue, May 10, 2016 at 10:38 AM, Alex Grönholm 
<alex.gronh...@nextday.fi <mailto:alex.gronh...@nextday.fi>> wrote:


So far the ONLY objective problems with YAML seems to be the
problematic implementation named PyYAML. If this is really the
case, I'd gladly help build a better one just to prevent TOML from
being chosen for this task. That we're even /considering/ building
something as important as this on an unstable standard is pretty
horrifying to me in itself.


​Just my two cents here: every time, but every every time,​ I have to 
google around about how to create a multi-line string in YAML. There 
are too many ways to write the same thing. And lets not forget those 
damn sexagesimal literals. The complexity of that language is beyond 
repair, it's not a *library* problem. Just look at insanities like 
this 
<http://stackoverflow.com/questions/3790454/in-yaml-how-do-i-break-a-string-over-multiple-lines/21699210#21699210> 
​or this <http://blog.teamlazerbeez.com/2009/04/15/yaml-gotchas/>​ .



I have no problem with any of the examples you linked to.

Thanks,
-- IonelCristian Mărieș, http://blog.ionelmc.ro


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

A few facts:

 * YAML is good enough for Salt, Ansible and numerous other common tools
 * The YAML standard has been stable for many years, unlike TOML which
   still hasn't even reached 1.0
 * YAML has widespread tooling support, unlike TOML

We all agree that JSON is not the solution. No comments, trailing commas 
etc.
TOML isn't much better than ConfigParser in terms of representing nested 
structures.
So far the ONLY objective problems with YAML seems to be the problematic 
implementation named PyYAML. If this is really the case, I'd gladly help 
build a better one just to prevent TOML from being chosen for this task. 
That we're even /considering/ building something as important as this on 
an unstable standard is pretty horrifying to me in itself.


10.05.2016, 06:37, Chris Barker kirjoitti:

Really?

writing Yet Another Markup Language (YAML :-) ) CAN'T be the simplest, 
best option.


> After further consideration, and pytoml's author's comment about the 
spec changing without a version increase, I think we might be better 
off rolling our own.



I like the general simplicity, and would stick with that, but I'd
be a lot more comfortable if we had our spec that was more consistent.


If we're going to do that, then why not the 'simple part of yaml'.

or Python literals. (if I recall, the main reason not to do that was 
that no other language has a lib to read it -- rolling out own does 
not solve that!)


Or just go with JSON -- I'm annoyed by it at times, but it's not SO bad.

(and you can kinda-sorta simulate comments with useless keys :-)

{ "comment": "this is just something i wanted to say here",
...
}

or we could do "JSON with comments" -- not hard to write a tiny 
pre-processor before passing it off to the json lib.


Anyway -- let's avoid the temptation to role your own everything, and 
use something standard!


-CHB

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov 


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

This is fine as long as developer convenience does not suffer. 
Underlying implementations can always be improved, but if we decide on a 
sucky format, we'll have to live with that for a long time.


08.05.2016, 08:07, Chris Barker kirjoitti:
On Sat, May 7, 2016 at 6:04 PM, Brett Cannon > wrote:


For both options I hear "pick a new format", which suggests we
might as well do it from the get-go for clear separation of the
new stuff and just bite the bullet instead of simply postponing a
decision; it isn't like our format options are going to
significantly change between now and later in the year.


Agreed. However, in another thread, I understood you to say that ALL 
we are talking about now is how to specify the build requirements. If 
that's the case, then we might a well as well just go with setup.cfg.


However, I'd rather we were setting the stage for grater things -- in 
which case, let's go with a new config file.


BTW, IIRC, there seemed to a consensus moving toward using a Python 
API, rather than a command line API for the mythical pluggable build 
systems


In which case, we can require python, and could use python literals 
for configuration. With the discussion of PyYaml, I"m thinking more 
and more that something that can be parsed with only the stdlib is a 
good idea.


-CHB

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov 


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

08.05.2016, 02:08, Donald Stufft kirjoitti:


On May 7, 2016, at 7:05 PM, Alex Grönholm <alex.gronh...@nextday.fi 
<mailto:alex.gronh...@nextday.fi>> wrote:


07.05.2016, 17:48, Nick Coghlan kirjoitti:



On 7 May 2016 13:00, "Nathaniel Smith" <n...@pobox.com> wrote:
>
> Here's that one-stop writeup/comparison of all the major configuration
> languages that I mentioned:
>
>https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f

Thanks for that, and "yikes" on the comment handling variations in 
ConfigParser - you can tell I've never even tried to use end-of-line 
comments in INI files, and apparently neither has anyone I've worked 
with :)


For YAML, my main concern isn't quirkiness of the syntax, or code 
quality in PyYAML, it's the ease with which you can expose yourself 
to security problems (even if *pip* loads the config file safely, 
that doesn't mean every other tool will). Since we don't need the 
extra power, the easiest way to reduce the collective attack surface 
is to use a strictly less powerful (but still sufficient) format.


Sounds like a far-fetched hypothetical problem. You're concerned 
about the custom tags provided by PyYAML? Do you happen to know a 
tool that defaults to loading files in unsafe mode?


Yea, pyYAML itself does (yaml.load() does it unsafely, you have to use 
yaml.safe_load()).


I don’t think it’s that big of a deal though, we could easily add a 
thing to PyPI that rejects any YAML file that can’t be parsed in safe 
mode. The bigger deal to me is just that the library to work with it 
is a bit of a bear to use as a dependency.
Sounds like we'd need an alternate implementation of YAML then (I'd love 
to see a "yaml" module in the standard library too, but PyYAML isn't a 
good candidate for that, agreed).


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 
3372 DCFA




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

07.05.2016, 17:48, Nick Coghlan kirjoitti:



On 7 May 2016 13:00, "Nathaniel Smith" > wrote:

>
> Here's that one-stop writeup/comparison of all the major configuration
> languages that I mentioned:
>
> https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f

Thanks for that, and "yikes" on the comment handling variations in 
ConfigParser - you can tell I've never even tried to use end-of-line 
comments in INI files, and apparently neither has anyone I've worked 
with :)


For YAML, my main concern isn't quirkiness of the syntax, or code 
quality in PyYAML, it's the ease with which you can expose yourself to 
security problems (even if *pip* loads the config file safely, that 
doesn't mean every other tool will). Since we don't need the extra 
power, the easiest way to reduce the collective attack surface is to 
use a strictly less powerful (but still sufficient) format.


Sounds like a far-fetched hypothetical problem. You're concerned about 
the custom tags provided by PyYAML? Do you happen to know a tool that 
defaults to loading files in unsafe mode?


For ast.literal_eval, we'd still need to come up with a way to do 
sections, key:value mappings and define rules for comments.


For completeness, I'll note that XML combines even more user 
unfriendly syntax than JSON with similar security risks to YAML.


So with the trade-offs laid out like that (and particularly the 
inconsistent comment and Unicode handling in ConfigParser), I'm 
prompted to favour following Rust in adopting TOML.


Cheers,
Nick.

P.S. I particularly like the idea of using extension sections to 
eventually consolidate other static config into a common file - that 
nicely addresses my concern with config file proliferation, since it 
opens the door to eventually subsuming other files like MANIFEST.in 
and setup.cfg as archiving and build systems are updated


>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> Distutils-SIG maillist  - Distutils-SIG@python.org 


> https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Alex Grönholm]

+1. I don't think the pathological cases of YAML syntax are of any 
concern in this context. Plus it has excellent tooling support, unlike TOML.


07.05.2016, 09:25, Fred Drake kirjoitti:

On May 6, 2016, at 10:59 PM, Nathaniel Smith  wrote:

Here's that one-stop writeup/comparison of all the major configuration
languages that I mentioned:

https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f

Thank you for this!  A very nice summary.

On Fri, May 6, 2016 at 11:14 PM, Donald Stufft  wrote:

While I personally prefer YAML to any of the options on a purely syntax based
level, when you weigh in all the other considerations for this I think that it
makes sense to go with TOML for it.

I expect either YAML or TOML would be acceptable, based on this.  I'll
admit that I'd not heard of TOML before, but it warrants consideration
(possibly for some of my projects as well).

I've spent a fair bit of time using YAML with Ansible lately, as well
as some time looking at RAML, and don't find myself worried about the
syntax.  Every oddness I've run across has been handled with an error
when the content couldn't be parsed correctly, rather than unexpected
behavior resulting from misunderstanding how it would be parsed.  It's
entirely possible I just haven't run across the particular problems
Donald has run across, though.

(The embedded Jinja2 in Ansible playbooks is another matter; let's not
make that mistake.)


   -Fred



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] moving things forward

[Alex Grönholm]

OK, so which setup() arguments do we want to leave out of the static 
metadata?


05.05.2016, 23:53, Daniel Holth kirjoitti:
This is a recurring point of confusion. setup.py is not ever going 
away. In general it is necessary for you to be able to write software 
to build your software, and there is no intention to take that feature 
away.


People repeatedly come to the conclusion that static metadata means 
the entire build is static. It's only the dependencies that need to be 
static to enable better dependency resolution in pip. The build does 
not need to be static.


The proposed feature means you will be able to have a simpler setup.py 
or no setup.py it by using something like flit or pbr that are 
configured with .cfg or .ini. setup.py is not going away.


Static metadata means the list of dependencies, author name, trove 
classifiers is static. Not the build itself.


Enforced staticness of the build script is right out.

On Thu, May 5, 2016 at 4:34 PM Alex Grönholm <alex.gronh...@nextday.fi 
<mailto:alex.gronh...@nextday.fi>> wrote:


I think it would be best to gather a few extreme examples of
setup.py files from real world projects and figure out if they can
be implemented in a declarative fashion. That at least would help
us identify the pain points.

For starters, gevent's setup.py looks like it needs a fair bit of
custom logic:
https://github.com/gevent/gevent/blob/master/setup.py

05.05.2016, 23:30, Chris Barker kirjoitti:

On Wed, May 4, 2016 at 7:45 PM, Nick Coghlan <ncogh...@gmail.com
<mailto:ncogh...@gmail.com>> wrote:

This configuration vs customisation distinction is probably worth
spelling out for folks without a formal software engineering or
computer science background, so:


fair enough -- good to be clear on the terms.

Configuration is different: you're choosing amongst a set of
possibilities that have been constrained in some way, and those
constraints are structurally enforced. 



That's a key point here -- I guess I'm skeptical that we can have
the flexibility we need with a purely configuration-based system
-- we probably don't WANT to constrain the options completely. If
you think about it, while distutils has it's many, many flaws,
what made it possible for it to be as useful as it is, and last
as long as it has because is CAN be customized -- users are NOT
constrained to the built-in functionality.

I suspect the idea of this thread is to keep the API to a build
system constrained -- and let the build systems themselves be as
customizable as the want to be. And I haven't thought it out
carefully, but I have a feeling that we're going to hit a wall
that way .. but maybe not.

Usually that enforcement is
handled by making the configuration declarative - it's in
some passive
format like an ini file or JSON, and if it gets too
repetitive then
you introduce a config generator, rather than making the
format itself
more sophisticated.


OK -- that's more or less my thought -- if it's  python that gets
run, then you've got your config generator built in -- why not?

The big advantage of configuration over customisation is that you
substantially increase the degrees of freedom in how
*consumers* of
that configuration are implemented - no longer do you need a full
Python runtime (or whatever), you just need an ini file
parser, or a
JSON decoder, and then you can look at just the bits you care
about
for your particular use case and ignore the rest.


Sure -- but do we care? this is about python packaging -- is it
too big a burden to say you need python to read the configuration?

-CHB

-- 


Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov <mailto:chris.bar...@noaa.gov>


___
Distutils-SIG maillist  -Distutils-SIG@python.org 
<mailto:Distutils-SIG@python.org>
https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  - Distutils-SIG@python.org
<mailto:Distutils-SIG@python.org>
https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] moving things forward

[Alex Grönholm]

I think it would be best to gather a few extreme examples of setup.py 
files from real world projects and figure out if they can be implemented 
in a declarative fashion. That at least would help us identify the pain 
points.


For starters, gevent's setup.py looks like it needs a fair bit of custom 
logic:

https://github.com/gevent/gevent/blob/master/setup.py

05.05.2016, 23:30, Chris Barker kirjoitti:
On Wed, May 4, 2016 at 7:45 PM, Nick Coghlan > wrote:


This configuration vs customisation distinction is probably worth
spelling out for folks without a formal software engineering or
computer science background, so:


fair enough -- good to be clear on the terms.

Configuration is different: you're choosing amongst a set of
possibilities that have been constrained in some way, and those
constraints are structurally enforced. 



That's a key point here -- I guess I'm skeptical that we can have the 
flexibility we need with a purely configuration-based system -- we 
probably don't WANT to constrain the options completely. If you think 
about it, while distutils has it's many, many flaws, what made it 
possible for it to be as useful as it is, and last as long as it has 
because is CAN be customized -- users are NOT constrained to the 
built-in functionality.


I suspect the idea of this thread is to keep the API to a build system 
constrained -- and let the build systems themselves be as customizable 
as the want to be. And I haven't thought it out carefully, but I have 
a feeling that we're going to hit a wall that way .. but maybe not.


Usually that enforcement is
handled by making the configuration declarative - it's in some passive
format like an ini file or JSON, and if it gets too repetitive then
you introduce a config generator, rather than making the format itself
more sophisticated.


OK -- that's more or less my thought -- if it's  python that gets run, 
then you've got your config generator built in -- why not?


The big advantage of configuration over customisation is that you
substantially increase the degrees of freedom in how *consumers* of
that configuration are implemented - no longer do you need a full
Python runtime (or whatever), you just need an ini file parser, or a
JSON decoder, and then you can look at just the bits you care about
for your particular use case and ignore the rest.


Sure -- but do we care? this is about python packaging -- is it too 
big a burden to say you need python to read the configuration?


-CHB

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov 


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] moving things forward

[Alex Grönholm]

Different files for what? Something not covered by PEP 508?

05.05.2016, 02:23, Ethan Furman kirjoitti:

On 05/04/2016 03:28 PM, Paul Moore wrote:

On 4 May 2016 at 23:11, Chris Barker wrote:




That basically repeats the mistake that was made with setup.py. We
explicitly don't want an executable format for specifying build
configuration.


Executable code or not, we need to be able to specify different files 
depending on the python version.


--
~Ethan~
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] moving things forward

[Alex Grönholm]

I certainly have no problem with Daniel's suggestion (and it would be 
much better than my solution) but would involve yet more standards work. 
Who's going to do that and when?


03.05.2016, 22:39, Donald Stufft kirjoitti:

On May 3, 2016, at 3:35 PM, Daniel Holth  wrote:

Who cares exactly why it doesn't work? We know how to fix it by doing something 
different (put build dependencies in a static file and have them installed by 
pip before running setup.py).


Presumably Alex would like to know why we can’t implement his suggestion.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] moving things forward

[Alex Grönholm]

As I pointed out in my previous post, "extended commands" are not the 
only use case for setup_requires -- upgrading setuptools itself would 
enable support for PEP 508 style conditional requirements. This 
currently does not work because if you have such requirements in your 
setup(), setuptools will fail when parsing those before it even has a 
chance to act on the minimum setuptools requirement specification in 
setup_requires.


03.05.2016, 21:33, Daniel Holth kirjoitti:
What happened is that only a half-dozen setuptools experts (I am not 
one of those six people) know how to write an extended command or 
whatever that would actually be able to take advantage of setup 
requirements as implemented by setuptools. Everyone else wants to 
"import x" at the top of setup.py and pass arguments to the setup() 
function. So it would be better to have the installer make that possible.


On Tue, May 3, 2016 at 2:29 PM Alex Grönholm <alex.gronh...@nextday.fi 
<mailto:alex.gronh...@nextday.fi>> wrote:


No, setuptools parses the install requirements before acting on
setup requirements. That is the source of the problem. If
setuptools only parsed and acted on setup requirements before even
parsing install requirements, this wouldn't be an issue.


03.05.2016, 21:26, Leonardo Rochael Almeida kirjoitti:



    On 3 May 2016 at 15:07, Alex Grönholm <alex.gronh...@nextday.fi
<mailto:alex.gronh...@nextday.fi>> wrote:

Having setuptools process the setup requirements before
parsing install requirements would be a good step forward.
Had that been done before, we could've just added a setup
requirement for a newer setuptools to enable PEP 508
conditional requirements.


Setuptools does process setup requirements before install
requirements. The "chicken and egg" issue with setuptools is
that, most of the time, setup requires are needed to calculate
information that is passed into the `setup()` call itself.

For example information on header files coming from the C api of
`numpy` which is used to build extensions.

This usually means importing code from the packages in "setup
requires" before setuptools has a chance to actually look at it.

A simple fix would be to allow `setup()` keywords to accept
functions as well as direct values and only invoke the functions
when the values are actually needed, but this idea never gained
traction.

Of course, even if this was implemented, it wouldn't help
directly with "setup requiring" a new version of setuptools
itself, unless setuptools detected this situation and reinvoked
setup.py from scratch.

Regards,

Leo


03.05.2016, 21:04, Daniel Holth kirjoitti:

We did separate build from install. Now we just want to be
able to build without [having to emulate] distutils; just
having some dependencies installed before setup.py runs
would also be a great boon.

I'm reading part of this conversation as "a simple
bdist_wheel bug is a reason to do a lot of work
standardizing file formats" which I find unfortunate.

If he is still up for it let Robert implement his own PEP as
the way forward for build system abstraction. The extra PEPs
are just delaying action.

On Tue, May 3, 2016 at 1:11 PM Paul Moore
<p.f.mo...@gmail.com <mailto:p.f.mo...@gmail.com>> wrote:

On 3 May 2016 at 17:47, Donald Stufft <don...@stufft.io
<mailto:don...@stufft.io>> wrote:
> It will likely get decided as part of the build system
PEP, whenever that
> gets picked up again.

Yes, but on 15th March

(https://mail.python.org/pipermail/distutils-sig/2016-March/028457.html)
Robert posted

> Just to set expectations: this whole process seems
stalled to me; I'm
> going to context switch and focus on things that can
move forward.
> Someone please ping me when its relevant to put effort
in again :).

And I think that's right. The whole build system PEP
issue appears
stalled from a lack of someone willing (or with the
time) to make a
call on the approach we take.

As far as I'm aware, the decision remains with Nick.
With the possible
exception of Donald's proposal (which AFAIK never got
formally
published as a PEP) everything that can be said on the
other proposals
has been said, and the remaining differences are ones of
choice of
approach rather than anything affecting capabilities.
(Robert's
message at

Re: [Distutils] moving things forward

[Alex Grönholm]

No, setuptools parses the install requirements before acting on setup 
requirements. That is the source of the problem. If setuptools only 
parsed and acted on setup requirements before even parsing install 
requirements, this wouldn't be an issue.


03.05.2016, 21:26, Leonardo Rochael Almeida kirjoitti:



On 3 May 2016 at 15:07, Alex Grönholm <alex.gronh...@nextday.fi 
<mailto:alex.gronh...@nextday.fi>> wrote:


Having setuptools process the setup requirements before parsing
install requirements would be a good step forward. Had that been
done before, we could've just added a setup requirement for a
newer setuptools to enable PEP 508 conditional requirements.


Setuptools does process setup requirements before install 
requirements. The "chicken and egg" issue with setuptools is that, 
most of the time, setup requires are needed to calculate information 
that is passed into the `setup()` call itself.


For example information on header files coming from the C api of 
`numpy` which is used to build extensions.


This usually means importing code from the packages in "setup 
requires" before setuptools has a chance to actually look at it.


A simple fix would be to allow `setup()` keywords to accept functions 
as well as direct values and only invoke the functions when the values 
are actually needed, but this idea never gained traction.


Of course, even if this was implemented, it wouldn't help directly 
with "setup requiring" a new version of setuptools itself, unless 
setuptools detected this situation and reinvoked setup.py from scratch.


Regards,

Leo


03.05.2016, 21:04, Daniel Holth kirjoitti:

We did separate build from install. Now we just want to be able
to build without [having to emulate] distutils; just having some
dependencies installed before setup.py runs would also be a great
boon.

I'm reading part of this conversation as "a simple bdist_wheel
bug is a reason to do a lot of work standardizing file formats"
which I find unfortunate.

If he is still up for it let Robert implement his own PEP as the
way forward for build system abstraction. The extra PEPs are just
delaying action.

On Tue, May 3, 2016 at 1:11 PM Paul Moore <p.f.mo...@gmail.com
<mailto:p.f.mo...@gmail.com>> wrote:

On 3 May 2016 at 17:47, Donald Stufft <don...@stufft.io
<mailto:don...@stufft.io>> wrote:
> It will likely get decided as part of the build system PEP,
whenever that
> gets picked up again.

Yes, but on 15th March
(https://mail.python.org/pipermail/distutils-sig/2016-March/028457.html)
Robert posted

> Just to set expectations: this whole process seems stalled
to me; I'm
> going to context switch and focus on things that can move
forward.
> Someone please ping me when its relevant to put effort in
again :).

And I think that's right. The whole build system PEP issue
appears
stalled from a lack of someone willing (or with the time) to
make a
call on the approach we take.

As far as I'm aware, the decision remains with Nick. With the
possible
exception of Donald's proposal (which AFAIK never got formally
published as a PEP) everything that can be said on the other
proposals
has been said, and the remaining differences are ones of
choice of
approach rather than anything affecting capabilities. (Robert's
message at
https://mail.python.org/pipermail/distutils-sig/2016-March/028437.html
summarised the state of the 3 proposals at the time).

I think this is something that should be resolved - we don't
appear to
be gaining anything by waiting, and until we have a decision
on the
approach that's being taken, we aren't going to get anyone
writing
code for their preferred option.

Nick - do you have the time to pick this up? Or does it need
someone
to step up as BDFL-delegate? Robert, Nathaniel, do you have
time to
spend on a final round of discussion on this, on the
assumption that
the goal will be a final decision at the end of it? Donald,
do you
have the time and interest to complete and publish your proposal?

Paul
___
Distutils-SIG maillist  - Distutils-SIG@python.org
<mailto:Distutils-SIG@python.org>
https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -Distutils-SIG@python.org 
<mailto:Distutils-SIG@python.org>
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] moving things forward

[Alex Grönholm]

Having setuptools process the setup requirements before parsing install 
requirements would be a good step forward. Had that been done before, we 
could've just added a setup requirement for a newer setuptools to enable 
PEP 508 conditional requirements.


03.05.2016, 21:04, Daniel Holth kirjoitti:
We did separate build from install. Now we just want to be able to 
build without [having to emulate] distutils; just having some 
dependencies installed before setup.py runs would also be a great boon.


I'm reading part of this conversation as "a simple bdist_wheel bug is 
a reason to do a lot of work standardizing file formats" which I find 
unfortunate.


If he is still up for it let Robert implement his own PEP as the way 
forward for build system abstraction. The extra PEPs are just delaying 
action.


On Tue, May 3, 2016 at 1:11 PM Paul Moore > wrote:


On 3 May 2016 at 17:47, Donald Stufft > wrote:
> It will likely get decided as part of the build system PEP,
whenever that
> gets picked up again.

Yes, but on 15th March
(https://mail.python.org/pipermail/distutils-sig/2016-March/028457.html)
Robert posted

> Just to set expectations: this whole process seems stalled to
me; I'm
> going to context switch and focus on things that can move forward.
> Someone please ping me when its relevant to put effort in again :).

And I think that's right. The whole build system PEP issue appears
stalled from a lack of someone willing (or with the time) to make a
call on the approach we take.

As far as I'm aware, the decision remains with Nick. With the possible
exception of Donald's proposal (which AFAIK never got formally
published as a PEP) everything that can be said on the other proposals
has been said, and the remaining differences are ones of choice of
approach rather than anything affecting capabilities. (Robert's
message at
https://mail.python.org/pipermail/distutils-sig/2016-March/028437.html
summarised the state of the 3 proposals at the time).

I think this is something that should be resolved - we don't appear to
be gaining anything by waiting, and until we have a decision on the
approach that's being taken, we aren't going to get anyone writing
code for their preferred option.

Nick - do you have the time to pick this up? Or does it need someone
to step up as BDFL-delegate? Robert, Nathaniel, do you have time to
spend on a final round of discussion on this, on the assumption that
the goal will be a final decision at the end of it? Donald, do you
have the time and interest to complete and publish your proposal?

Paul
___
Distutils-SIG maillist  - Distutils-SIG@python.org

https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Two ways to download python packages - I prefer one

[Alex Grönholm]

You make it sound like there's a plausible alternative to setuptools 
entry points -- is there?


02.05.2016, 10:14, Noah Kantrowitz kirjoitti:

The correct way to do that these days is `pip install -e .` AFAIK. Setuptools 
should be considered an implementation detail of installs at best, not really 
used directly anymore (though entry points are still used by some projects, so 
this isn't really a strict dichotomy).

--Noah


On May 2, 2016, at 12:03 AM, Thomas Güttler  
wrote:

I was told this:


`python setup.py develop` uses urllib2 to download distributions whereas pip 
uses requests

Souce: http://stackoverflow.com/a/36958874/633961

This can create confusing situations and I want to avoid this.

Is there a way to use only **one** way to install python packages?

Do wheels help here?

Or is there a way to use npm for python packages?

Regards,
  Thomas Güttler

--
Thomas Guettler http://www.thomas-guettler.de/
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] wheel including files it shouldn't

[Alex Grönholm]

To what end?

28.04.2016, 21:21, Barry Warsaw kirjoitti:

On Apr 27, 2016, at 10:00 PM, Alex Grönholm wrote:


The sdist should include all the source files, including tests and
documentation. In binary distributions, however, they are just dead
weight. Do you want the full documentation and test suites to be installed
for every single dependency when you deploy your application? I sure don't.

Shouldn't that be left to the package author to decide?

-Barry


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] wheel including files it shouldn't

[Alex Grönholm]

The sdist should include all the source files, including tests and 
documentation. In binary distributions, however, they are just dead 
weight. Do you want the full documentation and test suites to be 
installed for every single dependency when you deploy your application? 
I sure don't.


27.04.2016, 21:40, Ethan Furman kirjoitti:

On 04/27/2016 11:13 AM, Alex Grönholm wrote:


Are you seriously saying that you want your bdists to include tests,
documentation etc.?


However you and I agree or disagree on what should be in a bdist, the 
command I ran should have produced a bdist based on the sdists I just 
created in the same command.



Most developers would not agree with you, including yours truly.


Well, we disagree.  To me, the salient difference between an sdist and 
a bdist is whether binary artifacts are, um, already built.  I 
certianly enjoy having docs (so I know how to use the binaries I just 
installed) and tests (so I can assure myself the binaries work as 
advertised).


If a project is big enough I can see making separate packages for docs 
and/or tests, but mine are small.


And whichever way we decide to do the packaging, the tools should work 
for us, not us for the tools.


--
~Ethan~

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] wheel including files it shouldn't

[Alex Grönholm]

Are you seriously saying that you want your bdists to include tests, 
documentation etc.?

Most developers would not agree with you, including yours truly.

27.04.2016, 21:10, Ethan Furman kirjoitti:

On 04/27/2016 10:52 AM, Donald Stufft wrote:


This isn't really a problem with what you're doing. Rather it's an issue
with the toolchain and and open question whether or not wheels should
conceptually be able to be produced from a checkout, or if they should
only be produced from a sdist. Problems like this are why I advocate the
Checkout -> sdist -> wheel being the only path, but others feel 
differently.


As a simple user, my feelings are that the command I used should have 
generated three equivalent distributions, but did not.  That feels 
like a bug.  :(


Let me rephrase my question:  what command do I use to build the wheel 
from the sdist I just made?  For bonus points:  why can't setup do 
that for me?


--
~Ethan~

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Name arbitration on PyPI

[Alex Grönholm]

Here: https://pypi.python.org/pypi/PIL
It was downloadable until external downloads were disallowed.

19.04.2016, 01:49, Chris Barker - NOAA Federal kirjoitti:



Another high profile example of such a project: PIL.


Was PIL ever on PyPi? Anyway, yup, the solution there was to fork give 
it s new name -- Pillow was born.


CHB



19.04.2016, 00:56, Chris Barker kirjoitti:
On Mon, Apr 18, 2016 at 2:31 PM, Ian Cordasco 
> wrote:


>> 1.� PyYAML is a package that would be de-registered in such a 
scheme.� It

�

> and you don't think ANYONE would be willing to take on the miniscule 
amount
> of work to maintain the name? Plus there would be any number
of other
> schemes for determining whether a project name is abandoned.

I have in fact offered but the author refuses to accept help from
anyone. They're also the author of the C library (libyaml) and
they do
not maintain that either. It's actually quite frustrating as someone
who wants to fix some of the numerous bugs in the library +
improve it
and add support for YAML 1.2 which is years old at this point.


Interesting third case I hadn't considered -- the original author is 
still "active", but not actually maintaining the software or 
accepting help. I don't think there is anything PyPi policy can do 
about that -- too bad.


Time for a fork?�

-CHB


�
--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR � � � � � �(206) 526-6959�� voice
7600 Sand Point Way NE ��(206) 526-6329�� fax
Seattle, WA �98115 � � ��(206) 526-6317�� main reception

chris.bar...@noaa.gov 


___
Distutils-SIG maillist  -Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  - Distutils-SIG@python.org 


https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Name arbitration on PyPI

[Alex Grönholm]

Another high profile example of such a project: PIL.

19.04.2016, 00:56, Chris Barker kirjoitti:
On Mon, Apr 18, 2016 at 2:31 PM, Ian Cordasco 
> wrote:


>> 1.  PyYAML is a package that would be de-registered in such a scheme.  It

> and you don't think ANYONE would be willing to take on the miniscule 
amount
> of work to maintain the name? Plus there would be any number of
other
> schemes for determining whether a project name is abandoned.

I have in fact offered but the author refuses to accept help from
anyone. They're also the author of the C library (libyaml) and they do
not maintain that either. It's actually quite frustrating as someone
who wants to fix some of the numerous bugs in the library + improve it
and add support for YAML 1.2 which is years old at this point.


Interesting third case I hadn't considered -- the original author is 
still "active", but not actually maintaining the software or accepting 
help. I don't think there is anything PyPi policy can do about that -- 
too bad.


Time for a fork?

-CHB


--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov 


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The mypy package

[Alex Grönholm]

I've fallen into this trap as well, so +1 for the takeover.

It might be a good idea to come up with a standardized process for 
taking over old, unmaintained packages.


16.04.2016, 02:29, Guido van Rossum kirjoitti:

Brett suggested I ask the kind folks here.

As you may or may not know, there's an old unmaintained "mypy" package 
on PyPI that attracts a fair amount of downloads from people trying to 
download mypy the type checker. We then get bug reports and have to 
explain in our tracker that they have to use "pip install mypy-lang" 
instead.


Query: 
https://github.com/python/mypy/issues?utf8=%E2%9C%93=is%3Aissue++dbutils+


That mypy package was last updated in 2011, and it's a quite 
forgettable combination of copied open-source packages and a little 
bit of glue code presumably written by the package author. Both Donald 
and myself have approached the owner (zsp...@gmail.com 
) but not received any response. Is there a 
"higher authority" to whom we can appeal this, or are we just stuck 
with this situation?


As Brett wrote part of the problem, though, is the mypy project has 
2244 downloads in the last month which shows it's being used and we 
don't want to end up in an npm/left_pad situation. (But how many of 
those downloads are misguided attempts to install mypy-lang?)


One possibility, if people aren't happy with me or Jukka taking over 
owhership of the old mypy package, would be for someone (not me or 
Jukka) to take ownership of that package just so they can update the 
PyPI home page for that package with a prominent note telling people 
looking for Jukka's type checker to use mypy-lang instead.


--
--Guido van Rossum (python.org/~guido )


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] 'Invalid module name' creating package with setuptools

[Alex Grönholm]

If you get that error, it means you're not actually using python 3 to 
run setup.py.


15.04.2016, 09:52, Luí­s de Sousa via Distutils-SIG kirjoitti:

Hi Marius, thank you for the reply.

If I remove the line # coding=utf8, I get the following error:

$ python setup.py bdist_wheel --universal
  File "setup.py", line 18
SyntaxError: Non-ASCII character '\xc3' in file setup.py on line 18, 
but no encoding declared; see http://www.python.org/peps/pep-0263.html 
for details


I created main functions in each of these scripts and altered the 
entry points bit to:


entry_points={
'console_scripts': [
'scriptA=proj:scriptA:main',
'scriptB=proj:scriptB:main'
],
},

This way I get this error:

$ python setup.py bdist_wheel --universal
error in proj setup command: ("EntryPoint must be in 
'name=module:attrs [extras]' format", 'scriptA=proj:scriptA:main')


The __init__.py file is empty, by the way. Any other hints?

Thank you,

Luís


/Sent from ProtonMail , encrypted email based 
in Switzerland./




 Original Message 
Subject: Re: [Distutils] 'Invalid module name' creating package with 
setuptools

Local Time: April 15, 2016 8:24 AM
UTC Time: April 15, 2016 6:24 AM
From: mar...@gedmin.as
To: distutils-sig@python.org

On Thu, Apr 14, 2016 at 11:28:09AM -0400, Luí­s de Sousa via 
Distutils-SIG wrote:

> I have a project with these contents:
>
> proj
> ├── proj
> │ ├── scriptA.py
> │ ├── scriptB.py
> │ └── __init__.py
> ├── LICENCE
> ├── README.md
> └── setup.py
>
> The setup.py file looks like:
...
> entry_points={
> 'console_scripts': [
> 'scriptA=proj:scriptA',
> 'scriptB=proj:scriptB'

I'm not sure this is right -- an entry point should point to a python
module (not a package) and a function in that module, so something like

'scriptA=proj.scriptA:main',
'scriptB=proj.scriptB:main',

I've never tried to define entry points pointing to functions defined in
the __init__.py of a package, so I don't know if you're allowed to write

scriptX=proj:fn

or if you have to explicitly say

scriptX=proj.__init__:fn

> When I try to build I get the following error:
>
> $ python setup.py bdist_wheel --universal
> error in proj setup command: ('Invalid module name', 'proj')

(That is not a good error message indeed.)

Marius Gedminas
--
This loads a GDT entry into the "Task Register": that entry points to a
structure called the Task State Segment. Some comments scattered 
though the
kernel code indicate that this used for task switching in ages past, 
along

with blood sacrifice and astrology.
-- lguest source code
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pip install -e foo (without Repo URL)

[Alex Grönholm]

To my knowledge, metadata 2.0 does not have any reliable means to 
specify a repository URL.

So how do you propose to retrieve this information?

15.04.2016, 13:05, Thomas Güttler kirjoitti:



Am 14.04.2016 um 14:31 schrieb Ian Cordasco:


On Apr 14, 2016 2:20 AM, "Thomas Güttler" 
> 
wrote:

 >
 > I think it would be very cool if you could install a package editable
 > without the repo-url.
 >
 > The default repo-url can be defined in the meta-data of the package.
 >
 > Background: I came across this becaus saltstack prefers the branch 
"develop", but

 > most other repos use the branch "master".
 >
 > Yes, this is no big problem. I figured out the right repo url soon.
 >
 > Next use case: you use software third-party-foo-lib in your project.
 > Up to now you use it as package. You find a bug and want to fix it.
 > Wouldn't it be great if you could just type "pip install -e 
third-party-foo-lib"

 > and now you are read to create pull requests?

I don't understand how this makes me ready to submit pull requests. 
Can you explain a little more?



This:

 pip install -e third-party-foo-lib

should be the same as:

 pip install -e git+https:///@mydevbranch#egg=third-party-foo-lib

And the information needed to build the URL (in above example 
git+https:///@mydevbranch)

should be provided by setup.py.

Result: you have an editable checkout.

You are right, pull requests most often need a fork first. This is far 
out of scope.


But for custom (non public) projects hosted in a corporate lan, you 
get a ready-to-push

checkout.

Is this enough explanation? Please ask if you have further question.

Regards,
  Thomas Güttler





___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] 'Invalid module name' creating package with setuptools

[Alex Grönholm]

That setup.py worked fine here, so I'm guessing this is not the actual 
setup.py you're running. What is the exact value of the "name" argument 
in your real setup.py?


BTW, source encoding is utf-8 by default on Python 3 so the "# 
coding=utf8" is unnecessary.


14.04.2016, 18:28, Luí­s de Sousa via Distutils-SIG kirjoitti:

Hi everyone,

I have a project with these contents:

proj
├── proj
│   ├── scriptA.py
│   ├── scriptB.py
│   └── __init__.py
├── LICENCE
├── README.md
└── setup.py

The setup.py file looks like:

#!/usr/bin/python3
# coding=utf8

from setuptools import setup

setup(
name = "proj",
version = "0.2",
packages = ['proj'],
install_requires=[],
entry_points={
'console_scripts': [
'scriptA=proj:scriptA',
'scriptB=proj:scriptB'
],
},

# metadata for upload to PyPI
author = "Luís",
author_email = "l...@mail.com ",
description = "Some package",
license = "EUPL v1.1",
keywords = "pip package",
url = "https://some.place.com;,   # project home page, if any
classifiers = [
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Development Status :: 4 - Beta",
"Environment :: Console",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: European Union Public Licence 1.1 
(EUPL 1.1)",

"Operating System :: OS Independent",
"Topic :: Scientific/Engineering :: GIS"
],
)

When I try to build I get the following error:

$ python setup.py bdist_wheel --universal
error in proj setup command: ('Invalid module name', 'proj')

What am I doing wrong? Thank you,

Luís

/Sent from ProtonMail , encrypted email based 
in Switzerland./



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to deprecate a python package

[Alex Grönholm]

You can already modify the description and add the "Development Status 
:: 7 - Inactive" classifier. It would be really nice to filter these out 
of search results though.


06.04.2016, 02:32, Greg Ewing kirjoitti:

Geoffrey Spear wrote:
I don't have an answer to your actual question, but I'd not advocate 
people removing packages; we don't want a npm situation here. :(


Perhaps there should be a way of marking a package as
deprecated on PyPI, so that it shows a big red warning
flag?



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to deprecate a python package

[Alex Grönholm]

Implementing this on Warehouse and pip would have the added benefit of 
warning users who have a specific version pinned. As for pip letting 
stderr messages through, that'd be irrelevant if pip had direct support 
for this.


06.04.2016, 01:47, Alexander Walters kirjoitti:
I am not 100% sure if pip will let stderr messages though, but i THINK 
it does.  Warnings on import will work regardless.


Honestly, I don't care if its marginally easier (and it would only be 
marginally easier) to mark a package deprecated by flipping a bit on 
the site - it's the last thing they will ever do with the package.


On 4/5/2016 18:40, Alex Grönholm wrote:
Wouldn't my suggestion or Glyph's be easier for the maintainers? That 
way they wouldn't even have to make a new release, just modify a 
setting on the package settings page on PyPI.

Also, are you going you see the warning if it's emitted on installation?

06.04.2016, 01:37, Alexander Walters kirjoitti:



On 4/5/2016 18:34, Glyph wrote:
Perhaps, before anyone tries to make pip doing something mechanical 
about deprecations, we should just have the website itself do this 
sort of redirect? Removing the download would be drastic; giving 
people an interstitial that says "This package is no longer 
maintained, please use $X instead" would be very informative.


-glyph


I don't remember the last time I used the pypi website.  I use pypi 
every day.  I don't know if I am weird, but redirecting web views 
would do nothing for me.  Redirecting packages is pure evil.


I really think the best course of action is for the maintainer to 
release a final version of the package that warns on install and 
import.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to deprecate a python package

[Alex Grönholm]

Wouldn't my suggestion or Glyph's be easier for the maintainers? That 
way they wouldn't even have to make a new release, just modify a setting 
on the package settings page on PyPI.

Also, are you going you see the warning if it's emitted on installation?

06.04.2016, 01:37, Alexander Walters kirjoitti:



On 4/5/2016 18:34, Glyph wrote:
Perhaps, before anyone tries to make pip doing something mechanical 
about deprecations, we should just have the website itself do this 
sort of redirect?  Removing the download would be drastic; giving 
people an interstitial that says "This package is no longer 
maintained, please use $X instead" would be very informative.


-glyph


I don't remember the last time I used the pypi website.  I use pypi 
every day.  I don't know if I am weird, but redirecting web views 
would do nothing for me.  Redirecting packages is pure evil.


I really think the best course of action is for the maintainer to 
release a final version of the package that warns on install and import.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to deprecate a python package

[Alex Grönholm]

You make a valid point. This made me recall something -- there is a 
classifier "Development Status :: 7 - Inactive". As a quick fix, pip 
could be modified to emit a warning when a distribution containing this 
classifier is installed. But the problem seems more social than 
technical. The author of the package the OP referred to has not chosen 
to add any status classifier or any kind of warning in the description 
that would notify the user of its deprecated status.


06.04.2016, 01:05, Tres Seaver kirjoitti:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/05/2016 04:17 PM, Alex Grönholm wrote:


I think an ideal solution would be to add a feature to Warehouse that
would "redirect" any downloads of a library to another. Though I'm not
saying it would be simple.

Such a feature would be doing a huge disservice:  repeatability *matters*
for package consumers.  Unless an already-uploaded package is known to
contain malware, or the author is under force majeur compulsion
(governmental / legal injunction), removing a distribution is much worse
than giving the humans involved flexibility to deal with an issue.


Tres.
- -- 
===

Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBAgAGBQJXBDaKAAoJEPKpaDSJE9HYGYsP/AtJGhNFXPXjtUlTVoDHw6oz
ohzb5js31Dps86V/CQELl4cxFhfPMpPCPxcfA9z/E9B4Wk3HaFTPxOUunZKrbUJA
2MguOnsYOjeWCBSlBEOdjCSTHiYse07NRMV4NN+b0mfdnj6VvTk17uY9UW96GTiN
xigRgysRgN71JnE41ZNL+4qKjvCL/6dYFrga21rdvOGnZrTNUBwP8mbbACrdz9lh
jeOSVkbWMqKazAXIZB3y7KaByIHIfes5fguBnsjqpgdL9c9r8WsE5nhBCdlkUm8N
NAiNEpTy+5G5o0NhGz/4AXFtamkVLTGSZhWcUprHOtJUgjzer+b0WWijFcBtcQGY
Ugbijhotlbx+zI/QPqArqDemU++UhGr+oiI839KfyzV3viZ4MEr8jC/BchM88Jmn
8lR3Fyv25Tc2bDTC96hv8A5zcwM08i5FYHlPhW2a96xue5Vl9wZ6rmpRUTtqhErJ
vwPu/Yps/l1nXzRXPc8NcHTH8daDVIgaNNp8EeDHV+vYJgy066zzzSQ4dTJddWbZ
mcf6aFQDP50vrloZ81GaeByUJ1xlcVfyODdvpKj350YlqPqyv7y7uMJv026csRax
l/3DyhChbqzU/be9f6xaGL+KzJU3Xt2L0XY/annNkBWrsbRKISpiiGc+21rNo23P
EB9Sax3Uoa47h5GWQWH5
=CblP
-END PGP SIGNATURE-

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to deprecate a python package

[Alex Grönholm]

I think an ideal solution would be to add a feature to Warehouse that 
would "redirect" any downloads of a library to another. Though I'm not 
saying it would be simple.


05.04.2016, 22:59, Ionel Cristian Mărieș kirjoitti:
What's wrong with a new release that just depends on replacement 
(assuming there's identical API)? This might be of help: 
https://pypi.python.org/pypi/pypi-alias


If there are API changes or other breakages then maybe a release with 
a wrapper that emits warnings would be better and generate less 
surprise/anger.



Thanks,
-- IonelCristian Mărieș, http://blog.ionelmc.ro

On Tue, Apr 5, 2016 at 9:46 PM, Thomas Güttler 
> 
wrote:


I wasted some time because I used a deprecated python package.

I asked the maintainer to remove it, and he looked at the usage
statistics: I still gets
downloaded.

What is the official way to deprecate a python package?

Related discussion:


https://github.com/riklaunim/django-ckeditor/issues/60#issuecomment-205021579

Regards,
  Thomas Güttler

--
http://www.thomas-guettler.de/
___
Distutils-SIG maillist  - Distutils-SIG@python.org

https://mail.python.org/mailman/listinfo/distutils-sig




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Thank you for the ability to do `pip install git+https://...`

[Alex Grönholm]

I'm sorry if I offended anyone. I was just trying to point out that in 
the case of Github (or any other service that provides automated tarball 
generation) it's better to install from those rather than using the VCS 
integration plugins. Oh, and for the record, I just tested -- it does a 
deep clone at this time, which would be responsible for the slowness 
compared to installing from a tarball.


01.04.2016, 17:03, Nick Coghlan kirjoitti:

On 28 March 2016 at 22:54, Alex Grönholm <alex.gronh...@nextday.fi> wrote:

You could always point pip to the automatically generated zip
(https://github.com/nchammas/flintrock/archive/master.zip). That way you
won't have to wait for git to pointlessly download your entire version
history when you just want to install something.

I'd assume pip is doing shallow clones when installing from version
control, and if that currently isn't the case due to lack of time for
implementation and testing, it should be a relatively straightforward
PR for someone to submit as a performance optimisation.

It's also the case that the automatically generated zip archives are a
GitHub specific feature rather than a general property of version
control systems, so at least some folks are going to avoid depending
on that service specific feature and instead favour the
host-independent approach of using the relevant version control
protocol.

Regards,
Nick.

P.S. More generally: is a thread where someone is saying "Thanks for
this feature!" *really* the best place to be telling them "you
shouldn't be using that feature"? It's rare enough for maintainers of
any open source project to hear a heartfelt "thank you" that it's at
least worth considering taking other replies off list :)



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Thank you for the ability to do `pip install git+https://...`

[Alex Grönholm]

You could always point pip to the automatically generated zip 
(https://github.com/nchammas/flintrock/archive/master.zip). That way you 
won't have to wait for git to pointlessly download your entire version 
history when you just want to install something.


28.03.2016, 04:11, Nicholas Chammas kirjoitti:


Dunno how old/new this feature is, or what people did before it 
existed, but I just wanted to thank the people who thought of and 
built the ability to do installs from |git+https|.


It lets me offer the following to my users when they want the 
“bleeding edge” 
 version of 
my project:


|pip install git+https://github.com/nchammas/flintrock |

I also use this capability to install and test contributors’ branches 
when they open PRs against my project. For example:


|pip install git+https://github.com/contributor/flintrock@branch |

It’s a great feature and makes my work a bit easier. Thank you for 
building it.


I’m still waiting for when I can give the PyPA some money 
 for all the good and 
sorely needed work that y’all do…


Anyway, keep it up.

Nick

​


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Wheels and Python implementation markers

[Alex Grönholm]

What is the "correct" way to specify (for building wheels) in setup.py 
that a dependency should only be installed on CPython? If so, how should 
I go about it? Nothing I've tried seems to work right. The 
python_implementation marker worked up until packaging 16.5 and then 
platform_python_implementation does not work with older versions of pip.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] question about future pip capabilities

[Alex Grönholm]

10.12.2009 19:18, Darren Dale kirjoitti:

Hello,

I apologize for cross-posting. I am working on improving how I
distribute a few python packages, and in the process I have run into
some questions that involve easy_install, setuptools/distribute/, pip,
and the relationships between them.

I was originally planning on using easy_install to take advantage of
extras (easy_install foo[bar]) and automatically install dependencies,
but I just discovered that easy_install does not support
post-installation routines, which I need in order to add entries to
the windows start menu. Pip will, I think, support post_installation
routines if I implement a custom install command and pass it to
setup(). But pip does not yet support extras or installation from
windows binary installers, and does not support installation from
eggs. The pip documentation states:

* It cannot install from eggs. It only installs from source. (In the
future it would be good if it could install binaries from Windows .exe
or .msi -- binary install on other platforms is not a priority.)
* It doesn't understand Setuptools extras (like package[test]). This
should be added eventually.
* It is incompatible with some packages that customize distutils or
setuptools in their setup.py files.
* Maybe it doesn't work on Windows. At least, the author doesn't test
on Windows often.

Questions concerning pip:

* Is there a roadmap or timeline concerning installing from windows
exe or msi files? I didn't see anything at bitbucket.
* Is there a roadmap or timeline concerning extras? I didn't see
anything at bitbucket.
* Could anyone please expand on the comment that pip is incompatible
with some packages that customize distutils or setuptools in their
setup.py files, and that it maybe doesn't work on windows?

Questions concerning Distribute:

* The documentation states that easy_install will be deprecated, and
that we should use pip. Isn't this premature, given the current
discrepancy between the capabilities and platform support of pip and
easy_install?
   
The next major distribute release (the one that nixes easy_install) is 
relatively far in the future, and we (well, I at least) expect pip to 
have matured enough by then.

Questions for pip and Distribute:

* How closely coupled are these projects?
* I have a bit of time after hours, how can I help?

Darren
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
   


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Deleting build directory

[Alex Grönholm]

Tarek Ziadé kirjoitti:

On Sat, Nov 14, 2009 at 5:06 AM, Ben Finney ben+pyt...@benfinney.id.au wrote:
  

cool-RR cool...@cool-rr.com writes:



What I really want is never having to worry about the build directory
being around after doing any actions with `setup.py`. Do you have any
other suggestion?
  

If you have determined that ‘setup.py install clean’ does what you want,
then you can create a user-specific ‘setup.cfg’ with this section:

   [aliases]
   install = install clean




Semi-related:
I was wondering if a pre/post install hook to the install command
could be useful.
  
Distribute itself uses something like that, doesn't it? Having 
standardized pre/post-install hooks might be worth considering.

Tarek
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] People want CPAN :-)

[Alex Grönholm]

Andreas Jung kirjoitti:

Am 06.11.09 18:53, schrieb Guido van Rossum:
  

I just found this comment on my blog. People have told me this in
person too, so I believe it is real pain (even if the solution may be
elusive and the suggested solutions may not work). But I don't know
how to improve the world. Is the work on distutils-sig going to be
enough? Or do we need some other kind of work in addition? Do we need
more than PyPI?


My 2 cents after reading and ignoring the whole thread:

- PyPI provides a good functionality so far

What is annoying about PyPI:

 - some package maintainers have a certain ignorance
   and arrogance by misusing PyPI

   - for uploading packages without or broken metadata
   - for uploading packages of doubtful quality
   - for uploading packages to PyPI as a replacement
 for a private egg server

 - supports too much different versioning schemas. Both
   schema supported by setuptools and the one proposed
   by Tarek in some PEP are totally over-engineered.
   A simple and *enforced* versioning schema is what
   I want to see.

 - no more external hosting of packages. If people
   want their packages listed on Pypi, they should
   be required to upload their packages on PyPI
   (no more issues with non-available external server,
   no more issues with mirroring external servers,
   no more issues with wrong download URLs within
   package metadata)

 - better checks on uploaded packages. A source code
   release should be made using the 'sdist' command.
   We don't need source eggs of a package for
   Python 2.4-2.6 containing Python source code
   only.
  

+5

The solution for a better PyPI:

 - more checks, more restrictions

 - every package maintainer uploading something to PyPI
   should have a certain attitude that PyPI is a public
   resource where the content should met certain
   quality criteria and where each package has
   a certain responsibility to Python community.

  

+2

Andreas

  
___

Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.6c10 released

[Alex Grönholm]

P.J. Eby kirjoitti:

The long-overdue setuptools 0.6c10 update is now available on PyPI, at:

   http://pypi.python.org/setuptools/

Major updates and fixes include:

* Support for SVN 1.6 and Python 2.6
* Fix for the Python 2.6.3 build_ext API change
* Support for the most recent Sourceforge download link insanity
* Fix for Vista UAC errors running easy_install.exe or other
  installer-looking executables
* Fix for errors launching 64-bit Windows Python
According to the notice on the setuptools bug tracker, you did this by 
avoiding the use of LoadLibraryEx().
I don't see any changes in launcher.c in setuptools SVN yet, did you 
forget to commit? Or am I looking in the wrong place?

I'd like to see your solution.

* Stop crashing on certain types of HTTP error
* Stop re-trying URLs that already failed retrieval once
* Fixes for various dependency management problems such as looping
  builds, re-downloading packages already present on sys.path (but not
  in a registered site directory), and randomly preferring local -f
  packages over local installed packages
* Prevent lots of spurious already imported from another path warnings
  (e.g. when pkg_resources is imported late)
* Ensure C libraries (as opposed to extensions) are also built when
  doing bdist_egg

Other changes:

* Misc. documentation fixes
* Improved Jython support
* Fewer warnings under Python 2.6+
* Warn when 'packages' uses paths instead of package names (because it
  causes other problems, like spurious already imported warnings)
* Stop using /usr/bin/sw_vers on Mac OS (replaced w/'platform' module
  calls)

You can install the updated version using easy_install (or pip!), 
asking for setuptools==0.6c10.


(Note for users of Distribute: Distribute and setuptools use the same 
package name 'setuptools', and thus cannot both be present on the same 
sys.path (e.g. in the same virtualenv).  If you wish to switch a given 
environment from Distribute to setuptools or vice versa, you will need 
to completely uninstall one before installing the other.  If you 
currently have Distribute installed, please follow Distribute's 
uninstall instructions if you wish to reinstall setuptools.)


Please report any problems with setuptools to the setuptools bug 
tracker at:


  http://bugs.python.org/setuptools/

For faster response to questions, please use the distutils-sig mailing 
list, rather than the tracker.  Setuptools documentation can be found 
via links at 
http://pypi.python.org/pypi/setuptools#using-setuptools-and-easyinstall 
-- and the pages on the PEAK wiki now load much, much faster than they 
did a few months ago.  (They're static cached pages now, rather than 
dynamically generated, unless you're actually logged into the wiki.)


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] distribute equivalent of setup.py develop?

[Alex Grönholm]

Andreas Klöckner kirjoitti:

Hi there,

I've recently tried distribute, and much to my dismay found that it doesn't 
appear to contain an equivalent of setup.py develop. Is such a thing 
planned, or, even better, available now?


  

You must be mistaken. Distribute definitely has such a command.

Thanks,
Andreas
  



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] distribute equivalent of setup.py develop?

[Alex Grönholm]

Andreas Klöckner kirjoitti:

On Donnerstag 15 Oktober 2009, Andreas Klöckner wrote:
  

Hi there,

I've recently tried distribute, and much to my dismay found that it doesn't
appear to contain an equivalent of setup.py develop. Is such a thing
planned, or, even better, available now?



Ah, never mind. I had screwed up my install somehow--not sure. In any case, it 
seems all too easy to do that.


As a plea to everyone on this list: please keep in mind that what you decide 
affects large numbers of package authors and an even larger number of users. 
Please give us the packaging stability we need. The well-being of the Python 
ecosystem is at your mercy.


  

We are well aware of that, thank you :P

Be excellent to each other--and to your user community.

Thanks for all your work,
Andreas
  



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.6c10 release imminent; please test

[Alex Grönholm]

sstein...@gmail.com kirjoitti:


On Oct 12, 2009, at 4:25 PM, P.J. Eby wrote:


But before you do that, be sure to uninstall Distribute completely


Damn if I understand this...such a long time waiting for all these bug 
fixes...so little action, so much angst...


Then that all that effort going into the Distribute fork, so much 
effort expended carefully working around so that Distribute would be a 
bug-fixed, drop-in replacement for the orphaned setuptools.


Then suddenly out of the blue, everything is fixed in setuptools 06c10 
and we must uninstall Distribute completely get these fixes.

Makes sense because you can't have two setuptools packages in sys.path.



If the changes you made really were superior in any way, I trust that 
Tarek and co. are smart, and egoless enough to see that and will make 
those changes part of Distribute.

One such fix has already made it to distribute's repo.



Too little, too late, no thanks, I'll just be sticking with Distribute 
from now on.
Several developers and an open development process vs a lone coder with 
a closed codebase? That's not really a choice at all...


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata

[Alex Grönholm]

Ronald Oussoren kirjoitti:


On 11 Oct, 2009, at 16:27, Lennart Regebro wrote:


2009/10/11 Ronald Oussoren ronaldousso...@mac.com:
What about packages that are installed as a dependency of some other 
package

and then used in user scripts without an explict depency on them?

That is, I install SuperWebFramework==1.0 which happens to depend on
peak-rules. I later start using peak-rules in my own simple scripts 
(without
a setup.py or other explicit dependency information), and yet later 
decide
to uninstall SuperWebFramework.  If I understand the proposal 
correctly

the uninstallation of SuperWebFrameWork would break my scripts.


Yes of course.


IMHO that is a bad experience for the user, because it is very 
unintuitive that I have to explicitly install something that's already 
installed to ensure that it doesn't go away in the future. This might 
work in a serious development environment, but can end up to be very 
annoying and confusing for casual users.


The dpkg package manager (Debian) does not uninstall orphaned packages 
straight away, but notifies the user and allows them to clean up any 
orphans. Perhaps a similar approach would work here?

Ronald


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Packaging Distribute

[Alex Grönholm]

Chris Withers kirjoitti:

Tarek Ziadé wrote:

1- we ship 0.7 under a new name - e.g. like distribute2


You could even take the opportunity to rename it completely to 
something that makes sense ;-)


While I agree that the name was badly chosen (for reasons you outline 
below), a name change at this point would introduce even more confusion.
To re-iterate my argument: Distribute doesn't distribute anything. If 
anything was to be called Distribute, it'd be the PyPI software. 
Calling it Installer or Pacman might make more sense...


Chris


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata

[Alex Grönholm]

Carl Meyer kirjoitti:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Toshio Kuratomi wrote:
  

I would say REQUESTED due to my arguments for not recording
installed-as-package-dependency.



REQUESTED is fine, but I don't understand how the arguments apply, given
that I'm not proposing to record information like _which_ package it was
a dependency of. The same single bit (literally) of information is
tracked either way, it's just a question of whether the presence or
absence of a file signifies that bit.
  

And what if the package is a dependency for multiple packages?
Let's say we have packages A and B which are installed separately, in 
that order, and both depend on C.
C gets installed with information that it was required by A. Now if A is 
uninstalled, won't C also get uninstalled?

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFKzzeSFRcxmeyPUXIRAo0iAJ9QdQzxVUhTBO+rEGgCKmFg9zH7EwCcCfPc
lWgoZVc+Z+H2OcIw1V/wNLI=
=k12Y
-END PGP SIGNATURE-
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] the virtualenv-distribute mess

[Alex Grönholm]

kiorky kirjoitti:

Tarek Ziadé a écrit :
  

On Fri, Oct 9, 2009 at 2:43 PM, kiorky kio...@cryptelium.net wrote:


Hi tarek,

Tarek Ziadé a écrit :

  

The *whole* point of Distribute 0.6.x is to be backward compatible, meaning
that if virtualenv switch to it, you will not even notice it.


Living in my 0.6.x snail sandbox is not a solution.
As it seems that Distribute 0.7 won't for a long time.

setuptools based packages will be able to be installed via the distribute 0.6
branch but not compatible with distribute based stuff. Note that new things
will eventually be packaged with the new good way todo, aka with 0.7. There is
a great risk that they can't live together aside. NOGO
  

Why they can't ?



As i understood all those readings, packages for 0.6 and 0.7 will be installable
with the appropriate distribute version, thus side by side, but for me, they may
be incompatibles together.

  

0.7 packages wont be compatibles with setuptools installation/namespaces, so it
will be impossible to install a lot of setuptools based packages aside with
new stuff in with this way too. NOGO too.
  

Why will it be impossible ?



pep-0382 is not equivalent to setuptools's one for example. Can i have been
certified i will not have breakages when trying to import a setuptools based
namespace package from a 0.7 sharing the same namespace?

  

[...]


I appreciate what you folks are doing with the distribute sphere, i have not
that much problems with it, but i do not support that it breaks very badly the
retro compatibilty for all things already packaged today, today tomorrow or in
one year.
  

Again, you will be able to use 0.6 and 0.7 together. or 0.6 alone, or
0.7 alone.

Nothing will be broken in a distribution that uses 0.6.

0.6 stays maintained.



As i said ealier, there will be incompatibilities at some point.
And also, to use them together, what a hell. For package A i need 0.6 (hard
requirement), for package B i need 0.7 (hard requirement), for C i need 0.6. C
depend on A which depends on B. I also have no sort of control over the
maintenance of those products, think that the authors are dead.
So, i ll have to manually install B for A to fulfill its requirements then C
will install. Deployments will be simple :)
  
I fail to follow this logic. If you install C, then both 0.6 and 0.7 
will be installed as auto-dependencies (in case we decide to rename it 
to distribute2 or something), or just 0.7 (if we decide to provide a fat 
package).
  

Tarek



  



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[Alex Grönholm]

P.J. Eby kirjoitti:

At 11:53 AM 10/5/2009 +0200, Lennart Regebro wrote:

2009/10/5 Jeff Rush j...@taupro.com:
 Very unfortunate, as in, it should NOT have happened.  And 
*especially*

 without any announcement on python.org or mention on the
 python-committers list of something this major.

Well this major... It's a bug fix that breaks a setuptools 
monkey-patch.


Subclassing distutils commands != monkeypatching.  If, say, numpy's 
distutils extensions subclass build_ext and override that method, they 
could have had the same problem.  Same for anybody else's distutils 
extensions.


Setuptools subclasses distutils commands and then replaces the original 
classes with its own.

Example from setuptools/__init__.py:

import distutils.core
distutils.core.Command = Command

Isn't this exactly what monkey patching means?

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[Alex Grönholm]

Jeremy Sanders kirjoitti:

K. Richard Pixley wrote:

  

Ronald Oussoren wrote:


For beginners this issue is a showstopper that they cannot resolve
without help.
  
  

I'm a relative beginner to distutils/setuptools/distribute, but a long
time configuration/build/packaging professional.  You're mistaken if you
think that any of these technologies are suitable for beginners.  The
state of python package distribution resembles the state of linux
packages circa 1995, except that it isn't very well documented at all.



As a general question, is there any planned project to improve the state of 
distutils or replace it? It appears to be one of the weakest parts of the 
Python system and needs replacing with something much cleaner, better 
documented and more powerful.


Even making something like cmake the standard would help a lot.

  
There is a lack of consensus regarding how exactly they should work. If 
we are having this much trouble deciding how a third party tool should 
work, it is certainly not going to be merged into distutils until those 
issues have been resolved. Distutils is what houses (or should) the 
parts we all agree on. That said, I think that plenty of 
setuptools/distribute functionality should be moved to distutils (after 
the code has been cleaned up and the proper unit tests introduced).

Jeremy

  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[Alex Grönholm]

Bill Janssen kirjoitti:

Zooko Wilcox-O'Hearn zo...@zooko.com wrote:

  

I'm struggling to articulate something here.  When the maintainer of
the stable branch of a platform that I rely on says The fact that
upgrading to our recent stable release will break this critical
functionality is so-and-so's fault, not ours. this reduces my
confidence in that maintainer.  Not because he's wrong!  Maybe it
*is* so-and-so's fault.  But what I'm looking for in the maintainer
of a stable platform is someone who says Maybe this wasn't our
fault, but here are the steps we're taking to get you back on your
feet as soon as possible..



Zooko, I've struggled with this over the last year, in integrating a
dozen fairly complicated third-party Python extensions for my system.

I've come to the conclusion that the problem is setuptools, and I'm
trying hard to remove it entirely from my system.

I have no problem with the .egg format or the basic idea, or even the
implementation, which I think is pretty nice.  It's the structure of the
setuptools project that gives me pause.  There seems to be one
developer, and he seems to be too busy to fix the well-known bugs (like
having easy_install ruin the sys.path settings by putting stuff on it in
the wrong place -- is that one fixed yet?).

In addition, I think the mere existence of setuptools stifles progress
on distutils, which is where all the clever tricks of setuptools should
properly appear.  This would let the whole community of Python developers
work on the codebase.

I'd like to see a flag on PyPI marking whether the package relies on
setuptools, in which case I'll avoid it, or voluntarily entangles itself
with setuptools if present (as the only known way to create eggs), or is
setuptools-free (my preferred configuration).  Frankly, I'd also recommend
putting up a warning to developers on PyPI noting these problems with
setuptools.

  
Does your bug still exist in Distribute? If so, please report it at 
http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is 
operational, which it currently isn't)

Bill
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[Alex Grönholm]

Bill Janssen kirjoitti:

Alex Grönholm alex.gronh...@nextday.fi wrote:

  

Does your bug still exist in Distribute? If so, please report it at
http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is
operational, which it currently isn't)



Sorry, Alex, I don't know about Distribute, don't (particularly) care.
If you care, test for it and report it if it's there.  It's bug 53 in
the setuptools bug reporter.

Bill
  
If you are seriously expecting setuptools to be fixed, I can only assume 
you haven't been following the conversation on this list.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] SIP/PyQt and distutils

[Alex Grönholm]

Jeremy Sanders kirjoitti:

Zooko Wilcox-O'Hearn wrote:

  

I'm not sure that you really need to explicitly invoke g++ for
anything nowadays.  Modern gcc notices that the input is C++ and
compiles it accordingly.

Could you be more specific about what doesn't work?



It does work, but prints out warnings:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-
prototypes -fPIC -I/helpers/src -I/usr/include/qt4/QtCore -
I/usr/include/qt4/QtGui-I/usr/include/qt4 -I/usr/include/python2.6 -
Ihelpers/src -I/usr/include/python2.6 -c helpers/src/qtloops_helpers.cpp -o 
build/temp.linux-i686-2.6/helpers/src/qtloops_helpers.o
cc1plus: warning: command line option -Wstrict-prototypes is valid for 
Ada/C/ObjC but not for C++


It would be nice if it didn't use invalid C++ options.

I'd like to be able to exactly specify the arguments to the compiler as Qt 
requires particular options.


Jeremy

  

Compiling the extensions for pycryptopp also gives these kinds of warnings.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Installing stdeb on vanilla Ubuntu

[Alex Grönholm]

Olof Bjarnason kirjoitti:

Hi distutils!

So I'm quite sure stdeb is the tool for py2deb I'm looking for.

What are the correct steps of installing this tool, assuming
Ubuntu9.04. I guess I need this:

1. Python (check)
2. distutils (check)
  
Distutils is a part of the standard library, so you can't have Python 
without distutils anyway.

3. setuptools (dunno? how do I install?)
  
As a contributor to the distribute project (the successor to 
setuptools), I would advise you to download the distribute bootstrap 
script from:

http://nightly.ziade.org/distribute_setup.py
Then run it as root, and that's it!

4. stdeb (how do I install correctly?)

As all of you know by now, I'm fairly new to distutils, PyPI, eggs,
cheesecake, setuptools and stdeb. It's a whole new universe! :)

  


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig