Re: [Distutils] Setuptools 4.0 and 4.0.1 superseded by 3.8.1

2014-06-14 Thread Jeremy Stanley
On 2014-06-14 14:25:02 + (+), Jason R. Coombs wrote:
[...]
 I’ve removed all versions of Setuptools 4.x and closed that
 branch. [...] Please use Setuptools 3.8.1 or later.
[...]

Are most of the PyPI mirror tools out there smart enough to remove
packages which have been deleted from PyPI? People who are running
their own mirrors should probably check and take action as needed.
(I know I'm going to need to either delete copies from our CI
mirror, or more likely blacklist 4.x in all our requirements files
since I can't know the state of anyone else's mirrors.)
-- 
{ PGP( 48F9961143495829 ); FINGER( fu...@cthulhu.yuggoth.org );
WWW( http://fungi.yuggoth.org/ ); IRC( fu...@irc.yuggoth.org#ccl );
WHOIS( STANL3-ARIN ); MUD( kin...@katarsis.mudpy.org:6669 ); }
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] zc.buildout Docker container images

2014-07-03 Thread Jeremy Stanley
On 2014-07-03 12:24:32 +0200 (+0200), Reinout van Rees wrote:
[...]
 **Question/idea**: what about some mechanism to get this apt-get
 information out of a python package? [...] Does anyone do
 something like this?
[...]

Robert Collins wrote something called bindep as a proposal for
solving that problem:

http://git.openstack.org/cgit/stackforge/bindep/tree/README.rst

-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Other ideas from today's packaging meetup at EuroPython

2014-07-25 Thread Jeremy Stanley
On 2014-07-24 11:57:24 -0400 (-0400), Donald Stufft wrote:
 This is gonna make openstack sad I think… They were relying on the
 fact that pip prior to 1.4 didn’t install Wheels, and pip 1.4+ has
 the pre-releases are excluded by default logic to publish
 pre-releases safely to PyPI.
[...]

FWIW I don't think it'll be painful for very much longer... the need
to accommodate pip1.4 will dwindle now that Ubuntu has released an
LTS with a later version and RHEL 7 is out as well.

However it *would* be great if there was some official means of
safely uploading prerelease versions to PyPI which users of old pip
won't automatically pull down in an upgrade so we don't have to rely
on the current hack (which I agree is really just abusing unintended
behavior differences between older and newer pip versions), until a
majority of the ecosystem has stopped running with 1.4.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Immutable Files on PyPI

2014-09-30 Thread Jeremy Stanley
On 2014-09-30 07:26:32 -0400 (-0400), Donald Stufft wrote:
[...]
 I don’t personally believe it makes sense for a source
 distribution to have a build number.
[...]

I'm becoming less and less convinced it actually *is* a source
distribution any more. My constant interaction with downstream Linux
distro packagers shows a growing disinterest in consuming release
tarballs of software, that they would generally prefer to pull
releases directly from tags in the project's revision control
systems instead. Couple this with the fact that setup.py sdist can
(and often does) include autogenerated content in its output which
the packagers would rather strip or regenerate themselves, and I'm
of the opinion that the tarballs I create are only for PyPI/pip
consumption any longer. This effectively makes them a
channel-specific packaging format rather than a generally reusable
release source artifact.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Immutable Files on PyPI

2014-09-30 Thread Jeremy Stanley
On 2014-09-30 11:06:40 +0200 (+0200), M.-A. Lemburg wrote:
[...]
 You're regularly bringing up this argument.
 
 Let's just be fair here: external hosting of packages has been
 made so user unfriendly in recent pip releases, that this has
 pretty much become a non-option for anyone who wants to create a
 user friendly package installation environment.
[...]

And I'm seeing this argument regularly brought up as well. As a
heavy user of Python packages and someone who maintains very large
systems depending on them, I had a hard time trusting pip back when
it still would spontaneously grab software from wherever the
upstream author had decided to stick it that day (with whatever
hosting stability issues that implied). Our projects would regularly
audit our hundreds of requirements just to make sure nobody
*accidentally* added one which was hosted off PyPI, and that our
dependencies hadn't suddenly decided to start sticking new releases
off PyPI.

The suggestion that some developers want to control their release
process *so* tightly that they host their software in random places
without uploading them to the community package system or quietly
replace broken releases with new packages using the same version
numbers is a non-argument as far as I'm concerned. The software
authors I've talked to in these cases are pretty much always easily
convinced that those are troublesome behaviors (once it's pointed
out) and readily adjust their release processes to a more
user-friendly end result.

If there are a few who are so completely insistent on continuing in
this manner the projects I work on will not use them (for our own
sanity), and if pip and PyPI implement assurances against these
which have a side effect of driving *those particular* development
teams off of the community packaging channel then that can only be a
positive net effect in my opinion.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Immutable Files on PyPI

2014-09-30 Thread Jeremy Stanley
On 2014-09-30 09:22:29 -0600 (-0600), Carl Meyer wrote:
 On 09/30/2014 08:41 AM, Nick Coghlan wrote:
  Why is your setup.py sdist including autogenerated content? It
  shouldn't be doing that.
 
 Don't almost all sdists? At the very least egg-info is
 autogenerated, MANIFEST usually is too.

Bingo. Also in some cases I see files autogenerated from VCS
metadata to avoid double-entry... things like change histories,
authors lists, documentation indices, and even version numbers.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Immutable Files on PyPI

2014-09-30 Thread Jeremy Stanley
On 2014-09-30 12:07:23 -0400 (-0400), Barry Warsaw wrote:
 We had a discussion about this at the recently concluded Debian
 conference. There are folks who only want to use git tags as the
 consumption point for Debian packages, but this opinion was not
 the majority opinion.

Good to know. The Debian Developer packaging the majority of the
projects I work on must be in that minority.

 There's no guarantee that what you get from a tagged upstream
 source revision will match what comes in the sdist tarball.
[...]

Indeed, we've implemented quite a few workarounds specifically
requested by distro packagers who want to be able to ignore our
tarballs and use their own tools/workflow to generate them without
ever even running sdist. It seems backwards to me, but I'm not the
one doing their packaging work.

 Thus, in the Debian Python team our policy is that if upstream produces
 tarballs (as is the case for the vast majority of our packages, which are
 sourced from PyPI), then we want the Debian package to use tarballs.
[...]

Refreshing to hear!
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] py2-none-linux_x86_64 tag

2014-11-13 Thread Jeremy Stanley
On 2014-11-12 21:31:36 -0500 (-0500), Daniel Holth wrote:
[...]
 If there are no complaints I'll probably release as 0.25.0 in a
 few days.

I saw the pull request to fix data-only wheels (so that they're not
platform-specific) also got merged. Is it safe to assume that will
end up in 0.25.0 as well?
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] setuptools 8 changes are great, but ...

2014-12-16 Thread Jeremy Stanley
On 2014-12-16 02:25:33 -0500 (-0500), Donald Stufft wrote:
 Openstack won’t be providing one, they are adapting server to PEP
 440 syntax.
[...]

Confirmed, we're very close now and will likely be interoperating
with Setuptools 8 in our test infrastructure for all supported
branches later this week (depending on what direction we end up
going for embedded Git SHAs in unreleased dev sdist/wheel version
strings).
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] setuptools 8 changes are great, but ...

2014-12-16 Thread Jeremy Stanley
On 2014-12-16 18:11:12 +0300 (+0300), anatoly techtonik wrote:
 It would be more useful if the powers behind OpenStack could sponsor
 development of proper PEP scheme to untangle its own workflow instead
 of relying on imperfect PEP solutions that are being done by
 volunteers in their free time. I don't know where Donald finds his
 time for all his contributions, but in my world it is unreal even
 without full time job. Judging from activity in OpenStack community,
 maybe it can teach PSF a thing or two.

The way I see it, we're all volunteers. We voluntarily work on free
software and associated support infrastructure. Some of us are
simply lucky enough to find sponsors willing to pay us to do it full
time (and while I won't speak for Donald I get the impression he
feels the same, whether he's working on PSF or OpenStack projects).

As for PEP 440, I believe it's a necessary step forward in
untangling the previously ad-hoc state of versions in the Python
packaging ecosystem. Many of OpenStack's versioning decisions were
made specifically with Python tools and PyPI in mind, in a desire to
be a good Python community citizen, so it has been our (from the
standpoint of the people writing the integration tooling for it)
intent to continue to follow standards set by that community.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Local version identifiers from PEP 440 in practice

2014-12-16 Thread Jeremy Stanley
On 2014-12-17 00:53:08 +0100 (+0100), Maurits van Rees wrote:
[...]
   File .../pip/_vendor/pkg_resources.py, line 2583, in scan_list
 Expected ',' or end-of-list in,line,at,line[p:]
 ValueError: (Expected ',' or end-of-list in,
 'myproject==1.1+maurits.3', 'at', '+maurits.3')
[...]

This was basically why OpenStack/PBR opted not to switch to using an
LVI to encapsulate Git SHA details in unreleased dev version
strings (instead we've now moved to putting them into egg-info and
providing a separate tool to query that).
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] name of the dependency problem

2015-04-15 Thread Jeremy Stanley
On 2015-04-15 12:09:04 +0100 (+0100), Robin Becker wrote:
 After again finding that pip doesn't have a correct dependency
 resolution solution a colleage and I discussed the nature of the
 problem.
[...]

Before the discussion of possible solutions heads too far afield,
it's worth noting that this was identified years ago and has a
pending feature request looking for people pitching in on
implementation. It's perhaps better discussed at
https://github.com/pypa/pip/issues/988 so as to avoid too much
repetition.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] pip freeze not reporting pip or wsgiref

2015-06-22 Thread Jeremy Stanley
On 2015-06-22 12:20:19 +0100 (+0100), Chris Withers wrote:
 A couple of buglets I've noticed while using picky in anger:
 
 1. pip freeze doesn't include a line for pip itself, why is that?
[...]

You might want `pip list` for this instead. The old freeze behavior
omits common packages like pip and setuptools, but list does not.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] PyPi not allowing duplicate filenames

2015-10-14 Thread Jeremy Stanley
On 2015-10-14 09:43:25 -0700 (-0700), Chris Barker wrote:
[...]
> I propose that PyPi allow projects to replace existing files if
> they REALLY REALLY want to.
> 
> You should have to jump through all sorts of hoops, and make it
> really clear that it is a BAD IDEA in the general case, but it'd
> be good to have it be possible.
[...]

It used to be possible. If that possibility returns, I'm not looking
forward to more of the bugs I used to semi-regularly file explaining
to package authors why their assumptions about the safety of
silently replacing their broken releases were flawed.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Where should I put tests when packaging python modules?

2015-10-06 Thread Jeremy Stanley
On 2015-10-06 15:10:59 +0100 (+0100), Paul Moore wrote:
[...]
> That's 25%, which is certainly higher than I would have
> guessed. Of course, the implication is that 75% (or 54% in David's
> case) use test suites *not* installed with the package :-)
[...]

It seems rather optimistic to assume that 100% of projects have test
suites. More accurately, 75% either have no tests _or_ keep them
outside the package. ;)
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Making install a no-op

2015-07-10 Thread Jeremy Stanley
On 2015-07-10 15:11:32 -0700 (-0700), Ethan Furman wrote:
[...]
 The request came from someone who would like to have one wheel
 file for all Python2-3 versions; I know that in setup.py it's easy
 enough to check the version and add (or not) enum34 to the
 required (or dependent?) module list. Does this type of thing work
 with wheels?

If you want a real-world example you can look at argparse (I can
successfully pip install argparse in a Python 3.5 virtualenv even
though it's been in stdlib since 2.7).

If you want depending packages to do the work instead, they'll need
to implement environment markers with something like
;python_version='3.3' (which really needs setuptools 17.1 or later
to interpret it correctly).
http://pythonhosted.org/setuptools/history.html#id5
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Where should I put tests when packaging python modules?

2015-10-06 Thread Jeremy Stanley
On 2015-10-06 13:47:51 +0300 (+0300), Ionel Cristian Mărieș wrote:
[...]
> `pbr` just decides to alter your `setup.py sdist/bdist_*` output
> without being asked or invoked
[...]

Assuming you're talking about https://launchpad.net/bugs/1483067
then it was fixed in 1.7.0. If you're still seeing it in later
releases, a detailed bug report would be most appreciated since
that's not at all the intent.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils

2015-10-05 Thread Jeremy Stanley
On 2015-10-05 15:39:10 +0200 (+0200), Antoine Pitrou wrote:
[...]
> But why use two different formats for "source release" and "sdists"?
> Currently sdists fit the assumptions for a source release, why
> introduce some complexity and have the users deal with separate
> concepts (with all the confusion that will inevitably ensue)?

An sdist is an installable package which just happens to _look_ a
lot like a source release tarball, but trying to pretend that
downstream packagers will want to use it as such leads to a variety
of pain points in the upstream/downstream relationship. For better
or worse a lot of distros don't want generated files in upstream
source code releases, since they need to confirm that they also ship
the necessary tooling to regenerate any required files and that the
generated files they ship match what their packaged tooling
produces.

While this similarity was probably seen as a "Good Thing [TM]"
initially (hence standardizing on a .tar.gz extension), over time
both the generated content of a typical sdist and the concern most
distros have over shipping upstream-generated files has increased to
the point where they really need to be viewed as separate and
distinct release artifacts now.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Uploading to Warehouse

2016-06-13 Thread Jeremy Stanley
On 2016-06-12 01:48:40 +0200 (+0200), Maurits van Rees wrote:
[...]
> I had problems uploading to PyPI recently, getting an error although the
> upload seemed to have gone fine in reality.
> I did 8 uploads of Plone packages today using the warehouse url.
> All have gone fine.
> 
> Thanks a lot, Donald!

Agreed. I switched OpenStack's release upload automation to hit
Warehouse a week ago. Before that we were frequently getting HTTP
500 errors uploading releases of our various projects directly to
PyPI, and (aside from one day last week where a PyPI denial of
service attack was impacting uploads) the transition has very gone
smoothly (some 50+ releases so far). Thrilled to see Warehouse
coming along!
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] PEP 376, the INSTALLER file, and system packages

2016-01-22 Thread Jeremy Stanley
On 2016-01-23 08:09:47 +1300 (+1300), Robert Collins wrote:
[...]
> Having a interface to the system package manager as has been
> previously mooted - and I'm going to get back to that - might help a
> little, but uninstalls are quite different to installs. Uninstalls get
> blocked by things like 'app-foo requires requests', and I would be
> super suprised (and delighted!) if we were able to override that when
> upgrading requests via pip :)

Add to this the fact that whatever system-distributed files pip
removes/replaces will get quietly overwritten again the next time
you update your system packages and there's a new version of
whatever you've replaced... perhaps being able to set a hold state
through the distro package manager from pip at that point for
whatever package owned the files it replaced would be a helpful
middle ground (I know how I'd go about it on Debian derivatives,
less so on all those "other" distros).
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Thank you for the ability to do `pip install git+https://...`

2016-04-01 Thread Jeremy Stanley
On 2016-04-01 17:16:15 +0300 (+0300), Alex Grönholm wrote:
> I'm sorry if I offended anyone. I was just trying to point out
> that in the case of Github (or any other service that provides
> automated tarball generation) it's better to install from those
> rather than using the VCS integration plugins. Oh, and for the
> record, I just tested -- it does a deep clone at this time, which
> would be responsible for the slowness compared to installing from
> a tarball.

Whether this can work depends entirely on the project being
installed. Some don't check package metadata into their repos in a
form consumable by pip, and require an additional sdist build step
which may need information from the VCS itself or manually provided
during that step.

A prime example of this is projects using PBR, which will want
access to Git tag and commit details to determine the package
version. With a local clone of the project's Git repository (not
just a tarball/zip snapshot of its content) you can build a
pip-supported sdist, but otherwise you may need to manually
determine and set version information in the calling environment or
by editing to be able to then generate a usable sdist from the raw
source tree.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] PyPI and GPG Signatures

2016-05-12 Thread Jeremy Stanley
On 2016-05-12 07:41:21 -0400 (-0400), Donald Stufft wrote:
[...]
> What do folks think? Would anyone be particularly against getting
> rid of the GPG support in PyPI?

We have plans[*] in the OpenStack community to start autosigning our
sdist and wheel builds (and similar release artifacts we build for
other package ecosystems), so that we can track provenance and
integrity through part of our release pipeline. I'm hoping to have
that implemented in the next few months.

While also uploading these signatures to PyPI was seen as useful, we
do already have another primary location we can publish detached
signatures along with our release artifacts so I would probably just
ignore the PyPI/twine-specific part of the work if this goes away.

[*] 
http://specs.openstack.org/openstack-infra/infra-specs/specs/artifact-signing.html
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] can someone explain why I have started to get these pip failures

2016-07-14 Thread Jeremy Stanley
On 2016-07-14 16:01:23 +0100 (+0100), Robin Becker wrote:
> On 14/07/2016 15:41, Ian Cordasco wrote:
> >Try:
> >
> >pip install --force-reinstall setuptools -U
> >
> 
> I didn't do the force-reinstall and for some reason when I cleaned both
> ~/.cache/pip and ~/.pip the pip install -r requirements.txt did work.
> 
> I have tried various solutions proposed in the past eg
> 
> sudo apt-get install --reinstall python-pkg-resources
> 
> but nothing seems to work.
> 
> I did the cache cleanups in desperation mode.
> 
> I did try pip install -U setuptools, but it says it is up to date.
> 
> If this might be a case of the underlying python changing on the server I
> have turned off automatic security updates.
> 
> I would like to try and understand this happens as then I might have some
> wya of fixing it.

You really should avoid mixing pip-installed packages in the system
context with distro-provided Python libraries, otherwise you will
run into these sorts of issues constantly. I help maintain some
very, very large test infrastructure for Python-based tools and
libraries: in scenarios where we use pip to install anything
system-wide we first make sure to scrub every last distro-provided
Python library from the system along with any other Python-based
applications that might depend on them, and only then we bootstrap
pip completely independent of distro packaging (downloading and
running get-pip.py). Also whenever possible, we instead rely on pip
install within virtualenvs _without_ --system-site-packages, so that
there's no risk of interaction with any Python libraries that might
somehow get subsequently installed on the system.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] can someone explain why I have started to get these pip failures

2016-07-14 Thread Jeremy Stanley
On 2016-07-14 17:23:24 +0100 (+0100), Robin Becker wrote:
> not really sure why this is an issue. All of my problems are in virtual
> environments and I never use the --system-site-packages flag.
> 
> I have never used pip to install anything system wide (so far as I know).

Ahh, whoops, after rereading back through some of your earlier
messages in the thread I see this is indeed all in the scope of a
virtualenv.

> Step 0 in after setting up an environment is pip install -U pip setuptools
> 
> Of course you are right in that the python is a distro provided one; and
> also the pip and virtualenv etc etc.
[...]

I've noticed in the past that for some reason the system
pkg_resources can end up getting used by setuptools (on
Debian/Ubuntu it's unvendored and split into a separate
python-pkg-resources deb). I haven't had an opportunity to track it
down. You might try myenv/bin/python -c 'import sys;print sys.path'
and then check the results in order to see which the first one is to
provide pkg_resources.

Hunting around a bit, this looks similar to
https://github.com/pypa/setuptools/issues/497 so see if any of the
suggestions mentioned there help at all.

> When I run python -mvirtualenv I end up with an environment that has pip and
> setuptools already. Are you saying I should do
> 
> /usr/bin/python -mvirtualenv --no-pip --no-setuptools myenv
> myenv/bin/python get_pip.py
[...]

No, but you might try bootstrapping a newer version of virtualenv
into its own virtualenv, like:

virtualenv foo
foo/bin/pip install -U pip setuptools virtualenv
foo/bin/virtualenv myenv

Doing that on some systems where I'm forced to start from
distro-provided pip/setuptools/virtualenv has worked out
consistently well.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Proposal: "Install and save"

2016-07-31 Thread Jeremy Stanley
On 2016-07-30 20:23:14 -0500 (-0500), Wes Turner wrote:
> pbr also supports "environment markers"
> which we would want to preserve when round-tripping (reading in, modifying,
> and writing out) requirements.txt files;
> though IDK if environment markers are part of any Python Packaging Spec?
> 
> from http://docs.openstack.org/developer/pbr/#environment-markers :
> 
> argparse; python_version=='2.6'

I'm assuming you didn't follow the "conditional dependencies" link
from the description in that first paragraph, as it would have taken
you to the corresponding section of one:

https://www.python.org/dev/peps/pep-0426/#environment-markers

You should probably also see:

https://www.python.org/dev/peps/pep-0496/
https://www.python.org/dev/peps/pep-0508/#environment-markers

Quickly skimming relevant changelogs, environment markers have been
supported by Setuptools since 0.7, and directly in requirements
lists since pip 6.0.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Indexing modules in Python distributions

2017-02-09 Thread Jeremy Stanley
On 2017-02-08 18:14:38 + (+), Thomas Kluyver wrote:
[...]
> What I'm proposing differs in that it would need to download files from
> PyPI - basically all of them, if we're thorough about it. I imagine
> that's going to involve a lot of data transfer. Do we know what order of
> magnitude we're talking about?
[...]

The crowd I run with uses https://pypi.org/project/bandersnatch/ to
maintain a full PyPI mirror for our project's distributed CI system,
and du says the current aggregate size is 488GiB. Also if you want
to initialize a full mirror this way, plan for it to take several
days to populate.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] distlib and wheel metadata

2017-02-17 Thread Jeremy Stanley
On 2017-02-17 09:56:04 +0100 (+0100), Nick Coghlan wrote:
[...]
> So if we rely on a manual "publish with pinned dependencies", "get bug
> report from redistributor or app developer", "republish with unpinned
> dependencies", we'll be in a situation where:
> 
> - the affected app developer or redistributor is going to have a negative
> experience with the project
> - the responsible publisher is either going to have a negative interaction
> with an end user or redistributor, or else they'll just silently move on to
> find an alternative library
> - we relinquish any control of the tone used when the publisher is alerted
> to the problem
> 
> By contrast, if we design the metadata format such that *PyPI* can provide
> a suitable error message, then:
> 
> - publishers get alerted to the problem *prior* to publication
> - end users and redistributors are unlikely to encounter the problem
> directly
> - we retain full control over the tone of the error notification
[...]

It seems like the same could be said of many common mistakes which
can be identified with some degree of certainty through analysis of
the contents being uploaded. Why not also scan for likely security
vulnerabilities with a static analyzer and refuse offending uploads
unless the uploader toggles the magic "yes I really mean it" switch?
Surely security issues are even greater downstream risks than simple
dependency problems. (NB: I'm not in favor of that either, just
nudging an example in the reductio ad absurdum direction.)
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Module Installation Issues

2016-09-13 Thread Jeremy Stanley
On 2016-09-13 23:00:25 +0100 (+0100), Paul Moore wrote:
[...]
> And things get significantly worse if we allow upgrades from the
> Python prompt rather than just installs (e.g., if you have already
> imported something from the old version and then upgrade).
[...]

If you need it, and of course only if your software is actually
written to handle it (contextually), there is a function to handle
that:

https://docs.python.org/2/library/functions.html#reload
https://docs.python.org/3/library/importlib.html#importlib.reload

I've used it in some very specific situations where you need to
change out parts of a running daemon without restarting the process,
but you really need a lot of extra care to get out of contexts
utilizing the modules being reloaded if you want the replacements to
actually be used.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Current Python packaging status (from my point of view)

2016-11-05 Thread Jeremy Stanley
On 2016-11-05 17:43:48 +1000 (+1000), Nick Coghlan wrote:
[...]
> Putting my work hat back on for a moment, I actually wish more people
> *would* start saying that, as Red Hat actively want people to stop
> running their own applications in the system Python, and start using
> Software Collections (either directly or via the Docker images)
> instead. Sharing a language runtime environment between operating
> system components and end user applications creates all sorts of
> maintainability problems (not least of which is the inability to
> upgrade to new feature releases for fear of breaking end user
> applications and vice-versa), to the point where Fedora is planning to
> cede the "/usr/bin/python3" namespace to end users, and start
> migrating system components out to "/usr/bin/system-python":
> https://fedoraproject.org/wiki/Changes/System_Python
[...]

It's a grey area, complicated by the fact that many people are
writing their software with the intent of also packaging it for/in
common distros and so want to make sure it works with the "system
Python" present within them. There it looks like Fedora is splitting
their Python-using packages along some arbitrary line of "is it a
system component?" vs. "is it a user-facing application?" which is
probably tractable given the (relatively) limited number of packages
in their distribution. I can't fathom how to go about trying to
introduce a similar restructuring in, say, Debian.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Maintaining a curated set of Python packages

2016-12-08 Thread Jeremy Stanley
On 2016-12-08 10:05:47 -0600 (-0600), Wes Turner wrote:
[...]
> http://docs.openstack.org/developer/pbr/
> 
> - pbr does away with setup.py and install_requires in favor of just
> requirements.txt
[...]

It doesn't entirely "do away with setup.py" (it still relies on a
relatively minimal boilerplate setup.py which loads its setuptools
entrypoint), but does allow you to basically abstract away most
common configuration into declarative setup.cfg and requirements.txt
files (similar to some of the pyproject.toml use cases Donald et al
have for PEP 518).
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] A possible refactor/streamlining of PEP 517

2017-07-10 Thread Jeremy Stanley
On 2017-07-10 20:33:16 +1000 (+1000), Nick Coghlan wrote:
[...]
> we don't really *want* folks to be adding generated files to their
> sdist that they aren't keeping under source control - we'd prefer
> that such activities were postponed to "build_wheel" now that we
> have separate source and precompiled distribution formats.
[...]

This is a mildly naive view. The software I'm familiar with is
actually attempting to reflect metadata _from_ the source revision
control _into_ the sdist because while it's "tracked" there it's not
tracked as normal files (version information from tags, change logs
from the commit history, contributor lists from commit authorship).
The metadata in question is lost by just blindly tarring up tracked
files, so needs some mechanism to export and inject as untracked
files (from the source revision control perspective) for inclusion
in the sdist.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Getting dependecies of package from PyPiJSON

2017-07-20 Thread Jeremy Stanley
On 2017-07-20 14:55:53 +0200 (+0200), Krzysiek Płachno wrote:
[...]
> To make downloaded package working it's needed to install also
> dependencies. Is it possible to get dependencies information
> directly from PyPiJSON API? (e.g. by adding some request parameter
> or header in GET request)
> 
> Or is it possible to this data in any other way (apart from
> downloading package)?

Unfortunately, no, not with the current state of the Python package
ecosystem. Packages are allowed to define their own dependencies
dynamically and conditionally at the time of installation so there's
no good way for PyPI to know what dependencies each package has.

PEP 518 aims to solve this eventually:

https://www.python.org/dev/peps/pep-0518/

-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Getting dependecies of package from PyPiJSON

2017-07-20 Thread Jeremy Stanley
On 2017-07-20 11:11:38 -0300 (-0300), Leonardo Rochael Almeida wrote:
[...]
> A small clarification: Packages can define their own dependencies
> dynamically only at *build* time, not at *installation* time.
> 
> The difference is subtle (considering many packages (the ones with only
> sdist on PyPI) are built at the same time they're installed), but important:
> 
> In practice it means that if you have a wheel (or an egg), you can
> determine the dependencies without installing, just by looking at the
> metadata inside the package.

Agreed, I elided those details not knowing the extent of familiarity
of the question's author with nuances of Python packaging, but
you're right I should take more care to not overload the term
"installation" without clarifying the circumstances.

As many projects on PyPI lack wheels (and most non-pure-Python
projects lack wheels for at least some platforms), sdists are for
better or worse the most prevalent and portable "packaging" format
on PyPI. So while it might be possible to add some sort of feature
to inspect wheels at upload and then store the specific dependencies
declared therein and report those back via an API method, I expect
coverage across packages in general would be fairly low today. It
might be sufficient if sticking with some popular subset of packages
though.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Getting dependecies of package from PyPiJSON

2017-07-20 Thread Jeremy Stanley
On 2017-07-20 21:09:28 -0400 (-0400), John Thorvald Wodder II wrote:
> [Sending to the list this time]
> 
> On 2017 Jul 20, at 12:41, Jeremy Stanley <fu...@yuggoth.org> wrote:
> > So while it might be possible to add some sort of feature
> > to inspect wheels at upload and then store the specific dependencies
> > declared therein and report those back via an API method, I expect
> > coverage across packages in general would be fairly low today.
> 
> PyPI (both Legacy and Warehouse) actually does do this already;
> see the `requires_dist` field in, e.g.,
> <https://pypi.org/pypi/qypi/json>. However, this only seems to
> work if the maintainer uploads the wheel before uploading the
> sdist (unless the sdist is a .zip instead of a .tar.gz, then it
> can be uploaded first?  I'm not sure).

Indeed, I'd never noticed that. And the projects I work on upload
wheels before sdists so seem to have everything from our
install_requires reflected there (including extras and environment
markers... even the versioned pages work). Very neat, and glad to
learn it already exists.

I wonder though how it deals with projects that build multiple
wheels for different platforms with different install_requires. It
looks like that's a top-level field in the info dict so can't
reasonably be differentiated. Takes the first one uploaded I guess
and ignores the subsequent ones?

Anyway, this looks like it probably fulfils Krzysiek's need for
XWiki. Thanks for pointing it out! I very well may try to leverage
this for a few things myself.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] A possible refactor/streamlining of PEP 517

2017-07-05 Thread Jeremy Stanley
On 2017-07-05 12:40:08 -0400 (-0400), Donald Stufft wrote:
[...]
> That doesn’t solve the “I downloaded a archive from GitHub” or “I
> mounted my VCS checkout into a docker container without my VCS
> installed”, but those can only be solved by the backend tool
> itself and IMO it’s perfectly acceptable for the backend to fail
> with an appropriate error message if it needs something that the
> current directory or environment doesn’t provide.

That's exactly the approach PBR has taken since the beginning, and
it's worked just fine. Users do still wrongly assume from time to
time that they should be able to treat a "GitHub tarball" or similar
contextless pile of files like an sdist even if it hasn't been
correctly passed through the sdist build process first, but a
combination of sane fallback behaviors, environment variable
overrides and clear error messages keeps the invalid bug reports to
a manageable minimum.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Jeremy Stanley
On 2017-06-01 20:45:53 + (+), Brett Cannon wrote:
[...]
> I think *twine* is the tool that needs a way to specify how to
> produce an sdist. If we want to view twine as the tool to upload
> artifacts to PyPI then we need twine to know how to produce sdists
> and wheels in a PEP 517 world, not pip.
[...]

Why do you think that? Because traditionally you could call setup.py
to upload an sdist as well as build it?

One thing I really like about twine, as the tool I trust with my
PyPI creds, is that it's a very simple tool unencumbered by
unrelated features. While I agree that the tool which retrieves and
installs packages doesn't necessarily also need to be the tool which
builds packages, I don't see why the tool which securely uploads
packages should take on that function either. In the UNIX sense of
doing one thing well, I'd much rather see a separate tool for each
of these roles.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Jeremy Stanley
On 2017-06-01 21:09:57 -0400 (-0400), Donald Stufft wrote:
[...]
> I think a separate tool for each of these roles is somewhat user
> unfriendly TBH.
[...]

I'll do my best not to be offended that you don't consider me a user
(or representative of some broader class of users). ;)

At any rate, I think it depends on your definition of users. Some
users want one shiny kitchen-sink tool that does everything for
them, others want composable tools with well-considered bounds of
operation. It's possible a modular approach could satisfy both, but
then again if twine grows too many features I'm just as likely to
write a new lightweight API client instead so I can have something
auditable I can trust my credentials to which only knows how to
upload.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Jeremy Stanley
On 2017-06-02 15:12:19 +0100 (+0100), Paul Moore wrote:
[...]
> I'm struggling to understand why building a sdist in flit should need
> a VCS. It bothers me that I'm missing something important about how
> backends will work, that explains why (for example) you can't create a
> sdist from an export of a VCS branch (i.e., without the VCS metadata).
[...]

Unrelated to flit, but I have similar needs to be able to make sure
my sdist version has a 1:1 correspondence to the name of a release
tag in my VCS. Making a commit to embed a version number in a file
in the VCS and then tagging that commit with the same version number
is 1. racy and 2. duplication of information which can (and
frequently has) led to confusing mistakes. As a result, I either
need the VCS metadata present to be able to construct the version
number which will get included in PKG-INFO _or_ I need a complex
build wrapper which extracts the metadata prior to the filtered tree
copy happening and plumbs it through (with an envvar or maybe
spontaneously generating a file on disk) so that the sdist builder
will know what version to embed.

Present day, this works fine as a setuptools entrypoint which can
inspect the VCS metadata at sdist creation time. It would be
unfortunate to lose such flexibility in whatever hook implementation
we end up with.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Jeremy Stanley
On 2017-05-31 20:08:51 -0400 (-0400), Donald Stufft wrote:
[...]
> Both {name} and {version} MUST have any - characters escaped to a
> _ to match the escaping done by Wheel. Thus a sdist for a project
> named foo-bar with version 1.0-2 which is using a .tar.gz
> container for the sdist would produce a file named
> foo_bar-1.0_2.tar.gz.
[...]

While I agree with the reasoning, if this is going to end up
enforced by common tooling in the near future a warning period would
be appreciated as it implies a fairly significant behavior change
which will require quite a lot of adjustments to bespoke automation
(at least for some I maintain, and I'm pretty sure there are plenty
more out there too).
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Conditionless setup.py

2017-08-25 Thread Jeremy Stanley
On 2017-08-25 14:00:25 +0100 (+0100), Paul Moore wrote:
[...]
> I'm not that familiar with those tools, but if they enable that
> sort of use then that's great. I did get the impression that they
> were for more complex/specialised use cases, though, whereas flit
> (with PEP 517) is much more about simple configuration for the
> majority of (pure Python) projects that don't need complex
> behaviour.
[...]

Not really, no; pretty much everything in OpenStack is pure Python
as well and relies on PBR. I can't personally think of any non-pure
package I've seen using PBR, so it may not even be usable outside
pure Python packaging for all I know (I've certainly never tried
anyway).

PBR basically started with the primary goals of reducing copied code
by simplifying common setup.py file patterns and integrating
features to help avoid additional setup_requires which couldn't
easily be versioned (especially back in those days) without causing
all manner of breakage when they might end up conflicting with
transitive install_requires. With PEP 517 on the way now I consider
these earlier attempts at declarative package definitions will be
more of a historical footnote soon enough, but the intent was pretty
similar. Here's an example setup.cfg from a reasonably simple
command-line utility:

https://git.openstack.org/cgit/openstack-infra/bindep/tree/setup.cfg

Anyway, my point wasn't to advertise these tools specifically, but
rather to point out that there's nothing stopping projects who want
to extract their package metadata into static configuration and
bundle their setuptools logic into a separate reusable module (for
example by leveraging a setuptools entrypoint the way PBR does)
without having to wait around for PEP 517. Then they can always
revisit the design once PEP 517 support is more generally rolled out
in standard packaging tools.

> One thought - are the PBR and/or distutils2 teams looking at
> providing PEP 517 support? Assuming they are, have they had a
> change to review the PEP to ensure it suits their needs? And if
> they aren't, what is it about the PEP that makes them unwilling to
> do so?

As far as I know, Distutils2 has essentially been dead upstream for
~5 years, so I wouldn't expect it to receive any updates in that
regard.

I've had conversations with the primary maintainers of PBR about
what would need to be done to coexist with and eventually take
advantage of PEP 517 paradigms. On-the-fly generation of
pyproject.toml files during or, if necessary, preceding the sdist
build phase is probably the way they'll be looking to go there,
though the details have yet to be completely hashed out. (The
community around it is sensitive to gender diversity issues and
wants to avoid acquiring more of a "brogrammer" image, so some of us
worry that any conspicuous TOML files checked into revision control
repositories could be seen as a tacit endorsement of the author's
alleged behavior at GH a few years ago.)
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Conditionless setup.py

2017-08-25 Thread Jeremy Stanley
On 2017-08-25 16:16:39 -0700 (-0700), Nathaniel Smith wrote:
[...]
> I've wondered if we should fork it, rename it "the obvious minimal
> language", and release our own 1.0.

And make it a toggleable option to configparser? ;)

The basic format is pretty similar, granted TOML has a lot more
flexibility over configparser's classic loader.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] PBR/distutils2 with pep517 Support Was: Conditionless setup.py

2017-08-28 Thread Jeremy Stanley
On 2017-08-28 13:05:07 +0200 (+0200), Thomas Güttler wrote:
[...]
> Are there PBR/distutils2 hackers on this list here?
> 
> If yes: Do you support this pep?
> 
> If no: where can you meet PBR/distutils2 hackers?

See:

https://mail.python.org/pipermail/distutils-sig/2017-August/031315.html

-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] PBR/distutils2 with pep517 Support Was: Conditionless setup.py

2017-08-28 Thread Jeremy Stanley
On 2017-08-28 17:48:40 + (+), Daniel Holth wrote:
> Imo PBR is entirely a setuptools creature, without special
> concerns to operate in pep517 land. If I were them I'd rewrite it
> to not require setup.py and call that pbr2.
[...]

While it's true that PBR relies on setuptools entrypoints _today_,
the expectation is that it will also grow a PEP 517
build-system.build-backend compliant interface once the spec is
officially approved. At least one problem with "just rewrite it" is
that we expect to support people using old (pre-517) versions of pip
for years to come, so will likely end up with some sort of hybrid
implementation for as long as backwards compatibility needs to be
maintained.

It may also end up coupled with some manner of pre-build wrapper to
consume/transform existing static metadata and emit a suitable
pyproject.toml file, though that's for social reasons as much as
anything.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Conditionless setup.py

2017-08-25 Thread Jeremy Stanley
On 2017-08-25 10:50:11 +0100 (+0100), Paul Moore wrote:
[...]
> once PEP 517 is implemented and as flit gains popularity, I fully
> expect more and more projects to use a static data structure for
> their metadata (flit.ini, specifically).

This has also been possible for years already using either PBR or
distutils2. For example, hundreds of Python packages produced by the
OpenStack community use a branchless boilerplate setup.py which
declares a setup_requires on the pbr package, and then everything
else goes into an INI-formatted setup.cfg file (except for
install_requires which are drawn from requirements.txt instead).
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Conditionless setup.py

2017-08-25 Thread Jeremy Stanley
On 2017-08-25 07:37:23 -0500 (-0500), Ian Stapleton Cordasco wrote:
> Except that OpenStack frequently rejects outside use cases as I
> learned as an OpenStack developer who tried to improve PBR. Sadly
> it will never be seen as a global solution as long as that
> continues

Many projects frequently reject use cases regardless of their
proponents, if they don't fit with the scope as identified by their
maintainers. No project can be all things to all users and expect to
remain manageable in the long term.

I've not seen patches rejected out of hand because of who/where the
use case came from, and while I'm curious to know which change(s)
this was it's probably off-topic for this ML.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheel 1.0 roadmap

2017-10-30 Thread Jeremy Stanley
On 2017-10-31 00:20:34 +1000 (+1000), Nick Coghlan wrote:
[...]
> For folks that do want signatures on their build server -> deployment
> system connections (which is the problem this features aims to help with),
> they're currently more likely to use external GPG signatures (the way Linux
> distros and some container registries do) or a system like Notary/TUF (the
> way the Docker registry does), than they are a Python-specific format.
[...]

Agreed. For the hundreds of projects we publish on PyPI we have our
release automation generate detached OpenPGP signatures of sdsits
and wheels, and serve those signatures from our own release info
site since PyPI also seems to not want to support arbitrary
signature uploads over the long term. This satisfies the requests we
get from distribution package maintainers to provide proof of
provenance for our release artifacts; our release managers and
community infrastructure sysadmins sign the per-cycle release
automation keys, and regularly participate in key signing with
distro package maintainers in-person at conferences to establish a
sufficient web of trust. I understand this is probably untenable for
smaller projects, but at our scale it works fairly well (also easier
to generalize beyond merely Python-based software).

I'll be honest, when designing our artifact signing automation I
didn't even know the wheel spec suggested it should be a feature,
but without having consistent integration in other tooling for
signed sdists too it wouldn't have been much help to us anyway.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] What's the use case of testpypi?

2017-10-31 Thread Jeremy Stanley
On 2017-10-31 16:25:08 +1000 (+1000), Nick Coghlan wrote:
> Ideally we'd be recommending
> https://devpi.net/docs/devpi/devpi/stable/%2Bd/index.html to folks looking
> to develop a robust pre-release artifact testing workflow.
> 
> While we mention it a couple of times on packaging.python.org [1,2], and
> include it in the packaging related Non-PyPA Projects list [3], we don't
> really emphasise that it makes a much better platform for pre-release
> testing and private experimentation than PyPI itself does (see
> https://devpi.net/ for an example of a deployed instance with multiple
> distinct user namespaces).
> 
> Given Donald's comment about the current test PyPI not being particularly
> good at any of its roles, perhaps it would make sense to aim to replace it
> with a periodically purged DevPi instance?
[...]

Do the two share enough common code for successful uploading to a
devpi instance to be indicative of whether PyPI will accept or
reject on the grounds of, e.g., invalid trove classifiers (this one
in particular has been the most common preventable but otherwise
untestable upload failure our community encounters).
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] What's the use case of testpypi?

2017-10-31 Thread Jeremy Stanley
On 2017-10-31 13:09:34 + (+), Thomas Kluyver wrote:
[...]
> If anyone wants to adapt the code to use in other tools, it's
> here:
> 
> https://github.com/takluyver/flit/blob/ca08119/flit/inifile.py#L63-L108
[...]

Thanks for the pointer! That would make for a really great
stand-alone linting utility or maybe a flake8 plug-in (particularly
if it vendored a copy of the classifiers list and included an
option/flag to skip downloading).
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] library development patterns

2018-01-16 Thread Jeremy Stanley
On 2018-01-16 19:13:31 + (+), Brett Cannon wrote:
> On Tue, 16 Jan 2018 at 11:00 Chris Withers <ch...@withers.org> wrote:
[...]
> > I generally use pip install -e . in a checkout to set up a development
> > environment but beyond this I think things branch out a lot:
> >
> > How do you do axis development? (often python version, but can be a
> > major version of a dependency such as django, or operating system, or
> > for the lucky masochists out there: a dot product of each of those...)
> >
> > For me, I use travis-ci coupled with a few local virtualenvs for canary
> > versions. Some people like tox here, I never got on with it.
> >
> 
> This is part of what I would want us to come to a consensus on. For
> example, do people just create a venv per Python version they want to
> test/support, do they use pew or some other tool I don't know about? For VS
> Code we need to know how to detect what Python interpreters you might want
> to use with your workspace/folder so we know what interpreters to present
> to you to select from (and you *have *to select one for when you do things
> like want to execute a file or run tests).
[...]

At least with tox you get this more or less automagically (I know
plenty of people aren't tox fans, but it still merits pointing out).
For those unfamiliar, it has implicit environments defined for minor
(in the SemVer sense) Python versions so your project can define a
list of which versions it's intended to support and then anyone
running it by default gets tests executed under each of those for
which they have a viable interpreter on their path. My local dev
environment includes from-source builds of the latest point release
for each minor Python version my projects intend to support plus the
most recent alpha/beta/rc for the next unreleased Python 3.x, though
I'll admit that keeping up with the various build-deps and
compile-time optimizations for each of them is mildly time-consuming
(as is bootstrapping a new dev environment when the need arises).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


[Distutils] Re: Environment markers for GPU/CUDA availibility

2018-09-04 Thread Jeremy Stanley
On 2018-09-05 00:42:04 +1000 (+1000), Nick Coghlan wrote:
[...]
> I didn't actually realise the GPU tensorflow package was over 200 MB,
> though - that's large enough to be noticeably slow to extract and
> install even from a local zipfile, let alone if you're needing to
> download it first.
[...]

Yes. If you haven't tried running a mirror of PyPI lately you're
likely not to have noticed, but the various nightly builds for
tensorflow seem to be the majority of the data on PyPI now. I'm sure
it's a very neat and useful tool, but we basically had to switch
from mirroring PyPI in our CI system to using a caching proxy
because of this.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/JE5CDEDEDAZSOLK72JZFAD2DV7F32ZSM/


[Distutils] Re: Environment markers for GPU/CUDA availibility

2018-09-04 Thread Jeremy Stanley
On 2018-09-04 11:40:17 -0500 (-0500), Dustin Ingram wrote:
> On Tue, Sep 4, 2018 at 11:33 AM Jeremy Stanley  wrote:
> >
> > Yes. If you haven't tried running a mirror of PyPI lately you're
> > likely not to have noticed, but the various nightly builds for
> > tensorflow seem to be the majority of the data on PyPI now. I'm sure
> > it's a very neat and useful tool, but we basically had to switch
> > from mirroring PyPI in our CI system to using a caching proxy
> > because of this.
> 
> Side note: PyPI now provides a list of the largest packages by total
> filesize: https://pypi.org/stats/
> 
> Depending on what mirror you're using, you may be able to exclude
> these packages from your mirror if you don't need them, e.g. for
> bandersnatch: 
> https://github.com/pypa/bandersnatch/blob/master/docs/filtering_configuration.md#blacklist-filtering-settings

We played whack-a-mole blacklisting some of the largest offenders in
our bandersnatch config for a while, but really needed to rebuild
the mirror from scratch since there's no easy way to go back and
delete the now-blacklisted packages from before the blacklist
entries were added (and that's a week+ effort to bootstrap a new
mirror these days). In the end we just switched to a caching proxy
we already had on hand because it got us most of the benefit of
mirroring with a tiny fraction of the disk space, given we use fewer
than 1000 packaged Python library dependencies across our CI jobs
anyway.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/3SPP3O47YY7OO2UHADY6AA6PDJMKEFDS/


[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

2018-07-08 Thread Jeremy Stanley
On 2018-07-08 10:31:04 -0700 (-0700), Nathaniel Smith wrote:
[...]
> Unless I'm missing something, that's unrelated :-). The "build
> isolation" we're talking about here is a new feature where pip
> tries to use a clean python environment for the build, and that
> issue is about an old feature where pip tries to use a clean copy
> of the source directory for the build.
[...]

Ahh, yes, apologies! I had incorrectly assumed the new build
environment isolation was also going to have changes to build
directory isolation.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/EOQ65TFD4536VHZVIOVCQ7TFBWZUZKBA/


[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

2018-07-08 Thread Jeremy Stanley
On 2018-07-08 07:41:43 -0700 (-0700), Nathaniel Smith wrote:
> On Sun, Jul 8, 2018, 04:25 Paul Moore  wrote:
[...]
> > I'd like to find some way of assessing the impact before we
> > simply switch to full build isolation (we've already had a fair
> > number of bug reports on pip that are triggered by build
> > isolation, such as the ones that started this discussion).
> >
> 
> I think it would be helpful for this discussion if we could look
> at these bug reports – do does anyone have links?
[...]

The discussion around the attempts to resolve
https://github.com/pypa/pip/issues/2195 is probably relevant, if
you're looking for an example.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/WZXC7KHF3NZ7Q4MFPZCB3PEG2H5FP5BJ/


[Distutils] Re: Idea: perennial manylinux tag

2018-11-30 Thread Jeremy Stanley
On 2018-11-30 15:35:10 + (+), Paul Moore wrote:
[...]
> I certainly don't want to spark any sort of flamewar here, but I
> do feel a certain wry amusement that the term "DLL Hell" was
> invented as a criticism of library management practices on
> Windows, and yet in this context, library management on Windows is
> pretty much a non-problem, and it's Linux (that prided itself on
> avoiding DLL hell at the time) that is now struggling with library
> versioning complexity ;-)

You could look at it this way: "Linux" isn't an operating system,
it's just a kernel. GNU/Linux distributions are independent and
varied operating systems. If you needed to build packages which
could be installed on dozens of different competing Windows-based
operating systems all of whom recompiled Windows from source in
various ways with different features and random versions of system
libraries, the problem might look similar for the Windows ecosystem
as well. That Windows is a commercial product legally available
strictly in precompiled binary form from only one source is what
mostly saves it from this particular bit of fun.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/CU6MRFOVQONDW2YFNZWWWVEHNCAS2SMB/


[Distutils] Re: wrong SHA256 on setuptools

2019-03-21 Thread Jeremy Stanley
On 2019-03-20 14:52:47 -0700 (-0700), Martin Baker wrote:
> The SHA256 listed for setuptools-40.8.0.zip
> is 6e4eec90337e849ade7103723b9a99631c1f0d19990d6e8412dc42f5ae8b304d
> 
> But the actual SHA sum is 3547552b1009283f7ae31fded32ad33ed160e671

Both of the things you've stated here are true. The sha256sum
utility says
6e4eec90337e849ade7103723b9a99631c1f0d19990d6e8412dc42f5ae8b304d
while the sha1sum utility says
3547552b1009283f7ae31fded32ad33ed160e671. Make note that one is a
SHA2-256 checksum while the other is a SHA1 checksum.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/PWCBBFSMX4WT5H5K6DBQ4GZRGTLJRZHS/


[Distutils] Re: API for SHA-256 fingerprints

2019-02-12 Thread Jeremy Stanley
On 2019-02-12 12:42:27 -0500 (-0500), Wes Turner wrote:
[...]
> - cryptographically sign the SHA-256 checksums with a key and retrieve the
> corresponding key over a different channel
[...]

If you're going to use asymmetric cryptography with PKI to sign
something, you might as well just directly sign (a hash of) the
package file rather than merely signing (a hash of) its checksum.
Either way you're relying on the strength of your signing
implementation, so also having to rely on the strength of the
checksum is just added potential weakness and complexity.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/22DUNOXZSRCBQBT3CLTRJHICB5LKKSSI/


[Distutils] Re: API for SHA-256 fingerprints

2019-02-12 Thread Jeremy Stanley
On 2019-02-12 13:37:20 -0500 (-0500), Wes Turner wrote:
> MD5 is no longer suitable for verifying package integrity.
> 
> https://en.wikipedia.org/wiki/MD5#Security
> 
> > The security of the MD5 hash function is severely compromised. A
> > collision attack exists [...] there is also a chosen-prefix
> > collision attack
[...]

The difference between collision (or chosen-prefix collision) and
preimage (or second preimage) attacks is still very relevant. With
MD5 you can't trust that someone who provided you with an input and
a hash of that input hasn't carefully crafted that input so that
there is also a second input which results in the same hash. Or in
package terms, you can't trust that the package you've received
wasn't part of a contrived scheme on the part of someone you've
already decided to trust. You can still rest assured (for now
anyway) that the package you receive is the same one the person or
system providing the MD5 checksum intended for you to receive.

But because trying to explain this nuance to people is considerably
harder than just saying "MD5 bad" it's simply not worth trying to
have the discussion most of the time, and so easier instead to
replace it with a more modern alternative and move on with your
life.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/FXHHQGKMR6UHYDCBAXJTPFPFY52YVKQ5/


[Distutils] Re: API for SHA-256 fingerprints

2019-02-12 Thread Jeremy Stanley
On 2019-02-12 17:02:25 -0500 (-0500), Wes Turner wrote:
> On Tuesday, February 12, 2019, Wes Turner  wrote:
[...]
> > It is possible to find a nonce value that causes an arbitrary package to
> > have the same MD5 hash as the actual package.
> 
> e.g. browsers MUST NOT rely upon MD5 for x.509 certificate SSL/TLS/HTTPS
> fingerprints for exactly this reason.
[...]

I fear we're verging far into armchair crypto here, but you're
either making buzzword soup or have a severely flawed understanding
of the algorithms involved. There is no nonce in an IETF RFC 1321
(colloquially "MD5 checksum") implementation, so please at least
attempt to frame your assertions using terms found in the canonical
literature.

Creating a malicious package which computes to the same MD5 checksum
as an existing package of your choice would require that the second
preimage resistance of the MD5 algorithm is broken, or that you got
(time complexity 2^128) "lucky." Uses of MD5 elsewhere which mix in
attacker-controlled inputs to generate the reference output are
another story entirely, but as with the any of the information
security field the actual risk depends on your threat model.

I'm not about to recommend MD5 to anyone these days, don't get me
wrong. There are (at least marginally, again depending on your
threat model) better alternatives which require no additional effort
if you're designing a system from scratch. But let's not
mischaracterize the qualities of any algorithm, as it makes it
difficult for someone who does understand the differences to take us
seriously.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/6O3BHTLDPBHF3RNBXKJFXMH6X432C7AX/


[Distutils] Re: API for SHA-256 fingerprints

2019-02-12 Thread Jeremy Stanley
On 2019-02-12 18:42:29 -0500 (-0500), Wes Turner wrote:
[...]
> All it has to be is an archive containing a setup.py.
> 
> "MD5 considered harmful today:
> Creating a rogue CA certificate" (2008)
> https://www.win.tue.nl/hashclash/rogue-ca/

You keep trotting out PKI examples as if they have anything
whatsoever to do with checksumming, but I'm quickly getting the
distinct impression you don't actually know the difference so I'll
stop now as we've gone well off-topic for this list.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/WIOXLEBGNIUFB4267P6UR3GA27NGZ44O/


[Distutils] Re: Fwd: BitBounc BS Was: psycopg2/psycopg2-binary

2019-04-06 Thread Jeremy Stanley
On 2019-04-06 15:37:45 -0600 (-0600), Bert JW Regeer wrote:
> Is there a contact for the list owner so that people like this can
> get removed?
[...]

The footer of every message mentions:

> https://mail.python.org/mailman3/lists/distutils-sig.python.org/

Which in turn states, "To contact the list owners, use the following
email address: distutils-sig-ow...@python.org" (though as a
moderator/owner of lots of mailman listservs myself, I would half
expect anything you send there will get lost in a sea of @qq.com
spam).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/N7IEP4YTGPZAQQJXPR4K25SZUFWTAIHZ/


[Distutils] Re: Archive this list & redirect conversation elsewhere?

2020-07-29 Thread Jeremy Stanley
On 2020-07-30 07:17:03 +0530 (+0530), Pradyun Gedam wrote:
> TL;DR: OK to archive this mailing list? Reply by Aug 30th.
[...]

I find it disappointing that there will no longer be a mailing list
for discussions of Python packaging. Web forums with some E-mail
integration are hardly the same. But those of us who still use
E-mail (and worse, Usenet) eventually need to get out of the way of
the wheels of progress lest they run us over.

Many thanks to those who have maintained, moderated, and
collaborated through this list over the years. It has been much
appreciated.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/LJHH2CQA3RKSHYZH5C5LPXGFZ6P5KMWF/


[Distutils] Re: Archive this list & redirect conversation elsewhere?

2020-07-30 Thread Jeremy Stanley
On 2020-07-30 11:51:24 +0100 (+0100), Paul Moore wrote:
[...]
> 1. Will dropping distutils-sig mean that people who prefer
> interacting here lose their voice in packaging discussions?

Probably not. I think the fact that most of the list's prior
conversation has already moved to Discourse means this is already
the case, and so closing this ML is probably more a reflection of
the reality that those voices are effectively absent in relevant
conversations now.

> 2. Will having one fewer "recommended place" to start discussions
> make it easier for new participants to get involved?
[...]

Maybe. But as I've seen in many other communities, discussions start
organically in a variety of places and platforms, and are rarely
constrained by community consensus "recommendations" of venue.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/PCUVODMMWRAPGXA6PUJ5KHE5YU23T42O/


[Distutils] Re: Archive this list & redirect conversation elsewhere?

2020-07-30 Thread Jeremy Stanley
On 2020-07-29 22:51:37 -0400 (-0400), Sumana Harihareswara wrote:
[...]
> Jeremy, I'm not sure whether you were serious? If your
> disappointment is only out of nostalgia, then yeah, accepting
> change makes sense. But if your disappointment is because the
> Discourse experience is/will be worse for your participation, then
> it's totally fine to speak up and tell us how.
[...]

Robert also pretty accurately described my challenges with
Discourse, so I won't bother reiterating his points. I skim a good
1k mailing list messages daily across dozens of other communities
and reply with the press of a key; mailing lists make that workflow
easy compared conversation scattered between lots of different Web
forum sites with mediocre SMTP notification and response, poor
context quoting, et cetera. And I'm the sort of luddite who still
laments that many of these communities didn't stick with Usenet, but
remembering how the spam problem there got so out of control at a
time before we had reasonable technologies to deal with it, I don't
begrudge anyone the decision to abandon it.

Basically I've come to accept that most of the Python packaging
discussions lately have occurred entirely in a place where I've been
unable to follow them and reply easily, and that seems to be the
consensus choice of the Python community so I won't ask others to
cater to my outmoded workflows nor to necessarily value the fluid
sorts of cross-community participation they used to enable. It means
that my voice won't be included often, if ever, and that I'll be
less likely to find out tidbits like Setuptools 20.3 in October
turning on the new dep solver (and that the many projects I'm
involved in ought to test it and follow up with bug reports before
they begin to impact our users). Still, there are plenty of other
places I can spend my time and effort more effectively, so I'm not
particularly bitter about it.

> What if we bridged them, instead? Barry Warsaw in
> https://discuss.python.org/t/disappointed-and-overwhelmed-by-discourse/982/15
> suggested:
> 
> > My ultimate dream would be to add an IMAP and/or NNTP interface
> > directly to [Mailman 3/HyperKitty]. Then I could use my normal
> > mail application to catch up and interact with Mailman lists in
> > a very lightweight way, driven entirely by my own workflow. That
> > plus a Discourse bridge would be a pretty powerful and flexible
> > combination.
> 
> Is that something that other folks here who have trouble with
> Discourse would find fruitful? If so, we can start pushing to make
> it happen.

The idea there wasn't particularly fleshed-out so it's hard to say.
I'm assuming he meant a mechanism for mirroring/proxying
conversations between Discourse and MM3; if so then, yes, that might
help but there's really no way to know without a working prototype.
It also sounds like rather a lot of work and I'm certainly not going
to ask anyone else to expend effort just to cater to my personal
workflows. Discourse is not to my taste, but de gustibus non est
disputandum.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/KME5AJHBGFORUZTSS76J2OZ6VRLOTLOC/


[Distutils] Re: Archive this list & redirect conversation elsewhere?

2020-07-30 Thread Jeremy Stanley
On 2020-07-30 12:16:17 + (+), Jeremy Stanley wrote:
[...]
> I'll be less likely to find out tidbits like Setuptools 20.3 in
> October turning on the new dep solver
[...]

Er, clearly I meant pip. ;)

Also thanks for sending an update about it to the pypi-announce ML
just now!
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/EZUDONGU27R7GNGOUX6TVG3WGHSYDYNG/


[Distutils] Re: Installation Instructions for Python?

2020-12-11 Thread Jeremy Stanley
On 2020-12-08 18:29:33 + (+), Cort, Brian wrote:
> We were trying to create an automated installation process. As
> part of this, our software deployment team has requested the
> installation instructions, which I was unable to locate. Can you
> help by pointing me towards them?

The most official instructions I'm aware of are these:

https://www.python.org/downloads/release/python-391/
https://devguide.python.org/setup/
https://github.com/python/cpython/blob/master/README.rst#build-instructions

That said, most people obtain Python from a distributor and don't
build it themselves nor install it directly (unless maybe using
commercial operating systems which don't ship it by default).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/V3JOLJAR4EZWC7JVGMPMNZTLEHKYWMNR/


[Distutils] Re: Archive this list & redirect conversation elsewhere?

2021-05-09 Thread Jeremy Stanley
On 2021-05-07 15:21:36 -0400 (-0400), Sumana Harihareswara wrote:
> I figure now's a good time to revive this question, so that the new
> packaging community/project manager
> https://pyfound.blogspot.com/2021/04/the-psf-is-hiring-python-packaging.html
> can have a cleaner slate and potentially have fewer things to subscribe to
> when they come in!
[...]

On a whim I created an account on discuss.python.org and set my
preferences to enable mailing list mode. I'm assuming the
"packaging" category is the intended modern analog of the
distutils-sig mailing list, is that correct? Is there a guide for
former mailing list users moving to Discuss, which covers things
like limiting the subscription to a particular category or list of
categories, or how to construct your messages so that they end up in
the correct category when posting via E-mail?

More generally, is E-mail-only interaction with Discuss working well
for others?
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/QDW4B5IP4Y6MFCQS6ILUFM6FEVFCASMG/


[Distutils] Re: PEP 440 Issue

2021-04-27 Thread Jeremy Stanley
On 2021-04-27 20:32:07 +1000 (+1000), Nick Coghlan wrote:
[...]
> The simplest resolution would be to drop the ".taskslab11419" section, and
> just make the version 0.2.0 instead. Alternatively, if keeping the number
> is important, you could make a dev release (0.2.0.dev11419), or use a 4
> segment version number (0.2.0.11419).
[...]

And yet another option is to record it in custom PEP 566 metadata at
package build time. Modules like setuptools_scm and pbr use this to
store Git commit identifiers and similar information so that
programs can access the data at runtime (e.g. for annotating their
version output).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/4TA2OOWHLQBG6TSSWPD3LWOI47ZOYMJ4/


[Distutils] Re: Building Pre-releases with setup.cfg

2021-02-09 Thread Jeremy Stanley
On 2021-02-09 09:48:31 -0500 (-0500), Matthew Gilbert wrote:
> I'm wondering if it is possibly to build pre-release tags using only a
> setup.cfg file? From
> https://setuptools.readthedocs.io/en/latest/userguide/distribution.html#tagging-and-daily-build-or-snapshot-releases
> it seems like this is possible via setup.py, but I was unable to find
> anything related to setup.cfg.
[...]
> Possibly this can be configured using the --config-setting flag but I have
> been unable to get this to work or find any documentation on it. Ideally I
> would be able to pass a string so I can build a dev version for testing
> uploads to Test PyPI, e.g. "0.0.1devHASH"
[...]

I'm probably misunderstanding your question, but typically I've seen
something like PBR or Setuptools-SCM used to automatically calculate
the version at dist build time (including inferring something based
on the number of commits on the branch since the last tag, embedding
abbreviated Git commit IDs, or whatever).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/PUASLWVRUZOC37VOKKP2AZFYGHFNQDY2/


[Distutils] Re: Making setup.py run external command to build

2021-03-25 Thread Jeremy Stanley
On 2021-03-25 20:01:12 + (+), Thomas Kluyver wrote:
> I'm surprised it fails on the wheels - how exactly are you using
> pip?
> 
> If you're relying on PEP 517, then pip 18.1 is too old to install
> it from an sdist - PEP 517 support was added in 19.0
> (https://pip.pypa.io/en/stable/news/#id448 ). But that shouldn't
> be a problem for installing from a wheel that has already been
> made.

Unless you're making ABI3 wheels, which require pip 20.0 or later to
find them (and then they'll fall back to pulling the sdist which
they can't build because of no pyproject.toml support before 19.0).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/Y4DS6NLB4GMWSZUQBUJPFSH7ED326DFP/


[Distutils] Re: Making setup.py run external command to build

2021-03-25 Thread Jeremy Stanley
On 2021-03-25 19:07:50 + (+), Julian Smith wrote:
[...]
> I've come across a problem where an old pip-18.1 fails to install from
> an sdist or wheel, with error:
> 
>   ERROR: You must give at least one requirement to install (see "pip help 
> install")
> 
> pip-20.2.1 and pip-21.0.1 work fine, including installing from an sdist
> on test.py.pi.
> 
> Should i be worried about this? I'd like my package to install on
> anyone's machine, regardless of their pip version. But maybe users are
> expected to always run a recent pip version? After all it's easy enough
> to upgrade it.

According to https://pip.pypa.io/en/stable/news/#id448 support for
pyproject.toml files was first added in pip 19.0 (just over two
years ago). If you have users who rely on distribution-packaged pip
then this may not be old enough for them to have it yet. For example
the current Debian stable release provides pip 18.1, and the
next-most-recent Ubuntu LTS (18.04) provides 9.0.1.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/FEXQGFTNDL5NKTR2UZDV77BVULMFIBU7/


[Distutils] Re: Shutting down distutils-sig, for real

2021-12-30 Thread Jeremy Stanley
On 2021-12-30 09:55:34 -0500 (-0500), Andrew M. Kuchling wrote:
[...]
> let's shut the list down for 2021!
[...]

No objection from me at this point. While I find Discourse to
perform suboptimally as a mailing list server, all the packaging
discussions seem to have moved there (or into GitHub issues, which
are an even worse substitute for a mailing list), so it'll have to
do.

Thanks for all your hard work as a ML moderator over the years, and
to everyone else who made this list great!
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
-- 
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/IYIWFQ34DSZCPULQRFBBM2KSDBRLCB56/