[Distutils] Setuptools adopts distutils

[Jason R. Coombs]

I’m pleased to announce the release of Setuptools 48, which adopts the 
distutils codebase from CPython per 
pypa/setuptools#417 and 
pypa/packaging-problems#127.

Given the amount of change this effort involved, it’s likely unstable and thus 
the major version bump. Please report issues at the Setuptools issue tracker. 
I’ll be around today (IRC, Gitter, Slack) to either disable the functionality 
or add an escape hatch if needed.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/74WZ7D3ARF7B3N6MAV2JBV3DW6TRHFIV/

[Distutils] Re: tiny sprint Saturday, June 8th

[Jason R. Coombs]

I’ll be around for consulting. Feel free to ping me if I can elucidate any 
particularly dark corners.

> On 6 Jun, 2019, at 15:43, Sumana Harihareswara  wrote:
> 
> A few folks will be getting together on Saturday and doing a short in-person 
> sprint on some Python packaging & distribution tools, around 10am-4pm ET, at 
> a coworking space/lounge in New York City.
> 
> A few packaging/distribution folks, e.g., a Twine contributor, a pip bug 
> fixer/triager, and a Warehouse maintainer (me), are confirmed as coming. I 
> figure we'll review some open pull requests, triage bugs to find ones we can 
> close as no longer reproducible, and explain stuff to each other.
> 
> I think we've already run out of space for who can participate in person, but 
> please feel free to hang out and chat with us via IRC! I'll be on Freenode 
> IRC (#pypa-dev) as user "sumanah". And that way logs of our conversations 
> will also be available at http://kafka.dcpython.org/channel/pypa-dev .
> 
> (If you have never contributed to Python packaging/distribution tools before, 
> and you want to start, this is probably not the best event for you; let me 
> know, and I'll set up a more introductory event in the future.)
> 
> -- 
> Sumana Harihareswara
> Warehouse project manager
> Changeset Consulting
> https://changeset.nyc
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pypa-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pypa-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pypa-dev/5f46a872-5dd7-e66f-a781-d48d116e1c85%40changeset.nyc.

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/BFAVB2TJ5POQ73VXGYNXKVBSWKQ3SNNZ/

Re: [Distutils] Twine 1.10.0rc1 on Test PyPI

[Jason R. Coombs]

Thanks for working on this!

In my particular use-case, I rarely run twine myself, but instead rely on the 
Travis-CI DPL 
routine. 
Looking at that code, I don’t see any means I have to test a pre-release 
version.

Given the presumably broad impact this one use-case has, it would be nice if 
there were a way to test it against pre-release versions of twine (and maybe 
also wheel, pip, and setuptools). Perhaps it would be worthwhile to propose a 
hook to that project to enable the versions of those projects to be specified 
for selective testing.

Oh, I just had an idea - perhaps one could set the PIP_PRE environment variable 
and that would affect the install and allow the pre-release to be tested. I’ll 
give that a go.

On 3 Mar, 2018, at 11:06, Sumana Harihareswara 
> wrote:

Wrong URL (did I mention I'm new at this?). View 1.10.0rc1, including a
fairly spiffy new README, at:
https://test.pypi.org/project/twine/1.10.0rc1/ -- and please pass word
along to our downstreams.

-Sumana

On 03/02/2018 05:32 PM, Sumana Harihareswara wrote:
(So it turns out I've taken on a volunteer gig, which is that I'm now
one of the Twine maintainers. I may be wrong about how to do this -
please feel free to comment on https://github.com/pypa/twine/pull/314
which is where I'm pulling together a new release checklist for myself.)

https://test.pypi.org/manage/project/twine/release/1.10.0rc1/

This is a release candidate for Twine 1.10.0 which I'm planning to
release early next week.

This release improves project registration usage text (in some cases
removing it where inapplicable), and updates `--repository[-url]` usage
text, prints progress to `stdout` instead of `stderr`, improves the
progressbar, and reorganizes and improves user and developer documentation.

Please see the changelog
https://twine.readthedocs.io/en/latest/changelog.html for detailed notes
under "Next feature release".

I believe this is how you test it out:

 pip install --upgrade --pre --index-url https://test.pypi.org/simple/
--extra-index-url https://pypi.org/simple twine

Please check existing open issues at
https://github.com/pypa/twine/issues and open new ones if you have
problems. Thanks!


--
Sumana Harihareswara
Changeset Consulting
https://changeset.nyc

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Twine 1.10.0rc1 on Test PyPI

[Jason R. Coombs]

I tried but as you can see in this 
job<https://travis-ci.org/yougov/paradocx/jobs/348705862>, the environment 
variables aren’t honored, so it seems I cannot test a twine release in Travis. 
At this point, I think I’ll just wait for the official release.

On 3 Mar, 2018, at 11:17, Jason R. Coombs 
<jar...@jaraco.com<mailto:jar...@jaraco.com>> wrote:


This sender failed our fraud detection checks and may not be who they appear to 
be. Learn about spoofing<http://aka.ms/LearnAboutSpoofing>
Feedback<http://aka.ms/SafetyTipsFeedback>
Thanks for working on this!

In my particular use-case, I rarely run twine myself, but instead rely on the 
Travis-CI DPL 
routine<https://github.com/travis-ci/dpl/blob/master/lib/dpl/provider/pypi.rb>. 
Looking at that code, I don’t see any means I have to test a pre-release 
version.

Given the presumably broad impact this one use-case has, it would be nice if 
there were a way to test it against pre-release versions of twine (and maybe 
also wheel, pip, and setuptools). Perhaps it would be worthwhile to propose a 
hook to that project to enable the versions of those projects to be specified 
for selective testing.

Oh, I just had an idea - perhaps one could set the PIP_PRE environment variable 
and that would affect the install and allow the pre-release to be tested. I’ll 
give that a go.

On 3 Mar, 2018, at 11:06, Sumana Harihareswara 
<s...@changeset.nyc<mailto:s...@changeset.nyc>> wrote:

Wrong URL (did I mention I'm new at this?). View 1.10.0rc1, including a
fairly spiffy new README, at:
https://test.pypi.org/project/twine/1.10.0rc1/ -- and please pass word
along to our downstreams.

-Sumana

On 03/02/2018 05:32 PM, Sumana Harihareswara wrote:
(So it turns out I've taken on a volunteer gig, which is that I'm now
one of the Twine maintainers. I may be wrong about how to do this -
please feel free to comment on https://github.com/pypa/twine/pull/314
which is where I'm pulling together a new release checklist for myself.)

https://test.pypi.org/manage/project/twine/release/1.10.0rc1/

This is a release candidate for Twine 1.10.0 which I'm planning to
release early next week.

This release improves project registration usage text (in some cases
removing it where inapplicable), and updates `--repository[-url]` usage
text, prints progress to `stdout` instead of `stderr`, improves the
progressbar, and reorganizes and improves user and developer documentation.

Please see the changelog
https://twine.readthedocs.io/en/latest/changelog.html for detailed notes
under "Next feature release".

I believe this is how you test it out:

 pip install --upgrade --pre --index-url https://test.pypi.org/simple/
--extra-index-url https://pypi.org/simple twine

Please check existing open issues at
https://github.com/pypa/twine/issues and open new ones if you have
problems. Thanks!


--
Sumana Harihareswara
Changeset Consulting
https://changeset.nyc<https://changeset.nyc/>


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] setup.cfg - to interpolate or not to interpolate

[Jason R. Coombs]

In Setuptools #889 (https://github.com/pypa/setuptools/issues/889) and Python 
20754 (http://bugs.python.org/issue20754), setuptools and distutils are 
proposing a backward-incompatible change - to remove ConfigParser interpolation 
when parsing setup.cfg (and maybe other config) files.

I’d previously thought that it was an unintended regression from Python 2 to 
introduce interpolation, but on deeper investigation, interpolation has been 
present all along.

While it’s conceivable that no projects are relying on the interpolation 
behavior, I’d be surprised if none have at least worked around the 
interpolation behavior by escaping percent signs.

So I’m left with the sinking feeling there’s no easy transition from the status 
quo to the desired state (no interpolation).

I’d like feedback, in either ticket, about whether this change is worth the 
trouble and if so if there are any ways to mitigate the risks of introducing 
such a change.

If someone can assist on this effort, that would also be greatly appreciated. 
I’ll likely abandon the effort and close the tickets as won’t fix unless we can 
identify someone to champion the effort.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools moving away from self install - testing needed

[Jason R. Coombs]

In [setuptools 581](https://github.com/pypa/setuptools/issues/581), I’ve 
drafted a version of setuptools that essentially drops support for self 
installation, relying on pip install as the only supported mechanism for 
installing setuptools, both in the ez_setup.py bootstrap (for which there’s a 
minimally compatible shim for replacement) and in the codebase itself.

I invite everyone to install the pre-release linked in the ticket and provide 
feedback there.  I’ll give it at least a week before I cut an official release. 
If you need more time, just let me know.

Regards,
Jason
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Commit 17f89f4 "Update sdist to use sdist_add_defaults forward compatibility"

[Jason R. Coombs]

The discussion started here: https://github.com/pypa/setuptools/pull/750

Basically, I don’t understand exactly why that functionality was disabled, but 
I give my best guess in that ticket - mainly that I think data_files is 
incompatible with encapsulated installs, so I suspect it was explicitly, 
intentionally excluded, though I’ve found no evidence to that effect.

So for now, for compatibility, I’ve explicitly excluded that functionality with 
a comment.

I’ve also explicitly overridden _add_defaults_python to capture the overriding 
behavior, which while similar isn’t identical. I’m leaving it as a subsequent 
exercise to see if that implementation can be merged.

In my conclusion in that PR, I’ve suggested we go ahead and re-enable 
data_files, though I suggest we also add a test capturing that expectation.

> On 15 Oct, 2016, at 13:43, Brandon Casey  wrote:
> 
> Hi Jason,
> 
> I just started looking at the setuptools repo last night to see if I
> could figure out why setuptools doesn't place data_files inside an
> sdist and I saw your commit 17f89f4 to teach the sdist command to use
> the python 36 compatibility behavior.  But it overrides
> _add_defaults_data_files() which actually adds the data_files to the
> filelist, and replaces it with an empty function.
> 
> If I remove this function, so that the one from
> py36compat.sdist_add_defaults is used, then it seems to work correctly
> for me.  But that is the extent of my testing so far.
> 
> The doc comment for the new _add_defaults_data_files function just says:
> 
>   Don't add any data files, but why?
> 
> I see _add_defaults_python() was also overridden and is now only
> slightly different from the one in py36compat.sdist_add_defaults.  It
> looks like maybe you were just trying to make use of the py36compat
> functions without introducing any change in behavior.  Is that the
> only reason that _add_defaults_python() differs from the one in
> py36compat and why _add_defaults_data_files is disabled?  Or do you
> have some concerns about enabling this functionality?  Any thoughts of
> enabling it?  If there's some active discussion or explanation on a
> mailing list somewhere, please feel free to point me towards that.
> 
> Thanks,
> -Brandon

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Switch PyPA from IRC to Gitter or similar

[Jason R. Coombs]

In #pypa-dev, I raised the possibility of moving our PyPA support channels from 
IRC to another hosted solution that enables persistence. Although IRC has 
served us well, there are systems now with clear feature advantages, which are 
crucial to my continuous participation:

- always-on experience; even if one’s device is suspended or otherwise offline.
- mobile support — the in-cloud experience is essential for low power and 
intermittently connected devices.
- push notifications allow a project leader to remain largely inactive in a 
channel, but attention raised promptly when users make a relevant mention.
- continuous, integrated logging for catching up on the conversation.

Both Gitter and Slack offer the experience I’m after, with Gitter feeling like 
a better fit for open-source projects (or groups of them).

I’ve tried using IRCCloud, and it provides a similar, suitable experience on 
the same IRC infrastructure, with one big difference. While Gitter and Slack 
offer the above features for free, IRCCloud requires a $5/user/month 
subscription (otherwise, connections are dropped after two hours). I did reach 
out to them to see if they could offer some professional consideration for 
contributors, but I haven’t heard from them. Furthermore, IRCCloud requires an 
additional account on top of the account required for Freenode.

In addition to the critical features above, Gitter and Slack offer other 
advantages:

- For Gitter, single-sign on using the same Github account for authentication 
and authorization means no extra accounts. Slack requires one new account.
- An elegant web-based interface as a first-class feature, a lower barrier of 
entry for users.
- Zero-install or config.
- Integration with source code and other systems.

It’s because of the limitations of these systems that I find myself rarely in 
IRC, only joining when I have a specific issue, even though I’d like to be 
permanently present.

Donald has offered to run an IRC bouncer for me, but such a bouncer is only a 
half-solution, not providing the push notifications, mobile apps (IRC apps 
exist, but just get disconnected, and often fail to connect on mobile provider 
networks), or integrated logging.

I note that both Gitter and Slack offer IRC interfaces, so those users who 
prefer their IRC workflow can continue to use that if they so choose.

I know there are other alternatives, like self-hosted solutions, but I’d like 
to avoid adding the burden of administering such a system. If someone wanted to 
take on that role, I’d be open to that alternative.

I’d like to propose we move #pypa-dev to /pypa/dev and #pypa to /pypa/support 
in gitter.

Personally, the downsides to moving to Gitter (other than enacting the move 
itself) seem negligible. What do you think? What downsides am I missing?
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] on integrated docs in Warehouse and PyPI

[Jason R. Coombs]

I had some thoughts on documentation integration (and its deprecation in 
Warehouse), which I wrote up here: 
https://paper.dropbox.com/doc/Integrated-Docs-is-an-Essential-Feature-HEqnF8iWzCFwkCDaz0p8t

I include the full text below for ease of access and response.

Yesterday, as I was migrating Setuptools from PyPI to Warehouse due to this 
issue, Donald alerted me to the 
fact that Warehouse does not plan to support documentation 
hosting, 
and as a result, integrated documentation for the packaging infrastructure is 
deprecated.

At first blush, this decision seems like a sound one - decouple independent 
operations and allow a mature, established organization like 
RTD to support and manage Python Package 
documentation. I have nothing but respect for RTD; I reference them here only 
as the prime example of a third-party doc hosting solution.

After spending most of a day working on getting just one project documentation 
(mostly) moved from PyPI to RTD, I’ve realized there are several shortcomings 
with this approach. Integrated hosting provides several benefits not offered by 
RTD:


  *   Uniform custody - the person or team that owns/maintains the package also 
owns/maintains the documentation with a single point of management and 
authorization.
  *   Shared credentials - the accounts used to administer the packages are 
re-used to authenticate authorized users to the documentation. RTD requires a 
separate set of accounts for each user involved with the documentation.
  *   Shared naming - a name registered as a package is automatically reserved 
for the documentation.
  *   Automatic linkage - PyPI provides a Package Documentation link on each 
package that has documentation.
  *   Control over the build process - although RTD does provide excellent 
hooks for customization and control, the process is essentially out of the 
hands of the operator. Thus when issues like 
this arise (probably 
rarely), the user is at the mercy of the system. With PyPI hosting, it was 
possible to manually build and upload docs when necessary and to control every 
aspect of the build process, including platform, Python version, etc.
  *   One obvious choice - although RTD today is the obvious choice for 
hosting, I can see other prominent alternatives - using Github pages or self 
hosting or perhaps another service will emerge that’s more integrated with the 
packaging process. Having a sanctioned, integrated documentation hosting gives 
users confidence that it’s at least a good default choice if not the best one.
  *   API access - Although RTD hopes to provide support for project creation 
through an API, it currently only allows querying through the public API. 
Therefore, it’s not feasible through tools to mechanically configure projects 
in RTD. Each project has to be manually configured and administered.

For me, this last limitation is the most onerous. I maintain dozens of 
projects, many of them in collaboration with other teams, and in many of those, 
I rely on a model implementation that 
leverages PyPI hosting as part of the package release process to publish 
documentation. Moving each of these projects to another hosting service would 
require the manual creation and configuration of another project for each. As I 
consider the effort it would take to port all of these projects and maintain 
them in a new infrastructure, I’m inclined to drop documentation support for 
all but the most prominent projects.

The linkage provided by PyPI was a most welcome feature, and I’m really sad to 
see it go. I’d be willing to give up item 5 (Control) if the other items could 
be addressed.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools 18.0b1 available for testing (improved Cython build support).

[Jason R. Coombs]

I’m pleased to announce a fix for improved Cython support per Setuptools #288 
(https://bitbucket.org/pypa/setuptools/issue/288). I’ve published a beta 
release 18.0b1 in the Setuptools project downloads 
(https://bitbucket.org/pypa/setuptools/downloads/setuptools-18.0b1.zip) for 
pre-release testing. Release notes are here 
(https://bitbucket.org/pypa/setuptools/src/4249669040239bff20196b17ccd4d73870d37343/CHANGES.txt#cl-10).

Please, if you use any projects with Cython modules, give this install a trial 
run and report any issues in the Setuptools bug tracker. I hope to release 18.0 
in the next few weeks if there aren’t any emergent blockers.

Thanks!

Jason
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pip upgrade woes

[Jason R. Coombs]

Setuptools advertises in its description, Easily download, build, install, 
upgrade, and uninstall Python packages. It was intended to support the 
uninstall model, though at the time that was written, easy meant easier than 
finding and removing each of the files by hand.

If you're using eggs, the default install model for Setuptools, they can be 
uninstalled by removing any reference to it from easy-install.pth (which causes 
it not to be added to sys.path) or by removing the .egg from the site dir (in 
which case Setuptools will remove the easy-install.pth reference).

It's not a clean uninstall in all cases, because any scripts will still linger.

Nevertheless, it's still a model that works reasonably well.

Setuptools can't help with distutils-installed packages. Note that Setuptools 
will always install itself as an egg unless installed by another manager such 
as pip or apt.

-Original Message-
From: pypa-...@googlegroups.com [mailto:pypa-...@googlegroups.com] On Behalf Of 
Paul Moore
Sent: Tuesday, 24 March, 2015 14:29
To: Ionel Cristian Mărieș
Cc: pypa-dev; DistUtils mailing list
Subject: Re: pip upgrade woes

On 24 March 2015 at 18:10, Ionel Cristian Mărieș cont...@ionelmc.ro wrote:
 There's one issue with pip upgrade that annoys me occasionally, and 
 when it does it's very annoying.

 Every so often me or some customer has to upgrade some core packages 
 like pip, setuptools or virtualenv on some machine. Now this becomes 
 very annoying because said packages were installed there with either 
 easy_install or just `setup.py install`. Several upgrades like that 
 and now the machine has a hadful of eggs there. Lots of mistakes were 
 made but what's done is done.

 Now, if `pip upgrade pip setuptools virtualenv` it will run around a 
 bit, flap its wings and in the end it's not gonna fly like an eagle, 
 and won't be able to go beyond it's cursed fences. An so, I feel like 
 chicken farmer when I try to upgrade packages and pip can't upgrade 
 them. Cause those old eggs are still going to be first on sys.path. 
 And when I try to run pip it's still that old one.

 Sometimes few `pip uninstall` solve the issue, but most of the time I 
 have to manually remove files because pip can't figure out what files to 
 remove.

 One big issue is that pip uninstall only uninstalls the first package 
 it finds, and similarly, pip install will only uninstall the first 
 package it finds before coping the new files.

 This whole process becomes a whole lot more annoying when you have to 
 explain someone how to cleanup this mess and get latest pip and setuptools.

 So I'm wondering if there's a better way to cleanup machines like 
 that. Any ideas?

If I understand your problem correctly, the issue is that these machines have 
older installs of packages, added by tools that don't have uninstall 
capabilities, using formats that are not used by pip.
You're not able to get pip uninstall to work either because the uninstall data 
isn't in a form pip can handle, or because there is no uninstall data. I 
suspect the first on sys.path issue is caused by setuptools' .pth files, 
which are even messier to tidy up, from what I recall of my infrequent dealings 
with them.

It should be possible to write some sort of purge command that searches out 
and removes all traces of a package like that. I wouldn't want pip to do 
anything like that automatically, for obvious reasons.
I'm not even that keen on it being a new pip subcommand, because there's a lot 
of risk of breakage (I'd imagine the code would need a certain amount of 
guesswork to identify non-standard files and directories that belong to a 
package).

As a start, I'd suggest looking at writing some sort of independent 
purge-package command that you could use when you hit problems (pip install -U 
setuptools... weirdness happens, so purge-package setuptools; pip install 
setuptools). If all the scenarios that tool handles end up being clearly 
defined and low-risk, then it might be that there's scope for a pip purge 
command of some sort based on it, that removes all trace of a package even when 
the standard uninstall metadata isn't available.

Maybe someone else here with more understanding of the history of easy_install 
and how eggs used to work (maybe still do, for all I
know...) can offer something more specific. Sorry that I can't.
Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools 4.0 and 4.0.1 superseded by 3.8.1

[Jason R. Coombs]

Due to the regression reported in [Setuptools #218]( 
https://bitbucket.org/pypa/setuptools/issue/218/setuppy-develop-fails-with-some-packages),
 I've removed all versions of Setuptools 4.x and closed that branch.

I will continue to explore the underlying issue, [Setuptools #210]( 
https://bitbucket.org/pypa/setuptools/issue/210) to seek a proper solution, 
possibly without backward incompatibility.

Please use Setuptools 3.8.1 or later. If another backward-incompatible release 
is made for anything other than issue 210, it will tagged as Setuptools 5.0, 
bypassing 4.0 altogether.

Regards,
Jason
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] recommended setuptools version

[Jason R. Coombs]

To follow up, 3.1 has been released and the 3.0.x releases have been restored.

 -Original Message-
 From: Distutils-SIG [mailto:distutils-sig-
 bounces+jaraco=jaraco@python.org] On Behalf Of Marius Gedminas
 Sent: Friday, 07 March, 2014 08:15
 To: distutils-sig@python.org
 Subject: Re: [Distutils] recommended setuptools version
 
 On Fri, Mar 07, 2014 at 02:02:59AM -0800, Benedek Zoltan wrote:
  I'm a bit confused with the setuptools versioning.
  When I install it by ez_setup.py it installs the 3.0.2 version, but on pypi 
  the
 latest version is 2.2.
 
  Is there a difference between the two? Which is the recommended one?
 
 3.0 was released last night, broke a bunch of popular software, and was
 hurriedly pulled from PyPI.
 
 The current version of ez_setup.py available at
 https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
 also installs 2.2.
 
 I suggest you stick with 2.2 until the dust settles.
 
 Marius Gedminas
 --
 2B OR NOT 2B == FF
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] tourist here, with a dumb RTFM question

[Jason R. Coombs]

Thanks Mike for the report. I haven't read the rest of this thread yet, and I 
want to respond to your OP before I do. As a result, there will likely be some 
duplication.

You're absolutely right to be dismayed.

I had incorrectly assessed the potential impact of this change. I had expected 
the impact to be minor and isolated. I underestimated the number of systems 
that automatically upgrade Setuptools as well as the number of packages that 
could be using Features. I relied too heavily on the DeprecationWarning to have 
flagged the breakage (to package maintainers and their users). I had expected 
issues like you raised in this post to have been hashed out or at least raised 
long before I considered the release.

I hope my decision to backout the release serves as a strong signal that this 
type of breakage is unacceptable. I hope the tickets filed and discussion will 
lead to a better mechanism for communicating these types of changes in the 
future. In the future, I will leverage Twitter to announce in advance any 
backward-incompatible releases, giving additional visibility and oversight.

I do also want to ask that deployment teams consider the implications of always 
upgrading to the latest release of Setuptools. I'm encouraged that they do, and 
I do want to keep that as a viable deployment workflow. However, at the same 
time, there is some inherent risk with automatically upgrading to 
known-backward-breaking versions. The purpose of adopting semver for versioning 
is to allow the project to signal directly and unambiguously when a release is 
backward-breaking. I haven't yet decided when or where to raise this 
discussion, and it certainly doesn't excuse the challenges faced Thursday.

It's clear to me from your post that there are use-cases for Features that 
aren't covered by simply using extras. Namely, extras doesn't support 
exclusion, only inclusion. I'll make mention of that in the deprecation ticket 
(https://bitbucket.org/pypa/setuptools/issue/65/) and follow up after 
understanding the problem better.

I'm the one that needs to apologize, not you. Thanks for your patience and 
understanding as I and the rest of the PyPA work toward the best means of 
communication (that doesn't require every package maintainer to follow 
distutils-sig).

Best regards,
Jason

 -Original Message-
 From: Distutils-SIG [mailto:distutils-sig-
 bounces+jaraco=jaraco@python.org] On Behalf Of Michael Bayer
 Sent: Thursday, 06 March, 2014 12:56
 To: distutils-sig@python.org
 Subject: [Distutils] tourist here, with a dumb RTFM question
 
 Hi Distutils !
 
 I don't follow this list and haven't looked at it in a long time.   However, 
 I'm
 learning via twitter that a brand new setuptools release that's just gone out
 has just removed the Feature mechanism.
 
 Now as you're all rolling your eyes and preparing to bang out frustrated
 replies how this was well announced and deprecated and warned about and
 wow really didn't you know the term paper was due today, OK first off let me
 say I am sorry!  I am not a distutils/setuptools maintainer.I am just
 someone that uses it, and as I include setuptools in my setup.py, I am also
 getting thousands of other people who download my product to use it as
 well.And I don't read the setuptools changelog!  Or the setuptools blog.  
 Or
 this list.  I assume you guys have it under control (and you certainly do!).
 There seem to be other people like me (people who write very widely used
 software) who also seem surprised!   And that is surprising too (as I am
 usually the only person to be surprised by these things that should not be
 surprising).  So I hope you can hold back your frustration with my clueless
 RTFMness long enough to answer these questions:
 
 1. where was this announced?   I'm wondering why there weren't loud
 blaring blog posts and tweets all over the place stating that on March 6,
 dozens of major packages are going to all break.
 
 2. what is the plan for unmaintained packages and old versions of existing
 packages on pypi that import Feature and can no longer be installed?Is 
 it
 just the case that a large amount of pypi just won't install anymore?
 
 3. What, if any, is the recommended approach going forward to a Python
 package that wishes to specify a command-line flag during install.  Here's
 multiple choice:
 
 a. you can use new setuptools API some new way to add -flags
 
 b. you can roll it yourself in setup.py using hack X
 
 c. your setup.py should never accept any kind of flags as that interferes
 with up-and-coming use case Q
 
 d. other
 
 If choice c., then here is another question.  My library includes optional C
 extensions.   On certain platforms, these C extensions don't build (like on
 Pypy, or on Windows if you don't have compilation tools installed).  In this
 regard it gracefully degrades.  But also, I want to be able to have a command
 line option to determine this as well.  Because!  Maybe 

Re: [Distutils] tourist here, with a dumb RTFM question

[Jason R. Coombs]

From: Distutils-SIG [mailto:distutils-sig-bounces+jaraco=jaraco@python.org] 
On Behalf Of Marcus Smith
Sent: Thursday, 06 March, 2014 16:16
To: DistUtils mailing list
Subject: Re: [Distutils] tourist here, with a dumb RTFM question



On Thu, Mar 6, 2014 at 10:37 AM, Michael Bayer 
mike...@zzzcomputing.commailto:mike...@zzzcomputing.com wrote:

On Mar 6, 2014, at 1:07 PM, Daniel Holth 
dho...@gmail.commailto:dho...@gmail.com wrote:

 pje said:

 The Feature() facility was never completely implemented or
 supported, and even if it were, it should be deprecated now, as it
 will not be compatible with the coming packaging systems based on PEP
 426. If you need separate features, use separate distributions and
 extras instead.
wait, ok this is that thing.  separate distributions means


SQLAlchemy-0.9.3-nocext.tar.gz
SQLAlchemy-0.9.3-cext.tar.gz

I'm new to understanding setuptools Feature myself, so don't crank too much, 
if I get anything wrong.

afaik, it means something like:

SQLAlchemy-X.Y.tar.gz
SQLAlchemy-cext-X.Y.tar.gz

i.e. isolating the feature into a separate project.

and then instead of having the handy  --with-cext option,  it has to be  pip 
install SQLAlchemy[SQLAlchemy-cext]

PJ, Jason: can you clarify what the alternative is supposed to look like?

I'm new to understanding Features as well, but based on PJ's post, I would 
agree with Marcus here. This is how I interpreted the use of extras to 
supersede Features. I see a lot of advantages to this approach over build-time 
selection. Because it decouples the main project from the C-extension speedups, 
it gives the installer control over what behavior is present. It also gives 
deployment tools visibility over what capability is present (with features, 
it's harder to tell if the C-extension speedups are present; with extras, one 
can query pkg_resources).

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools 3.0b1 available for preview

[Jason R. Coombs]

I'm pleased to announce that Setuptools 3.0b1 is now available for download 
from the project downloads page:

https://bitbucket.org/pypa/setuptools/downloads

This backward-incompatible release contains the changes detailed in the 
CHANGES.txt file:

https://bitbucket.org/pypa/setuptools/src/3.0b1/CHANGES.txt?at=default

Please give this beta release a try or review the changes for any impact this 
might have on your package. I anticipate most users will find these changes of 
no consequence, but if there are any issues that might affect your package that 
don't have an obvious solution, please file a ticket.

Regards,
Jason
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools 2.0 Released

[Jason R. Coombs]

The PyPA is pleased to announce the Setuptools 2.0 release.



This backward-incompatible release drops support for Python 2.4 and Python 2.5, 
but is otherwise compatible with 1.x releases.



The 1.x series will continue to be supported for bug and security fixes for the 
foreseeable future. This means that any systems with older versions of Python 
should install setuptools2dev or use the legacy bootstrap script, available 
at https://bitbucket.org/pypa/setuptools/raw/bootstrap-py24/ez_setup.py for 
Python 2.4 and Python 2.5 users. That link is also published in the README as 
install instructions.



Please report any emergent issues at 
https://bitbucket.org/pypa/setuptools/issues.



Jason

PyPA

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to handle launcher script importability?

[Jason R. Coombs]

 -Original Message-
 From: Steve Dower [mailto:steve.do...@microsoft.com]
 Sent: Monday, 12 August, 2013 15:03

 Jason R. Coombs wrote:
  6. Two to three files to do the job of one. In fact, the job isn't
  much more than to invoke code elsewhere, so it seems ugly to require
  as many as three files to do the job. Then multiply that by the
  Python-specific version and you have up to six files for a single script.

 While I can understand this from the POV of the implementer/maintainer,
 I've never heard a single Windows user mention it. And with an embedded
 manifest, it's no more than one .py and one .exe-per-Python-version.

Thanks Steve (for this and other comments).

I do want to remind that silence is not consent. The multiple files per script 
has always bugged me (as a user, not implementer), but up until now, I've 
always considered it a necessary evil. And now that it's not necessary, it's 
just evil. Over the past day, I've realized/recalled even more problems 
stemming from side-by-side executables (some of which also apply to other 
non-executable side-by-side solutions such as markers and manifests):

1. Renames, deletes, and other actions must be synchronized. There's an 
implicit connection between the files, but it's implicit. And while it's 
relatively easy to imagine how one can manage the synchronicity, in practice, 
it's harder. For example, to delete a script, one has to be careful to delete 
{script_name}-script.py and {script_name}.exe. If one wants to rename, two 
renames have to occur. What would otherwise be a simple, intuitive operation 
now has a semantic and technical burden.
2. Discoverability is diminished. Imagine for example that you want to delete 
all scripts that reference a particular package (as one is wont to do when 
that package is removed). If there's any side-by-side content, it's not 
sufficient to grep the files and delete the matching files, but one must 
instead write a routine or otherwise resolve the matches to their side-by-side 
equivalents and perform the same operation on them.
3. A directory listing is distracting and unnecessarily twice as long, as 
there's two files per script. This necessarily means more scrolling and actual 
human time spent parsing and organizing that structure. And since there's not 
exactly two files per script (only those with a launcher), the pairing isn't 
consistent. With one file per script, the listing matches the essence of the 
directory's content.
4. Updates to the launcher won't apply to existing scripts. If the launcher is 
updated, the side-by-side versions will remain out-of-date until their scripts 
are re-installed. If the launcher is associated with the file type, then the 
state of the launcher can be managed independently of the scripts.
5. Files in use can't be replaced. Because a Windows executable that's in use 
is not allowed to be overwritten, it's not possible to use a script to update 
itself. For example, running 'easy_install -U setuptools' will result in an 
error because the easy_install.exe is in use.
6. There are potentially privilege issues and security aspects to the separate 
files that haven't yet been uncovered. On a multi-user system, there are 
considerations about ownership and permissions on the files. When there are 
multiple files per script, it's less obvious how permissions should be 
assigned. It's not obvious to me this poses a significant issue, but it sure 
seems easier to manage these issues on a single file rather than multiple.

All of these issues except for (6) have impacted my ability to work 
effectively with the side-by-side launchers currently on the system. It's one 
of the things I love about working on the Unix systems. Now that I feel like 
we're so close to a similarly-elegant solution on Windows, I want to see it 
employed (even if it's only opt-in, although I would prefer we work toward a 
solution that ultimately defaults to a single file mode for scripts).


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to handle launcher script importability?

[Jason R. Coombs]

 

 

From: Distutils-SIG
[mailto:distutils-sig-bounces+jaraco=jaraco@python.org] On Behalf Of
Paul Moore
Sent: Tuesday, 13 August, 2013 17:28



In the interests of getting more concrete data, can I suggest that
setuptools add an off-by-default option, which can be set globally using a
config file or environment variable or something (so that it can be used
transparently by tools like pip) to use scripts rather than exe wrappers?
People can then try it and see whether they hit issues with it. Otherwise, I
think we're going to remain forever stuck with theorising and guesswork.

 

That's exactly what setuptools 1.0 does, but I want to get the first draft
right, so I'm seeking comment on the technique (using env var to enable),
the proper extension to use (.py has problems), and any other suggestions.
I'm glad you mentioned the issues on powershell with 4 characters. I'm
going to explore that issue to confirm that something like .pygs isn't
viable (I use powershell and PATHEXT quite a bit).



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to handle launcher script importability?

[Jason R. Coombs]

 -Original Message-
 From: Distutils-SIG [mailto:distutils-sig-
 bounces+jaraco=jaraco@python.org] On Behalf Of PJ Eby
 Sent: Monday, 12 August, 2013 11:22

 On Mon, Aug 12, 2013 at 10:32 AM, Paul Moore p.f.mo...@gmail.com
 wrote:
  On 12 August 2013 14:01, PJ Eby p...@telecommunity.com wrote:
 
  As far as zipped Python applications are concerned (pyz), these can be
  created by just using a pys file containing a #! line prepended to the
  zip file. Certainly, it's a binary file with a filename that would
  normally indicate a text file format, but is that any less true on
  Unix when users create these files? I don't know what the user
  experience with zipped Python applications on Unix is like - I doubt
  it's *that* much better than on Windows. Probably the reality is that
  nobody uses zipped applications anyway, so the problems haven't been
  identified yet. Maybe the pyz PEP would bet better rewritten to
  propose providing tools to create and manage zipped Python
  applications, but *not* to require new extensions, merely to reuse
  existing ones (pys on Windows, no extension on Unix) with binary (zipped)
 content.

 Seems reasonable...  but then somebody will need to write another PEP for
 the file extension(s) issue.

My preference is to reject the idea of the side-by-side executable launcher. 
There are several downsides that I'm trying to avoid by moving away from the 
executable:

1. Disparity with Unix. Better parity means cleaner code, easier 
documentation, and less confusion moving from platform to platform.
2. Executables that look like installers. If a launcher executable is used and 
Windows detects that it looks like an installer and it's a 32-bit executable 
and it doesn't have a manifest to disable the functionality, Windows will 
execute the file in a separate UAC context (often in a separate Window).
3. Additional files are needed. In particular, due to (2), a manifest must be 
provided for 32-bit executables.
4. Word size accounting. It's not clear to me what word size is needed. 32-bit 
may be sufficient, though 64-bit seem to have some advantages: a manifest is 
not needed, and it can match the word size of the installed Python executable 
(for consistency). Setuptools currently provides both (and installs the one 
that matches the Python executable).
5. Platform support. We're fortunate that Windows is one of the most stable 
binary platforms out there. Nevertheless, Setuptools recently got support for 
AMD binaries in the launcher. By relying on an external launcher, the launcher 
becomes responsible for platform support.
6. Two to three files to do the job of one. In fact, the job isn't much more 
than to invoke code elsewhere, so it seems ugly to require as many as three 
files to do the job. Then multiply that by the Python-specific version and you 
have up to six files for a single script.
7. Obfuscation of purpose. A single script pretty directly communicates its 
purpose. When there are multiple files, it's not obvious why they exist or 
what their purpose is. Indeed, I went years without realizing we had an open 
issue in Distribute due to a missing manifest (which was fixed in Setuptools), 
all because I used the 64-bit executable. While it may take some time for the 
community to learn what a '.pyl' is, it's easily documented and simple to 
grasp, unlike the subtle and sometimes implicit nuances (and fragility) of a 
side-by-side executable.
8. Unwanted content. Some Unix users have complained about finding Windows 
executables in their Linux packages, so now Setuptools has special handling to 
omit the launchers when installed on Unix systems. This is far from beautiful.

 I think the issue of too many extensions vs. source/binary confusion is
 going to boil down to a BDFL judgment call, whether it's by Nick, Guido, or
 some more Windows-specific BDFL For One PEP.

 If we go with One Extension To Rule Them All, I would actually suggest 
 '.pyl'
 (for PyLauncher), since really all that extension does is say, hey, run 
 this as a
 console app via PyLauncher, not that it's a script (which would be
 assumed to be text).  And that all you can be sure of is that a .pyl files 
 will
 start with a #! line, and launch whatever other program is specified there, 
 on
 the contents of the file
 -- which may actually be a zipfile.

If it's '.py*', I don't see why it's not reasonable to allow omission of the 
shebang, and assume the default python. After encountering and now 
understanding the subtle import semantics, I'm hoping that this new extension 
can also be used in my personal 'scripts' collection to serve the same purpose 
it does for setuptools console entry points. I guess one could require 
#!/usr/bin/python in each, but that seems superfluous on Windows. I don't feel 
at all strongly on this point.

  PS Either the ref file marker approach, or a new Python command line
  argument with appropriate behaviour, could avoid the need for even the
  pys/pws 

[Distutils] How to handle launcher script importability?

[Jason R. Coombs]

In Setuptools 1.0 (currently in beta), I've added an experimental, opt-in
feature to install pure Python launcher scripts on Windows instead of
installing a launcher executable for each script, with the intention that
these scripts will be launched by pylauncher or Python directly, eventually
obviating the need for a launcher executable in setuptools at all.

This means that instead of installing, for example:

  Scripts\my-command.exe
  Scripts\my-command-script.py
  Scripts\my-command.exe.manifest

Instead Setuptools just installs:

  Scripts\my-command.py

This technique is much like the scripts that get installed to bin/ on Unix,
except that due to the nature of launching commands on Windows, the .py
extension is essentially required.

One problem with this technique is that if the script is also a valid module
name, it can be imported, and because Python puts the script's directory at
the top of sys.path, it _will_ be imported if that name is imported.

This happens, for example, after installing Cython. Cython provides a
command, 'cython', and a (extension) module called 'cython'. If one launches
cython using the script launcher, the 'cython' module will not be importable
(because import cython will import the launcher script). Presumably, this
is why '-script' was added to the launcher scripts in previous versions.

This is a rather unfortunate situation, and I'd like to solicit comments for
a way to avoid this situation. I see a few options:

1. Have the setuptools-generated launcher scripts del sys.path[0] before
launching.
2. Accept the implementation and file bugs with the offending projects like
Cython to have them either rename their script or rename their internal
module.
3. Continue to generate the script names with '-script.py' appended,
requiring invocation to always include -script.py on Windows.
4. Continue to generate executables, duplicating the effort of pylauncher,
and dealing with the maintenance burden of that functionality.

I don't see (2), (3), or (4) as really viable, so my proposal is to move
forward with (1) if there aren't any better suggestions.

If we move forward with (1), there are a few concerns that come to mind.

First, this approach would not apply to package-supplied scripts. What
should be done about those (if anything)?

Second, this would apply to Unix and Windows scripts (I'd rather not make
the distinction if possible) - are there any concerns about removing
sys.path[0] in the launch scripts on Unix? 

Third, is it possible some users are depending on the presence of
sys.path[0] or the assumed positions of other items in sys.path that should
be protected by this change? Would it make sense to replace sys.path[0] with
a non-existent directory instead?

My instinct is that these concerns are not of sufficient importance to
account for them, and that we should just implement (1) as simply as
possible.

Your comments and suggestions are appreciated.


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to handle launcher script importability?

[Jason R. Coombs]

 -Original Message-
 From: PJ Eby [mailto:p...@telecommunity.com]
 Sent: Sunday, 11 August, 2013 12:17
 
 
 Here's another problem with #1: you will break single-directory standalone
 portable app installs, where you use easy_install -mad somedir to
install all
 of an app's dependencies to a single directory that the app can then be
run
 from (assuming Python is available).
 
 In order to work around this issue, you'd need to hardcode sys.path
entries
 for the dependencies, or do something else more complicated in order to
 ensure that dependency resolution will pick up the adjacent distributions
 before searching anything else on sys.path.
 
 
  Third, is it possible some users are depending on the presence of
  sys.path[0]
 
 Absolutely.  It's a documented feature of Python that the script directory
is
 always first on sys.path, so that you can provide modules and packages
 adjacent to it.  That's how portable app installs work with easy_install.

All good points to be considered. Thanks.
 
 May I suggest an option 5 instead?  Use the new .pyz (or .pyzw for non-
 console apps) as a zipped Python application.  .pyz files aren't
importable,
 but *are* executable.  That's basically all that's needed to prevent
importing
 -- a file extension that's launchable but not importable.

This sounds like a suitable idea, but as you mention in a subsequent
message, this format has issues with sys.path assumptions as well. In this
case, I'm inclined to suggest yet another option (7) - create another
extension to specifically represent executable but not importable scripts,
perhaps .pys/.pysw (or .pycs/.pygs to more closely match console script and
gui script).

It sounds as if there is a fundamental need for Python to define an
extension that distinguishes a script from a module.
 
 (There's also an option 6, which is to use import system hooks to prevent
the
 script modules from being found in the sys.path[0] entry, but that's
rather
 hairier.)

Agreed, this sounds like it has its own subtle challenges.

 Using option 5 means the feature can only work with versions of Python on
 Windows that install the necessary PATHEXT support to allow that extension
 to work, but you're kind of limited to that anyway, because by default .py
 files aren't findable via PATH on Windows.
 
 Your post doesn't make it clear whether you're aware of that, btw:
 IIUC, on most Windows setups, executing a .py file via PATH doesn't work
 unless you've set up PATHEXT to include .py.  So your feature's going to
break
 until that's fixed, and AFAIK there is *no* Windows Python that fixes
this,
 with the possible exception of 3.4 alpha, possibly a future alpha that
hasn't
 been released yet, because last I saw on Python-Dev it was still being
 discussed *how* to update PATHEXT safely from the installer.
 
 In short: dropping .exe wrappers is not supportable on *any* current
version
 of Python for Windows, in the sense that anybody who uses it will not yet
be
 able to execute the scripts if they are currently doing so via PATH (and
 haven't manually fixed their PATHEXT).  (This was one of the main reasons
 for using .exe wrappers in the first
 place.)
 
 The .pyz approach of course has the same drawback, but at least it should
be
 viable for future Python versions, and doesn't have the sys.path[0]
problems.
 I think you are going to have to keep .exe wrappers the default for all
Python
 versions  3.4.

I am aware of the PATHEXT factor. I personally add .py and .pyw to the
PATHEXT (for all users) on my systems, so I was unsure if Python 3.3 did add
those or if pylauncher would add them (if installed separately). I was
_hoping_ that was the case, but it sounds like it is not. I did include in
the documentation notes about this requirement
(https://bitbucket.org/pypa/setuptools/src/1.0b1/docs/easy_install.txt#cl-98
).

I do want to explore the possibility of setuptools facilitating this
configuration such that it's easy for a Windows user to enable these
settings even if Python does not.

The nice thing about (7) is it would define an extension that's specifically
meant to be executable, so would be an obvious (and potentially less
controversial) choice to include in PATHEXT.


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to handle launcher script importability?

[Jason R. Coombs]

 -Original Message-
 From: Nick Coghlan [mailto:ncogh...@gmail.com]
 Sent: Sunday, 11 August, 2013 17:14

 We actually have a proposal on import-sig to allow module specific import
 path manipulation (including the ability to say don't import this module
 from this directory, even though it looks like it is here). I'd favour that
 mechanism over a new not importable file extension.

I don't believe this mechanism would suffice. My previous example was 
over-simplified to the general problem, which is that any script could 
potentially be imported as a module of the same name. So if I were to launch 
easy_install.py, it would set sys.path[0] to Scripts\ and if it were then to 
import cython (which it does), it would import Scripts/cython.py as cython, 
unless there were some way to globally declare all installed scripts somewhere 
so they're excluded from import.

 If that doesn't make it into 3.4, the proposed zipapp extensions would also
 serve a similar purpose, with some straightforward sys.path manipulation in
 __main__.py (as PJE pointed out).

Regardless what solution might be made available for Python 3.4, I'd prefer to 
work toward a solution that leverages pylauncher under older Pythons. After 
all, one of the huge benefits of pylauncher is that it supports multiple 
Pythons. If zipapp is the preferred mechanism for that, then so be it. I do 
agree that we should devise a best approach within the context of Python 3.4, 
and consider the backward-compatibility implications separately.

My feeling is that zipapp is somewhat too convoluted in that it alters the 
sys.path, but then has to alter it back to simulate not being a zipapp. It's 
also a file within a file, meaning it can't be readily edited with a text 
editor, but requires a routine even just to inspect it. I guess that's more 
transparent than an executable, though.

This approach also means that the script generation is not congruent with that 
on Unix systems. Using a zipapp means that the whole script generation needs 
to be special-cased for Windows. One of great benefits of using a simple 
script was that the code becomes largely unified (only requiring appending of 
an extension when on Windows). That is, unless zipapps can be made executable 
on Unix as well.

Given these obstacles, do you still feel that zipapp is the best approach?


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

[Jason R. Coombs]

There are two versions of launchers primarily because of my naiveté when
addressing the UAC issue. 64-bit launchers were exempt from the UAC
restrictions that caused them to launch in a separate window. I believed
this to be a proper fix, when in fact those still using 32-bit launchers
were still experiencing the problem. See
https://bitbucket.org/tarek/distribute/issue/143/easy_install-opens-new-cons
ole-cant-read for more detail.

 

So I agree, it would probably be sufficient to only supply 32-bit
executables. However, my preference would be to supply
architecture-appropriate executables rather than relying on a compatibility
layer.

 

Furthermore, I don’t believe the ARM architecture has a compatibility layer
(meaning 64-bit executables are required for 64-bit ARM builds), so
architecture and word size distinction is necessary.

 

I believe you’re right about leveraging the py launcher. I’d like for
setuptools to not have to supply launchers at all but depend on py launcher
instead. The py launcher is bundled with Python 3.3 so should become
ubiquitously available soon. I believe setuptools can begin to rely on it
and not supply a launcher at all. The scripts currently installed by
setuptools are suitable for launching by py launcher, so all that will need
to happen is to stop supplying its own launcher.

 

At least, that’s how I imagine it happening.

 

From: Paul Moore [mailto:p.f.mo...@gmail.com] 
Sent: Tuesday, 16 July, 2013 09:32
To: Distutils; Jason R. Coombs
Subject: Re: Wheels and console script entry point wrappers (Was: Replacing
pip.exe with a Python script)

 

On 16 July 2013 14:08, Paul Moore p.f.mo...@gmail.com
mailto:p.f.mo...@gmail.com  wrote:

PS There is still the proviso that I haven't tested my assumption that the
separate 32 and 64 bit wrappers are *needed* (setuptools and distlib use
them, so I think it's a valid assumption, but I need to test). I will try to
get time to check that ASAP.


Hmm. I just did a quick test, and then based on the results checked the
setuptools source code. I can see no reason why there needs to be 32 and 64
bit launcher exes. The launchers simply use CreateProcess to launch a
separate Python process using the #! line of the script. So there's no DLL
loading going on, and no reason that I can see for needing separate 32 and
64 bit builds.

 

Jason - can you shed any light on why there are separate builds for 32 and
64 bits?

 

Actually, the launcher is essentially identical to the py launcher for
Windows, except that it gets a script name to execute from the name of the
launcher. I'm wondering whether the correct approach here would be to
enhance the launcher one more time to look for a suitably named script and
auto-run it if it's present (i.e. merge the wrapper functionality into the
launcher). Then we have a standard wrapper that everyone can use and not
reinvent their own.

 

Paul



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] buildout/setuptools/distribute unhelpful error message (0.7.x issue?)

[Jason R. Coombs]

Hi Chris,
It looks like something is trying to install Setuptools 0.7.2, possibly 
with 
a temporary version of distribute or one that's not visible by default in your 
Python environment.

When you get that error message, I suggest you upgrade away from 
distribute. 
The easiest way to do this if you have distribute installed is to 
'easy_install -U distribute', which will grab distribute 0.7.3 and install 
setuptools=0.7. If this doesn't work (as it may not if you in fact don't have 
Distribute), you may be able to pro-actively avoid the problem by installing 
the latest Setuptools (0.9 at the time of this writing) using the published 
installation instructions:

https://pypi.python.org/pypi/setuptools/0.9

I hope that helps. Please report back if that doesn't get you going.

Regards,
Jason

 -Original Message-
 From: Distutils-SIG [mailto:distutils-sig-
 bounces+jaraco=jaraco@python.org] On Behalf Of Chris Withers
 Sent: Sunday, 07 July, 2013 03:09
 To: distutils sig
 Subject: [Distutils] buildout/setuptools/distribute unhelpful error message
 (0.7.x issue?)

 Hi All,

 What is this exception trying to tell me?

 Downloading
 https://pypi.python.org/packages/source/s/setuptools/setuptools-
 0.7.2.tar.gz
 Extracting in /tmp/tmpJNVsOY
 Now working in /tmp/tmpJNVsOY/setuptools-0.7.2 Building a Setuptools egg
 in /tmp/tmpBLZGeg /tmp/tmpBLZGeg/setuptools-0.7.2-py2.6.egg
 Traceback (most recent call last):
File bootstrap.py, line 91, in module
  pkg_resources.working_set.add_entry(path)
File /usr/lib/python2.6/dist-packages/pkg_resources.py, line 451, in
 add_entry
  self.add(dist, entry, False)
File /usr/lib/python2.6/dist-packages/pkg_resources.py, line 542, in 
 add
  self._added_new(dist)
File /usr/lib/python2.6/dist-packages/pkg_resources.py, line 705, in
 _added_new
  callback(dist)
File /usr/lib/python2.6/dist-packages/pkg_resources.py, line 2727, in
 lambda
  add_activation_listener(lambda dist: dist.activate())
File /usr/lib/python2.6/dist-packages/pkg_resources.py, line 2227, in
 activate
  self.insert_on(path)
File /usr/lib/python2.6/dist-packages/pkg_resources.py, line 2328, in
 insert_on
  with distribute. Found one at %s % str(self.location))
 ValueError: A 0.7-series setuptools cannot be installed with distribute.
 Found one at /tmp/tmpBLZGeg/setuptools-0.7.2-py2.6.egg

 I don't see any distribute in there, and I don't know where it might be...

 $ python2.6
 Python 2.6.8 (unknown, Jan 29 2013, 10:05:44) [GCC 4.7.2] on linux2 Type
 help, copyright, credits or license for more information.
   import setuptools
 Traceback (most recent call last):
File stdin, line 1, in module
 ImportError: No module named setuptools

 cheers,

 Chris

 --
 Simplistix - Content Management, Batch Processing  Python Consulting
 - http://www.simplistix.co.uk
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools 0.8 and Distribute 0.7.3 (legacy wrapper) now released

[Jason R. Coombs]

The PyPA is excited to announce the public release of Setuptools 0.8. This
release of setuptools provides no additional functionality over Setuptools
0.7.x except that it no longer requires 2to3 to build/install on Python 3.
What this means for packaging is that tools like pip and virtualenv can now
invoke setuptools directly on all supported Python versions (currently
2.4+). This build enables more natural upgrades and helps address many of
the bugs that the 2to3 conversion process triggered.

 

Additionally, Distribute 0.7.3 has also been released to PyPI. Distribute
0.7 was designed to ease the upgrade process from Distribute 0.6.x to
Setuptools 0.7. This new version, 0.7.3, is a re-release of the legacy
wrapper 0.7, but additionally bundles the Setuptools 0.8 code for the
purposes of bootstrapping the upgrade. This version specifically eases
upgrades on systems running older systems. Now, one can readily upgrade any
environment with Distribute 0.6 by simply upgrading (using pip or
easy_install) to Distribute 0.7.3, which will replace the 'distribute'
package with an empty shell leaving setuptools = 0.7 (probably 0.8)
installed.

 

Enjoy, and please report any issues with either of these packages at the
Setuptools project page (https://bitbucket.org/pypa/setuptools).



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7.2 and Distribute 0.7 (compatibility wrapper) on PyPI

[Jason R. Coombs]

Donald,

What's the next step to support this bootstrap script? Do we
need a ticket for PyPI to support this mechanism? How will PyPI resolve the
'latest' virtual link when there are multiple non-hidden versions (as with
setuptools 0.7.4 and 0.6c11), or does the uploader make that determination?

 

From: Donald Stufft [mailto:don...@stufft.io] 
Sent: Monday, 10 June, 2013 14:15
To: Jason R. Coombs
Cc: Tres Seaver; Distutils-Sig@Python.Org
Subject: Re: [Distutils] Setuptools 0.7.2 and Distribute 0.7 (compatibility
wrapper) on PyPI

 

 

On Jun 10, 2013, at 2:55 PM, Jason R. Coombs jar...@jaraco.com
mailto:jar...@jaraco.com  wrote:





There seems to be a dominant opinion that the file should be on PyPI, and I
don't disagree.

One issue is that the file must be mutable. It necessarily contains a
reference to the preferred version to be downloaded. That version changes
over time. It's typically incremented, but sometimes decremented.

It's possible the script could be updated to otherwise discover the most
appropriate version. As it's currently implemented however, the file must be
expected to change over time.

Because the file must change during each release, the system needs to be
able to accept an updated version. This is why the Bitbucket Link was
referenced, because it can include that version information.

I'm open to suggestions on how we can create a perma link to a file that
changes over time on PyPI.

 

Put a version in the filename/path, and have a latest tag that just 301
redirects to whatever is latest.

 

/bootstrap/setuptools/latest/ez_setup.py - 301 Redirect to

/bootstrap/setuptools/0.7.2/ez_setup.py






For now, the most authoritative place I could think of for that file was the
bit bucket downloads. Let's identify a mechanism to do the same on PyPI and
get that into set of tools for the next release.

Sent from my comm


-

Donald Stufft

PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

 



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7.2 and Distribute 0.7 (compatibility wrapper) on PyPI

[Jason R. Coombs]

There seems to be a dominant opinion that the file should be on PyPI, and I 
don't disagree.

One issue is that the file must be mutable. It necessarily contains a reference 
to the preferred version to be downloaded. That version changes over time. It's 
typically incremented, but sometimes decremented.

It's possible the script could be updated to otherwise discover the most 
appropriate version. As it's currently implemented however, the file must be 
expected to change over time.

Because the file must change during each release, the system needs to be able 
to accept an updated version. This is why the Bitbucket Link was referenced, 
because it can include that version information.

I'm open to suggestions on how we can create a perma link to a file that 
changes over time on PyPI.

For now, the most authoritative place I could think of for that file was the 
bit bucket downloads. Let's identify a mechanism to do the same on PyPI and get 
that into set of tools for the next release.

Sent from my comm

On 2013-06-09, at 22:42, Tres Seaver tsea...@palladion.com wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On 06/09/2013 10:28 PM, Donald Stufft wrote:
 
 On Jun 9, 2013, at 9:51 PM, Nick Coghlan ncogh...@gmail.com wrote:
 
 On 10 June 2013 03:49, Jason R. Coombs jar...@jaraco.com wrote:
 I'm leaning toward uploading it to BitBucket downloads as part of
 the release script.
 
 I'll put ez_setup.py for 0.7.2 in downloads for now. You can
 assume that's where the permalink will be.
 
 I think that's a good way to publish it securely to the PSF 
 infrastructure team (at least for now)
 
 If anyone has a better suggestion, please raise it.
 
 As others have suggested, we should work with Donald and Noah to
 get the bootstrapping script an official home somewhere on 
 pypi.python.org.
 If you want it under pypi.python.org it'll probably need manually
 sent to myself for the time being. I believe it could easily be hosted
 via python hosted.org though without needing manual intervention.
 
 I'm not wedded to anyplace in particular.  Having the bootstrap needed to
 make effective use of PyPI hosted on PyPI itself makes a lot of sense to
 me.  Maybe under a URL like this one?
 
  https://pypi.python.org/bootstraps/ez_setup.py
 
 The main key is that the URL should be immutable (so we can doucment it
 widely, and use it in scripts) and have availability as good as the
 cheeseshop (I'm not knocking Bitbucket in particular).
 
 If the file were copied into the Sphinx docs uploaded to
 pythonhosted.org, that would probably work just as well.
 
 
 Tres.
 - -- 
 ===
 Tres Seaver  +1 540-429-0999  tsea...@palladion.com
 Palladion Software   Excellence by Designhttp://palladion.com
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.11 (GNU/Linux)
 Comment: Using GnuPG with undefined - http://www.enigmail.net/
 
 iEYEARECAAYFAlG1PO8ACgkQ+gerLs4ltQ64WQCeO0tVhPBjlmzvtzdBx0C4V5Hr
 lVAAniVJ4XuAnPfV7NXT7KvcextrrTaJ
 =ILKd
 -END PGP SIGNATURE-
 
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig
 


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools 0.7.2 and Distribute 0.7 (compatibility wrapper) on PyPI

[Jason R. Coombs]

On behalf of the PYPA, I'm excited to announce that Setuptools 0.7 is now
official and complete.

 

Released to PyPI, Setuptools 0.7.2 is now available for all to see by
default (https://pypi.python.org/pypi/setuptools/). Users of Setuptools 0.6
may upgrade by simply running `easy_install -U setuptools`.

 

Additionally, I released Distribute 0.7 to PyPI. This means that users who
have Distribute 0.6.x can upgrade to setuptools 0.7 by simply invoking
`easy_install -U distribute` (or `easy_install distribute=0.7`). Windows
users should use `easy_install-script -U distribute` (to avoid file in use
errors on easy_install.exe).

 

The documentation for Setuptools 0.7.2 has been uploaded to
https://pythonhosted.org/setuptools and includes the notes about the merge
in addition to the official documentation.

 

Please report any issues at the project page
(https://bitbucket.org/pypa/setuptools). 

 



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7.2 and Distribute 0.7 (compatibility wrapper) on PyPI

[Jason R. Coombs]

Is the bitbucket 'raw' source not suitable? It uses the version tag to 
indicate precisely which file is appropriate, which means we don't have to 
redundantly host that file elsewhere.

Oh, I think I see - there's no link that doesn't change - you want a permalink 
for the latest stable release. I hadn't considered that use case, but you're 
right - we do want that. Perhaps the file could be added to pythonhosted.org 
alongside the documentation. Or maybe it belongs on bitbucket in `downloads`.

I need to figure out which is more straightforward w.r.t. authentication. The 
distribute process was messy because it involved SSH and keys that had to be 
manually managed.

I'm leaning toward uploading it to BitBucket downloads as part of the release 
script.

I'll put ez_setup.py for 0.7.2 in downloads for now. You can assume that's 
where the permalink will be.

If anyone has a better suggestion, please raise it.

 -Original Message-
 From: Distutils-SIG [mailto:distutils-sig-
 bounces+jaraco=jaraco@python.org] On Behalf Of Tres Seaver
 Sent: Sunday, 09 June, 2013 13:14
 To: Distutils-Sig@Python.Org
 Subject: Re: [Distutils] Setuptools 0.7.2 and Distribute 0.7 (compatibility
 wrapper) on PyPI

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 On 06/09/2013 12:22 PM, Jason R. Coombs wrote:
  On behalf of the PYPA, I'm excited to announce that Setuptools 0.7 is
  now official and complete.
 
  Released to PyPI, Setuptools 0.7.2 is now available for all to see by
  default (https://pypi.python.org/pypi/setuptools/). Users of
  Setuptools 0.6 may upgrade by simply running `easy_install -U
  setuptools`.
 
  Additionally, I released Distribute 0.7 to PyPI. This means that users
  who have Distribute 0.6.x can upgrade to setuptools 0.7 by simply
  invoking `easy_install -U distribute` (or `easy_install
  distribute=0.7`). Windows users should use `easy_install-script -U
  distribute` (to avoid file in use errors on easy_install.exe).
 
  The documentation for Setuptools 0.7.2 has been uploaded to
  https://pythonhosted.org/setuptools and includes the notes about the
  merge in addition to the official documentation.
 
  Please report any issues at the project page
  (https://bitbucket.org/pypa/setuptools).


 Thanks for all the work that went into this release.  I'm working to get
 buildout 2.0 to use the new setuptools, and need to encode a more-or-less
 permalink URL for 'ez_setup.py' -- do we not have a better URL for it than
 the Bitbucket 'raw' source?



 Tres.
 - --
 ==
 =
 Tres Seaver  +1 540-429-0999  tsea...@palladion.com
 Palladion Software   Excellence by Designhttp://palladion.com
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.11 (GNU/Linux)
 Comment: Using GnuPG with undefined - http://www.enigmail.net/

 iEYEARECAAYFAlG0t74ACgkQ+gerLs4ltQ6Q5wCg2hMoLBxfv6NIR/DTOUjdBNe
 i
 kVUAnjZ9ua2gRpwXyeRw/i1CIBPeV9e0
 =Ksc7
 -END PGP SIGNATURE-

 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools/Distribute error with 0.7.2

[Jason R. Coombs]

Hi Liam,

Sorry for the trouble. The cause is rooted in the latest
updates to Setuptools and Distribute on PyPI which were launched today.

 

I believe what's happening here is pip is installing
Distribute 0.7, which triggers the uninstallation of Distribute, but
Distribute 0.7 (a compatibility wrapper) depends on a setuptools module to
be in place to install.

 

I'm unsure about the failed rollback. That looks like a
separate issue in pip.

 

Based on your report, I'm going to remove the Distribute-0.7
wrapper from PyPI until this issue is resolved. That is done.

 

In order to repair those envs, you may simply be able to
re-run the chef code over the environments and they may repair naturally. If
they don't, you may need to re-initialize the virtualenv (run virtualenv
again on that path) in order to revive distribute in that environment.

 

I hope the pip or virtualenv maintainers can comment as
well.

 

Regards,

Jason

 

From: Distutils-SIG
[mailto:distutils-sig-bounces+jaraco=jaraco@python.org] On Behalf Of
Liam Kirsher
Sent: Sunday, 09 June, 2013 13:24
To: distutils-sig@python.org
Subject: [Distutils] Setuptools/Distribute error with 0.7.2

 

Hi --
I'm hoping for some help with an error I'm suddenly getting with distribute,
setuptools.
It had been working fine, and seemingly overnight it stopped working.  I
really don't think I changed anything.
This is part of a chef recipe, which also installs virtualenv.
It appears that a new version 0.7.2 is downloaded, but it fails to install,
and then the rollback fails, as well.

Not sure quite what is wrong here and how to fix it!

Liam




Using version 0.7.2 (newest of versions: 0.7.2, 0.7.2, 0.7.1, 0.7)
  Downloading from URL
https://pypi.python.org/packages/source/s/setuptools/setuptools-0.7.2.tar.gz
#md5=de44cd90f8a1c713d6c2bff67bbca65d (from
https://pypi.python.org/simple/setuptools/)
  Running setup.py egg_info for package setuptools

running egg_info
creating pip-egg-info/setuptools.egg-info
writing requirements to pip-egg-info/setuptools.egg-info/requires.txt
writing pip-egg-info/setuptools.egg-info/PKG-INFO
writing top-level names to
pip-egg-info/setuptools.egg-info/top_level.txt
writing dependency_links to
pip-egg-info/setuptools.egg-info/dependency_links.txt
writing entry points to
pip-egg-info/setuptools.egg-info/entry_points.txt
writing requirements to pip-egg-info/setuptools.egg-info/requires.txt
writing pip-egg-info/setuptools.egg-info/PKG-INFO
writing top-level names to
pip-egg-info/setuptools.egg-info/top_level.txt
writing dependency_links to
pip-egg-info/setuptools.egg-info/dependency_links.txt
writing entry points to
pip-egg-info/setuptools.egg-info/entry_points.txt
writing manifest file 'pip-egg-info/setuptools.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found


reading manifest file 'pip-egg-info/setuptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pip-egg-info/setuptools.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/setuptools has version 0.7.2, which
satisfies requirement setuptools=0.7 (from distribute-supervisor)
  skipping extra ssl:sys_platform=='win32'
  skipping extra ssl:sys_platform=='win32' and python_version=='2.4'
  skipping extra certs
  skipping extra ssl:python_version in '2.4, 2.5'
Installing collected packages: distribute, setuptools

  Found existing installation: distribute 0.6.45

Uninstalling distribute:

  Removing file or directory
/usr/local/lib/python2.7/dist-packages/distribute-0.6.45-py2.7.egg
  Removing pth entries from
/usr/local/lib/python2.7/dist-packages/easy-install.pth:
  Removing entry: ./distribute-0.6.45-py2.7.egg
  Successfully uninstalled distribute

  Running setup.py install for distribute

Running command /usr/bin/python -c import
setuptools;__file__='/tmp/pip-build-root/distribute/setup.py';exec(compile(o
pen(__file__).read().replace('\r\n', '\n'), __file__, 'exec')) install
--record /tmp/pip-5tzG3v-record/install-record.txt
--single-version-externally-managed
Traceback (most recent call last):

  File string, line 1, in module

ImportError: No module named setuptools

Complete output from command /usr/bin/python -c import
setuptools;__file__='/tmp/pip-build-root/distribute/setup.py';exec(compile(o
pen(__file__).read().replace('\r\n', '\n'), __file__, 'exec')) install
--record /tmp/pip-5tzG3v-record/install-record.txt
--single-version-externally-managed:

Traceback (most recent call last):

  File string, line 1, in module

ImportError: No module named setuptools



  Rolling back uninstall of distribute

  Replacing
/usr/local/lib/python2.7/dist-packages/distribute-0.6.45-py2.7.egg
Exception:
Traceback (most recent call last):
  File

Re: [Distutils] Setuptools/Distribute error with 0.7.2

[Jason R. Coombs]

Hi Julian,

I believe for those packages that call `use_setuptools()`
from `distribute_setup.py`, the only option to support them is to install
Distribute 0.7 (the stub package), which I believe will satisfy the
use_setuptools() call so it doesn't attempt to install Distribute 0.6.

 

As mentioned in another post, I've temporarily pulled
Distribute 0.7 from PyPI until we can determine a way to avoid the pip
upgrade issue. In the meantime, if you install Distribute 0.7 from
https://bitbucket.org/pypa/setuptools/downloads, does that solve your issue?

 

Regards,

Jason

 

From: Distutils-SIG
[mailto:distutils-sig-bounces+jaraco=jaraco@python.org] On Behalf Of
Julian Berman
Sent: Sunday, 09 June, 2013 14:21
To: distutils-sig@python.org
Subject: Re: [Distutils] Setuptools/Distribute error with 0.7.2

 

(I was also getting this until I used easy_install to upgrade)

 

What is supposed to happen with stuff that used to do `use_setuptools()` in
their setup.py now?

 

I upgraded to setuptools 0.7.2, but now if I try to install a thing that has
that in the setup.py, it seems to want to try and install distribute, and I
get:

 

Downloading/unpacking pudb (from -r
/Users/Julian/.local/share/virtualenvs/requirements-every-virtualenv.txt
(line 2))

  Using download cache from
/Users/Julian/Library/Caches/pip/https%3A%2F%2Fpypi.python.org
http://2Fpypi.python.org
%2Fpackages%2Fsource%2Fp%2Fpudb%2Fpudb-2013.2.tar.gz

  Running setup.py egg_info for package pudb

Downloading
http://pypi.python.org/packages/source/d/distribute/distribute-0.6.35.tar.gz

Extracting in /var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3

Now working in
/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribute-0.6.35

Building a Distribute egg in
/Users/Julian/.local/share/virtualenvs/great/build/pudb

Traceback (most recent call last):

  File setup.py, line 45, in module

exec(init_file.read(), d)

  File string, line 8, in module

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/setuptools/__init__.py, line 2, in module

from setuptools.extension import Extension, Library

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/setuptools/extension.py, line 5, in module

from setuptools.dist import _get_unpatched

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/setuptools/dist.py, line 6, in module

from setuptools.command.install import install

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/setuptools/command/__init__.py, line 8, in module

from setuptools.command import install_scripts

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/setuptools/command/install_scripts.py, line 3, in module

from pkg_resources import Distribution, PathMetadata,
ensure_directory

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/pkg_resources.py, line 2825, in module

add_activation_listener(lambda dist: dist.activate())

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/pkg_resources.py, line 710, in subscribe

callback(dist)

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/pkg_resources.py, line 2825, in lambda

add_activation_listener(lambda dist: dist.activate())

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/pkg_resources.py, line 2257, in activate

self.insert_on(path)

  File
/private/var/folders/zc/bzrwlgxd51594q6lt1yrx5v0gn/T/tmp8NepZ3/distribu
te-0.6.35/pkg_resources.py, line 2358, in insert_on

with distribute. Found one at %s % str(self.location))

ValueError: A 0.7-series setuptools cannot be installed with distribute.
Found one at
/Users/Julian/.local/share/virtualenvs/great/lib/python2.7/site-packages/set
uptools-0.7.2-py2.7.egg

 
/Users/Julian/.local/share/virtualenvs/great/build/pudb/distribute-0.6.35-py
2.7.egg

Traceback (most recent call last):

  File string, line 16, in module

  File
/Users/Julian/.local/share/virtualenvs/great/build/pudb/setup.py, line 5,
in module

use_setuptools()

  File distribute_setup.py, line 152, in use_setuptools

return _do_download(version, download_base, to_dir, download_delay)

  File distribute_setup.py, line 132, in _do_download

_build_egg(egg, tarball, to_dir)

  File distribute_setup.py, line 123, in _build_egg

raise IOError('Could not build the egg.')

IOError: Could not build the egg.

Complete output from command python setup.py egg_info:

Downloading

Re: [Distutils] Setuptools/Distribute error with 0.7.2

[Jason R. Coombs]

I think it would be highly desirable to add support for pip to handle the
upgrade from distribute 0.6 to 0.7. You'll note that because 0.7 depends on
setuptools 0.7 that 0.7 has already been downloaded. Perhaps a shim like you
propose would work, or it's also conceivable that distribute 0.7 could
include a setuptools 0.7 source tree which pip could leverage (but not
install).

 

Yes, PEP 439 will be awesome, but I think this important milestone for
setuptools is also essential to simplify the landscape in order to help
migrate users in a sustainable way to the new tools, so we've got to find a
way to make it happen for the good of the eco system.

 

From: qwe...@gmail.com [mailto:qwe...@gmail.com] On Behalf Of Marcus Smith
Sent: Sunday, 09 June, 2013 17:54
To: Jason R. Coombs
Cc: Liam Kirsher; distutils-sig@python.org
Subject: Re: [Distutils] Setuptools/Distribute error with 0.7.2

 

 

It was only working in python2 before now anyway.
(https://github.com/pypa/pip/issues/650)

pip is fundamentally dependent on setuptools to perform upgrades.

 

with distribute-0.6.X upgrades, even though pip uninstalled distribute as
part of the upgrade, it ran the install subprocess with the cwd equal to the
build dir, so it could import setuptools from the unpacked download.

but now,  distribute-0.7 is just a shell, that contains no importable
setuptools that pip can use.

we could *make* it work I guess, by enforcing non-standard logic when
dealing with distribute (i.e. get setuptools-0.7 installed first; by
default, setuptools gets queued to be installed after the distribute
upgrade)

but like I said before, it's much more motivating to think about PEP439,
than this dreary setuptools/distribute legacy headache stuff.

 

Marcus

 

 

the best fix I think is to move faster on making pip PEP439 compliant

   - i.e. have pip be able to at least install from wheels w/o needing
setuptools, which would remove the bootstrap headache

   - see https://github.com/pypa/pip/issues/863

   - this could likely involve pip replacing it's use of pkg_resources with
distlib (https://github.com/pypa/pip/pull/909)

 

Marcus   

 



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools 0.7 final available for download

[Jason R. Coombs]

I'm pleased to announce the official release of Setuptools 0.7, now
available for download from the project page
https://bitbucket.org/pypa/setuptools .

 

Nothing has changed from the 0.7b4 pre-release. This is the version that
will be uploaded to PyPI after we work out the technique to deploy to PyPI
without interfering with the setuptools 0.6 releases.

 

For convenience, I've also added experimental .msi installers for Windows
for Python 3.3 and Python 2.7. Work may continue on these in the future, but
as the documentation states, the recommended installation procedure is to
use ez_setup.py.

 

To install this latest release, follow the canonical install instructions
(using ez_setup.py), but direct the script to use bitbucket instead of PyPI:

 

python ./ez_setup.py
--download-base=https://bitbucket.org/pypa/setuptools/downloads/

 

I'll send another announcement when the official release has been uploaded
to PyPI and the ez_setup.py script can be used directly.



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

[Jason R. Coombs]

See https://bitbucket.org/pypa/setuptools/src/0.7/CHANGES.txt and 
https://bitbucket.org/pypa/setuptools/src/0.7/MERGE.txt for details on the 
changes.

 

From: anatoly techtonik [mailto:techto...@gmail.com] 
Sent: Sunday, 02 June, 2013 13:15
To: Jason R. Coombs
Cc: distutils-sig@python.org
Subject: Re: [Distutils] Setuptools 0.7 final available for download

 

On Sun, Jun 2, 2013 at 6:42 PM, Jason R. Coombs jar...@jaraco.com 
mailto:jar...@jaraco.com  wrote:

I’m pleased to announce the official release of Setuptools 0.7, now available 
for download from the project page https://bitbucket.org/pypa/setuptools .

 

Nothing has changed from the 0.7b4 pre-release. This is the version that will 
be uploaded to PyPI after we work out the technique to deploy to PyPI without 
interfering with the setuptools 0.6 releases.

 

Nice. This is a significant event. Is there any Changes page to see what is so 
awesome in the release that took about more than 3 years to appear according to 
PyPI page (https://pypi.python.org/pypi/setuptools)?



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

[Jason R. Coombs]

Good suggestions. Historically, Setuptools has not followed semver. I intend 
for Setuptools to start to follow semver (at least in spirit) starting with 
0.7.



From: anatoly techtonik [mailto:techto...@gmail.com]
Sent: Sunday, 02 June, 2013 14:50
To: Jason R. Coombs
Cc: distutils-sig@python.org
Subject: Re: [Distutils] Setuptools 0.7 final available for download



On Sun, Jun 2, 2013 at 8:19 PM, Jason R. Coombs jar...@jaraco.com 
mailto:jar...@jaraco.com  wrote:

See https://bitbucket.org/pypa/setuptools/src/0.7/CHANGES.txt and 
https://bitbucket.org/pypa/setuptools/src/0.7/MERGE.txt for details on the 
changes.

Thanks for the links. I expected something more massive, but as people said 
there appeared to be a plenty of releases over this period, so MERGE.txt 
covers my needs,



Major number increments mean that some features are added according to 
semver.org http://semver.org  and even without it it is convenient to read 
about features separately from fixes.

http://sourceforge.net/p/roundup/code/ci/default/tree/CHANGES.txt







smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.7 final available for download

[Jason R. Coombs]

 

 

From: Paul Moore [mailto:p.f.mo...@gmail.com] 
Sent: Sunday, 02 June, 2013 14:03



 

One other question that I can't see covered by the CHANGES and README files.
Does setuptools 0.7 support dist-info directories the same way that recent
versions of distribute did? The dist-info support is critical to wheel
support in pip.

 

 

Yes, unless otherwise specified in the merge docs, Setuptools 0.7 includes
all of Distribute.



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Sooner or later, we're going to have to be more formal about how we name packages.

[Jason R. Coombs]

 

 

From: Distutils-SIG
[mailto:distutils-sig-bounces+jaraco=jaraco@python.org] On Behalf Of
Donald Stufft
Sent: Saturday, 01 June, 2013 15:30
To: holger krekel
Cc: distutils sig
Subject: Re: [Distutils] Sooner or later, we're going to have to be more
formal about how we name packages.

 

 

On Jun 1, 2013, at 3:21 PM, holger krekel hol...@merlinux.eu
mailto:hol...@merlinux.eu  wrote:





On Sat, Jun 01, 2013 at 11:57 -0400, Jim Fulton wrote:




For a while, many of us have been pretty careful to use namespaces
for new packages to mitigate this issue.  For example, the zc namespace
is a shorter version of com.zope, but at some point, it won't be fair
for us to claim zc for ourselves.


I wonder if we could allow people/groups to apply (to humans) for a
namespace which they can subsequently control, like the zc.* one.

 

So for example if the django community wants to introduce the concept
of vetted plugins/addons, they could move to manage dj.* or so.



 

I think this example highlights some of the challenges with
registering/controlling namespaces - who owns what and what is the meaning
of a (distribution) package name? For example, what is the namespace used
for an endorsed django plugin written by zope corporation?

 

This problem is not present now, as the author can choose the domain which
is most relevant to that plugin and its users. If there's some expectation
that it should appear in a namespace managed by another organization, that
necessitates a coordination between the namespace owner/manager and the
project author.

 

I think more people would claim namespaces when namespaces are better
supported in Python. My expectation is Python 3.3 namespace package support
will ease that challenge (when it becomes a dominant version).

 

My inclination is to say it's not a huge problem, and later is preferable
than sooner.



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Announcing Setuptools 0.7b3

[Jason R. Coombs]

Setuptools and Distribute are now merged, and the new home can be found at
https://bitbucket.org/pypa/setuptools.

 

Issues for Distribute are still being maintained at its old home, but
releases are now made from the 'distribute' fork on the setuptools repo.
Future releases of Distribute (if any) will be maintenance only.

 

The new setuptools, which is designed to be largely compatible both with
Distribute and Setuptools can now be downloaded on the 'downloads' page of
the PyPA site:

 

  https://bitbucket.org/pypa/setuptools/downloads

 

For those familiar with setuptools/distribute, the upgrade process is pretty
straightforward: remove your old versions of distribute and/or setuptools
and install the latest. To install the latest, extract the .tar.gz, and run
setup.py in the target environment. Windows .msi installers have been
provided as well. Ez_setup.py will not yet work until the official release
is made to PyPI.

 

I encourage all of those with experience or interest in packaging to try out
the new setuptools in your environments. Test it against your tool chain.
Provide feedback and file issues in the project site.

 

This limited public release is provided as an advance release prior to the
full public release. Assuming there are no reported blocking issues, a final
0.7 release will be made to PyPI in the coming weeks.

 

Regards,

Jason



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcing Setuptools 0.7b3

[Jason R. Coombs]

I'll be sure to address that before the final release.

I have been tempted to just support sdist installs and follow the technique 
that Distribute used, except for this one comment I heard in person by a newer 
Windows user recently, that they still hear that Setuptools is still 
considered friendlier than Distribute.

But if PJE is on board with source installs, now is a good time for that 
switch, so let's do it.

 -Original Message-
 From: PJ Eby [mailto:p...@telecommunity.com]
 Sent: Saturday, 25 May, 2013 18:45
 To: Jason R. Coombs
 Cc: distutils-sig@python.org
 Subject: Re: [Distutils] Announcing Setuptools 0.7b3

 On Sat, May 25, 2013 at 4:40 PM, Jason R. Coombs jar...@jaraco.com
 wrote:
https://bitbucket.org/pypa/setuptools/downloads

 You may be already aware of this, but the installation instructions don't
 match the available downloads.  I was previously thinking we should just
 drop binary distributions of setuptools altogether anyway, and make
 ez_setup.py the standard way to install setuptools on all platforms.  (Which
 would help address things like installation conflicts w/existing Distribute, 
 etc.)



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Setuptools-Distribute merge announcement

[Jason R. Coombs]

As PJE mentioned in his e-mail, he and I have been working on a merge of the
code lines of Setuptools and Distribute. I'm excited about this transition
and I hope you are too.

 

In this message, I will provide some answers based on questions that he and
I encountered in our discussions and subsequent merge activity. If you have
further questions, please direct them to both of us and we intend to answer
promptly and also update the FAQ at the wiki
(https://bitbucket.org/jaraco/setuptools/wiki/Setuptools%20and%20Distribute%
20Merge%20FAQ).

 

- Jason R. Coombs

 

Where does the merge occur?

 

The merge is occurring between the heads of the default branch of Distribute
and the setuptools-0.6 branch of Setuptools. The Setuptools SVN repo has
been converted to a Mercurial repo hosted on Bitbucket. The work is still
underway, so the exact changesets included may change, although the
anticipated merge targets are Setuptools at 0.6c12 and Distribute at 0.6.35.

 

What happens to other branches?

 

Distribute 0.7 was abandoned long ago and won't be included in the resulting
code tree, but may be retained for posterity in the original repo.

 

Setuptools default branch (also 0.7 development) may also be abandoned or
may be incorporated into the new merged line if desirable (and as resources
allow).

 

What history is lost/changed?

 

As setuptools was not on Mercurial when the fork occurred and as Distribute
did not include the full setuptools history (prior to the creation of the
setuptools-0.6 branch), the two source trees were not compatible. In order
to most effectively communicate the code history, the Distribute code was
grafted onto the (originally private) setuptools Mercurial repo. Although
this grafting maintained the full code history with names, dates, and
changes, it did lose the original hashes of those changes. Therefore,
references to changes by hash (including tags) are lost.

 

Additionally, any heads that were not actively merged into the Distribute
0.6.35 release were also omitted. As a result, the changesets included in
the merge repo are those from the original setuptools repo and all
changesets ancestral to the Distribute 0.6.35 release.

 

What features will be in the merged code base?

 

In general, all features added in distribute will be included in
setuptools. Where there exist conflicts or undesirable features, we will be
explicit about what these limitations are. Changes that are
backward-incompatible from setuptools 0.6 to distribute will likely be
removed, and these also will be well documented.

 

Bootstrapping scripts (ez_setup/distribute_setup) and docs, as with
distribute, will be maintained in the repository and built as part of  the
release process. Documentation and bootstrapping scripts will be  hosted at
python.org, as they are with distribute now. Documentation at  telecommunity
will be updated to refer or redirect to the new, merged  docs.

 

On  the whole, the merged setuptools should be largely compatible with the
latest releases of both setuptools and distribute and will be an easy
transition for users of either library.

 

Who is invited to contribute? Who is excluded?

 

While we've worked privately  to initiate this merge due to the potential
sensitivity of the topic,  no one is excluded from this effort. We invite
all members of the  community, especially those most familiar with Python
packaging and its  challenges to join us in the effort.

 

We have lots of  ideas for how we'd like to improve the codebase, release
process,  everything.  Like distribute, the post-merge setuptools will have
its  source hosted on bitbucket.  (So if you're currently a distribute
contributor, about the only thing that's going to change is the URL of  the
repository you follow.)  Also like distribute, it'll support Python  3, and
hopefully we'll soon merge Vinay Sajip's patches to make it run  on Python 3
without needing 2to3 to be run on the code first.

 

Why Setuptools and not Distribute or another name?

 

We do understand that this announcement might be unsettling for some.  The
setuptools name has been subjected to a lot of deprecation in recent  years,
so the idea that it will now be the preferred name instead of distribute
might be somewhat difficult or disorienting for some.   We considered use of
another name (Distribute or an entirely new name),  but that would serve to
only complicate matters further. Instead, our goal is to simplify the
packaging landscape but without losing any hard-won advancements.  We hope
that the people who worked to spread the first message will be equally
enthusiastic about spreading the new one, and we especially look forward to
seeing the new posters and slogans celebrating the new setuptools.

 

What is the timeframe of release?

 

There are no hard timeframes for any of this effort, although progress is
underway and a draft merge is underway and being tested privately.   As an
unfunded volunteer effort, our time to put

Re: [Distutils] Moving distribute_setup.py

[Jason R. Coombs]

Sounds good to me. See release.py in the root of distribute which uploads
that file as part of the release process. It would be nice if that continues
to work after the move. How is access controlled to that resource?

 -Original Message-
 From: Distutils-SIG [mailto:distutils-sig-
 bounces+jaraco=jaraco@python.org] On Behalf Of Jim Fulton
 Sent: Thursday, 14 February, 2013 12:33
 To: Tarek Ziadé
 Cc: distutils-sig@python.org
 Subject: Re: [Distutils] Moving distribute_setup.py
 
 On Thu, Feb 14, 2013 at 11:41 AM, Tarek Ziadé ta...@ziade.org wrote:
  Hey
 
  As Marc-André suggested, it would be a good thing to move the
  bootstrapping file to
  :https://pypi.python.org/packages/source/d/distribute/distribute_setup
  .py
 
 +1
 
 I need to do something similar for buildout bootstrap files.
 How are you accomplishing this?
 
 
  so we can use the PSF certificate and provide a better security scheme
 
 Yay!
 
  I guess we can do the change in zc.buildout and distribute and once
  it's working, add a permanent redirect on the old address
 
 You mean from the old to the new?
 
  That'll also take care of the bad uptimes I have on my cheap server
 
 
  Any concerns ?
 
 Not from me.
 
 Jim
 
 --
 Jim Fulton
 http://www.linkedin.com/in/jimfulton
 Jerky is better than bacon! http://zo.pe/Kqm
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] an approach for inspectable setup scripts

[Jason R. Coombs]

In
http://lists.idyll.org/pipermail/testing-in-python/2011-October/004447.html,
Barry Warsaw kicked off a discussion on testing various packages within the
Python Package Index (pypi). As part of this discussion, I mentioned a
technique I've been using internally for a while to write setup scripts (old
distutils/setuptools style) such that they're more robust and
introspectable.

 

Instead of writing the following in my scripts:

 

from setuptools import setup

setup(

  name = 'foo',

  install_requires = 'bar',

)

 

I write the following:

 

setup_params = dict(

  name = 'foo',

  install_requires = 'bar',

)

 

if __name__ == '__main__':

  from setuptools import setup

  setup(**setup_params)

 

While slightly more verbose, this technique has a couple of benefits. First,
it means that code traversal algorithms (such as test discovery) don't
inadvertently execute the setup script. Second, it allows the script to be
read via import or execfile without necessarily invoking the setup()
function. This allows a third-party product, such as the Cheese Taster to
open up a project and easily inspect its setup parameters. Also, you'll note
the setuptools requirement is deferred until the script is run, and isn't
necessary to construct the parameters.

 

Furthermore, if there is other side-effect behavior, it can be invoked from
inside the __main__ block.

 

 

I share this with the community for your feedback. Is there any reason this
technique shouldn't be adopted in general? Also, how can a third-party
product detect whether a setup script is safe in this way? I don't think it
would be possible in general, but perhaps packagers could include a
directive near the head to indicate such. Consider:

 

# -*- script-disposition: import-safe -*-

 

Or similar. Alternatively, a tool could be built to compile and statically
analyze the code to detect the presence of setup_params, though would be
more likely to encounter false positives. 

 

What downsides am I missing? How could this technique be improved? Would it
be difficult to take these parameters and generate package metadata
(DistributionMetadata) from it?

 

I look forward to any feedback you have.

 

Regards,

Jason R. Coombs

 

 

 



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] How to customize egg_info behavior

[Jason R. Coombs]

I'm using setuptools 0.6c11.

 

I'd like to programmatically customize the way the egg_info command is run.
That is, in my setup.py, I would like to run some functions to determine the
tag-build, tag-date, and tag-svn-revision parameters to egg_info. Are there
parameters

 

How is this task best accomplished? Do I subclass the egg_info Command? If
so, do I just specify cmdclass={'egg_info':my_custom_egg_info} ?  Or would
you recommend another approach?

 

Regards,

Jason



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How to customize egg_info behavior

[Jason R. Coombs]

Thanks for the tip, pje. Your suggestion was very helpful. Perhaps I can ask
a follow-up.

In addition to calculating the tag at package-time (when setup.py is run for
sdist/bdist*), I am calculating the version at package-time.

The problem I have with this approach is with source distributions (sdists).
Since the sdists get the same setup.py file, the calculations for version
and tag are done both at packaging time and at install time. I would like to
configure my setup.py such that when sdist is run, the version and tag are
calculated and added to the egg_info, but when a consumer of that sdist goes
to install it, the version and tag are not calculated and just consumed from
the egg_info (similar to what happens when installing a bdist_egg).

This leads to two questions:

1. Is there a way to determine that I'm installing from an sdist, so I can
bypass any package-time calculations and defer to the already-calculated
PKG_INFO present in the sdist zip? I can probably elicit that information
from the environment, but if there's an indicator in setuptools, that would
be helpful.
2. Once the setup script has determined that it is installing from an sdist,
how can it call setup() to actually defer to the existing PKG_INFO for
installation?

I suspect there may be a better approach; any suggestions are appreciated.
Using bdist_egg does exactly what I need, but I'd prefer to use sdist to
avoid the need to distribute versions for each Python version. I can almost
imagine an sdist_egg command that creates a source-only distribution
suitable for easy_install.

If it helps to have a concrete example, the project I'm working on is
[hgtools][1] and here is the [current state of the setup.py script][2].

Thanks!

Jason

[1] http://bitbucket.org/jaraco/hgtools
[2] http://bitbucket.org/jaraco/hgtools/src/6e18ec0cea4a/setup.py

 -Original Message-
 From: P.J. Eby [mailto:p...@telecommunity.com]
 Sent: Saturday, 31 July, 2010 19:01
 To: Jason R. Coombs; distutils-sig@python.org
 Subject: Re: [Distutils] How to customize egg_info behavior
 
 At 12:39 PM 7/31/2010 -0700, Jason R. Coombs wrote:
 I'm using setuptools 0.6c11.
 
 I'd like to programmatically customize the way the egg_info command is
 run. That is, in my setup.py, I would like to run some functions to
 determine the tag-build, tag-date, and tag-svn-revision parameters to
 egg_info. Are there parameters
 
 How is this task best accomplished? Do I subclass the egg_info Command?
 If so, do I just specify cmdclass={'egg_info':my_custom_egg_info} ?  Or
 would you recommend another approach?
 
 You can just do this in your setup script:
 
 setup(
  ...
  options = dict(egg_info = dict(tag_build = whatever, ...)),
  ...
 )
 
 Replacing whatever, of course, with your programmatic calculation.
 



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] setuptools 0.7a for Python 3 fails to build on Windows

[Jason R. Coombs]

Thanks to Lennart Regebro for putting together a build of setuptools for
Python 3 here:
http://regebro.wordpress.com/2009/02/01/setuptools-and-easy_install-for-pyth
on-3/

Unfortunately, the package fails to build on Windows.  Refer to the blog
post (and comments) for details.

The problem occurs when get_script_args calls resource_string for the
'cli.exe' launcher.

It turns out that in Python 3, the following command yields the same error:

 open('setuptools/cli.exe', 'rt').read()

resource_string ultimately executes the same command, which is why the build
fails.

The correct thing to do in this particular case is to open the file in
binary mode, which returns `bytes` instead of a Unicode `str`.
Unfortunately, I'm not sure what negative impact would come from altering
pkg_resources.DefaultProvider._get to always read bytes, and that doesn't
strike me as the correct solution.

So, I'm writing here to ask: what should be done about pkg_resources in
Python 3 to support getting a package resource that's binary data?

As I see it, there are a few options,
- always have pkg_resources providers return bytes.
- read the bytes and try converting to str, falling back to bytes on
failure.
- require a parameter to indicate what type of content is expected.

There are probably others.  Any suggestions are appreciated.

Regards,
Jason


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setuptools 0.7a for Python 3 fails to build on Windows

[Jason R. Coombs]

Lennart,
That seems to have done the trick.  Between that fix and using the
64-bit cli.exe and gui.exe from http://bugs.python.org/setuptools/issue2, I
was able to install setuptools on Python 3.0.1 64-bit on Windows Vista SP1
and run easy_install.  For that build in particular, a Windows installer is
available at
http://dl.getdropbox.com/u/54081/setuptools-0.7a1.win-amd64.exe. I'll try to
put together a 32-bit build also later.

Thanks so much for doing the Python 3 port.  I'll be sure to report
back if I encounter any further problems.

Now I can get started porting my namespace packages to Python 3.

Jason

 -Original Message-
 From: Lennart Regebro [mailto:rege...@gmail.com]
 Sent: Wednesday, 08 April, 2009 15:53
 To: Jason R. Coombs
 Cc: distutils-sig@python.org
 Subject: Re: [Distutils] setuptools 0.7a for Python 3 fails to build on
 Windows
 
 On Wed, Apr 8, 2009 at 21:12, Jason R. Coombs jar...@jaraco.com
 wrote:
  Unfortunately, I'm not sure what negative impact would come from
 altering
  pkg_resources.DefaultProvider._get to always read bytes, and that
 doesn't
  strike me as the correct solution.
 
 No, if you do that everything explodes, and you can't even run
 python3.0 setup.py anymore. So that's definitely not right. :-)
 
  So, I'm writing here to ask: what should be done about pkg_resources
 in
  Python 3 to support getting a package resource that's binary data?
 
  As I see it, there are a few options,
  - always have pkg_resources providers return bytes.
  - read the bytes and try converting to str, falling back to bytes on
  failure.
 
 That change does still run all the tests at least, so it's worth a
 try. I changed DefaultProvider._get to:
 
 def _get(self, path):
 stream = open(path, 'rb')
 try:
 data = stream.read()
 return data.decode()
 except UnicodeDecodeError:
 return data
 finally:
 stream.close()
 
 Try if that makes a difference.
 
  - require a parameter to indicate what type of content is expected.
 
 Tricky, since _get is called from somewhere else than where the fact
 that it's cli.exe thet should be opened is set. There is too much
 iterators and indirection in pkg_resources for my small brain. I
 usually get a headache. :-)
 
 --
 Lennart Regebro: Pythonista, Barista, Notsotrista.
 http://regebro.wordpress.com/
 +33 661 58 14 64


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Error downloading pythonutils from pypi: Can't process plain .py files

[Jason R. Coombs]

Phillip,
Thanks for your response.

 -Original Message-
 From: Phillip J. Eby [mailto:p...@telecommunity.com]
 Sent: Thursday, 18 December, 2008 21:59
 
 ...  And
 it's not reasonable to expect it to grow special cases for every site
 that wants to hide the filename in the query string attached to a CGI
 script.

I absolutely agree it's not reasonable to grow special cases... but the RFCs
do provide a standard for describing the content disposition.  Perhaps it
would be worthwhile to handle that general case.  Specifically, one could
request the headers for a given URL to determine if a content-disposition
filename is supplied.

I've put together some sample code which is short, robust, and clean (I
hope).  This code demonstrates how one might go about querying the
content-disposition.  See it at:

http://paste.turbogears.org/paste/19813

With that code, the filename_test code could go something like this:

filename = get_content_disposition_filename(url) or
get_filename_the_old_way(url)
test_valid_egg_distro(filename)

... or alternatively, one could only perform the content-disposition
retrieval when the filename _appears_ to be invalid.

 
 If someone needs to have their files processed by a script, they
 could always make the path something like:
 
downman.py/pythonutils-0.2.3.zip
 
 Using either PATH_INFO or mod_rewrite or something similar, at which
 point easy_install can tell it's looking at a source zipfile, rather
 than a .py script.

While I agree it's unreasonable for setuptools to have to grow to handle
nonstandard cases, I would say the same argument applies to the content
providers.  They shouldn't have to alter their deployment architecture to
accommodate limitations in setuptools, particularly when they're following
the web standards.

The above solution handles the general case where files can be served
through many different content management systems and not just as static
file references in the URL.  This allows for special urls like
http://www.mycompany.com/myproject/latest_distro.jsp, which might generate
the content dynamically.  Note that the voidspace author is not attempting
to hide the filename in the get parameters.  He's probably using a sort of
content management system.

Would you consider a patch that incorporates this more robust behavior?

Regards,
Jason


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Error downloading pythonutils from pypi: Can't process plain .py files

[Jason R. Coombs]

Downloading pythonutils from pypi fails even though it is downloading a
perfectly good .zip source file.

 easy_install pythonutils==0.3.0
Searching for pythonutils==0.3.0
Reading http://pypi.python.org/simple/pythonutils/
Couldn't find index page for 'pythonutils' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://pypi.python.org/simple/
Reading http://pypi.python.org/simple/Pythonutils/
Reading http://www.voidspace.org.uk/python/modules.shtml#pythonutils
Reading
http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0.
2.3.zip
Reading
http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0.
2.5.zip
Reading
http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0.
3.0.zip
No local packages or download links found for pythonutils==0.3.0
error: Could not find suitable distribution for
Requirement.parse('pythonutils==0.3.0')

If I download the .zip file manually and easy_install the zip file, the
install goes great.  I think the problem is related to the .py name
appearing in the filename.

 easy_install
http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0.
3.0.zip
Downloading
http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0.
3.0.zip
error: Can't process plain .py files without an '#egg=name-version' suffix
to enable automatic setup script generation.

I'm guessing that whatever easy_install uses for downloading the file ends
up naming it downman.py (or something similar).

I checked the headers on the response from voidspace, and I see voidspace is
in fact supplying the content-disposition header:

content-disposition: attachment; filename= pythonutils-0.3.0.zip

So it appears that whatever downloader easy_install is using isn't honoring
the content-disposition header. or the check for python-ness is otherwise
getting a false positive.

This problem was observed in Windows Vista 64-bit with Python 2.5.2 32-bit
and setuptools 0.6c9.

Should I post a bug in bugtracker on this issue?

Regards,
Jason


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] new behavior of parse_version('dev')?

[Jason R. Coombs]

It appears to me as if the behavior of parse_version('dev') has changed with
setuptools-0.6c9, but this change is not documented on the setuptools
documentation.  Assuming pv=parse_version, the documentation implies that
pv('a')  pv('b')  pv('c') == pv('rc')  pv('d') == pv('dev'). but it turns
out that pv('dev') == pv('@')  pv('a').

 

I suggest the documentation say something like the following:

 

A pre-release tag is a series of letters that are alphabetically before
final. Some examples of prerelease tags would include alpha, beta, a, c,
dev, and so on. You do not have to place a dot before the prerelease tag if
it's immediately after a number, but it's okay to do so if you prefer. Thus,
2.4c1 and 2.4.c1 both represent release candidate 1 of version 2.4, and are
treated as identical by setuptools.

In addition, there are three special prerelease tags that are treated as if
they were the letter c: pre, preview, and rc. So, version 2.4rc1, 2.4pre1
and 2.4preview1 are all the exact same version as 2.4c1, and are treated as
identical by setuptools.

Furthermore, dev is a special tag that is always less than any other
character tag.  It is treated as the equivalent of the character '@'.

Any objections?



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Subversion 1.5 entries not parsed successfully

[Jason R. Coombs]

Subversion 1.5 is released and official now.  Any chance we can get the
patch suggested at
http://mail.python.org/pipermail/python-bugs-list/2008-May/051721.html or
any other fix applied to the 0.6 branch of setuptools?  I acknowledge this
fix may not fit into the grand scheme of the development, but it seems to me
to be a straightforward and logical fix to address what will be an
increasingly prominent problem.

 

Sincerely,

Jason



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] ImportError running 'test' on namespace package - other package in namespace not found

[Jason R. Coombs]

Phillip,
You're right.  I was running the tests on my two Windows Vista
environments, and now I discover the problem only exhibits itself under
Windows.  I expanded the packages to my Ubuntu 8.04 box, and after
installing nose (as you pointed out), the tests also ran (minimally) without
incident.

I then copied the packages to a brand-new Windows Vista install.  I
installed Python 2.5.2, setuptools-0.6c8, and nose 0.10.3.  I then installed
myns.projA using develop, and attempted to run setup.py test on
myns.projB, and the error did exhibit itself.  Additionally, I tested the
sample on Windows XP (Python 2.5.2) and it also exhibits the error.

Thanks to your help, I've further narrowed down the problem.

I have filed a ticket with the nose tracker:
http://code.google.com/p/python-nose/issues/detail?id=197

Any further suggestions are always appreciated.

Regards,
Jason

-Original Message-
From: Phillip J. Eby [mailto:[EMAIL PROTECTED] 
Sent: Sunday, 29 June, 2008 21:21
To: Jason R. Coombs; distutils-sig@python.org
Subject: Re: [Distutils] ImportError running 'test' on namespace package -
other package in namespace not found

I can't reproduce this using the archive you uploaded.  After running 
develop on projA, I ran test on projB and got an error message about 
nose not being available, so I added tests_require=nose to projB, 
and then I got a run that said zero tests ran.



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] ImportError running 'test' on namespace package - other package in namespace not found

[Jason R. Coombs]

It seems that error messages are stripped... so I've uploaded the test case to 
here: http://www.filedropper.com/myns and below is the error message I 
mentioned was attached previously.

Jason

PS C:\Users\jaraco\projects\distutils_namespace\myns.projB ./setup test
running test
running egg_info
creating src\myns.projB.egg-info
writing requirements to src\myns.projB.egg-info\requires.txt
writing src\myns.projB.egg-info\PKG-INFO
writing namespace_packages to src\myns.projB.egg-info\namespace_packages.txt
writing top-level names to src\myns.projB.egg-info\top_level.txt
writing dependency_links to src\myns.projB.egg-info\dependency_links.txt
writing entry points to src\myns.projB.egg-info\entry_points.txt
writing manifest file 'src\myns.projB.egg-info\SOURCES.txt'
reading manifest file 'src\myns.projB.egg-info\SOURCES.txt'
writing manifest file 'src\myns.projB.egg-info\SOURCES.txt'
running build_ext
Failure: ImportError (No module named projA.modA) ... ERROR

==
ERROR: Failure: ImportError (No module named projA.modA)
--
Traceback (most recent call last):
  File c:\program files 
(x86)\python\lib\site-packages\nose-0.10.3-py2.5.egg\nose\loader.py, line 
364, in loadTestsFro
mName
addr.filename, addr.module)
  File c:\program files 
(x86)\python\lib\site-packages\nose-0.10.3-py2.5.egg\nose\importer.py, line 
39, in importFromP
ath
return self.importFromDir(dir_path, fqname)
  File c:\program files 
(x86)\python\lib\site-packages\nose-0.10.3-py2.5.egg\nose\importer.py, line 
84, in importFromD
ir
mod = load_module(part_fqname, fh, filename, desc)
  File 
C:\Users\jaraco\projects\distutils_namespace\myns.projB\test\testBasic.py, 
line 1, in module
from myns.projA.modA import funcA
ImportError: No module named projA.modA

--
Ran 1 test in 0.014s

FAILED (errors=1)


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] ImportError running 'test' on namespace package - other package in namespace not found

[Jason R. Coombs]

I've got two projects:  mynamespace.myprojectA and mynamespace.myprojectB
myprojectB depends on myprojectA.  I'm using setuptools 0.6c8 to manage both
projects.
Both projects are registered using 'setup develop'.  Both projects are
accessible from an interactive interpreter:

PS C:\Users\me\projects python
Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)]
on win32
Type help, copyright, credits or license for more information.
 import mynamespace.myprojectA
 import mynamespace.myprojectB
 from mynamespace.myprojectA import mymoduleZ

However, when I run 'setup test' in myprojectB, the tests fail with

File .mymoduleZ.py, line NNN, in [some context]
from mynamespace.myprojectA.mymoduleZ import MyClassC
ImportError: No module named myprojectA.mymoduleZ

In setup.py, the test_suite is nose.collector.

I searched and couldn't find anyone else with this problem.  Is this a
supported configuration?  Is there something I can do to make tests work
with interdependent projects with the same root namespace?

If there's not something obvious I should be doing differently, I'm happy to
put together a minimal test case that reproduces the problem.  Any
suggestions are appreciated.

Sincerely,
Jason R. Coombs


smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig