[Distutils] Re: Should an sdist/MANIFEST.in include docs and tests?
Bert JW Regeer writes: > Speaking as a maintainer of various different packages for the Pylons > project, we include the following in our sdists: > > - source code for the package > - tests for the package > - documentation for the package > > and of course the license/history/changelog/everything you'd > theoretically need to create a fork (minus .git). Our sdists are > pretty big as a result. > > In our wheels we ship: > > - source code for the package/software > > And nothing else, tests are not included in the wheel. That seems like an eminently sensible scheme: The ‘wheel’ is for installation and should be targeted only to that; the ‘sdist’ is the source distribution and should contain all the source. -- \ “You can stand tall without standing on someone. You can be a | `\ victor without having victims.” —Harriet Woods, 1927–2007 | _o__) | Ben Finney -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/OPIZIKGLQD3UVZQ2JK7M7YYVZN6JJ7VT/
[Distutils] Re: Should an sdist/MANIFEST.in include docs and tests?
Wayne Werner writes: > It depends on whether or not you require docs or tests with your software. > Most of the libraries that I've written require neither - nobody is going > to be running the tests How can you know that? If someone wants to try some changes based on the source distribution (the ‘sdist’) they downloaded, why do you assume they won't run the test suite? Rather, I think you should assume that the source distribution is likely to be used by at least some recipients as a starting point to make and test and redistribute changes to your work (maybe many years later, when the VCS hosting has gone away). For that reason you must not be so certain no-one will want to have those files in the source distribution. > and if they want the documentation then they're probably going to > check out the library page on readthedocs. Quite commonly I have wanted to access documentation for a package that I have the ‘sdist’, but there's no longer any online hosting of the documentation. The distribution should have included the documentation as it existed, in the source distribution. -- \ “There are no chaplains in foxholes.” —Sergeant Justin | `\ Griffith, 2011-07-27 | _o__) | Ben Finney -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/6PJ6FNGAHPGJBZXVK35BOAE26ETPVNRS/
[Distutils] Re: Should an sdist/MANIFEST.in include docs and tests?
Segev Finer writes: > What should really be included in an sdist via MANIFEST.in? The ‘sdist’ name is derived from “source distribution (of this Python package/distribution)”. It is of practical benefit to treat it as “the state of the source needed for developing this Python distribution, as of this release”. The release of a Python distribution (e.g. ‘sampleproject’) marks a snapshot of the source code at a point in time. The source code as of that specific release is beneficial for some recipients, in order to be able to make changes and build a modified complete distribution from that. > A package's source tree, more often than not, includes other files such as > documentation, tests, examples, a random assortment of other text files, > etc. Should docs & tests, in particular, be included in an sdist via > MANIFEST.in? Should other files be added too? Yes, all those should be in the source distribution. PyPI is a good, stable over time, and easily-discovered repository of these source distributions. The source distribution (‘sdist’) should reflect the state of the source code – all of it – that a recipient might need for making modifications, testing them, updating documentation, etc. based on that specific release, by version, sourced from the same location that has the wheels or other ‘bdists’. > Or maybe the sdist should be kept to a minimum? For a minimal installable, we already have the ‘wheel’ format. I don't think ‘sdist’ needs to be kept minimal, when it is more useful as a snapshot of the source for a distribution. > This is not clearly discussed in the packaging guide: > https://packaging.python.org/guides/distributing-packages-using-setuptools/#manifest-in. > The sampleproject (https://github.com/pypa/sampleproject) does seem to > include tests (Well a no-op test) and doesn't include them in the > sdist. Yes, I would like the packaging guide to reflect what I described above. -- \ “Repetition leads to boredom, boredom to horrifying mistakes, | `\ horrifying mistakes to God-I-wish-I-was-still-bored, and it | _o__) goes downhill from there.” —Will Larson, 2008-11-04 | Ben Finney -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/JRA4FXY2MXGW4JPVQE623EPNLWIMGXWX/
[Distutils] Re: Packaging Advice for EFF's Certbot
Brad Warren writes: > Wow! Thanks for all the feedback everyone. Thank you for seeking it, and for the work on making Certbot viable. > > On Jul 23, 2018, at 11:10 PM, James Bennett wrote: > > > > On Mon, Jul 23, 2018 at 8:17 PM, Alex Walters > > wrote: > > As a user of certbot, docker, conda, nix, and guix are non-starters. > > […] > > You're better off with cx_freeze or pyinstaller binaries downloaded > > from a website or a PPA-like-system to add to system package > > managers, which are not perfect solutions either. > > > > I would emphasize this point. > > Not wanting to install a lot of extra software to use Certbot is > certainly fair and we’d obviously prefer our packaging solution to be > as lightweight as possible. Thanks for bringing this up as a > consideration. > > With that said, our current approaches aren’t working for us. We’re a > small development team and continuing to maintain our custom > certbot-auto installer which installs dependencies through your OS > package manager and creates a virtual environment tucked away in /opt > is a significant drain on our resources. Another approach is to get a good handle on the third-party dependency libraries needed for Certbot, and encourage those dependencies to be part of the OS package repositories. That way, you don't take on the huge maintenance burden of trying to keep the third-party library code *and* Certbot in good shape. Just focus on Certbot, and cheer from the sidelines as the OS distributions do the work of third party packages. Yes, that's a different set of problems (for example, keeping Certbot compatible with those versions of the libraries the OS repositories provide). But it is likely to be more manageable, for a team that wants to focus on Certbot's code base primarily. > If there’s existing tools or projects which can make this easier for > us, we’d like to consider them so we can focus our efforts on Certbot > and its features rather than packaging and distribution. The OS distributions – the free-software ones, anyway – have communities that make it their business to do that job well. Engage with them and ask what it'll take to have Library Foo kept up to date in the OS, so you can have Certbot just declare a dependency on that. In other words, I advise: Don't look for tools to do this automatically, engage with the people who already know how to do this in each OS community. > Our main use case has been the long tail of individuals or small teams > of sysadmins who maintain servers and need or want help and automation > around maintaining a secure TLS configuration. For what it's worth, I certainly concur that most people in the group you describe will thank you to not have a bundle of custom dependencies from outside the OS repository, and instead make Certbot work with (and/or be advocates to introduce) the dependencies as OS-provided library packages. > This is definitely our preferred approach to building native packages > right now. To be honest, no one on my team has any experience building > .debs and .rpms and we’re happy to learn what we need to if we go with > this approach, but the more reliable automation around the process we > can use the better. For Debian, instead of becoming packaging experts yourselves, instead we ask that you make Certbot easier for *us* to package and maintain in Debian. See https://wiki.debian.org/UpstreamGuide>; other OS projects likely have similar documentation. -- \ “There's a fine line between fishing and standing on the shore | `\ looking like an idiot.” —Steven Wright | _o__) | Ben Finney -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/YWX6XKZLOD4IKBD5KMPSE7JEWGOYY77H/
Re: [Distutils] Improving Communication
Wes Turner <wes.tur...@gmail.com> writes: > On Friday, April 20, 2018, Donald Stufft <don...@stufft.io> wrote: > > > Currently in the packaging space, we have a number of avenues for > > communication […] > > GitHub Team Discussions [1] > […] > - Unfortunately, they're not real time like IRC/Gitter Another unfortunate thing about GitHub as a forum is that it's centralised, not community-owned, and is closed for those who don't want to have that specific gatekeeper corporation mediating their communication with the community. Feeding data silos is something we should be discouraging (i.e. make effort to not go further along that path) for community-owned projects like Python. So, while we currently have some commitment to that particular data silo, we should IMO not increase that and instead should move to decentralised, community-owned platforms for community discussion. -- \ “Facts are meaningless. You could use facts to prove anything | `\that's even remotely true!” —Homer, _The Simpsons_ | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distributing packages to offline machines
Nick Coghlan <ncogh...@gmail.com> writes: > Keep a requirements.txt file or `Pipfile` in source control, then run > CI jobs based on that repo […] What is a “Pipfile”? -- \ “Software patents provide one more means of controlling access | `\ to information. They are the tool of choice for the internet | _o__) highwayman.” —Anthony Taylor | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 517 - is it ready?
Nick Coghlan <ncogh...@gmail.com> writes: > I've heard back off-list from the folks I was waiting to hear from, > and I'm very happy to say that I'm provisionally accepting PEP 517, as > defined in > https://www.pypa.io/en/latest/specifications/#provisional-acceptance Good news. You might want to update the PEP document again, though; <URL:https://www.python.org/dev/peps/pep-0517/#rrovisional-acceptance> has the title “Rrovisional acceptance” :-) -- \ “I know you believe you understood what you think I said, but I | `\ am not sure you realize that what you heard is not what I | _o__) meant.” —Robert J. McCloskey | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] setup.cfg - to interpolate or not to interpolate
Nick Coghlan <ncogh...@gmail.com> writes: > That would make the following the lowest impact resolution that still > fixes the original reported problem: > > - update the ConfigParser (3.x) and SafeConfigParser (2.x) docs to > explicitly say that "%%" works as an escape > - update setuptools and distutils to use SafeConfigParser in 2.7 > - update the setuptools and distutils docs to say that interpolation > is enabled when reading setup.cfg and "%%" works as an escape (except > on older versions of Python 2.x with either distutils or an older > version of setuptools). > > So I'll retract my initial response as being ill-informed (since I > didn't click through and read the original bug report). I'm glad to see this result, thank you for taking all that into account. -- \ “Repent before ring 31 moves.” —Damon Anton Permezel, hanoi.el | `\ _Tower of Hanoi_ source code, Unix doomsday version, 1985 | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Upload documentation to be hosted at ‘pythonhosted.org’
"Alex Walters" <tritium-l...@sdamon.com> writes: > While the topic of shutting down pythonhosted.org has come up, on and > off, for a few years, there is one certain thing: The recommended > documentation hosting platform of the python community is > readthedocs.org. The problem at the moment is that this is poorly communicated on the sites themselves. As I demonstrated, <URL:https://pythonhosted.org/> gives an impression quite contrary to the recommendation you stated. -- \ “Whenever you read a good book, it's like the author is right | `\ there, in the room talking to you, which is why I don't like to | _o__) read good books.” —Jack Handey | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Upload documentation to be hosted at ‘pythonhosted.org’
Howdy all, Is ‘pythonhosted.org’ still a recommended service for hosting documentation for Python distributions? It certainly seems so, from the page at <URL:https://pythonhosted.org/>: This site hosts documentation uploaded by authors of packages in the Python Package Index. To upload documentation, go to your package edit page (http://pypi.python.org/pypi?%3Aaction=pkg_edit=yourpackage), and fill out the form at the bottom of the page. Should that change for the fact that ‘pypi.python.org’ is in the process of deprecation for uploads? -- \ “If nature has made any one thing less susceptible than all | `\others of exclusive property, it is the action of the thinking | _o__) power called an idea” —Thomas Jefferson, 1813-08-13 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] ANNOUNCE: Sunsetting of uploading to the legacy PyPI/TestPyPI
Randy Syring <ra...@thesyrings.us> writes: > […] would it hurt to have an "provisional" area to the key projects > list that makes people aware of the projects without "blessing" them > as Key Projects. It might hurt, yes. I am already quite confused by what is meant to be done in the Brave New World described in this thread. Pointing to a document that gives a set of tools is helpful – now I have only the confusion of figuring out whether all my development and deployment environments can use those – but that document becomes *less* useful as you add more decisions I need to make. > The whole point of that section of the guide is to raise awareness, > yes? That describes the purpose of an announcement like this. I'm now aware of the change that's looming. On the conitrary, the point of the section of the guide, for a user like me, is to *simplify* the actions I need to take to survive the transition. Adding more options to decide between, however well-intentioned, will work against that purpose. -- \ “You could augment an earwig to the point where it understood | `\ nuclear physics, but it would still be a very stupid thing to | _o__) do!” —The Doctor, _The Two Doctors_ | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] ANNOUNCE: Sunsetting of uploading to the legacy PyPI/TestPyPI
Donald Stufft <don...@stufft.io> writes: > Umm. Twine 1.8.0, Python 3.4.6, 3.5.3, 3.6.0, Not sure on 2.7, the > Github interface seems to be broken and not listing the tags for that > one. Thanks. What version of Setuptools is needed? I'm hoping that the above list will be prominently published with the documentation that we point people toward for announcing we should be using those tools. (“Use the latest version of the tool” doesn't cut it; we'll need to check what versions our operating system provides, and know whether it's in or out of compliance with the above list.) > > What is the cross-over period: ‘pypi.org’ as the primary > > recommendation, while ‘pypi.python.org’ continues to work? > > I’m not sure what specifically you’re asking here? Are you looking for > how long they’re both going to remain active or the process we’re > going to migrate things over with? Or something else? I'm working on the assumption the sequence will include: * Now: Uploading to ‘pypi.python.org’ is officially recommended; it goes to the same data store as uploading to ‘pypi.org’. Both work fine. * Crossover: Uploading to ‘pypi.org’ is officially recommended; it goes to the same data store as uploading to ‘pypi.python.org’. Both work fine. * Warehouse Rules Alone: Uploading to ‘pypi.org’ is recommended, because uploading to ‘pypi.python.org’ doesn't work. So, what is the period – when does it start and when does it end – that the Crossover state will active? -- \ “… correct code is great, code that crashes could use | `\ improvement, but incorrect code that doesn’t crash is a | _o__)horrible nightmare.” —Chris Smith, 2008-08-22 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] ANNOUNCE: Sunsetting of uploading to the legacy PyPI/TestPyPI
Nick Coghlan <ncogh...@gmail.com> writes: > On 23 June 2017 at 20:44, Ben Finney <ben+pyt...@benfinney.id.au> wrote: > > What is the cross-over period: ‘pypi.org’ as the primary > > recommendation, while ‘pypi.python.org’ continues to work? > > Full migration = pypi.python.org becomes a CNAME for pypi.org. Thank you, that is the state at the *end* of the cross-over period. But that's not what I asked. What is the cross-over period, during which we have ‘pypi.org’ as the primary recommendation, while ‘pypi.python.org’ continues to work as it currently does? -- \ “If you always want the latest and greatest, then you have to | `\ buy a new iPod at least once a year.” —Steve Jobs, MSNBC | _o__) interview 2006-05-25 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] ANNOUNCE: Sunsetting of uploading to the legacy PyPI/TestPyPI
Donald Stufft <don...@stufft.io> writes: > […] rewrite PyPI in order to solve a lot of long standing problems. > For those who aren't aware, that is currently available at > https://pypi.org/ Thank you to everyone involved in bringing this to fruition! > For a awhile now, Python, setuptools, and twine have all defaulted to > using this new code base for uploading artifacts to PyPI. Where can we find the exact versions of each that default to ‘pypi.org’ for uploads? > [‘pypi.org’] uses the same database that "Legacy" PyPI does, so the > two are essentially just different views over the same data. At what point does the big red warning banner (“ This is a pre-production deployment of Warehouse.”) come down? What is the cross-over period: ‘pypi.org’ as the primary recommendation, while ‘pypi.python.org’ continues to work? -- \ “You can never entirely stop being what you once were. That's | `\ why it's important to be the right person today, and not put it | _o__) off until tomorrow.” —Larry Wall | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Which commercial vendor?
Thomas Güttler <guettl...@thomas-guettler.de> writes: > Let's look back. How was this in the past? Maybe five years ago? That's a very vague question. What kind of answer do you want? Is it one you have an answer for already; and if so, what is the point of your question here? I don't doubt that you *have* a point. What I doubt is the value to others here of asking something that demands a lot of to even find out what you mean. -- \ “[T]he question of whether machines can think … is about as | `\ relevant as the question of whether submarines can swim.” | _o__) —Edsger W. Dijkstra | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] GnuPG signatures on PyPI: why so few?
Paul Moore <p.f.mo...@gmail.com> writes: > One tool that needs improvement to be easier to use for this to happen > is GPG itself. No disagreement from me on that. And indeed, the GnuPG project's chronic under-funding eventually drew attention from the new Core Infrastructure Initiative <URL:https://www.coreinfrastructure.org/gnupg> to improve it faster than was historically the case. This is thanks in large part to the amazing work of Nadia Eghbal <URL:http://nadiaeghbal.com/oss> in drawing attention to how critical free software, such as GnuPG, benefits society enormously and must receive reliable funding from the organisations who benefit. If anyone reading this works for any organisation that wants to ensure such critical free-software infrastructure continues to be consistently funded and maintained, encourage regular financial contribution to the Core Infrastructure Initiative <URL:https://www.coreinfrastructure.org/> or similar projects. > As a Windows user, I've "played" with it in the past, and found it > frustratingly difficult. I hope many people here will find the guide published by the FSF, Email Self-Defense <URL:https://emailselfdefense.fsf.org/>, a useful walk through how to set it up properly. -- \ “I must say that I find television very educational. The minute | `\ somebody turns it on, I go to the library and read a book.” | _o__) —Groucho Marx | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] GnuPG signatures on PyPI: why so few?
(Ian, your messages are failing to properly quote material you're responding to. The message you posted has no quote leaders on my material, which looks like it was written by you; see the message at <URL:https://mail.python.org/pipermail/distutils-sig/2017-March/030245.html>. If this is some mangling done by GMail, you may need to change its configuration or post using something else until it's fixed.) Ian Cordasco <graffatcolmin...@gmail.com> writes: > If you can find a tool that is easy to install on Linux, Windows, and Mac, > which solves the problems above by virtue of having very good defaults, and > is accessible to anyone with less than a few hours to waste on it... Then > maybe I would collaborate to make it a requirement. No-one here has argued that it be a requirement as things stand now. I'm talking about encouraging it as a norm, by improving tool support to make it easier. -- \“The fact of your own existence is the most astonishing fact | `\you'll ever have to confront. Don't dare ever see your life as | _o__)boring, monotonous, or joyless.” —Richard Dawkins, 2010-03-10 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python 3.5
Adh <amine.djill...@gmail.com> writes: > Hello, I do not arrive to install python 3.5 in the terminal, I use > Ubuntu. You should ask general usage questions in the main user forum for Python, <URL:https://mail.python.org/mailman/listinfo/python-list>. Please subscribe there, tell them which Ubuntu version you are using, what command you type and what is the result. They will help you from there. -- \“Simplicity is prerequisite for reliability.” —Edsger W. | `\ Dijkstra | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Can't upload sdist: "File already exists"
Nick Timkovich <prometheus...@gmail.com> writes: > Generally, is it OK (recommended/socially acceptable) to upload > universal wheels only or is the sdist still nice to have available? I > guess maybe for pip, but it considering universal, pure-Python wheels > as "binaries" seems quirky. None the less, the Wheel format *is* a format intended for installation (so a “binary” format in that terminology). A wheel is not suitable for making changes and re-building from source. So the source distribution still needs to be uploaded as well, to support that important freedom for the recipient. -- \ “Liberty, n. One of imagination's most precious possessions.” | `\ —Ambrose Bierce, _The Devil's Dictionary_, 1906 | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Different purposes for Setuptools requirements versus Pip requirements (was: Maintaining a curated set of Python packages)
Jeremy Stanley <fu...@yuggoth.org> writes: > [the ‘pbr’ library] does allow you to basically abstract away most > common configuration into declarative setup.cfg and requirements.txt > files Hmm. That description sounds like a mistaken conflation of two things that should be distinct: * Declaration in Setuptools metadata of what versions of dependencies this distribution is *compatible with*. This purpose is served by Distutils ‘install_requires’ (and ‘test_requires’, etc.). It is characterised by specifying a range of versions for each dependency, for allowing dependency resolvers to have options to choose from. foo >=1.2, <3.0 * Declaration in Pip metadata for what *exact version* of each dependency I want to deploy. This purpose is served by Pip ‘requires.txt’ input. It is characterised by pinning a *single* version of each dependency, for a deterministic, repeatable deployment. foo == 1.4.7 If we're saying ‘pbr’ encourages the use of a single set of declarations for those quite different purposes, that sounds like an attractive nuisance. For those who haven't read it, see this post from Donald Stufft for why those purposes need to be kept distinct: There’s a lot of misunderstanding between setup.py and requirements.txt and their roles. A lot of people have felt they are duplicated information and have even created tools to handle this “duplication”. <URL:https://caremad.io/posts/2013/07/setup-vs-requirement/> -- \ “Corporation, n. An ingenious device for obtaining individual | `\ profit without individual responsibility.” —Ambrose Bierce, | _o__) _The Devil's Dictionary_, 1906 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Interrogate distribution for an entry point command path
Thomas Kluyver <tho...@kluyver.me.uk> writes: > If the entry point looks like: > > foo=foomod:main > > Then you can invoke it in a subprocess by running: > > subprocess.Popen([sys.executable, '-c', 'import foomod; foomod.main()']) That will invoke the program. I'll probably try that. One disadvantage there is the process won't have an informative name; instead of being named ‘/foo/bar/lipsum’, it will be named ‘python’ which is less useful. > This avoids the need to work out where the 'foo' script has been > installed to. So, I'm still wanting to know from Setuptools itself at run time, what filesystem path Setuptools installed the command to. -- \“I took it easy today. I just pretty much layed around in my | `\underwear all day. … Got kicked out of quite a few places, | _o__) though.” —Bug-Eyed Earl, _Red Meat_ | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Interrogate distribution for an entry point command path
Nick Timkovich <prometheus...@gmail.com> writes: > 1. include the shell scripts (could also be binaries) in the package & > manifest > (https://github.com/nicktimko/autolycus/blob/master/MANIFEST.in#L3) No, I'm using ‘[…] install --install-scripts=APPLICATION_SCRIPTS_PATH’ at install time. > 2. use pkg_resources to get where the .sh was so I could run it with > subprocess. What I need is for ‘pkg_resources’ to tell me at run time where ‘--install-scripts’ put the scripts for this application. -- \“I don't accept the currently fashionable assertion that any | `\ view is automatically as worthy of respect as any equal and | _o__) opposite view.” —Douglas Adams | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Interrogate distribution for an entry point command path
Nick Timkovich <prometheus...@gmail.com> writes: > Usually that entry point is on the PATH […] It's not, because I'm deliberately specifying that it shouldn't be, at install time. This is an executable that is private to the application and not for general availability on the host. > If you want to call that entry point from your code, the clean way > (same environment/version, and especially if you don't need to bother > multiprocessing it) would be to import the corresponding entry point > function & call that. I'm modifying an existing application that invokes the program as a subprocess, so I'm wanting to find that program as an external command. > I might not be answering your question directly, but hopefully there's > a workaround there. What's your use-case for grabbing the exec path? Existing code assumes it is an external command on the shell PATH, but I'm changing that so that it's not on PATH. I need to make a minimal change and want to ensure that I get the right filesystem path based on what the distribution knows about itself. -- \ “I like to fill my bathtub up with water, then turn the shower | `\ on and pretend I'm in a submarine that's been hit.” —Steven | _o__) Wright | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Interrogate distribution for an entry point command path
Howdy, How can a Python application discover at run-time where on the filesystem its own ‘entry_points’ programs are available? The Setuptools ‘entry_points’ are available at run-time to the distribution, via the ‘pkg_resources’ API for entry points <URL:https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points>. I don't see there how to interrogate for which filesystem path to invoke for a specific entry point's command. How can I reliably ask ‘pkg_resources’ for “where is the entry point named ‘foo’ installed?” such that I get a filesystem path of a command to invoke? -- \ “Geeks like to think that they can ignore politics. You can | `\leave politics alone, but politics won't leave you alone.” | _o__) —Richard M. Stallman, 2002-07-26 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Consumers that care about the contents of an sdist (was: abstract build system PEP update)
Nathaniel Smith <n...@vorpus.org> writes: > Ah-ha, this is useful. The reason this hasn't been considered, at > least in my proposal, is that I think this is the first I've heard > that there is anything that cares about what's in an sdist besides > setup.py :-). Please know that OS distributions also very much care about the contents of the sdist, because that's what the distributor claims is the corresponding source for the distribution. So they'll want to start from the sdist and build from that, to verify the sdist constitutes the source. -- \“The fact of your own existence is the most astonishing fact | `\you'll ever have to confront. Don't dare ever see your life as | _o__)boring, monotonous, or joyless.” —Richard Dawkins, 2010-03-10 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Ensuring source availability for PyPI entries (was: PEP: Build system abstraction for pip/conda etc)
Paul Moore <p.f.mo...@gmail.com> writes: > But as I said in my response to Nathaniel, it may be that all that is > needed is some context in the PEP explaining how we require[1] people > to upload source to PyPI in the new world where we support build > systems which don't have a "sdist" command like setuptools does. > > Paul > > [1] I say "require" in the sense of "you have to follow these rules if > pip is to be able to use your source", not "you must upload source" - > although I hope that the number of people actually preferring to *not* > include source in their PyPI uploads is vanishingly small... Don't underestimate the number of people who don't wish to put source in PyPI. Especially, don't rely merely on hope that such numbers remain vanishingly small. After all, as you may recall from the discussion in 2014 [0], there are core Python members who wish PyPI had remained an index only, with distributions allowed to have *no* files necessarily hosted at PyPI. It seems reasonable to infer that their preferences are shared by others, and some people would wish to avoid putting source for a distribution on PyPI if that were to become easier. [0] https://mail.python.org/pipermail/distutils-sig/2014-May/024224.html -- \ “DRM doesn't inconvenience [lawbreakers] — indeed, over time it | `\ trains law-abiding users to become [lawbreakers] out of sheer | _o__)frustration.” —Charles Stross, 2010-05-09 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] version conflict
Robin Becker <ro...@reportlab.com> writes:
> pkg_resources.VersionConflict: (certifi 2015.11.20.1
> (/Library/Python/2.7/site-packages),
> Requirement.parse('certifi==2015.11.20'))
This is the hazard of specifying a strict no-earlier-no-later version
requirement. Presumably ‘certifi’ at version “2015.11.20.1” would serve
just fine, but the strict requirement on “certifi==2015.11.20” rejects
that.
> the full traceback is visible here
> https://bitbucket.org/rptlab/reportlab/issues/73/versionconflict
Sadly, Setuptools (in this case the ‘pkg_resources’ library) gives no
help to identify where the failed requirement is specified. This is IMO
worth a bug report on Setuptools: it should catch that exception and
give a graceful error message with useful information, not a traceback
dump.
--
\ “I tell you the truth: this generation will certainly not pass |
`\ away until all these things [the end of the world] have |
_o__) happened.” —Jesus, c. 30 CE, as quoted in Matthew 24:34 |
Ben Finney
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Building and installing packages on a (unix) system lacking network accesss
KM <nal...@fea.st> writes: > This works up until a certain point, where pip (or something launched > by pip) tries to download from the internet. I tried adding > ''--global-option '--no-index -f '" to my options That specifies a *single* option to be passed to Distutils, containing spaces: “--no-index -f ”. > but this failed. I get the error: option --no-index -f not > recognized. Right, there is no “--no-index -f ” option recognised by Distutils. The “--global-option” option takes a single argument, which it passes to the ‘setup.py’ invocation. If you want multiple options passed that way, I think you need to use ‘--global-option’ several times. Once you use that, though, you'll find that Distutils also doesn't recognise “--no-index” nor “--find-links” options. What are you hoping those will do? -- \ “Pinky, are you pondering what I'm pondering?” “I think so, | `\ Brain, but if they called them ‘Sad Meals’, kids wouldn't buy | _o__)them!” —_Pinky and The Brain_ | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] The future of invoking pip
Paul Moore <p.f.mo...@gmail.com> writes: > Unfortunately, pip is orders of magnitude more frequently used than > the other commands that suffer from this problem, so it *feels* like a > pip issue. But it isn't really, it's either a Python issue or an OS > issue, depending on how you view it. +1. Addressing this by insisting on ‘python -m foo’ is not a solution. It's a plaster over a problem that will remain until the underlying conflict is resolved. That's not to say PyPA should ignore the issue, certainly there are things that can be done to help. But ‘python -m foo’ is an ugly wart, and I really want the rhetoric to acknowledge that instead of considering it a satisfactory end point. -- \“We cannot solve our problems with the same thinking we used | `\ when we created them.” —Albert Einstein | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Please don't impose additional barriers to participation
Robert Collins <robe...@robertcollins.net> writes: > On 28 October 2015 at 18:02, Ben Finney <ben+pyt...@benfinney.id.au> wrote: > > Please keep the conversation on a mailing list where one can > > participate without needing to sign up with any particular service > > provider. > > One has to sign up with the mailing list, so there's no functional > difference there. You've stated that point in the past, and it's still untrue. The functional difference is that the mailing list is provided by effectively the same organisation that one is wanting to participating in discussion with. An objector to the terms of service of the mailing list is much less likely to want to also participate as part of this community. The same is certainly not true of GitHub. Please don't make a GitHub account a requirement for participating in this community's discussions on an equal footing with other participants. > > Your proposal would have the effect of excluding people from the > > conversation if they don't agree to have a GitHub account. I think > > it's valuable to avoid that barrier to entry, needing only an email > > account. > > They can always reply to emails. As first-class citizens, like today? Or as outsiders looking in? The proposal is to move the primary discussion forum *away* from this forum, and make GitHub participants first-class citizens of that forum. Either there is no significant advantage to having the discussion forum take place primarily on the walled garden of GitHub, in which case the proposal seems pointless on its face. Or there is some significant advantage being proposed, in which case my objections stand: don't restrict that advantage to only those who are willing to use a GitHub account to do so. -- \ “If you go flying back through time and you see somebody else | `\ flying forward into the future, it's probably best to avoid eye | _o__) contact.” —Jack Handey | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Time for a setuptools_lite??
Nick Coghlan <ncogh...@gmail.com> writes: > That said, for Fedora RPMs, we don't need sdist per se - we only need > ready access to the "original sources". There's a temporal element to that, too. While “ready access” to the source might be clear enough at the moment of the wheel's release, it is less clear in ten years time when the original source for a package still in Fedora is needed again. In practice, a URL to (what one hopes is) the source is not enough to provude assurance the source will be available long in the future. Only a known tarball (or equivalent fixed single-file archive form) of the actual source is going to provide that. > It's just that wheels don't count, since they're a nominally binary > format, and "C ABI = None, Platform = None" in the filename isn't > currently an entirely reliable indicator of a pure Python wheel file. One needn't say that wheel is “nominally” anything; it is sufficient to ask “is this what a recipient would need to have the source in a form suitable for further modification and redistribution?” A wheel distribution is fairly clearly *not* the preferred form of the work for a recipient to have to exercise freedom to modify and redistribute. The wheel distgribution was generated from source files edited by the developer, and so *those* file are the source form of the work, the wheel distribution is not. > A HTTPS source control URL together with a commit hash *can* count, > though, even in the absence of a source archive uploaded to PyPI, as > the VCS information is enough for us to retrieve the original sources > and put them in the SRPM. As I point out above, that's only reliably true if it is immediately turned into a more reliably-archived form than a URL to some hosting provider somewhere. So, in practice, the URL is not enough to provide good assurance one has the source form of the work for redistribution. > (I'm not as familiar with Debian's policies as I am with those for > Fedora et al, but as far as I am aware, they want a reference to the > original sources for similar reasons of build reproducibility, license > compliance, and code auditability). Not merely a reference, but (as explained above) The actual source form of the work, persistent over long periods of time as the known corresponding source form of that version of the work. > Right, any recommended solution needs to provide access to the > original sources *in addition to* any already built wheel files. Yes, thanks. -- \ “Try adding “as long as you don't breach the terms of service – | `\ according to our sole judgement” to the end of any cloud | _o__) computing pitch.” —Simon Phipps, 2010-12-11 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Please don't impose additional barriers to participation (was: build system abstraction PEP)
Marcus Smith <qwc...@gmail.com> writes: > 1) *Please*, *please*, *please* let's start doing PEP conversations as > PRs to pypa/interoperability-peps : ) Please keep the conversation on a mailing list where one can participate without needing to sign up with any particular service provider. Your proposal would have the effect of excluding people from the conversation if they don't agree to have a GitHub account. I think it's valuable to avoid that barrier to entry, needing only an email account. -- \ “'Tis strange, — but true; for truth is always strange; / | `\Stranger than fiction.” —“Lord” George Gordon Noel Byron, _Don | _o__) Juan_ | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] red, green, refactor OR red, green, new feature?
Thomas Güttler <guettl...@thomas-guettler.de> writes: > Hi distutils friends, > > which workflow do you prefer? > > 1. red (test fails) > 2. green (test and code works) > 3. refactor (clean up, deprecate old stuff) More accurately: * Write a failing test for the new behaviour. (red) * Change the system-under-test until the test passes. (green) * Clean up, remove redundancies, change no behaviour (refactor) > OR I disagree with that dichotomy. > 1. red > 2. green > 3. new feature. The latter isn't a workflow I recognise. It appears to confuse the levels of working. Making a new feature is much higher level than the rest of those steps; it entails many iterations of “red, green, refactor”. > I, like most other programmers, love new features You're thinking that is incompatible with test-driven development. Actually, from the TDD point of view, adding new features is like any other change to the system under test: follow the TDD workflow (red, green, refactor) until the new feature works. > Are you willing to clean up and deprecate old stuff? Everyone who works with code should be *willing* to do it. What TDD does is allow the programer to do that with confidence. -- \“Intellectual property is to the 21st century what the slave | `\ trade was to the 16th.” —David Mertz | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PyPi not allowing duplicate filenames
Ben Finney <ben+pyt...@benfinney.id.au> writes: > Apache License 2.0 imposes no additional reastrictions on the > recipient of a work licensed under GNU GPL v3. So that meets the > conditions of both. Apache License 2.0 does impose further restrictions not found in GPL v2 though. Sorry for the confusion. Nathaniel Smith <n...@pobox.com> writes: > The FSF believes that Apache 2 is incompatible with GPLv2 (but v3 is > OK) Yes, I'd neglected that. <URL:https://gnu.org/licenses/license-list.html#apache2> -- \“My house is made out of balsa wood, so when I want to scare | `\ the neighborhood kids I lift it over my head and tell them to | _o__) get out of my yard or I'll throw it at them.” —Steven Wright | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PyPi not allowing duplicate filenames
Nathaniel Smith <n...@pobox.com> writes: > That's under Apache 2, so it can't be used by GPLv2 packages, or any > package that might be used by GPLv2 packages. I'm not sure what the claim is here, but it seems false to me. Apache License 2.0 permits a derived work to be released under a different license (even a non-free license). See Apache License 2.0 §4. So a work BAR can be released under GNU GPL (e.g. “version 3 or later”) that derives from work FOO released under Apache License 2.0. The GNU GPL version 3 allows combination with other works with different license conditions, provided those conditions don't impose further restrictions on recipients. See GNU GPL v3 §7, §10, §12. (GPL v2 has different wording with similar effect.) Apache License 2.0 imposes no additional reastrictions on the recipient of a work licensed under GNU GPL v3. So that meets the conditions of both. FOO as a distinct work retains its license conditions, and all recipients of FOO or BAR must comply with its conditions. The work BAR as a whole carries the GNU GPL version (for example) 3 conditions, and all recipients of BAR must comply with its conditions. The two licenses are compatible, as far as I can see. The effective result is a work BAR, licensed as free software under GPL version (e.g.) 3 or later. Does that not match your understanding? Have I misunderstood your claim? -- \ “Corporation, n. An ingenious device for obtaining individual | `\ profit without individual responsibility.” —Ambrose Bierce, | _o__) _The Devil's Dictionary_, 1906 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Where should I put tests when packaging python modules?
Thomas Güttler <guettl...@thomas-guettler.de> writes: > Where should I put tests when packaging python modules? When packaging them? The same place they go when not packaging them :-) > I want a "cowpath", an "obvious way" For me, the obvious way is to have: > outside the module like this: > https://github.com/pypa/sampleproject/tree/master/tests and have ‘setup.py’ ensure the tests are distributed with the source package, but not installed. > I think there is no need to hurry. Let's wait one week, and then check > which one is preferred. More important than which is preferred, we should use the one which is best (regardless how popular it may be). So instead of just hearing votes, we should examine reasoned argument in favour of good options. -- \“Visitors are expected to complain at the office between the | `\ hours of 9 and 11 a.m. daily.” —hotel, Athens | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Where should I put tests when packaging python modules?
Barry Warsaw <ba...@python.org> writes: > I'm a big fan of putting the tests inside the package. I've often > looked at a package's tests to get a better understanding of something > that was unclear for the documentation, or didn't work the way I > expected. Having the tests there in the installed package makes it > easier to refer to. That doesn't follow, or I'm not understanding you. If you have the tests in the source package, as is being advocated, you have the tests available for reference. So the *relative location* of the tests, within that source tree, doesn't argue for what you're saying. Are you arguing the separate point of whether tests should be *installed* with the package? -- \ “We now have access to so much information that we can find | `\ support for any prejudice or opinion.” —David Suzuki, 2008-06-27 | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Where should I put tests when packaging python modules?
Barry Warsaw <ba...@python.org> writes: > On Oct 06, 2015, at 06:21 AM, Donald Stufft wrote: > > >FreeBSD relies on ``python setup.py test`` as it's preferred test > >invocation > > Oh how I wish there was a standard way to *declare* how to run the > test suite, such that all our automated tools (or the humans :) didn't > have to guess. I think the above describes the standard way of declaring the test runner: The ‘setup.py test’ command. Now, I lament that more Python projects don't *conform to* that standard, but at least it exists. -- \ “I have never made but one prayer to God, a very short one: ‘O | `\ Lord, make my enemies ridiculous!’ And God granted it.” | _o__) —Voltaire | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Where should I put tests when packaging python modules?
Ionel Cristian Mărieș <cont...@ionelmc.ro> writes: > I've seen some projects that lump up lots of test data and crazy files > in their packages tests and that created install issues. > > On the other hand, if the user really wants to run the tests he can > just get the sources (that would naturally include everything)? Yes, this is a sensible approach: * The source package contains all the source files a developer would use to make further changes and test them. * The package for installation contains only those files useful run-time users, plus metadata (e.g. copyright information). I highly recommend it, and I would like the PyPA to also recommend the above approach. It does, though, require the acknowledgement of a separate *build* step in the development-and-release process. The build step is prior to the packaging step, and it generates run-time files from the collection of source files. That separation of a discrete build step is crucial for many good practices: generate documentation, integration testing, OS packaging, etc. -- \“The restriction of knowledge to an elite group destroys the | `\ spirit of society and leads to its intellectual | _o__)impoverishment.” —Albert Einstein | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Where should I put tests when packaging python modules?
Barry Warsaw <ba...@python.org> writes: > On Oct 07, 2015, at 08:54 AM, Ben Finney wrote: > > >Barry Warsaw <ba...@python.org> writes: > > > >> I'm a big fan of putting the tests inside the package. I've often > >> looked at a package's tests to get a better understanding of something > >> that was unclear for the documentation, or didn't work the way I > >> expected. Having the tests there in the installed package makes it > >> easier to refer to. > > > >That doesn't follow, or I'm not understanding you. > >[…] > > > >Are you arguing the separate point of whether tests should be > >*installed* with the package? > > Yes. We've had this conversation before in the context of Debian package > sponsorship. I know and respect that you disagree. Okay. That's quite an orthogonal dimension, though, to the *relative location* of tests within the source tree. So “I'm a big fan of putting tests inside the [Python] package [directory]” can't be motivated by “Having the tests there in the installed package”. The two aren't related, AFAICT. -- \“There was a point to this story, but it has temporarily | `\escaped the chronicler's mind.” —Douglas Adams | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Working toward Linux wheel support
Nick Coghlan <ncogh...@gmail.com> writes: > On 7 September 2015 at 02:09, Marcus Smith <qwc...@gmail.com> wrote: > > For example, down the road when there's Wheel 2.0, what's the "Current > > Specs" for wheel? > > Yes, but I think that's easy enough to handle by having a default URL > that always goes to the latest version of the spec, and moving > previous versions to URLs that include the version number. Or consistently publish each spec version to a predictable URL with the version number, and have the default URL *redirect* to whatever is the current-versioned spec. That way, the URL works as people expect, *and* the resulting destination gives a URL that (when inevitably copy-and-pasted) will retain its meaning over time. -- \ Moriarty: “Forty thousand million billion dollars? That money | `\must be worth a fortune!” —The Goon Show, _The Sale of | _o__) Manhattan_ | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Reviving PEP 470 - Removing External Hosting support on PyPI
Donald Stufft don...@stufft.io writes: Removal of the “External Repository Discover” feature. […] I think that the feature itself is a bad feature and I think it presents a poor experience for people who want to use it, so I’ve removed it from the PEP and instead focused the PEP on explicitly recommending that all installers should implement the ability to specify multiple repositories and deprecating and removing the ability for finding anything but files hosted by the repository itself on /simple/. +1, thank you for this improvement. -- \“Human reason is snatching everything to itself, leaving | `\ nothing for faith.” —Bernard of Clairvaux, 1090–1153 CE | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Version management for devpi (was: devpi-{server-2.2.2, web-2.4.0, client-2.3.0} releases)
holger krekel hol...@merlinux.eu writes: We just released devpi-server-2.2.2, devpi-web-2.4.0 and devpi-client-2.3.0, core parts of the private pypi package management and testing system. Thank you for the release, and for continuing to work on this code base. What reason is there to have different code bases, with different versions, for a collection that is so clearly all related and makes sense as a single code base? It's very confusing to know what versions relate to other versions of other parts of devpi. The versions don't help to know the timeline, which defeats one of the primary purposes of versions on a code base. Since the code tends to be ready for release at the same time, why not make a single version string for the whole code base, and release it as a unit? -- \ “Reichel's Law: A body on vacation tends to remain on vacation | `\unless acted upon by an outside force.” —Carol Reichel | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Accessing package data files in wheels
Justin Uang justin.u...@gmail.com writes: What is the recommended way to get access to a data file in my package if I'm using a wheel? pkg_resources seems like it's mainly useful because eggs might be used in a zipped form, but for wheels, I'm guaranteed that the data will be unpacked into a directory structure right? In that case, should I just use __file__ to manually find the location of the file? An advantage of querying package resources using the ‘pkg_resources’ API is that their lockation *doesn't* need to be known by your program. Let the installation process put them in a sensible location for the system, and discover that location at run-time using ‘pkg_resources’. I'm assuming I can still use pkg_resources, but it feels like I'm adding an unnecessary dependency on setuptools, given that setuptools isn't needed during installation but only when building the wheel. You'll be doing the recipient a favour by allowing the package resources to be in a different place from the executable files. -- \ “Philosophy is questions that may never be answered. Religion | `\ is answers that may never be questioned.” —anonymous | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PyPI and Uploading Documentation
Donald Stufft don...@stufft.io writes: Ok, so unless someone comes out against this in the near future here are my plans: 1. Implement the ability to delete documentation. +1. 2. Implement the ability to add a (simple) redirect where we would essentially just send /project/(.*) to $REDIRECT_BASE/$1. 3. Implement the ability to point the documentation URL to something that isn't pythonhosted.org Both of these turn PyPI into a vector for arbitrary content, including (for example) illegal, misleading, or malicious content. Automatic redirects actively expose the visitor to any malicious or mistaken links set by the project owner. If you want to allow the documentation to be at some arbitrary location of the project owner's choice, then an explicit static link, which the visitor must click on (similar to the project home page link) is best. -- \ “I find the whole business of religion profoundly interesting. | `\ But it does mystify me that otherwise intelligent people take | _o__)it seriously.” —Douglas Adams | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PyPI is a sick sick hoarder
Donald Stufft don...@stufft.io writes: On May 15, 2015, at 2:57 PM, Robert Collins robe...@robertcollins.net wrote: If folk consider PyPI a sort of historical archive then perhaps we could have a feature to select 'supported' versions by the author, and allow a query parameter to ask for all the versions. It’s common for deployments people have to use a requirements.txt file like ``foo==1.0`` and to just continue to pull from PyPI. Deleting the old files breaks anyone doing that, so it would require either having people bundle their deps in their repositories or some way to get at those old versions. Personally I think that we shouldn’t go deleting the old versions or encouraging people to do that. Yes, it's common to consider PyPI as a repository of all versions ever released, and to treat it as an archive whose URLs will continue to make available the historical versions. -- \ “If history and science have taught us anything, it is that | `\ passion and desire are not the same as truth.” —E. O. Wilson, | _o__) _Consilience_, 1998 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Dynamic linking between Python modules (was: Beyond wheels 1.0: helping downstream, FHS and more)
Chris Barker chris.bar...@noaa.gov writes: On Tue, Apr 14, 2015 at 8:41 AM, Nick Coghlan ncogh...@gmail.com wrote: The point where I draw the line is supporting *dynamic* linking between modules - I'm confused -- you don't want a system to be able to install ONE version of a lib that various python packages can all link to? That's really the key use-case for me Agreed. A key pain point for Python distributions is the lack of support for installing *one* instrance of a Python library, and other Python modules able to discover such installed libraries which meet their declared dependency. For example: * Python distribution ‘foo’, for Python implementation “D” on architecture “X“, declares dependency on “bar = 1.7”. * Installing Python distribution ‘bar’ version 1.8, on a host running Python “D” for architecture “X”, goes to a single instance for the ‘bar’ library for that architecture and Python implementation. * Invoking the ‘foo’ code on the same host will go looking (dynamically?) for the dependency ‘bar’ and find version 1.8 already installed in the one instance on that host. It uses that and all is well. I'm in agreement with Chris that, while the above example may not currently play out as described, that is a fault to be fixed by improving Python's packaging and distribution tools so that it *is* a first-class use case. Nick, you seem to be arguing against that. Can you clarify? -- \ “Natural catastrophes are rare, but they come often enough. We | `\ need not force the hand of nature.” —Carl Sagan, _Cosmos_, 1980 | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Beyond wheel 1.0: more fine-grained installation scheme
David Cournapeau courn...@gmail.com writes: My suggestion for a better scheme would be to use an extended version of the various default directories defined by autotools. The extension would handle windows-specifics. This is a good direction to go, IMO. The expectations of Windows and *nix, when it comes to where files should be installed, are so disparate that abstracting the locations to functional labels which mean the same on all platforms will inevitably result in a lot of such labels. So, if we agree that such abstraction is worthwhile to pursue, the large set of labels (and perhaps even leaving it open to growing even more) should be embraced. # Suggested variables The goal of supporting those variables is to take something that is flexible enough to support almost any installation scheme, without putting additional burden on the developer. People who do not want/need the flexibility will not need to do anything more than what they do today. A point to note is that “people” in that statement can be different people with conflicting wants/needs. The developer who writes the Distutils metadata may want to have as little in there as possible, at the expense of flexibility in installation. The system integrator who needs to specify many different locations for the various files in the distribution will want the opposite. They still need to work on the same code base. So perhaps one consideration is that the distribution metadata should be extensible by the integrator, without extensive patches to the Python code; and the developer's code as written still finds the files in the resulting filesystem locations. Is that feasible? The variables I would suggest are every variable defined in https://github.com/cournape/Bento/blob/master/bento/core/platforms/sysconfig.py#L10, except for destdir which is not relevant here. That looks like a lot of named locations. I think that is inevitable, as argued above. For now, I would be happy to just make a proof of concept not caring about backward compatibility in a pip branch. Does that sound like a workable basis to flush out an actual proposal ? Looking forward to it. Thanks! -- \ “Teach a man to make fire, and he will be warm for a day. Set a | `\ man on fire, and he will be warm for the rest of his life.” | _o__) —John A. Hrastar | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip/warehouse feature idea: help needed
Nick Coghlan ncogh...@gmail.com writes: Yep, Guido's keynote was the genesis of the thread. I can't find it online, can you give a URL so we can see the talk? Past suggestions for social features have related to providing users with a standard way to reach maintainers and each other, and I'd prefer to leave maintainers in full control of that aspect of the maintainer experience. I'm not alone in feeling that way, hence why such features tend not to be viewed especially positively. Thanks for this detailed response differentiating this proposal from previous ones, it's exactly what I was asking for. -- \ “For mad scientists who keep brains in jars, here's a tip: why | `\ not add a slice of lemon to each jar, for freshness?” —Jack | _o__) Handey | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip/warehouse feature idea: help needed
Trishank Karthik Kuppusamy trish...@nyu.edu writes: Yeah, I think Guido said something to this effect in his keynote. Apparently I'm missing that context, then. The original post didn't help me understand why this proposal is significantly different from past “add a bunch of social to PyPI” rejected in the past. Can someone help by distinguishing this from past proposals of that kind? -- \ “I am as agnostic about God as I am about fairies and the | `\ Flying Spaghetti Monster.” —Richard Dawkins, 2006-10-13 | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip/warehouse feature idea: help needed
Nick Coghlan ncogh...@gmail.com writes: On 11 Apr 2015 12:22, Alexander Walters tritium-l...@sdamon.com wrote: Is the package index really the best place to put this? This is a very social-networking feature for the authoritative repository of just about all the third party module, and it feels like either it could corrupt the 'sanctity' of the repository (in the absolute worst case) If you're concerned that this feature might weaken the comforting illusion that PyPI published software is contributed and maintained by faceless automatons rather than living, breathing human beings, then yes, encouraging folks to think more about where the software they use is coming from would be a large part of the point of adding such a feature. I can't speak for Alexander, but I'm also −1 to have this *on PyPI*. I'm all for such features existing. What is at issue is whether PyPI is the place to put them. We have been gradually improving the function of PyPI as an authoritative *index* of packages; that's possible because it is a repository of uncontroversial facts, not opinions (i.e. “what is the packaging metadata of this distribution”, “where is its documentation”, “where is its VCS”, etc.). I am not saying the PSF shouldn't do this, but is pypi REALLY the best part of python.org to put it? I personally believe so, yes - sustaining software over the long term is expensive in people's time, but it's often something we take for granted. The specific example Guido brought up in his keynote was the challenge of communicating a project's openness to Python 3 porting assistance. The people doing the work of maintaining PyPI have said many times in recent years that there just isn't enough person-power to add a whole bunch of features that have been requested. Why would we think moderating a social-networking rating, opinion, discussion, or other non-factual database is something reasonable to ask of the PyPI maintainers? Conversely, if we are under the impression that adding ratings, feedback, reviews, discussion, and other features to PyPI is *not* going to be a massive increase in workload for the maintainers, I think that's a foolish delusion which will be quite costly to the reputation PyPI has recently gained through hard effort to clarify its role. By all means, set up a well-maintained social ecosystem around Python packages. But not on PyPI itself: The Python Package Index is feasible in part because it has a clear and simple job, though, and that's not it. -- \“If you can't hear me sometimes, it's because I'm in | `\ parentheses.” —Steven Wright | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] it's happened - wheels without sdists (flit)
Nick Coghlan ncogh...@gmail.com writes: I see it as more a matter of eventually migrating to a devdir - sdist - wheel - installed build deployment pipeline Yes. Increasingly often these days, that first distinction (dev working tree is not the same as source for distribution) gets ignored, and actively obliterated. Getting many GitHub or Bitbucket projects to distribute a source release, that doesn't have a mess of build artifacts in it, is an exercise in great frustration – largely because those platforms provide no convenient means to distribute tarballs that are *not* an undifferentiated bundling of everything in the repository. where the tools used at each stage are only required to support the transition to the next stage For the “dev tree → sdist” transition, part of the battle is going to be educating developers that such a distinction actually exists. The common bad practice of including build artifacts in VCS obliterates the separate “package the source for distribution” step, which means that step is rarely reproducible or reliable. -- \ “Simplicity and elegance are unpopular because they require | `\ hard work and discipline to achieve and education to be | _o__)appreciated.” —Edsger W. Dijkstra | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] setup_requires for dev environments
Robert Collins robe...@robertcollins.net writes: For context, the main use I have for setup_requires these days is projects with a version contained within the project, and for the use of pbr in openstack (and some other git hosted) projects. […] The openstack projects have a nice solution I think, which is that they write the egg metadata file and then read that back - both at runtime via pbr helpers and at build time when pbr takes over the build. I'm not using ‘pbr’, but yes, a big bundle of “create an egg-info file and read it back” custom Setuptools code was the clumsy solution I ended up with for this in ‘python-daemon’ 2.x. If that boilerplate could be removed, and “this is a dependency for build actions only” could just work for all users, I would be quite happy. -- \“The problem with television is that the people must sit and | `\keep their eyes glued on a screen: the average American family | _o__) hasn't time for it.” —_The New York Times_, 1939 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments
Nick Coghlan ncogh...@gmail.com writes: CPython uses the Reitveld instance integrated with bugs.python.org, and has the same problem as pip: incremental changes are a pain to publish, review, and merge, so we review and accept monolithic patches instead (cf the problem statement in https://www.python.org/dev/peps/pep-0462/) Fair enough. I don't know of a good code review tool for Mercurial. While the main UI is very busy, I've actually quite liked my own experience with Gerrit for http://gerrit.beaker-project.org/ My understanding is that Gerrit makes it tedious to review a sequence of revisions, in proportion to the number of revisions in the sequence. If I understand correctly, such a sequence must have separate reviews for every revision, and an aggregate of all the changes is not available to the reviewer. I'm impressed by GitLab's code review tool UI; see an example at URL:https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/344/diffs. The merge request page has tabs for the discussion, the commit log, and the overall diff – and you choose from inline diff or side-by-side diff. GitLab is free software, including all its tools; anyone can set up a GitLab instance and the project data can move from one instance to another without loss. For the purposes of the past thread where some proposed migrating to the proprietary lock-in site GitHub, those objections don't exist with GitLab: a project can migrate to a different host and keep all the valuable data it accumulated. A move to GitLab would be unobjectionable, in my view. That it has good code review features would help the issues in this thread too. If anyone knows of equivalent hosting for Mercurial with equivalent code review tools under free-software terms with no lock-in, that would be even better I think. -- \ “Don't be misled by the enormous flow of money into bad defacto | `\standards for unsophisticated buyers using poor adaptations of | _o__) incomplete ideas.” —Alan Kay | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments
Ben Finney ben+pyt...@benfinney.id.au writes: Donald Stufft don...@stufft.io writes: Sadly with how the code in pip is written, sometimes it’s just not reasonable to make small PRs because things are not well factored and changing things requires touching a lot of different areas. I've seen a number of other projects enforce “small revisions only, otherwise your change gets accepted”. If actually enforced, it is a highly successful way to get meaningful review of changes, and does not appear to limit the scope of the eventual change. That's “small changes only, otherwise your change gets rejected”, of course. The policy for Linux that I alluded to is in §3 of this document: 3) Separate your changes. - Separate each _logical change_ into a separate patch. […] The point to remember is that each patch should make an easily understood change that can be verified by reviewers. Each patch should be justifiable on its own merits. If one patch depends on another patch in order for a change to be complete, that is OK. Simply note this patch depends on patch X in your patch description. […] URL:https://www.kernel.org/doc/Documentation/SubmittingPatches I would be happy to see more projects adopt this, and enforce it, for change contributions. -- \ “Reality must take precedence over public relations, for nature | `\ cannot be fooled.” —Richard P. Feynman, _Rogers' Commission | _o__) Report into the Challenger Crash_, 1986-06 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Implementing large changes in small increments (was: Getting more momentum for pip)
Donald Stufft don...@stufft.io writes: Sadly with how the code in pip is written, sometimes it’s just not reasonable to make small PRs because things are not well factored and changing things requires touching a lot of different areas. I've seen a number of other projects enforce “small revisions only, otherwise your change gets accepted”. If actually enforced, it is a highly successful way to get meaningful review of changes, and does not appear to limit the scope of the eventual change. What does end up happening in such projects (e.g., Linux) is the community learns how to – and teaches newcomers how to – implement large changes as smaller refactorings, each of which results in a working system. I think the Pip developers should not fear the loss of large changes. Large changes can always be implemented as a series of small, understandable changes, if skill and design effort are brought to bear. The resulting large changes also end up being better examined and better designed. -- \ “Well, my brother says Hello. So, hooray for speech therapy.” | `\ —Emo Philips | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Upload signature (and signing key) after package upload
Howdy all, How can I upload an OpenPGP signature (and the signing key) for a version, after the upload of the distribution is complete? I have recently been informed of the ‘--sign’ and ‘--identity’ options to the ‘upload’ command. As described here: Signing a package is easy and it is done as part of the upload process to PyPI. […] URL:https://www.davidfischer.name/2012/05/signing-and-verifying-python-packages-with-pgp/ Can it be done, not “as part of the upload process”, but subsequent to the upload of the distribution? How? -- \ “Try adding “as long as you don't breach the terms of service – | `\ according to our sole judgement” to the end of any cloud | _o__) computing pitch.” —Simon Phipps, 2010-12-11 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Upload signature (and signing key) after package upload
Richard Jones rich...@python.org writes: Sorry, there's no facility at present for signing a file that's already uploaded. Thanks. I can now stop futilely trying to find it :-) -- \ “It's up to the masses to distribute [music] however they want | `\… The laws don't matter at that point. People sharing music in | _o__)their bedrooms is the new radio.” —Neil Young, 2008-05-06 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Documenting the Distutils sdist format
(Ping; I didn't see a response and would like to start this work soon.) Nick Coghlan ncogh...@gmail.com writes: For projects under the PyPA banner, including packaging.python.org, license in = license out is fine. That text is CC-BY-SA, so including Apache Licensed text shouldn't pose any problems. Okay. So with which repository from URL:https://bitbucket.org/pypa/ should I start? Where would the documentation go, where would the regression tests go? You appear to be distinguishing the license for executable code versus non-executable documentation. What license conventions should I be aware of? If these are answered in a “hacking” document somewhere, a response pointing me to that document would be fine. -- \ “Quidquid latine dictum sit, altum viditur.” (“Whatever is | `\ said in Latin, sounds profound.”) —anonymous | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] How to implement ‘setup.py’ functionality that itself needs third-party distributions
Ben Finney ben+pyt...@benfinney.id.au writes: How can I ‘import docutils’ in ‘setup.py’ without creating a circular dependency: * The correct way to get Docutils installed is to run ‘setup.py’ and have it install the dependencies. […] What I need is a way to express “ensure Docutils is installed before continuing with other Setuptools actions” in ‘setup.py’. I don't know of a neat way to tell Setuptools that. Thanks to George V. Reilly for a much neater solution: # setup.py from setuptools import (dist, setup, find_packages) # Declare a requirement for Docutils in a throw-away distribution. # Surprisingly, Setuptools will magically ensure this is installed, # or raise an exception if it can't. dist.Distribution(dict(setup_requires=docutils)) # By the time we get here, the following import will work. import docutils # … other code that needs Docutils … setup(…) Should this be in the Setuptools documentation? My time over the past several weeks would have been *much* less frustrating if this was the documented One Obvious Way™ to early-load a dependency in setup. -- \ “Free thought is a necessary, but not a sufficient, condition | `\ for democracy.” —Carl Sagan | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] How to implement ‘setup.py’ functionality that itself needs third-party distributions
Marius Gedminas mar...@gedmin.as writes: You shouldn't ever define an entry point that points to a package or module that won't be installed. This has now become clear. Thanks to everyone in this series of threads who has explained this. My preference of course was to avoid entry points, but it seemed a way out of the circular dependency. I now see that this is not a solution either. I'll attempt some suggestions: Thanks very much for making a concerted attempt to find a solution. 2. Implement metadata extraction using custom command classes[*] and setup_requires. This seems to be the only one which might be feasible (and without needless duplication of information, which is part of the whole point of this exercise). I will learn more and try it. -- \“I was in Las Vegas, at the roulette table, having a furious | `\ argument over what I considered to be an odd number.” —Steven | _o__) Wright | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] How to implement ‘setup.py’ functionality that itself needs third-party distributions
Daniel Holth dho...@gmail.com writes: Have you had a go with https://bitbucket.org/dholth/setup-requires ? Thanks for the pointer. My version of setup-requires replaces setup.py with a short script Why two separate files? Why not add the code in ‘setup.py’, prior to the ‘setup()’ call? that uses pip to install the setup requirements as specified in a special section in setup.cfg file This seems even more fragile than what I'm currently faced with, to be honest. I'm trying to get away from hard dependencies on specific ways of obtaining and installing packages and instead use declarative “I need this situation, make it so” that remain relevant without alteration as Distutils continues to evolve. Adding explicit parsing of the current Setuptools config format, and explicit calls to specific ‘pip’ command lines, seems to make the packaging more dependent on fast-changing APIs, not less. Perhaps I'm misunderstanding? -- \“If you go parachuting, and your parachute doesn't open, and | `\you friends are all watching you fall, I think a funny gag | _o__) would be to pretend you were swimming.” —Jack Handey | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] How to implement ‘setup.py’ functionality that itself needs third-party distributions
Carl Meyer c...@oddbird.net writes: On 01/20/2015 07:28 PM, Ben Finney wrote: What I need is a way to express “ensure Docutils is installed before continuing with other Setuptools actions” in ‘setup.py’. I don't know of a neat way to tell Setuptools that. That is the precise purpose of the `setup_requires` kwarg. That's too late though. What I need is to be able to instruct the build process to install Docutils so I can get stuff done *before ‘setup()’ is called*. Is there an API within Distutils or Setuptools that allows me to say “I haven't declared anything yet but go get and install distribution FOO right now”? I could do that before importing the module which needs Docutils. -- \ “The long-term solution to mountains of waste is not more | `\ landfill sites but fewer shopping centres.” —Clive Hamilton, | _o__)_Affluenza_, 2005 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] How to implement ‘setup.py’ functionality that itself needs third-party distributions
Donald Stufft don...@stufft.io writes: setuptools does not offer the ability to have only build time entry points, it assumes the installed project supplies those entry points. I'm now convinced that Setuptools entry points are not suitable to this purpose. spin it out into it’s own thing that can be installed on it’s own. Presumably this leads back to ‘setup_requires’ and ‘install_requires’? The right way to handle this is to either import it in setup.py and do it there I'd love to. How can I ‘import docutils’ in ‘setup.py’ without creating a circular dependency: * The correct way to get Docutils installed is to run ‘setup.py’ and have it install the dependencies. * Running ‘setup.py’ performs ‘import docutils’, and so will fail if Docutils is not already installed. What I need is a way to express “ensure Docutils is installed before continuing with other Setuptools actions” in ‘setup.py’. I don't know of a neat way to tell Setuptools that. -- \ “I'm beginning to think that life is just one long Yoko Ono | `\ album; no rhyme or reason, just a lot of incoherent shrieks and | _o__) then it's over.” —Ian Wolff | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] How to implement ‘setup.py’ functionality that itself needs third-party distributions (was: Module from install breaks subsequent install of different distribution)
Tres Seaver tsea...@palladion.com writes: setuptools itself is extensible by means of entry points. Both entry points in your example (as cited by Marius) demonstrate that: one adds support for a new keyword argument to 'setup()'[1], and the other defines a new writer for 'egg-info'[2]. By design, both of those are supposed to be loaded / available for any invocation of 'setup()' in a Python where the are installed (not merely for packages which mention them). What recourse do I have, then? I'm using entry points because it seems to be the only way I can declare functionality that resides in a module alongside the ‘setup.py’ which itself needs third-party packages. * During the distribution build stage (‘./setup.py build’ or earlier), I want to parse a reST document and derive some of the distribution metadata from that. * The ‘setup.py’ is the wrong place for this; it's complex and deserves its own module which can be imported for unit testing. * This also is totally unrelated to the functionality this distribution installs, so it doesn't belong in any of the packages to distribute for installation. * So I place it in a separate top-level module, ‘version’, only for use within ‘setup.py’. * That module itself needs a third-party distribution (‘docutils’) from PyPI. So I need that distribution to be installed *before* the ‘version’ module can be used. Apparently the ‘setup_requires’ option is the only way to do that. * Then, in order to declare how that functionality is to be used for commands like ‘./setup.py egg_info’, I have no other way than declaring a Setuptools entry point. * Declaring a Setuptools entry point makes Setuptools think the distribution-specific module must be imported by every other distribution in the same run. Of course it's not available there so those distributions fail to install. * It even thwarts the installation of ‘docutils’, the very distribution that is needed for ending this circular dependency. What am I missing? How can I implement complex functionality specific to packaging this distribution, without making an utter mess? -- \“The whole area of [treating source code as intellectual | `\property] is almost assuring a customer that you are not going | _o__) to do any innovation in the future.” —Gary Barnett | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Module from install breaks subsequent install of different distribution (was: Python module for use in ‘setup.py’ but not to install)
Ben Finney ben+pyt...@benfinney.id.au writes:
That has worked for me. But it appears to cause problems for some
others, related to this module which should not be installed.
This discussion was prompted, for me, by this bug report
URL:https://alioth.debian.org/tracker/index.php?func=detailaid=314948group_id=100328atid=413098.
Can someone please inspect the log output from Pip and help me
understand why an unrelated package is breaking when installing
‘python-daemon’ 2.0.3?
The ‘python-daemon’ 2.0.3 distribution declares ‘docutils’ as a
dependency with ‘setup_requires’ and ‘install_requires’. So it's
expected that asking Pip to install ‘python-daemon’ will attempt to
install ‘docutils’.
But why does the dependency install of ‘docutils’ then fail with::
=
Downloading/unpacking docutils (from python-daemon)
[…]
Downloading from URL
https://pypi.python.org/packages/source/d/docutils/docutils-0.12.tar.gz#md5=4622263b62c5c771c03502afa3157768
(from https://pypi.python.org/simple/docutils/)
Running setup.py (path:/home/pwj/.virtualenvs/venv/build/docutils/setup.py)
egg_info for package docutils
[…]
Installing collected packages: python-daemon, lockfile, docutils
Running setup.py install for docutils
Running command /home/pwj/.virtualenvs/venv/bin/python -c import
setuptools,
tokenize;__file__='/home/pwj/.virtualenvs/venv/build/docutils/setup.py';exec(compile(getattr(tokenize,
'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))
install --record /tmp/pip-7iBtrf-record/install-record.txt
--single-version-externally-managed --compile --install-headers
/home/pwj/.virtualenvs/venv/include/site/python2.7
[…]
Running docutils-0.12/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-XO8spm/docutils-0.12/egg-dist-tmp-0EsqLD
Traceback (most recent call last):
[…]
File
/home/pwj/.virtualenvs/venv/local/lib/python2.7/site-packages/pkg_resources.py,
line 2147, in load
['__name__'])
ImportError: No module named version
=
Why on earth would the presence of ‘version.py’ in this distribution
cause the install of ‘docutils’ to fail? Or is some other correlation
going on? I have had other reports that subsequent attempts to install
packages also fail with the same error.
--
\ “The fact that I have no remedy for all the sorrows of the |
`\ world is no reason for my accepting yours. It simply supports |
_o__) the strong probability that yours is a fake.” —Henry L. Mencken |
Ben Finney
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python module for use in ‘setup.py’ but not to install
Nick Coghlan ncogh...@gmail.com writes: I actually misunderstood your question. If you're just after the ability to say I want to include this file in the sdist, but not in the built wheel file or installed distribution (as I now believe you are) Correct, that's the goal here. then you're in the implementation defined world of the significantly underspecified sdist format. I believe setting that up actually *is* possible already, but have no idea what incantation you'll need to pass to setuptools to make it do it (and the docs are unlikely to be a reliable guide). My understanding, based on an answer received elsewhere [0], is that omitting the file from ‘setup.py’ but adding it to ‘MANIFEST.in’ causes it to be included in the sdist but omitted from install targets. [0] URL:http://stackoverflow.com/questions/27930124/python-module-use-during-setup-but-not-to-be-installed/27931088#27931088 That has worked for me. But it appears to cause problems for some others, related to this module which should not be installed. Unfortunately this kind of problem (trouble post-install from a PyPI package) is difficult to test. How can I test the behaviour of ‘pip’ in this regard, without thrashing many releases of a package upload to PyPI? -- \ “The internet's completely over.… Anyway, all these computers | `\and digital gadgets are no good. They just fill your head with | _o__) numbers and that can't be good for you.” —Prince, 2010-07-05 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Documenting the Distutils sdist format
Nick Coghlan ncogh...@gmail.com writes: For projects under the PyPA banner, including packaging.python.org, license in = license out is fine. That text is CC-BY-SA, so including Apache Licensed text shouldn't pose any problems. Okay. So with which repository from URL:https://bitbucket.org/pypa/ should I start? Where would the documentation go, where would the regression tests go? You appear to be distinguishing the license for executable code versus non-executable documentation. What license conventions should I be aware of? If these are answered in a “hacking” document somewhere, a response pointing me to that document would be fine. -- \ “Quidquid latine dictum sit, altum viditur.” (“Whatever is | `\ said in Latin, sounds profound.”) —anonymous | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Module from install breaks subsequent install of different distribution
Marius Gedminas mar...@gedmin.as writes: (It would be a bit easier if Alioth didn't require creating a guest account and logging in just to read a public bug report.) You're right, I wasn't aware of that. I have filed a feature request URL:https://alioth.debian.org/tracker/index.php?func=detailaid=314950group_id=1atid=350001 asking that Alioth not do this. We'll see whether it is implemented. -- \ “Skepticism is the highest duty and blind faith the one | `\ unpardonable sin.” —Thomas Henry Huxley, _Essays on | _o__) Controversial Questions_, 1889 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Documenting the Distutils sdist format (was: Python module for use in ‘setup.py’ but not to install)
Nick Coghlan ncogh...@gmail.com writes: If you'd like to volunteer for the task of reverse engineering and properly documenting how sdists work (with regression tests!), that would be quite awesome. Not necessarily *fun* from your point of view, but definitely awesome from my point of view :) I'll embark on that work and see how far I get. Thanks for identifying that as a desirable area to work on. The contribution will be licensed under Apache License version 2.0 equally to all recipients (i.e. not signing a PSF-specific agreement). Is there a way that can be accepted? -- \ “What is it that makes a complete stranger dive into an icy | `\ river to save a solid gold baby? Maybe we'll never know.” —Jack | _o__) Handey | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python module for use in ‘setup.py’ but not to install
Nick Coghlan ncogh...@gmail.com writes: If you have a build/install time only dependency that you want to distribute, you *have* to separate it out into a separate component if you don't want it to also be present at runtime. So, to be clear: if this module is needed during build-time for the distribution but does not have a stable API yet, you're saying it nevertheless needs to go to a repository for download as an independent distribution? That places a large burden on developing such components. Placing them in a repository for independent download strongly implies they have a reliable public API, which in this case is not yet true. The API for this component isn't stable, and so far isn't needed beyond the packaging for this one distribution. That's why I don't want to separate it out yet. We do not, and will not, support selective installation of subcomponents, as it's too hard to audit later. Instead, such components need to be separated out into distinct packages so that the segmentation of functionality and availability is clear to both the automated tools and to other humans. That's your authority as PyPA, of course. I hope it's clear why I'm not satisfied by the reasoning though. -- \ “Anyone who puts a small gloss on [a] fundamental technology, | `\ calls it proprietary, and then tries to keep others from | _o__) building on it, is a thief.” —Tim O'Reilly, 2000-01-25 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python module for use in ‘setup.py’ but not to install
Donald Stufft don...@stufft.io writes: I’m confused what this actually is. Please look at the example I cited, ‘python-daemon’ version 2.0.3. If it’s just a file you don’t want installed… then don’t specify it in your setup.py’s setup() function in either the py_modules or the packages keyword. That doesn't have the desired effect; it is installed (unwanted) to the site packages along with the packages I do want. -- \ “Faith is the determination to remain ignorant in the face of | `\ all evidence that you are ignorant.” —Shaun Mason | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python module for use in ‘setup.py’ but not to install
Donald Stufft don...@stufft.io writes: On Jan 17, 2015, at 7:47 PM, Ben Finney ben+pyt...@benfinney.id.au wrote: As it stands (‘python-daemon’ [0] version 2.0.3), the ‘version.py’ file is correctly included in the source distribution, correctly used by the ‘egg_info.writers’ entry point; but then ends up incorrectly installed to the run-time packages library. This causes problems for subsequent import of unrelated modules that happen to share the same name. How can I specify to Setuptools that the file is needed in the source distribution, is needed by the entry points for Setuptools, but should not be installed along with the binary distribution? setup_requires? Did you want to say anything else about that, for instance, how it applies to the question or what specifically you suggest I do? As it stands, I can only quote the documentation: setup_requires A string or list of strings specifying what other distributions need to be present in order for the setup script to run. […] URL:https://setuptools.pypa.io/en/latest/setuptools.html The module in question is part of the same code base, and is not an “other distribution”. So I don't know why you suggest this option. -- \ “Every valuable human being must be a radical and a rebel, for | `\ what he must aim at is to make things better than they are.” | _o__) —Niels Bohr | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Python module for use in ‘setup.py’ but not to install
Howdy all, How can I specify to Distutils (Setuptools) that module ‘foo’ needs to be available for use by ‘setup.py’, but should not be installed with the binary distribution? In the ‘python-daemon’ distribution, I have refactored a bunch of functionality to a separate top-level module (‘version’). That module is required to perform Setuptools actions – the ‘egg_info.writers’ entry point specifically – but is not needed at all by the resulting installation and should not be installed. So it's not clear how to specify this dependency. I have a ‘packages=find_packages(exclude=[test])’ specification; but that module isn't a package and so should not (?) be collected. I have the file included in ‘MANIFEST.in’; but that only specifies what to include in the source distribution, and should not add any files to the binary. As it stands (‘python-daemon’ [0] version 2.0.3), the ‘version.py’ file is correctly included in the source distribution, correctly used by the ‘egg_info.writers’ entry point; but then ends up incorrectly installed to the run-time packages library. This causes problems for subsequent import of unrelated modules that happen to share the same name. How can I specify to Setuptools that the file is needed in the source distribution, is needed by the entry points for Setuptools, but should not be installed along with the binary distribution? [0] URL:https://pypi.python.org/pypi/python-daemon/; version 2.0.3 at URL:https://pypi.python.org/pypi/python-daemon/2.0.3. -- \ “The entertainment industry calls DRM security software, | `\ because it makes them secure from their customers.” —Cory | _o__) Doctorow, 2014-02-05 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Distribution metadata: build timestamp of the distribution
Howdy all, Where can I find the build timestamp when a distribution was built by ‘distutils.setup’? The application I'm writing has a need to get a distribution and use the timestamp when that distribution was built:: import pkg_resources distribution_name = foo distribution = pkg_resources.get_distribution(distribution_name) distribution_timestamp = distribution.build_timestamp That doesn't work because the attribute doesn't exist. Where can I find the timestamp the specified distribution was built? -- \“The right to search for truth implies also a duty; one must | `\ not conceal any part of what one has recognized to be true.” | _o__) —Albert Einstein | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Package's declared latest version ignored by Warehouse
Howdy all, The Warehouse is ignoring the feature of PyPI which sets particular versions of a package visible or not visible. It makes all versions visible regardless. This is a problem when, for example, a package has been uploaded but should not be shown by default. An example is the ‘python-daemon’ package. At PyPI, the latest visible version is 1.5.5, as requested by the package maintainer. That is reflected as the default version shown at URL:https://pypi.python.org/pypi/python-daemon/. Also, when viewing other versions, the “Latest version” link appears, and correctly shows that the latest version is 1.5.5. But at Warehouse, the settings for which versions should be hidden are ignored by the application, and a different version is shown by default URL:https://warehouse.python.org/project/python-daemon/. That page should show version 1.5.5, as selected by the package maintainer in the PyPI database. -- \ “He who allows oppression, shares the crime.” —Erasmus Darwin, | `\ grandfather of Charles Darwin | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package's declared latest version ignored by Warehouse
Donald Stufft don...@stufft.io writes: Can you go into some detail about what the use case is for having something other than the latest version be the default version shown? Not much detail needed: The package version was released before it was realised that it breaks many people's systems on upgrade. To address this, without re-writing history, it is desirable that the default version be an earlier, known-working version. I didn't realise that package fetch-and-install tools fail to honour that setting. So now I don't have a good solution. -- \“All opinions are not equal. Some are a very great deal more | `\ robust, sophisticated, and well supported in logic and argument | _o__) than others.” —Douglas Adams | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package's declared latest version ignored by Warehouse
Donald Stufft don...@stufft.io writes: You can always remove a version from PyPI if it’s breaking things for people. I ond't consider that a good option, since I do want some testers to get it from the published location. You can also of course revert whatever changes people are being broken by and upload that as a new version. Obviously, yes. That leaves the undesirable version online until such a fix is available. But before a fixed version is available, a quick “this version is not for public consumption (now)” is what I looked for, and believed I had found with the “hide this version” feature. -- \ “Faith is the determination to remain ignorant in the face of | `\ all evidence that you are ignorant.” —Shaun Mason | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Call for information - What assumptions can I make about Unix users' access to Windows?
Paul Moore p.f.mo...@gmail.com writes: On 7 November 2014 16:52, Ben Finney ben+pyt...@benfinney.id.au wrote: If I was required to provide packages for MS Windows, the only viable solutions would be those that don't involve me obtaining an MS Windows instance myself. For that usage […] the license costs […] I didn't mention monetary costs at all. My understanding is that changing the cost doesn't in any way affect the terms of the license one is bound by. -- \ “I have never imputed to Nature a purpose or a goal, or | `\anything that could be understood as anthropomorphic.” —Albert | _o__)Einstein, unsent letter, 1955 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Call for information - What assumptions can I make about Unix users' access to Windows?
Steve Dower steve.do...@microsoft.com writes: Ben Finney wrote: The restrictions of the license terms make MS Windows an unacceptable risk on any machine I'm responsible for. Just out of interest, which restrictions would those be? It has been a long time since I bothered to read any of the numerous license texts from Microsoft, so I can't cite specific clauses. From memory, unacceptable restrictions include: * Restricting the instance to specific hardware, instead of leaving it up to the recipient to run the work they paid for on any hardware they choose. * Forbidding reverse-engineering of the OS to see how it behaves. * Forbidding collaboration with other recipients to discover how the OS behaves. * Refusal to disclose the source code for the running OS to the recipient. * Forbidding the recipient from getting their choice of vendor to make improvements to the OS and collaborate with other recipients on the improvements. * Arrogating control of the running OS to a party other than the license recipient, including the ability to (at Microsoft's sole discretion) deny applications to run, and to disable features of the OS. * Arrogating data collection to Microsoft and undisclosed third parties, tracking broad classes of activity on the OS and sending the logs to a server not of the recipient's choosing. Does this prevent you from creating a VM on a cloud provider on your own account? If I need to accept restrictions such as the above, I don't see that the location of the instance (nor the fees charged) has any affect on these concerns. The risks discussed above are not mitigated. If the licensing is a real issue, I'm in a position where I can have a positive impact on fixing it, so any info you can provide me (on- or off-list) about your concerns is valuable. Thank you for this offer, I am glad to see willingness expressed to solve these restrictions. I hope you can achieve software freedom for all recipients of Microsoft operating systems. Until then, the risk is too great to anyone to whom I have professional responsibilities, and my advice must continue to be that they avoid accepting such restrictions. -- \“Simplicity is prerequisite for reliability.” —Edsger W. | `\ Dijkstra | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] DRAFT PEP 396 - module version number
Barry Warsaw ba...@python.org writes: #. The ``__version__`` attribute's value SHOULD be a string. I may be fighting against the tide here; but this screams to me that the PEP should not be talking at all about “version number” (except to point out that they're strings, not numbers). Instead, the term should be “version string” throughout. Much damage is done to understanding version strings by calling them “version numbers”, with all the baggage about parsing and sequencing etc. that entails. We don't have to make the same mistake in this PEP. -- \ “He may look like an idiot and talk like an idiot but don't let | `\ that fool you. He really is an idiot.” —Groucho Marx | _o__) | Ben Finney pgpzMM8LaKN5W.pgp Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] DRAFT PEP 396 - module version number
Barry Warsaw ba...@python.org writes: #. When a module (or package) includes a version number, the version SHOULD be available in the ``__version__`` attribute. #. For modules which live inside a namespace package, the sub-package name SHOULD include the ``__version__`` attribute. The namespace module itself SHOULD NOT include its own ``__version__`` attribute. I still find a little ambiguity in “the ``__version__`` attribute”, but it's much improved, thanks. I think those paragraphs are good. -- \ “It is clear that thought is not free if the profession of | `\ certain opinions makes it impossible to earn a living.” | _o__) —Bertrand Russell, _Free Thought and Official Propaganda_, 1928 | Ben Finney pgpluWOIsEqsP.pgp Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] DRAFT PEP 396 - module version number
Fred Drake fdr...@acm.org writes: On Wed, Jun 8, 2011 at 6:14 PM, Barry Warsaw ba...@python.org wrote: #. For modules which live inside a namespace package, the sub-package name SHOULD include the ``__version__`` attribute. The namespace module itself SHOULD NOT include its own ``__version__`` attribute. I've no idea what you're saying here. What's a sub-package? Moreover, how can a “sub-package name” include an attribute? I agree that needs to be clarified. If you're referring to a package like zope.testing, I'd just call that a package; there's nothing special about that. I'd expect the __version__, if it exists, to be present in the file zope/testing/__init__.py. Yes, but how to specify that? The ‘__init__.py’ file is a module. What's that, then; a package module? Ben Finney wrote: I may be fighting against the tide here; but this screams to me that the PEP should not be talking at all about “version number” (except to point out that they're strings, not numbers). Instead, the term should be “version string” throughout. I'd rather we just say 'version' instead of 'version number' or 'version string'. Natural use of natural language is... natural. A separate sentence can state simply that versions are expressed as strings. −1. A version is a state of the code base at a point in its development; what we're talking about here are strings which act as identifiers for versions of the code. I would argue for “version identifier”, but that has virtually no actual use :-) -- \ “I was sleeping the other night, alone, thanks to the | `\ exterminator.” —Emo Philips | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] DRAFT PEP 396 - module version number
Barry Warsaw ba...@python.org writes: Hi Éric, On Jun 07, 2011, at 05:30 PM, Éric Araujo wrote: Just two things I thought about while perusing my archives yesterday. #. For modules which are also packages, the module namespace SHOULD include the ``__version__`` attribute. I’m still not sure “module namespace” will be clear to everyone. […] Agreed. Damn English and its ample opportunities for ambiguity. So, what *should* we call this thing? Note that in the PEP, I'm specifically talking about the namespace, not the file. Maybe module's namespace would be clearer? +1. I think the grammar could be even clearer: For modules which are also packages, the ``__version__`` attribute SHOULD be in the module's namespace. -- \ “The opposite of a correct statement is a false statement. But | `\ the opposite of a profound truth may well be another profound | _o__) truth.” —Niels Bohr | Ben Finney pgpRaz71oLCew.pgp Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] the usecase of continious integration based release-management (which could use more support on the ds2 side)
Greg Ewing greg.ew...@canterbury.ac.nz writes: In my projects I tend to keep the definitive version number in the Makefile, and have a make target that generates a version.py file from it. This is convenient because the Makefile often needs the version number for other things like creating release tarballs. Tagging vcs commits would be another use. I often do something similar, but IMO simpler: the definitive version string (note: version strings are rarely single numbers!) is kept as the sole content of a file at the top of the project tree, named ‘version’. That way, it's available equally to anything that can read text content from a file – the Makefile, any program code, even many configuration files. -- \ “I went to a general store. They wouldn't let me buy anything | `\ specifically.” —Steven Wright | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] stdeb gzipping some data_files
Lee Harr miss...@hotmail.com writes: stdeb is working quite well, but some of the files that I mention in the data_files part of the setup.py are being gzipped at some point in the process. Yes. The Debian packaging policy mandates some files are gzipped; see the policy document for more, otherwise leave the tool to do its work. Is there any way to prevent (hopefully from setup.py, setup.cfg, or stdeb.cfg) the alteration of those files? Consider it an opportunity to learn about the Debian packaging policy, if you're interested in why :-) -- \ “First they came for the verbs, and I said nothing, for verbing | `\weirds language. Then, they arrival for the nouns and I speech | _o__) nothing, for I no verbs.” —Peter Ellis | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] docs/ vs doc/
anatoly techtonik techto...@gmail.com writes: There are some arguments for 'doc/': 1. there is no plural for 'documentation' 2. bin/ is a common name for storing executables, although there is plural for 'binary' 3. we still use src/ instead of sources/ * ‘doc/’ is consistent with the standardised directory name on many operating systems (those that inspired, or are based on, the FHS) I don't think a vote is appropriate (I think the choice should be made on merit, not popularity), but if anyone's counting, I prefer ‘doc/’. -- \ “Progress might have been all right once, but it's gone on too | `\long.” —Ogden Nash | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Including tests in source package
Stanisław Pitucha virap...@gmail.com writes: If I do: setup( packages = find_packages(exclude=['tests']), test_suite = tests, ... this doesn't package tests anywhere. […] Essentially I want either tests.py, or tests/__init__.py to be distributed in sdist, but not in bdist. I am using effectively the same as you describe above, but I get the result you are seeking: the sdist contains the test suite, and the binary dist does not. Have a look at Gracie URL:http://pypi.python.org/pypi/gracie for an example that works for me. Perhaps it can help you find what's different that causes the problem you're seeing. -- \ “Few things are harder to put up with than the annoyance of a | `\ good example.” —Mark Twain, _Pudd'n'head Wilson_ | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] setuptools' Feature feature
P.J. Eby p...@telecommunity.com writes: That's because it's essentially deprecated - an experimental thing that turned out to be, well, a failed experiment. An experiment that produces a reliable result is a success. If you can reliably say that the feature is not worth it, that's a successful experiment :-) -- \ “In economics, hope and faith coexist with great scientific | `\ pretension and also a deep desire for respectability.” —John | _o__)Kenneth Galbraith, 1970-06-07 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Design rationale for the egg format ?
David Cournapeau courn...@gmail.com writes: I have a few questions about the egg format implementation, and was hoping people who designed it could answer them: […] I tried to find a rationale for those, but could not find much info on distutils-sig or setuptools doc, You might find it useful to read this archived thread: I've noticed that there seems to be a lot of confusion out there about what setuptools is and/or does, at least among Python-Dev folks, so I thought it might be a good idea to give an overview of its structure, so that people have a better idea of what is and isn't magic. […] URL:http://mail.python.org/pipermail/python-dev/2006-April/064145.html It goes into some amount of detail on the background and rationale for Setuptools and its early design decisions. -- \“Sane people have an appropriate perspective on the relative | `\ importance of foodstuffs and human beings. Crazy people can't | _o__) tell the difference.” —Paul Z. Myers, 2010-04-18 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip vs easy_install vs distutils2
Barry Warsaw ba...@python.org writes: On May 29, 2010, at 09:12 PM, Tarek Ziadé wrote: That's the point I am trying to express: it's implicit, embed, installers in project's setup.py vs an installer globally installed, knowing how to install projects that follows a given standard I feel pretty firmly that the choice of installer should /not/ be left up to the project's setup.py. That's within the domain of the user, or by proxy, their operating system. That is the ideal, yes. We need to get to the point where the packaging data stops being an executable program, and is instead purely declarative metadata with a well-defined format and semantic speification. -- \ “Software patents provide one more means of controlling access | `\ to information. They are the tool of choice for the internet | _o__) highwayman.” —Anthony Taylor | Ben Finney pgpG2HMbifVHI.pgp Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] install --dry-run broken?
Greg Ewing greg.ew...@canterbury.ac.nz writes: Jim Fulton wrote: The documentation says that when --dry-run is used: don't actually do anything. This is not the actual behavior. I'd be happy to see the option go away. I wouldn't! I find it useful for testing a setup.py that I'm about to distribute, to make sure it isn't going to fall over due to some stupid bug as soon as someone tries to use it. Are you saying that the feature does work as described? If not, what does it do that you find useful? -- \ “Our products just aren't engineered for security.” —Brian | `\ Valentine, senior vice-president of Microsoft Windows | _o__) development | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] include special filetypes and subdirectories
Assem Chelli assem...@gmail.com writes: how to change setup.py to include special filetypes and subdirectories recursively in python eggs The manifest file allows specifying globs and recursive includes URL:http://docs.python.org/distutils/sourcedist.html#specifying-the-files-to-distribute. -- \ “Reichel's Law: A body on vacation tends to remain on vacation | `\unless acted upon by an outside force.” —Carol Reichel | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distribute 0.6.10 and convert_2to3_doctests
P.J. Eby p...@telecommunity.com writes:
At 04:05 PM 1/29/2010 -0500, Barry Warsaw wrote:
The important thing is to have exactly one place to set the package's
version number.
Put it in setup.py, then. If you absolutely must have a __version__
at runtime, use this to extract it from the installation metadata:
__version__ = pkg_resources.require('MyProject')[0].version
Mostly, though, I don't bother with having a __version__ in my modules
or packages any more, since you can just do the above if you want to
check the installed version of something.
That assumes that the only things that will need to query the package's
version are Python modules. That's often not the case, especially when
there are other tools (e.g. shell programs, make files, or any program
not written in Python) that are part of the same package.
Better would be to have a *non-executable* data file containing the
version string and other such package meta-data. “Query the metadata”
should not necessarily imply “parse or execute a bunch of Python code”.
--
\“None can love freedom heartily, but good men; the rest love |
`\ not freedom, but license.” —John Milton |
_o__) |
Ben Finney
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] packaging terminology confusion
David Lyon david.l...@preisshare.net writes: If you have a zip/archive file, you can put anything in it. No reason why 'everything' can't go in it. A L'Oeuf incredible might include a Python 2.x and Python 3.x code set, make code for linux, .pyd for windows. It would be so un-confusing to have an egg like that. I disagree entirely with that last statement. However, you're now talking about changing the package format, and not the terminology of what we already have. So I'm dropping this sub-thread. -- \“Simplicity is prerequisite for reliability.” —Edsger W. | `\ Dijkstra | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Catalog-sig] packaging terminology confusion
Zooko Wilcox-O'Hearn zo...@zooko.com writes: On Saturday, 2010-01-09, at 11:47 , Brad Allen wrote: Maybe it's just wrong to call the __init__.py directories packages', because they are really just a piece of what is getting packaged. Bingo! A package is something that you deliver to someone else and they use it. You don't deliver __init__.py directories to people. __init__.py directories are not packages. +1. I think the above idea, of renaming Python's term for “collection of modules with a namespace”, is the most likely route to clearing up confusion over the term “package”. It is also the only option so far presented that IMO has any hope of actually succeeding, since it's addressing a Python-specific term. Expecting the rest of the software community to treat the term “package” specially in the context of Python is futile. The term is established, and Python's meaning conflicts with the established meaning. We need distinct terms for both of these concepts, since they're both involved in communications about distributing software. Until that change happens, we can expect the confusion to continue indefinitely. I hope no-one wants that. -- \ “If you can do no good, at least do no harm.” —_Slapstick_, | `\ Kurt Vonnegut | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Catalog-sig] packaging terminology confusion
Lennart Regebro rege...@gmail.com writes: If you say I need to install this package it may technically be unclear if you mean the package foo.bar or the distribution foo.bar-3.6.tgz, but that difference is not in that case significant. Installing the distribution and installing the package is in that case the same thing. That case is far from the only one to consider, though. As already pointed out, there are existing examples of ‘foo-3.6.tar.gz’ that contain zero collection-of-module packages, and there are other such examples that contain multiple collection-of-module packages. So it's not the same thing (and I'm not saying you didn't know that). It's needlessly confusing to expect people discussing these entities to distinguish the name depending on how the modules are collected together internally. Such a thing will *still* be called “a package” by programmers familiar with the wider programming world, regardless what name Python has chosen for them. I think it behooves us to choose terminology that acknowledges that, even if it means asking the Python community to change its terminology to be a little less parochial. -- \ “Science is a way of trying not to fool yourself. The first | `\ principle is that you must not fool yourself, and you are the | _o__) easiest person to fool.” —Richard P. Feynman, 1964 | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Catalog-sig] packaging terminology confusion
Lennart Regebro rege...@gmail.com writes: On Sun, Jan 10, 2010 at 10:01, Glyph Lefkowitz gl...@twistedmatrix.com wrote: This is precisely what I meant to recommend: Parcel would be a good replacement word for the Python-specific meaning of distribution. I'm sorry if I was ambiguous and created confusion. I think it sounds too much like a python-specific word, like egg. Agreed. At this point, people are just going to keep calling this entity a “package”, consistent with the majority of other languages and systems out there. We'd have more chance, I think, of changing Python's concept “namespace that behaves like a module and can have sub-modules” — currently called “package” — to a different word and encouraging adoption of that new term. Neither terminology change would be easy, but trying to change a raft of other concepts to Python-specific terms seems less likely to succeed than picking a new term for a concept that is *already* Python-specific. -- \ “For every complex problem, there is a solution that is simple, | `\ neat, and wrong.” —Henry L. Mencken | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Catalog-sig] packaging terminology confusion
Lennart Regebro rege...@gmail.com writes: On Sun, Jan 10, 2010 at 12:40, Ben Finney ben+pyt...@benfinney.id.au wrote: At this point, people are just going to keep calling this entity a “package”, consistent with the majority of other languages and systems out there. I'm not sure that's a problem. When you are talking about installation and distribution, the distribution and the package are almost identical. I can't understand this because of the ambiguity of the term “package”. You might be saying something tautologically true: When the terms “package” and “distribution” are used to refer to the same thing, their referents are identical. Well yes, of course. The other alternative seems to be that you're saying something we know to be false: it has already been demonstrated that an entity that Python terminology calls a “distribution” can contain zero, one, or multiple entities that Python calls “package”. We'd have more chance, I think, of changing Python's concept “namespace that behaves like a module and can have sub-modules” — currently called “package” — to a different word and encouraging adoption of that new term. I don't think it's realistic to change the name of one of the fundamental concepts of Python. It's an option to be considered, I think. Especially since the alternative proposal seems to be to change the name of one of the fundamental concepts already in established use *outside* Python. I sure as heck is not a question for distutils-list. :-) Probably not. -- \ “I'm not a bad guy! I work hard, and I love my kids. So why | `\ should I spend half my Sunday hearing about how I'm going to | _o__)Hell?” —Homer Simpson | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Catalog-sig] packaging terminology confusion
Tarek Ziadé ziade.ta...@gmail.com writes: On Sun, Jan 10, 2010 at 12:40 PM, Ben Finney ben+pyt...@benfinney.id.au wrote: At this point, people are just going to keep calling this entity a “package”, consistent with the majority of other languages and systems out there. Who is people. AFAIK, most Python developers use the word package for python package and when they use the word package for distribution, they will stop doing it if they are told that the proper word is distribution imho. That's exactly opposite to my experience. The established term for this concept is “package”, and that will remain true and continue to be reinforced by the general programming culture. Expecting programmers to use a Python-specific term, when there's already a wider-use term that fits exactly the same concept, is foolhardy in my opinion. -- \ “I cannot be angry at God, in whom I do not believe.” —Simone | `\ De Beauvoir | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
