[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires
Hello, I would vote for option 2, again in the spirit of explicit better than implicit (as maintainer of tox). pip should raise an error I think and fail the build. That being said, we don't want to start breaking existing packages. So what about doing a release of pip that raises a warning first, this gives a grace period of 1 month for people to fix their packages? And then switch over to breaking. Bernat On Fri, Jun 22, 2018 at 5:34 PM Pradyun Gedam wrote: > Hey everyone! > > In PEP 518, it is not clearly specified how a project that has a > pyproject.toml > file but has no build-system.requires should be treated (i.e. build-system > table). > > In pip 10, such a pyproject.toml file was allowed and built with > setuptools > and wheel, which has resulted in a lot of projects making releases that > assumed > that such a pyproject.toml file is valid and they use setuptools and wheel. > I understand that at least pytest, towncrier and Twisted might have done > so. > This happened since these projects have included configuration for some > tool in > pyproject.toml (some of which use only pyproject.toml for configuration -- > black, towncrier). > > There's a little bit of subtlety here, in pip 10's implementation: adding a > pyproject.toml file enables a new code path that does the build in > isolation > (in preparation for PEP 517; it's a good idea on it's own too) with only > the > build-system.requires packages available. When the build-system.requires > key > is missing, pip falls back to assuming it should be ["setuptools", > "wheel"]. > The in-development version of pip currently prints warnings when the key is > not specified -- along the lines of "build-system.requires is missing" + > "A future version of pip will reject pyproject.toml files that do not > comply > with PEP 518." and falls back to legacy behavior. > > Basically, pip 10 has a distinction between a missing pyproject.toml and > build-system.requires = ["setuptools", "wheel"] and the PEP doesn't. > However, > the PEP's precise wording here would help inform the debate about how pip > should behave in this edge case. > > I can think of at least 2 options for behavior when build-system.requires > is > missing: > > 1. Consider a missing build-system.requires equivalent to either a missing >pyproject.toml or build-system.requires = ["setuptools", "wheel"]. > > 2. Making the build-system table mandatory in pyproject.toml. > > I personally think (2) would be fine -- "Explicit is better than implicit." > > It'll be easy to detect and error out in this case, in a way that it's > possible > to provide meaningful information to the user about what to do here. > However, > this does mean that some existing releases of projects become > not-installable, > which is concerning; I do think the benefits outweigh the costs though. > > Thoughts on this? > > Cheers, > Pradyun > > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/7A4QHWCHR54TSIO2DQZUVNZHZS6ZPBLY/ > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/TNSKKGT3IJ5K6EFQWBGDI55VJARTIG7K/
[Distutils]PEP 518 - pyproject.toml with no build-system.requires
Fair enough :+1: Let's go ahead with this then. > For tools (such as pip) where 2a and 2b behave differently, yes it would. But > that's a pip implementation decision which can be made independently of the > resolution of this discussion (the PEP allows either behaviour). The details > of pip's behaviour have been discussed both on this thread and on the related > pip issue that's referenced in this thread. I don't think it's worth > rehashing that discussion again here. If you want to argue that pip should > switch to behaviour 2a, you should pick up the discussion on the pip issue > tracker. But be warned, I'll argue against you over there :-) (Not least > because the legacy behaviour is available using --no-build-isolation, which > is *precisely* for the case where you want to manage the build tools manually > the way you are doing above). -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/46N2F7WWSGMIQ4JHYKLW424NRQ3QPPGK/
[Distutils]PEP 518 - pyproject.toml with no build-system.requires
Sorry to bump into this so late, but: Would not 2a be more backward compatible than 2b? I mean people may have build environments/installs doing: - pip install setuptools-scm setuptools pbr - pip install os-system-level This way you can bypass the setup requires of setuptools to use easy_install. With 2b there is no way to avoid setup requires elements to not trigger easy install; other than os-system-level (which requires pbr for setup) to switch to pyproject.toml. > OK, thanks for confirming that. I think everyone who made comments on my > proposal was ultimately saying something along the lines of "but I'm not > sufficiently bothered to make a fight over it". So in the interests of > bringing this discussion to a conclusion and moving on, I'm going to say that > we go with what I proposed: 1. It should be legal for pyproject.toml to *not* > contain a [build-system] section. 2. If a [build-system] section is present, > the requires key must be present. Tools should behave as follows: 1. If > [build-system] is present but requires is missing, they should raise an > error. 2. If [build-system] is missing, they should take one of the following > two approaches: a) Act as if pyproject.toml is missing altogether b) Act as > if [build-system] is present, with a requires value of ["setuptools", > "wheel"] Whether there is any practical difference between cases 2a and 2b is > tool-dependent. (It's not ideal that we don't make a recommendation between > 2a and 2b, but getting i nto the details of pip's isolation strategy hasn't really helped make the choice more obvious, and I doubt that any new tools without pip's backward compatibility requirements would behave differently in the two cases anyway). Brett - you offered to update the PEP, so I'll leave that step to you. Thanks to everyone for the helpful discussions and willingness to compromise on the final outcome :-) Paul -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/XLQILTH4E3JF52EZWJYVO63CLQEE5PVZ/ -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/PTNKGMJ6VGMTUXUWDQKI56FGZJCBDW3O/
[Distutils] Re: Adopting virtualenv package maintenance
I'm maintainer of tox and tox is heavily relying on virtualenv. I would be up for being a co maintainer, but for now I don't have availability for heavy undertakings. I've had a discussion with Donald at Pycon us about it and there certainly is a desire and place for virtualenv. the issue is mostly qualified people willing to invest time into it. On Thu, Sep 6, 2018, 21:44 Alex Becker wrote: > Another +1 to the utility of a maintainer. I am also working on package > management and have found that venv is not a full replacement for > virtualenv--for example I don't believe the environment can be entered > programatically, while virtualenv provides activate_this.py which can be > exec'd. I'm sure there are many other limitations, so I don't think python > can give up on virtualenv soon. > > On Thu, Sep 6, 2018 at 12:48 AM Tzu-ping Chung wrote: > >> I don’t have the authority to do it, but I would really love to see >> virtualenv getting maintenance. >> >> Pipenv still depends on virtualenv because for not only Python 2. I am >> already working on switching >> to venv for Python 3.5+, but we will continue to need virtualenv for >> quite a while due to some >> compatibility issues, even on Python 3. We occasionally get bug reports >> about virtual environment >> creation, and those generally are due to some long standing bugs in >> virtualenv. It would be awesome >> if we could point reporters somewhere to actually get things fixed, >> instead of say sorry with a shrug. >> >> I am not suggesting in particular who should be the maintainer(s) (but >> also not objecting to Sorin’s >> proposal), but virtualenv really needs a maintainer. >> >> TP >> >> >> > On 05/9, 2018, at 16:56, Sorin Sbarnea wrote: >> > >> > As it seems that virtualenv package is in need of some maintenance >> effort, focused mostly on doing reviews, closing or merging them and >> eventually doing a new release once a month. >> > >> > I know that virtualenv is in deprecation mode as its would be no longer >> needed when Python2 will no longer be used. The reality is that Python 2.x >> will still be in production after January 1st, 2020 because there are >> deployed products with LTS contracts which will need some time to get >> updated to newer versions that use py3. This automatically translates to >> the need to have a working virtualenv for testing them. I am part of the >> OpenStack team and I am sure that, even if I like it or not, I would have >> to deal with some amount of py2 even after the magic date. >> > >> > The current situation with virtualenv is pretty bad because there are >> lots of open pull-requests which are not reviewed or merged, mostly because >> there is nobody available to do that boring extra work. I had few changes >> that were improving the CI testing of virtualenv which soon will be one >> year old,... most of them without any feedback. Even finding whom to ping >> by email or irc was a challenge as I got two responses: no response at all >> or someone else telling me that they are not maintainers of the virtualenv >> package. Example >> https://groups.google.com/forum/#!topic/pypa-dev/YMVsRbNoVpg >> > >> > For these reasons I would like to become a maintainer for virtualenv, >> preferably working with two others on keeping it alive for a couple of >> years till we could organize a big wake ceremony for it. >> > >> > It would be preferable if two others would join the maintenance >> "taskforce" because merging a change should almost always involve at least >> two reviewers. >> > >> > While I cannot make any guarantees regarding dealing with all reported >> bugs, I can commit on assuring that there are no PRs that are not reviewed >> for longer than 30 days (aiming for one week). Now there are ~75 open PRs. >> I have being doing open source for a long time and I respect all the time >> and efforth put by project maintainers and at the same time I always tried >> to do my best dealign with incoming PRs because if someone spended his time >> trying to make a contribution that is passing CI, they probably deserve at >> least a review. >> > >> > https://github.com/pypa/virtualenv/pulls >> > >> > Thanks >> > Sorin Sbarnea >> > @ssbarnea on irc/github/... >> > -- >> > Distutils-SIG mailing list -- distutils-sig@python.org >> > To unsubscribe send an email to distutils-sig-le...@python.org >> > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ >> > Message archived at >> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/EOXOL3742HVDLAIQDODL36UNRGU4R6SG/ >> -- >> Distutils-SIG mailing list -- distutils-sig@python.org >> To unsubscribe send an email to distutils-sig-le...@python.org >> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ >> Message archived at >> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/TFUYTDYSOLT3KJ2SRPM7Y7K6GNDK5TMW/ >> > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an
[Distutils] Adopting virtualenv package maintenance
So as a conclusion to this, I think we can say everyone agrees virtualenv should have some maintenance for a next few years at least. Sorin and I offered to help with this, can we get some PyPa people help with this (e.g. give rights on repo) ? Thanks! > It should probably be noted that the "activate_this" script (and indeed, all > of the activation scripts, etc) are not anything special. Users can > relatively easily write their own, or a customised version. Sure, providing > the scripts is a convenience, and they are somewhat tricky to get right, but > the fact that venv doesn't provide activate_this isn't a showstopper. -- > Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an > email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message > archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/OXJPWRHZJG4QAZNDDEFSLOB3KLYAAFE5/ -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/GUFJOP23D7FG3ARYYW5DQAYL5BRHYJDC/
[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires
In the pep it's stated tools can use the tool section https://www.python.org/dev/peps/pep-0518/#id28 and at no point says build tools only. So I don't think at all strange that towncrier uses it. It follows the words of the pep quite rigourously. On Thu, Jun 28, 2018, 18:40 Thomas Kluyver wrote: > On Thu, Jun 28, 2018, at 6:27 PM, Brett Cannon wrote: > > The file was originally meant to target building wheels for libraries. It > just happens that folks have pushed that out to include apps as well. So if > the purpose of the file expands to include apps as well then that changes > what the PEP should require to be in the file. > > > It even expanded to include tools not directly related to packaging at all > - towncrier was given as an example, and that's a developer tool for > producing release notes. I guess we implicitly invited this generalisation > by calling it 'pyproject.toml' rather than anything explicitly tied to > building or packaging. > > In any case, I think it's too late to say such tools shouldn't use it, and > as Nathaniel pointed out, it's pretty surprising that using a tool like > towncrier would suddenly change how pip deals with your package. So I think > the best option is to say that the presence or absence of pyproject.toml is > not significant for build frontends - rather it's the presence of > build-system information in that file that matters. > > Thomas > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/CB65HN6QPPFAVTNGPWYMRKOEA52JQXXQ/ > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/KTDEESRGM5BQVOZ36URRKYCTGT7KF26X/
[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires
I don't consider building from source a niche use case per se, given that pip is most widely adopted PEP-517/518 build frontend. What other build frontends there are or are to be recommended for usage (with setuptools/flint/poetry being mostly backends)? On Thu, Jun 28, 2018 at 8:35 AM Paul Moore wrote: > On 28 June 2018 at 02:37, Nathaniel Smith wrote: > > On Wed, Jun 27, 2018 at 2:25 PM, Paul Moore wrote: > >> On 27 June 2018 at 22:09, Pradyun Gedam wrote: > >>> > >>> On Wed, Jun 27, 2018 at 9:15 PM Paul Moore > wrote: > > >> But Brett clearly stated that he views PEP 518 as stating that the > >> build-system.requires key in pyproject.toml is *not* optional. And I > >> think that's the correct reading of the PEP. > > > > FWIW, I'm a co-author on the PEP, and I've apparently been shipping > > software without this key for most of a year: > > https://github.com/python-trio/trio/blob/master/pyproject.toml > > Clearly I should have read my PEP more closely... > > Oh! I'm really sorry - I thought you'd seen the discussion and the > fact that you hadn't responded to Brett's assertion directly meant you > were OK with it. I think I must have been skimming at the time and not > fully understood the point you were making in your post. My error, and > thanks for making me aware of it. > > >> OK. But I'm going to take the view that having explicitly requested > >> clarification on distutils-sig, if we want to do anything other than > >> reject a pyproject.toml with a missing build-system.requires as > >> invalid, we need to first of all get the PEP changed. And for that > >> we'll have to have a good use case to justify the change. Personally, > >> I don't see the value. > > > > If pip starts erroring out when pyproject.toml is missing > > build-system.requires, then 'pip install twisted' will stop working > > and everyone will start screaming on twitter: > > https://github.com/twisted/twisted/blob/trunk/pyproject.toml > > > > (Trio won't actually be affected b/c it has a universal wheel, so pip > > always uses that instead of the sdist. But twisted only has wheels for > > python 2 on Windows; all other configurations build from source, and > > thus will break.) > > I can see the practicality argument here. I don't think (correct me if > I'm wrong!) that anyone particularly anticipated when we were drafting > PEP 518 that the tools section would be used by anything other than > build tools - I know I hadn't anticipated that usage. In hindsight > it's an obvious thing to do, but it does mean that pyproject.toml is > more of a "general project configuration" file than purely a build > configuration one. > > > We probably need to update the PEPs some here anyway, if only for > clarity. > > Agreed > > > Also, right now PEP 517 says "If the pyproject.toml file is absent, or > > the build-backend key is missing, the source tree is not using this > > specification, and tools should fall back to running setup.py." But > > thinking about it again, we probably don't want to do this, because it > > adds Yet Another build configuration that pip has to handle: > > > > 1. no pyproject.toml -> legacy non-isolated build via legacy setup.py > support > > 2. pyproject.toml without build-backend specified -> new isolated > > build via legacy setup.py support > > 3. pyproject.toml with build-backend specified -> new isolated build > > via new build-backend support > > > > Once pip and setuptools both have PEP 517 build backend support (which > > should hopefully happen soon?), then option (2) will become weird and > > unnecessary. It would be nice to get rid of it. So I think we'll be > > having a similar discussion in a few months about how to handle > > pyproject.toml without build-backend keys. At that point we'll have a > > few options: > > > > - We could say that build-system.build-backend is mandatory, and error > > out if it's missing. But that will break the world (probably even more > > so than making build-system.requires mandatory now will break the > > world, since by the time this happens there will be more > > pyproject.toml files in the wild). > > > > - We could shrug and say that updating PEPs is a lot of work so let's > > just go with what we wrote way back when. But that forces pip to > > forever carry around code to implement both option (2) and (3), which > > do essentially the same thing except with extra code and probably > > there will be weird bugs that show up in one configuration or the > > other, etc. > > > > - We could say that when build-system.build-backend is missing, it > > defaults to "setuptools". But it's *really weird* to have a default > > value for build-system.build-backend while not having one for > > build-system.requires, because, effectively, the interpretation of > > build-system.build-backend depends on the value of > > build-system.requires. The build backend is an object that gets looked > > up inside those required packages. If we're going to be like "we have > > no idea what
[Distutils] setuptools configuration in pyproject.toml
I'm aware this might be a controversial subject, so let's have the initial discussion about it here first for full disclosure and see what people think about it. Should setuptools support pyproject.toml as configuration source or not (alternative to setup.cfg which it already does - https://setuptools.readthedocs.io/en/latest/setuptools.html#configuring-setup-using-setup-cfg-files )? The main benefit of having this would be to decrease configuration files and have build dependencies and other types of dependencies in one location. Furthermore many other packaging projects (flit, poetry) already do define their dependencies inside pyproject.toml; so would create one unified location where to look for such in the future. The counter-argument is that "a big part of pyproject.toml was keeping that file clean" and would furthermore increase the size of that file down the line. So what do people think? Should we encourage or discourage to have a single python project file? I'm personally supporting build/code quality tools supporting pyproject.toml as their main configuration file. Thanks -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/C3JEBOCQEILLPXK4FDQPADCFO6WWW6JT/
[Distutils] Re: New virtualenv maintainer
Hi all, Let me first say thank you for the opportunity, I consider it an honour to take up this challenge. I'm excited to make virtual environments as simple and as seamless to use as possible (as tox maintainer we get exposed to some of the virtualenv bugs, will try to make that as few as possible in the long term). That being said to quote Donald, *virtualenv code base is gnarly. *As such, will spend the next month on trying to get a good grasp on virtualenv/venv code base plus improving the CI of virtualenv. Hopefully, by the PyPi sprint on end of October, I can have a good grasp on what needs to be done next and what not from the all the PRs/issues open at the moment. Bare with me while I get up to speed. All the best, Bernat On Thu, Sep 20, 2018 at 2:31 PM Paul Moore wrote: > On Thu, 20 Sep 2018 at 14:25, Donald Stufft wrote: > > > > We don’t typically bother to make big announcements on distutils-sig for > maintainers of projects, however I’m making an exception in this case > because of the recent threads on virtualenv maintenance. The current core > team of virtualenv agrees that it is an important package, and deserves > additional maintenance, but have somewhat stepped back from it in recent > times for a variety of reasons (many ephemeral in nature). > > > > We’ve been happy to have a number of people step forward to offer help > in maintaining the project, and after some thought and discussion, we’ve > decided to extend an invitation of a commit bit to Bernat Gabor, and I’m > happy to say that he has graciously accepted, and is now part of the team. > For those unaware, Bernat is currently a developer for the tox project, > which is one of the major consumers of virtualenv which made him a great > fit for maintainership since it gives another real world perspective > (consuming virtualenv as an API, instead of as a tool) that the current > team currently lacked. > > > > So Welcome Bernat! > > Welcome Bernat :-) > > Paul > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/CHEWKWT523PO3XEZRWH26U7XSH2NE5QE/ > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/IUCW4KDF3EOOEIAJO642LWXCWGGKUU5E/
[Distutils] Re: setuptools configuration in pyproject.toml
Yes, furthermore PEP-517 as sdist is implemented in tox and under
development for pip.
On Mon, Sep 24, 2018, 20:26 Pradyun Gedam wrote:
>
>
> On Mon, 24 Sep 2018, 21:46 Daniel Holth, wrote:
>
>> You could probably implement this outside of setuptools as an extension.
>> Clients would add a load-toml line to setup.py. Do build requirements work
>> yet?
>>
>
> Yep. PEP 518 is supported by the latest version of pip (18.0).
>
>
>> One obstacle might be reconciling the all-strings nature of .cfg with
>> typed toml.
>>
>> On Mon, Sep 24, 2018, 12:06 RonnyPfannschmidt <
>> opensou...@ronnypfannschmidt.de> wrote:
>>
>>> I believe contributions in the directions would be welcome,
>>> for now having something like
>>> tool.setuptools.{metadata,options,command.*} might be interesting to grow
>>> and experiment with
>>>
>>> but whats really missing is a setuptools conributor with more than just
>>> thinly stretched time.
>>>
>>> -- Ronny
>>> Am 24.09.18 um 17:30 schrieb Bernat Gabor:
>>>
>>> I'm aware this might be a controversial subject, so let's have the
>>> initial discussion about it here first for full disclosure and see what
>>> people think about it. Should setuptools support pyproject.toml as
>>> configuration source or not (alternative to setup.cfg which it already does
>>> -
>>> https://setuptools.readthedocs.io/en/latest/setuptools.html#configuring-setup-using-setup-cfg-files)?
>>>
>>>
>>> The main benefit of having this would be to decrease configuration files
>>> and have build dependencies and other types of dependencies in one
>>> location. Furthermore many other packaging projects (flit, poetry) already
>>> do define their dependencies inside pyproject.toml; so would create one
>>> unified location where to look for such in the future.
>>>
>>> The counter-argument is that "a big part of pyproject.toml was keeping
>>> that file clean" and would furthermore increase the size of that file down
>>> the line.
>>>
>>> So what do people think? Should we encourage or discourage to have a
>>> single python project file?
>>>
>>> I'm personally supporting build/code quality tools supporting
>>> pyproject.toml as their main configuration file.
>>>
>>> Thanks
>>>
>>> --
>>> Distutils-SIG mailing list -- distutils-sig@python.org
>>> To unsubscribe send an email to
>>> distutils-sig-leave@python.orghttps://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
>>> Message archived at
>>> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/C3JEBOCQEILLPXK4FDQPADCFO6WWW6JT/
>>>
>>> --
>>> Distutils-SIG mailing list -- distutils-sig@python.org
>>> To unsubscribe send an email to distutils-sig-le...@python.org
>>> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
>>> Message archived at
>>> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/VBC35YTBWZYJCFPTRQRLSFBTV5FZV4OR/
>>>
>> --
>> Distutils-SIG mailing list -- distutils-sig@python.org
>> To unsubscribe send an email to distutils-sig-le...@python.org
>> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
>> Message archived at
>> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/KV72IYDNA2M3WXHKWEV5RPX4UJ4OACA7/
>>
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/XI6TKYZGIVZ32BJHKUXJNPVUH66M5PD7/
[Distutils] Re: PEP 517 - source dir and sys.path
It feels to me that importing from setup.py level is a code smell and
should be avoided (I mean what happens if you move the source into src
layout, do you now want tools to also append the src folder additionally if
exists?). Can't people just inject at the start of setup.py into sys path
the cwd if they really want to go with it? (feels superior over pinning pip
to 18.0).
On Sun, Jan 27, 2019 at 7:48 PM Paul Moore wrote:
> On Sun, 27 Jan 2019 at 14:39, Thomas Kluyver wrote:
> > I think the rule about the CWD not being on sys.path only applies to
> loading a proper PEP 517 build backend when that's specified in
> pyproject.toml.
>
> I'm not entirely sure what you're intending when you refer to a
> "proper PEP 517 build backend". The setuptools backend is a perfectly
> acceptable one, the only difference here is that we default to it in
> pip when the project hasn't specified a backend. The reason we do that
> is simple - if we don't, we're unlikely to be able to remove the
> legacy setuptools code any time in the foreseeable future, as there
> will be essentially no incentive for existing projects to specify a
> setuptools backend. Also, we'd get basically no testing of the new PEP
> 517 code, for the same reason. (Yes, it would have been better to test
> before release, but we've never had any success getting beta testers
> for pip). We could have made PEP 517 entirely opt-in, but (a) that
> would have drastically slowed down adoption, and (b) implicit in
> everything we'd discussed was the assumption that there would be a
> setuptools backend that was semantically equivalent to running
> setup.py the way we currently did, so there was no need to. Clearly
> that assumption turned out to be wrong, and that's where the issues
> for our users arose.
>
> Having said this, I don't have any problem with changing the default
> backend used by pip. In fact, if the setuptools project don't view it
> as a goal of their existing backend to replicate setup.py behaviour,
> then I think it's pretty much essential that we change. Pip needs to
> default to a PEP 517 backend that behaves the same as pip's legacy
> code - that's essential for backward compatibility if we're to remove
> or deprecate the legacy setup.py code path. The only problem is that
> right now there doesn't appear to be a backend that suits our
> requirements.
>
> Paul Ganssle suggested that setuptools could provide an alternative
> ("legacy") setuptools backend that preserved full setup.py
> compatibility (including having the current directory on sys.path).
> That seems to me to be a reasonable solution, and I appreciate the
> offer. In practice, the legacy setuptools backend doesn't *have* to be
> implemented by setuptools itself - it could be a separate project if
> setuptools don't want to maintain it themselves. But IMO it would be
> better if the setuptools team are OK with maintaining it.(At a pinch,
> pip could even implement that backend internally, but adding pip to
> every build environment seems a bit heavyweight).
>
> So in terms of practical resolution of the problems people are hitting
> since pip 19.0 was released, I don't think there's a significant issue
> here. We need a new pip release, sure, and someone needs to write the
> new backend, but I don't think either of those tasks are huge.
>
> What is more difficult is the question of whether the PEP should
> change. As Chris pointed out, the previous discussion ended up saying
> that the build directory should not be on sys.path, but acknowledged
> that mandating that might cause issues. So the question now is, are
> the issues we've seen big enough that we want to change PEP 517 to say
> that the build directory *should* be on sys.path? We don't have to
> answer that question urgently (the urgent resolution is with a legacy
> backend, as noted above) but we do have to decide one way or another.
> And here the question is whether enforcing projects to adopt the "good
> practice" of not relying on being able to import the project at build
> time is worth the negative impact of requiring what appears to be a
> non-trivial number of projects to change their build code. Or whether
> there's an "intermediate" option (the discussions Chris linked to
> included talk of adding another pyproject.toml value which would be a
> list of directories the frontend should add to sys.path when calling
> hooks, for example). But that discussion shouldn't be made under a
> false impression of urgency - it's not on the critical path for fixing
> pip's immediate issue.
>
> Paul
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/distutils-sig@python.org/message/SG2VB3RFGHTUHYB2CMTS2WN3RUDTS3KB/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to
[Distutils] Re: Update PEP 508 to allow version specifiers
I'm not sure I agree that setting up a private repository is that hard. Devpi does an excellent job for open source stuff, or otherwise if you use Artifactory inside your company that also has similar plugin. On Tue, Jan 29, 2019 at 1:30 PM Jan Musílek wrote: > Nick Coghlan wrote: > > So URL specifiers replaced the part of dependency links that we > > actually wanted to keep: letting projects *temporarily* depend on VCS > > repos and other URLs while waiting for a release containing the > > feature that they needed, while focusing on abstract dependencies > > outside those cases (and deliberately eliminating the ability to add > > arbitrary new repositories to the dependency resolution process). > > Thank you for the explanation! This makes a lot of sense for public > packages. > > However, when you work with private packages, it's not as straightforward. > You can setup your own warehouse, but that's a lot of overhead for small > personal projects. What would you recommend to use in this case? > > Cheers, > Jan > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/archives/list/distutils-sig@python.org/message/PRLJLJVQ3AOAYU4I3HZ6SWB6HKNCQF3P/ > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/JJFCEXT7GO5XU2GMIJISMMQJEGIM4JCF/
[Distutils] Re: WIll custom setup.py with (set library-prefix have post install actions) continue to be supported ?
It will exist only as a legacy thing, and as setuptools build system configuration. PEP-517 and PEP-518 defines what pip will support going forward. On Sun, May 26, 2019, 15:25 Stuart Axon via Distutils-SIG < distutils-sig@python.org> wrote: > I'm making a python package for something python Gedit/Pluma/Xed plugin. > > Like other lots of other packages that use Gtk, I use glib-compile-schemas > [1] at install time. > > I do this by using a custom setup.py > - similar to this file this from flux: > https://github.com/xflux-gui/fluxgui/blob/master/setup.py#L81 > (other app, like Meld to something similar) > > > It looks like there are a lot of changes happening to setuptools, will > this continue to work or will it break one day ? > > > This isn't the only file I'd like to generate at install time, but might > be the hardest to work around if I can't. > > > The other custom action I'm going looking at is setting `--install-lib` to > the editors plugin directory (this varies depending on: Operating system > and is the user root ?) > > > I'm using use pip for this, as the install story for most python plugins > for these editors is nonexistant - people have to copy the files to some > place manually, and instructions usually miss out Windows and OSX. > > > [1] glib-compile-schemas takes an XML file and generates a binary GVariant > file. > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/archives/list/distutils-sig@python.org/message/EWR4XLHRVMXH5ZO776UNL5S4POIXJMUT/ > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/2V3M3UR4WYIN4WIGU7SBYV2YNSBMCVG7/
[Distutils] Re: Seeking a new maintainer for packaging.python.org and Twine.
I'll be okay in chiming in on twine All the best Thea! On Fri, Jul 19, 2019, 23:38 Dustin Ingram wrote: > Sorry to see you go, Thea! You've done a great job. > > I'd be happy to help maintain both these projects. > > On Fri, Jul 19, 2019 at 5:33 PM Thea Flowers via Distutils-SIG < > distutils-sig@python.org> wrote: > >> Hi friends! >> I'm stepping away from several things in the Python community. I've >> served as the maintainer of packaging.python.org and Twine for a while, >> but it's time for me to move on. >> >> I'm looking for someone to take on primary responsibility for >> packaging.python.org and another person to help share the load for Twine. >> >> If you or someone you know is interested, please reply to this thread and >> let me know. >> >> -Thea Flowers >> magicalg...@google.com >> -- >> Distutils-SIG mailing list -- distutils-sig@python.org >> To unsubscribe send an email to distutils-sig-le...@python.org >> https://mail.python.org/mailman3/lists/distutils-sig.python.org/ >> Message archived at >> https://mail.python.org/archives/list/distutils-sig@python.org/message/M7VRNT5KP4YQ6UPVI4MN4IIWM2Z3IXCH/ >> > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/archives/list/distutils-sig@python.org/message/GNEQW6JB44ED575NWD47SAFILOEFJY7A/ > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/IMC7ZFBHMNGKCTGLTLFYZ4GQUKNGBCDO/
[Distutils] Virtualenv 20.0.0 beta1 is available
More information available at https://discuss.python.org/t/virtualenv-20-0-0-beta1-is-available/3077 -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/KLUZUTWO3QQL2LABKZQB5K3MZMWHTD4E/
[Distutils] Re: replacement for tests_require?
With the caveat that one should not use setuptools to satisfy those dependencies, but instead use some kind of test runner such as tox/nox (which then delegate to pip to acquire packages). On Mon, Mar 9, 2020 at 11:10 AM Xavier Fernandez wrote: > Hello, > > you might want to use the "test" extra: cf > https://packaging.python.org/specifications/core-metadata/#provides-extra-multiple-use > : > "Two feature names test and doc are reserved to mark dependencies that > are needed for running automated tests and generating documentation, > respectively" > > -- > Xavier > > On Mon, 9 Mar 2020 at 12:02, Felix Schwarz > wrote: > >> Hi, >> >> what is the recommended to solution to declare dependencies which are only >> required to run tests? >> >> In the past there was "tests_require" but as far as I know this is >> deprecated. >> >> I tried to read obvious github issues but maybe I missed something so >> please >> bear with me if this has come up way too often. I appreciate pointers. >> >> Felix >> -- >> Distutils-SIG mailing list -- distutils-sig@python.org >> To unsubscribe send an email to distutils-sig-le...@python.org >> https://mail.python.org/mailman3/lists/distutils-sig.python.org/ >> Message archived at >> https://mail.python.org/archives/list/distutils-sig@python.org/message/L52T3OKIGUCEESLV4OKD33KVJNR7E7PL/ >> > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/archives/list/distutils-sig@python.org/message/2PNZPSFGDGCLMOFRSSNRWNJ7EHC7V4NJ/ > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/N64K5XYWC6ZCWTO4OLR6G7TEDUAKRCFZ/
[Distutils] How should virtualenv handle the seed pip version
Please vote (and share your thoughts) on how you think virtualenv should handle what pip you get by default on new virtual environment creation: https://discuss.python.org/t/how-should-virtualenv-behave-respective-of-seed-packages-e-g-pip-by-default/4146 Thanks, Bernat -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/DPEVTHXTXSO7RVK3ZSTDP3YZ46A34WM6/
[Distutils] Re: Archive this list & redirect conversation elsewhere?
I think it's natural that people on the mailing list will favour the mailing list over Discourse, just as when asked this question on Discourse voted for the forum over the mailing list. Taking a more practical look at the issue, I want to note that 99% of the discussion about packaging and its future/peps has happened on the Discourse forum for the last 6 months. The amount of chatter on this mailing list is almost zero, so having the mailing list can be confusing to new users, as it might give the wrongful impression that this is the platform to discuss things, while in practice it's the Discourse. This is why we were thinking to remove the mailing list, because active development and design discussion moved off from it. Bernat On Thu, Jul 30, 2020 at 12:39 PM Ian Stapleton Cordasco < graffatcolmin...@gmail.com> wrote: > On Thu, Jul 30, 2020 at 5:39 AM Robert Collins > wrote: > > > > > > > > On Thu, 30 Jul 2020 at 14:52, Sumana Harihareswara > wrote: > >> > >> On 7/29/20 10:14 PM, Jeremy Stanley wrote: > >> > On 2020-07-30 07:17:03 +0530 (+0530), Pradyun Gedam wrote: > >> >> TL;DR: OK to archive this mailing list? Reply by Aug 30th. > >> > [...] > >> > > >> > I find it disappointing that there will no longer be a mailing list > >> > for discussions of Python packaging. Web forums with some E-mail > >> > integration are hardly the same. But those of us who still use > >> > E-mail (and worse, Usenet) eventually need to get out of the way of > >> > the wheels of progress lest they run us over. > >> > > >> > Many thanks to those who have maintained, moderated, and > >> > collaborated through this list over the years. It has been much > >> > appreciated. > >> > >> Jeremy, I'm not sure whether you were serious? If your disappointment is > >> only out of nostalgia, then yeah, accepting change makes sense. But if > >> your disappointment is because the Discourse experience is/will be worse > >> for your participation, then it's totally fine to speak up and tell us > how. > > > > > > Discourse requires about 10x the effort to participate in the community. > It's "mailing list mode" sends garbled fragments of interwoved WYSIWYG > documents that are unintelligible - I tried it for some months when the > Pyython Discourse started up but had to turn it off after a while as I > really couldn't effectively tell what was being said in a conversation that > comes in via it. > > I definitely agree with this. > > > > > And its so siloed, I may as well not be in the conversation at all: I > have to actively go and log into the website to look and see what new > conversations are happening, which in my time poor situation just doesn't > happen. So, for all intents and purposes, I'm not participating in any > conversation in Discourse at all, except for a rare helicopter drop-in when > someone pings me on email or Twitter or Slack or Discord to say 'hey, you > should comment on '. > > With email, I don't have to monitor heavily what topics I'm subscribed > to and if someone starts a discussion I'm uninterested in, I can mute > it. With Discourse it hasn't respected my desires to unsubscribe from > a thread's notifications and I still get emails from a thread from > Rust's Discourse from 2 years ago. > > I have generally the same sentiment as Jeremy and Rob though. I'm more > or less an emeritus member of this community (although I still try to > collaborate on Twine and keep up with general packaging discussions > and direction). Things have already started moving to Discourse though > and surprising me on twine because no one remembered me as part of the > team. So, Discourse has already made me want to work on Twine less. > > I will say that my impression of the ability to moderate Discourse is > lower friction/less difficult than to moderate a mailing list > (although mailman 3 is pretty great). I don't think that's a point in > Discourse's favor to be overlooked. > > > > > I full well recognise the advantage that these properties have when > dealing with a bulk of (largely) newcomers whose community use case is to > sample discussion: to find one or two things that have been said previously > via search, ask some questions to get a problem solved, and then move on. > > > > Relatively few of those users will be publishing packages though; even > with the rise of docker : consuming Python in a local script or workbook is > still the majority use case I think, so the bulk of the work we do affects > a (large) fraction of users, and most of those users are experienced by the > time they need our assistance. > > > >> Pradyun, thanks for starting this conversation. > >> > >> I am definitely interested in consolidating our conversational channels > >> and reducing fragmentation, but I have substantial reservations about > >> taking this particular step: > >> > >> * The majority of information overwhelm in my PyPA-related life is > >> because of GitHub repo and issue sprawl -- if we're going to put energy > >> into pruning
[Distutils] Re: Critical problem in PyCharm caused by the removal of "--build-dir" in 2020.3
Hello, You want to raise the issue directly at the pip issue tracker, https://github.com/pypa/pip. This mailing list is reserved mostly for general packaging related topics. Thanks, On Mon, Nov 30, 2020 at 6:47 PM Mikhail Golubev via Distutils-SIG < distutils-sig@python.org> wrote: > Hi there! > > I'm a member of the PyCharm team, and we faced a rather critical issue > with the release of pip 2020.3 ( > https://youtrack.jetbrains.com/issue/PY-45712). It turned out that we > actually still used the "--build-dir" option in our internal scripts for > installing packages through pip, and we completely missed the fact that it > was deprecated. To make the matter worse, a few releases back we > implemented automatic upgrading of packaging tools in all virtual > environments created in the IDE, so now it's impossible to install anything > in a new project unless pip is explicitly downgraded first. > > Judging by how pip now operates regarding building of packages, this > option can be safely dropped in our wrappers, but we still need some time > to prepare and test patches for a few previous releases and upcoming > 2020.3. Is it possible to give us some time to soften the blow for the > users? > > The second question is about the behavior of this option. It appears that > we initially started using it because in the past packages were not built > in a temporary directory by default. Could you please point me to the exact > version of pip where it changed? -- I couldn't find it in a changelog. It > would help us decide whether we need to keep some compatibility layer for > interpreters with an old version of pip installed. > > Thanks a lot! > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/archives/list/distutils-sig@python.org/message/6Z5DQGRDGOU4OCO3JSS4FVYWV5JBUPVP/ > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/MX6RBICVYCZMSNNWUXKW2CQQX7WI5L57/
[Distutils] Re: Making setup.py run external command to build
The only interface for the backend (setuptools) talking with the frontend
(pip) is https://www.python.org/dev/peps/pep-0517/#build-backend-interface.
The PEP itself is the documentation. Lack of documentation on how to fully
automate backend parts it's up to the backend. distutils is deprecated and
will be removed from the python standard library in two years. No
development planned on it. If you want distutils/setuptools like
functionality reach out to the setuptools issue tracker (same for
documentation on how to do things - note though this is an open-source
community project, so the answer well might be there's no such at the
moment, but you're free to contribute it).
Bernat,
On Tue, Mar 23, 2021 at 11:31 AM Julian Smith wrote:
> I've recently returned to this issue after a while away.
>
> I've ended up with a custom setup.py that runs the external build
> system as required, and allows pip install/sdist/bdist_wheel to work.
>
>
> However as a result of all this. i have some general questions.
>
> Basically i'm confused about the lack of information that is available
> on how to write a setup.py for packaging a project.
>
> If a project is pure python, or it uses compiled code and it's possible
> to specify how to build it using the swig/compile/link facilities
> provided by distutils or setuptools, then it's easy enough to use
> distutils or setuptools.
>
> However if, like me, one needs to use an external build system, it's
> very difficult to figure out what to do.
>
> There seem to be two approaches:
>
> Approach 1 is to pass callbacks to distutils.core.setup() or
> setuptools.setup(). However there doesn't appear to be documentation in
> either of these modules about what such callbacks should do or how/when
> they are called. The only way to figure things out seems to be to look
> at the distutils or setuptools source, which i'm finding pretty opaque.
>
> As far as i can tell, callbacks are only given information about the
> original command line arguments rather than more abstract information
> such as where to put output files, so distutils and setuptools don't
> seem to be giving any added value here.
>
> Approach 2 is to give up on distutils and setuptools, and instead write
> setup.py from scratch, directly handling the command-line arguments
> from pip.
>
> Given how fundamental the pip tool is to Python packaging, i was hoping
> that the command-line arguments that pip passes to setup.py would be
> standardised and documented, but this doesn't seem to be the case.
>
> One can gather some information by running pip commands and have
> setup.py print out sys.argv, but this is hardly a robust way of doing
> things. For example there's no guarantee that future versions of pip
> won't start using different command-line arguments that setuptools
> already knows about.
>
>
> So as far as i can tell, there are two levels of abstraction at which
> on can implement customised Python packaging (the setuptools.setup()'s
> callbacks or the setup.py command line), but neither one seems to be
> documented or standardised.
>
> Is that right? Or am i missing something fundamental here?
>
>
> Many thanks for any help.
>
> - Jules
>
>
> On Mon, 14 Dec 2020 13:29:28 -0500
> Daniel Holth wrote:
>
> > You would have to have a high tolerance for learning SCons. I'm aware
> that
> > this is not for everyone. Then you could write a SConstruct with
> dependent
> > tasks in a normal build system way. e.g.
> >
> > target = env.Command("a task", ...)
> > platlib = env.Whl("platlib", target, root=".")
> > whl = env.WhlFile(platlib)
> >
> > and have each step re-build only when its sources change.
> >
> > Here is a stackoverflow answer about custom distutils commands.
> > https://stackoverflow.com/questions/1710839/custom-distutils-commands
> >
>
>
>
> --
> http://op59.net
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/distutils-sig@python.org/message/77Z7LLYBBW5T4DKCFUTDEG5PM4N5DSIW/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at
https://mail.python.org/archives/list/distutils-sig@python.org/message/NCUTRTAHKYOVJDNURNWI7NA7DTWJY6WK/
