Re: [Python-Dev] BDFL delegate for Daniel Holth's packaging PEPs?
On 12/31/12 12:44 PM, Antoine Pitrou wrote: On Mon, 31 Dec 2012 15:40:39 +1000 Nick Coghlan ncogh...@gmail.com wrote: Does anyone object to my naming myself as BDFL-Delegate for Daniel Holth's packaging PEPs? PEP 425 Compatibility Tags for Built Distributions PEP 426 Metadata for Python Software Packages 1.3 PEP 427 The Wheel Binary Package Format 0.1 I've mentioned doing so before, but IIRC it was in the depths of a larger thread, so I figured I should make a separate post before claiming them in the PEPs repo. Ok for me. It would be nice if one of the past disutils maintainers gave their approval too, but they don't seem very active. FWIW I think Nick is perfect for this job. Cheers Tarek -- Tarek Ziadé · http://ziade.org · @tarek_ziade ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] distlib updated with resources API
On 9/28/12 12:55 AM, Antoine Pitrou wrote: Last but not least, distlib is the plan forward endorsed by python-dev, Is it? I haven't seen a PEP or an official decision about that. Just because someone proposed it on a mailing-list doesn't mean it is endorsed by python-dev. We discussed about this with Vinay, Nick and al on python-dev, based on Nick's document that describes what 'distlib' is. The document has changed since then, http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html But the idea was to create a subset of 4 or 5 modules that implement the various PEPs. Vinay started to work on this and made progress. When I said endorsed, I mean that most of the people in python-dev that care about packaging agreed or did not disagree. Now, if you disagree please say it. Or if you need an official decision, we need to first declare who is the current packaging BDFL maybe ? And since you seem interested in the topic maybe you could take that role ? Cheers Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging location ?
On 9/14/12 5:12 PM, Vinay Sajip wrote: Nick Coghlan ncoghlan at gmail.com writes: I like distcore or distlib, though. I have set up a BitBucket repo called distlib, at https://bitbucket.org/vinay.sajip/distlib/ This has the following bits of distutils2 / packaging, updated to run on 2.x and 3.x with a single codebase, and including tests (though not docs, yet): version.py - version specifiers as per PEP 386 metadata.py - metadata as per PEPs 345, 314 and 241 markers.py - environment markers as per PEP 345 database.py - installed distributions as per PEP 376 depgraph.py - distribution dependency graph logic glob.py - globbing functionality The code was taken at around the time packaging was removed, and may not have more recent changes. Regards, oh, cool ! maybe we could copy it at hg.python.org ? Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging location ?
On 9/13/12 11:38 AM, Antoine Pitrou wrote: On Thu, 13 Sep 2012 11:14:17 +1000 Nick Coghlan ncogh...@gmail.com wrote: On Thu, Sep 13, 2012 at 8:43 AM, R. David Murray rdmur...@bitdance.com wrote: When the removal was being pondered, the possibility of keeping certain bits that were more ready than others was discussed. Perhaps the best way forward is to put it back in bits, with the most finished (and PEP relevant) stuff going in first. That might also give non-packaging people bite-sized-enough chunks to actually digest and help with. This is the plan I'm going to propose. The previous approach was to just throw the entirety of distutils2 in there, but there are some hard questions that doesn't address, and some use cases it doesn't handle. So, rather than importing it wholesale and making the stdlib the upstream for distutils2, I believe it makes more sense for distutils2 to remain an independent project, and we cherry pick bits and pieces for the standard library's new packaging module as they stabilise. How is that going to be useful? Most people use distutils / packaging as an application, not a library. If you provide only a subset of the necessary features, people won't use packaging. Regards Antoine. Yeah but we've been too ambitious. Here's my proposal - actually it's Nick's proposal but I want to make sure we're on the same page wrt steps, and I think that addresses Antoine concerns 1. create a new package, called pkglib (or whatever), located at hg .python.org as a new project that just strictly contains : - the PEP implementations - non controversial features like files parser, pypi index browser etc it's doable - since that's what we have done in distutils2. the modules that implements those PEPs are standalone Let's avoid by all means to put the old distutils command logic there. Let's have a strict process on every new thing we're adding there. 2. make pkglib python 2 *and* python 3 compatible - natively, not w/ 2to3 3. make distutils2, distribute, pip, bento, etc. use that and try to share as many bits as possible 4. ask each project to pour in pkglib anything that can be reused by others when 3.4 comes around, I guess we can decide if pkglib can go in or not. That way, we won't have the usual controversy about distutils' command machinery. People will use whatever tool and hopefully this tool will be based on pkgutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging location ?
On 9/13/12 2:45 PM, Paul Moore wrote: 4. ask each project to pour in pkglib anything that can be reused by others +1, although again it'll be down to the projects whether they do actually contribute. Also this somewhat contradicts the be strict point above, which is why I'm lukewarm on be strict. Practicality vs purity - getting contributors/users is more important than insisting that everything be standardised before it can be implemented. Let me take back 'strict process' and replace it with: Everything added in pkglib should be a basic feature/implementation that does not force the tools to change the way *they* see their build/release/UI processes. The most sophisticated feature I am thinking about is the set of apis in distutils2.pypi They are APIs to browse and interact with PyPI, and they are useful to many projects, so I think it fits my definition. otho, anything relating to compilation should not be added there, unless it's APIs to get some aggregated info, like on the top of platform/sys/etc Cheers Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] packaging location ?
Hello I was wondering if anyone knows if the removed Lib/packaging directory landed in some other places after it was removed. We have http://hg.python.org/distutils2 be the 'packaging' version is a full py3-renamed version we need to keep mirrored Cheers Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/22/12 7:05 AM, Nick Coghlan wrote: .. - I reject setup.cfg, as I believe ini-style configuration files are not appropriate for a metadata format that needs to include file listings and code fragments I don't understand what's the problem is with ini-style files, as they are suitable for multi-line variables etc. (see zc.buildout) yaml vs ini vs xxx seems to be an implementation detail, and my take on this is that we have ConfigParser in the stdlib ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/22/12 9:11 AM, Nick Coghlan wrote: On Fri, Jun 22, 2012 at 4:42 PM, Tarek Ziadéta...@ziade.org wrote: On 6/22/12 7:05 AM, Nick Coghlan wrote: I don't understand what's the problem is with ini-style files, as they are suitable for multi-line variables etc. (see zc.buildout) yaml vs ini vs xxx seems to be an implementation detail, and my take on this is that we have ConfigParser in the stdlib You can't do more than one layer of nested data structures cleanly with an ini-style solution, and some aspects of packaging are just crying out for metadata that nests more deeply than that. The setup.cfg format for specifying installation layouts doesn't even come *close* to being intuitively readable - using a format with better nesting support has some hope of fixing that, since filesystem layouts are naturally hierarchical. A JSON based format would also be acceptable to me from a functional point of view, although in that case, asking people to edit it directly would be cruel - you would want to transform it to YAML in order to actually read it or write it. I still think this is an implementation detail, and that ini can work here, as they have proven to work with buildout and look very clean to me. But I guess that's not important -- looking forward for you changes proposals on packaging. I am now wondering why we don't have a yaml module in the stdlib btw :) Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/21/12 11:08 AM, Dag Sverre Seljebotn wrote: ... David Cournapeau's Bento project takes the opposite approach, everything is explicit and without any magic. http://cournape.github.com/Bento/ It had its 0.1.0 release a week ago. Please, I don't want to reopen any discussions about Bento here -- distutils2 vs. Bento discussions have been less than constructive in the past -- I just wanted to make sure everybody is aware that distutils2 isn't the only horse in this race. I don't know if there are others too? That's *exactly* the kind of approach that has made me not want to continue. People are too focused on implementations, and 'how distutils sucks' 'how setuptools sucks' etc 'I'll do better' etc Instead of having all the folks involved in packaging sit down together and try to fix the issues together by building PEPs describing what would be a common set of standards, they want to create their own tools from scratch. That will not work. And I will say here again what I think we should do imho: 1/ take all the packaging PEPs and rework them until everyone is happy (compilation sucks in distutils ? write a PEP !!!) 2/ once we have a consensus, write as many tools as you want, if they rely on the same standards = interoperability = win. But I must be naive because everytime I tried to reach people that were building their own tools to ask them to work with us on the PEPs, all I was getting was distutils sucks!' It worked with the OS packagers guys though, we have built a great data files managment system in packaging + the versions (386) -- Dag Sverre Seljebotn ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 2:53 PM, Nick Coghlan wrote: On Wed, Jun 20, 2012 at 9:31 PM, Tarek Ziadéta...@ziade.org wrote: Yeah maybe this subset could be left in 3.3 and we'd remove packaging-the-installer part (pysetup, commands, compilers) I think it's a good idea ! OK, to turn this into a concrete suggestion based on the packaging docs. Declare stable, include in 3.3 -- packaging.version — Version number classes packaging.metadata — Metadata handling packaging.markers — Environment markers packaging.database — Database of installed distributions I think that's a good subset. +1 on all of the things you said after If you succeed on getting the sci people working on PEP: Distutils replacement: Compiling Extension Modules it will be a big win. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/21/12 2:45 PM, Dag Sverre Seljebotn wrote: Guido was asked about build issues and scientific software at PyData this spring, and his take was that if scientific users have concerns that are that special, perhaps you just need to go and do your own thing. Which is what David is doing. Trailing QA session here: http://www.youtube.com/watch?v=QjXJLVINsSA if you know what you want and have a tool that does it, why bother using distutils ? But then, what your community will do with the guy that create packages with distutils ? just tell him he suck ? The whole idea is *interoperability*, not the tool used. Generalizing a bit I think it's web developers and scientists typically completely failing to see each others' usecases. I don't know if that bridge can be crossed through mailing list discussion alone. I know that David tried but came to a point where he just had to unsubscribe to distutils-sig. I was there, and sorry to be blunt, but he came to tell us we had to drop distutils because it sucked, and left because we did not follow that path Sometimes design by committee is just what you want, and sometimes design by committee doesn't work. ZeroMQ, for instance, is a great piece of software resulting from dropping out of the AQMP committee. That will not work. And I will say here again what I think we should do imho: 1/ take all the packaging PEPs and rework them until everyone is happy (compilation sucks in distutils ? write a PEP !!!) I think the only way of making scientists happy is to make the build tool choice arbitrary (and allow the use of waf, scons, cmake, jam, ant, etc. for the build). After all, many projects contains more C++ and Fortran code than Python code. (Of course, one could make a PEP saying that.) Right now things are so horribly broken for the scientific community that I'm not sure if one *can* sanely specify PEPs. It's more a question of playing around and throwing things at the wall and see what sticks -- 5 years from now one is perhaps in a position where the problem is really understood and one can write PEPs. Perhaps the web developers are at the PEP-ing stage already. Great for you. But the usecases are really different. If you sit down and ask your self: what are the information a python project should give me so I can compile its extensions ? I think this has nothing to do with the tools/implementations. And if we're able to write down in a PEP this, e.g. the information a compiler is looking for to do its job, then any tool out there waf, scons, cmake, jam, ant, etc, can do the job, no ? Anyway: I really don't want to start a flame-war here. So let's accept up front that we likely won't agree here; I just wanted to clarify my position. After 4 years I still don't understand what we won't agree means in this context. *NO ONE* ever ever came and told me : here's what I want a Python project to describe for its extensions. Just we won't agree or distutils sucks :) Gosh I hope we will overcome this lock one day, and move forward :D ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/21/12 5:50 PM, Chris McDonough wrote: A minor backwards incompat here to fix that issue would be appropriate, if only to be able to say hey, that issue no longer exists to folks who condemn the entire ecosystem based on that bug. At least, that is, if there will be another release of setuptools. Is that likely? or simply do that fix in distribute since it's Python 3 compatible -- and have setuptools officially discontinued for the sake of clarity. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/21/12 6:44 PM, Chris McDonough wrote: Yes. At the very least, there will be updated development snapshots (which are what buildout uses anyway). (Official releases are in a bit of a weird holding pattern. distribute's versioning scheme leads to potential confusion: if I release e.g. 0.6.1, then it sounds like it's a lesser version than whatever distribute is up to now. OTOH, releasing a later version number than distribute implies that I'm supporting their feature enhancements, and I really don't want to add new features to 0.6... but don't have time right now to clean up all the stuff I started in the 0.7 line either, since I've been *hoping* that the work on packaging would make 0.7 unnecessary. And let's not even get started on the part where system-installed copies of distribute can prevent people from downloading or installing setuptools in the first place.) Welp, I don't want to get in the middle of that whole mess. But maybe the distribute folks would be kind enough to do a major version bump in their next release; e.g. 1.67 instead of 0.67. That said, I don't think anyone would be confused by overlapping version numbers between the two projects. Oh yeah no problem, if Philip backports all the things we've done like Py3 compat, and bless more people to maintain setuptools, we can even discontinue distribute ! If not, I think you are just being joking here -- we don't want to go back into the lock situation we've suffered for many years were PJE is the only maintainer then suddenly disappears for a year, telling us no one that is willing to maintain setuptools is able to do so. (according to him) It's known that they have been diverging for a while. Yeah the biggest difference is Py3 compat, other than that afaik I don't think any API has been removed or modified. In my opinion, distribute is the only project that should go forward since it's actively maintained and does not suffer from the bus factor. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/21/12 7:49 PM, PJ Eby wrote: On Thu, Jun 21, 2012 at 1:20 PM, Tarek Ziadé ta...@ziade.org mailto:ta...@ziade.org wrote: telling us no one that is willing to maintain setuptools is able to do so. (according to him) Perhaps there is some confusion or language barrier here: what I said at that time was that the only people who I already *knew* to be capable of taking on full responsibility for *continued development* of setuptools, were not available/interested in the job, to my knowledge. Specifically, the main people I had in mind were Ian Bicking and/or Jim Fulton, both of whom had developed extensions to or significant chunks of setuptools' functionality themselves, during which they demonstrated exemplary levels of understanding both of the code base and the wide variety of scenarios in which that code base had to operate. They also both demonstrated conservative, user-oriented design choices, that made me feel comfortable that they would not do anything to disrupt the existing user base, and that if they made any compatibility-breaking changes, they would do so in a way that avoided disruption. (I believe I also gave Philip Jenvey as an example of someone who, while not yet proven at that level, was someone I considered a good potential candidate as well.) This was not a commentary on anyone *else's* ability, only on my then-present *knowledge* of clearly-suitable persons and their availability, or lack thereof. Yes, so I double-checked my sentence, I think we are in agreement: you would not let folks that *wanted* to maintain it back then, do it. Sorry if this was not clear to you. But let's forget about this, old story I guess. I would guess that the pool of qualified persons is even larger now, but the point is moot: my issue was never about who would maintain setuptools, but who would *develop* it. And I expect that we would at this point agree that future *development* of setuptools is not something either of us are seeking. Rather, we should be seeking to develop tools that can properly supersede it. This is why I participated in Distutils-SIG discussion of the various packaging PEPs, and hope to see more of them there. I definitely agree, and I think your feedback on the various PEPs were very important. My point is just that, we could (and *should*) in my opinion, merge back setuptools and distribute, just to have a py3-enabled setuptools that is in maintenance mode, and work on the new stuff in packaging besides it. the merged setuptools/distribute project could also be the place were we start to do the work to be compatible with the new standards That's my proposal. Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/21/12 4:26 PM, Dag Sverre Seljebotn wrote: project should give me so I can compile its extensions ? I think this has nothing to do with the tools/implementations. If you sit down and ask your self: what are the information a python I'm not sure if I understand. A project can't give the information needed to build it. The build system is an integrated piece of the code and package itself. Making the build of library X work on some ugly HPC setup Y is part of the development of X. To my mind a solution looks something like (and Bento is close to this): Step 1) Some standard to do configuration of a package (--prefix and other what-goes-where options, what libraries to link with, what compilers to use...) Step 2) Launch the package's custom build system (may be Unix shell script or makefile in some cases (sometimes portability is not a goal), may be a waf build) Step 3) Some standard to be able to cleanly install/uninstall/upgrade the product of step 2) An attempt to do Step 2) in a major way in the packaging framework itself, and have the package just declare its C extensions, would not work. It's fine to have a way in the packaging framework that works for trivial cases, but it's impossible to create something that works for every case. I think we should, as you proposed, list a few projects w/ compilation needs -- from the simplest to the more complex, then see how a standard *description* could be used by any tool And if we're able to write down in a PEP this, e.g. the information a compiler is looking for to do its job, then any tool out there waf, scons, cmake, jam, ant, etc, can do the job, no ? Anyway: I really don't want to start a flame-war here. So let's accept up front that we likely won't agree here; I just wanted to clarify my position. After 4 years I still don't understand what we won't agree means in this context. *NO ONE* ever ever came and told me : here's what I want a Python project to describe for its extensions. That's unfortunate. To be honest, it's probably partly because it's easier to say what won't work than come with a constructive suggestion. A lot of people (me included) just use waf/cmake/autotools, and forget about making the code installable through PyPI or any of the standard Python tools. Just because that works *now* for us, but we don't have any good ideas for how to make this into something that works on a wider scale. I think David is one of the few who has really dug into the matter and tried to find something that can both do builds and work through standard install mechanisms. I can't answer for why you haven't been able to understand one another. It may also be an issue with how much one can constructively do on mailing lists. Perhaps the only route forward is to to bring people together in person and walk distutils2 people through some hairy scientific HPC builds (and vice versa). Like versions scheme, I think it's fine if you guys have a more complex system to build software. But there should be a way to share a common standard for complation, even if people that uses distutils2 or xxx, are just doing the dumbest things, like simple C libs compilation. Just we won't agree or distutils sucks :) Gosh I hope we will overcome this lock one day, and move forward :D Well, me too. The other thing is, the folks in distutils2 and myself, have zero knowledge about compilers. That's why we got very frustrated not to see people with that knowledge come and help us in this area. So, I reiterate my proposal, and it could also be expressed like this: 1/ David writes a PEP where he describes how Bento interact with a project -- metadata, description files, etc 2/ Someone from distutils2 completes the PEP by describing how setup.cfg works wrt Extensions 3/ we see if we can have a common standard even if it's a subset of bento capabilities Dag ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/21/12 7:56 PM, Chris McDonough wrote: ... think any API has been removed or modified. In my opinion, distribute is the only project that should go forward since it's actively maintained and does not suffer from the bus factor. Yeah the biggest difference is Py3 compat, other than that afaik I don't I'm not too interested in the drama/history of the fork situation You are the one currently adding drama by asking for a new setuptools release and saying distribute is diverging. so I don't care whether setuptools has the fix or distribute has it or both have it, but being able to point at some package which doesn't prevent folks from overriding sys.path ordering using PYTHONPATH would be a good thing. It has to be in Distribute if we want it in most major Linux distros. And as I proposed to PJE I think the best thing would be to have a single project code base, working with Py3 and receiving maintenance fixes with several maintainers. Since it's clear we're not going to add feature in any of the projects, I think we can safely trust a larger list of maintainers, and just keep the project working until the replacement is used - C ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/21/12 10:46 PM, Dag Sverre Seljebotn wrote: ... I think we should, as you proposed, list a few projects w/ compilation needs -- from the simplest to the more complex, then see how a standard *description* could be used by any tool It's not clear to me what you mean by description. Package metadata, install information or description of what/how to build? I hope you don't mean the latter, that would be insane...it would effectively amount to creating a build tool that's both more elegant and more powerful than any option that's currently already out there. Assuming you mean the former, that's what David did to create Bento. Reading and understanding Bento and the design decisions going into it would be a better use of time than redoing a discussion, and would at least be a very good starting point. What I mean is : what would it take to use Bento (or another tool) as the compiler in a distutils-based project, without having to change the distutils metadata. But anyway, some project types from simple to advanced: - Simple library using Cython + NumPy C API - Wrappers around HPC codes like mpi4py, petsc4py - NumPy - SciPy (uses Fortran compilers too) - Library using code generation, Cython, NumPy C API, Fortran 90 code, some performance tuning with CPU characteristics (instruction set, cache size, optimal loop structure) decided compile-time I'd add: - A Distutils project with a few ExtensionsThe other thing is, the folks in distutils2 and myself, have zero knowledge about compilers. That's why we got very frustrated not to see people with that knowledge come and help us in this area. Here's the flip side: If you have zero knowledge about compilers, it's going to be almost impossible to have a meaningful discussion about a compilation PEP. It's very hard to discuss standards unless everybody involved have the necessary prerequisite knowledge. You don't go discussing details of the Linux kernel without some solid C experience either. Consider me as the end user that want to have his 2 C modules compiled in their Python project. The necessary prerequisites in this case is not merely knowledge of compilers. To avoid repeating mistakes of the past, the prerequisites for a meaningful discussion is years of hard-worn experience building software in various languages, on different platforms, using different build tools. Look, these problems are really hard to deal with. Myself I have experience with building 2-3 languages using 2-3 build tools on 2 platforms, and I consider myself a complete novice and usually decide to trust David's instincts over trying to make up an opinion of my own -- simply because I know he's got a lot more experience than I have. Theoretically it is possible to separate and isolate concerns so that one set of people discuss build integration and another set of people discuss installation. Problem is that all the problems tangle -- in particular when the starting point is distutils! That's why *sometimes*, not always, design by committee is the wrong approach, and one-man-shows is what brings technology forwards. I am not saying this should be designed by a commitee, but rather - if such a tool can be made compatible with simple Distutils project, the guy behind this tool can probably help on a PEP with feedback from a larger audience than the sci community. What bugs me is to say that we live in two separate worlds and cannot build common pieces. This is not True. So, I reiterate my proposal, and it could also be expressed like this: 1/ David writes a PEP where he describes how Bento interact with a project -- metadata, description files, etc 2/ Someone from distutils2 completes the PEP by describing how setup.cfg works wrt Extensions 3/ we see if we can have a common standard even if it's a subset of bento capabilities bento isn't a build tool, it's a packaging tool, competing directly with distutils2. It can deal with simple distutils-like builds using a bundled build tool, and currently has integration with waf for complicated builds; integration with other build systems will presumably be added later as people need it (the main point is that bento is designed for it). I am not interested in Bento-the-tool. I am interested in what such a tool needs from a project to use it = It can deal with simple distutils-like builds using a bundled build tool = If I understand this correctly, does that mean that Bento can build a distutils project with the distutils Metadata ? If this is the case it means that there a piece of function that translates Distutils metadata into something Bento deals with. That's the part I am interested in for interoperability. Dag ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/21/12 11:55 PM, David Cournapeau wrote: I think there is a misunderstanding of what bento is: bento is not a compiler or anything like that. It is a set of libraries that work together to configure, build and install a python project. Concretely, in bento, there is - a part that build a packager description (Distribution-like in distutils-parlance) from a bento.info http://bento.info (a bite like setup.cfg) - a set of tools of commands around this package description. - a set of backends to e.g. use waf to build C extension with full and automatic dependency analysis (rebuild this if this other thing is out of date), parallel builds and configuration. Bento scripts build numpy more efficiently and reliable while being 50 % shorter than our setup.py. - a small library to build a distutils-compatible Distribution so that you can write a 3 lines setup.py that takes all its info from bento.info http://bento.info and allow for pip to work. Now, you could produce a similar package description from the setup.cfg to be fed to bento, but I don't really see the point since AFAIK, bento.info http://bento.info is strictly more powerful as a format than setup.cfg. So that means that *today*, Bento can consume Distutils2 project and compiles them, just by reading their setup.cfg, right ? And the code you have to convert setup.cfg into bento.info is what I was talking about. It means that I can create a project without a setup.py file, and just setup.cfg, and have it working with distutils2 *or* bento That's *exactly* what I was talking about. the setup.cfg is the *common* standard, and is planned to be published at PyPI statically. Let people out there use their tool of their choice to install a project defined by a setup.cfg so 2 questions: 1/ does Bento install things following PEP 376 ? 2/ how does the setup.cfg hooks work wrt Bento ? and last one proposal: how a PEP that defines a setup.cfg standard that is Bento-friendly, but still distutils2-friendly would sound ? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/19/12 11:46 PM, Éric Araujo wrote: ... I don’t think (a) would give us enough time; we really want a few months (and releases) to hash out the API (most notably with the pip and buildout developers) and clean the bdist situation. Likewise (c) would require developer (my) time that is currently in short supply. (b) also requires time and would make development harder, not to mention probable user pain. This leaves (d), after long reflection, as my preferred choice, even though I disliked the idea at first (and I fully expect Tarek to feel the same way). Yeah I feel the same way. +1 for (d). I had unfortunately no time lately. Thanks for picking up things. We want a solid distutils replacement, and I think we wrote solid PEPs and seemed to have find consensus for most issues in the past two years. So I prefer to hold it and have a solid implementation in the stldib. The only thing I am asking is to retain ourselves to do *anything* in distutils and continue to declare it frozen, because I know it will be tempting to do stuff there... Cheers Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:05 AM, Dirkjan Ochtman wrote: On Wed, Jun 20, 2012 at 10:55 AM, Tarek Ziadéta...@ziade.org wrote: So I prefer to hold it and have a solid implementation in the stldib. The only thing I am asking is to retain ourselves to do *anything* in distutils and continue to declare it frozen, because I know it will be tempting to do stuff there... That policy has been a bit annoying. Gentoo has been carrying patches forever to improve compilation with C++ stuff (mostly about correctly passing on environment variables), and forward-porting them on every release gets tedious, but the packaging/distutils2 effort has made it harder to get them included in plain distutils. I understand there shouldn't be crazy patching in distutils, but allowing it to inch forward a little would make the lives of the Gentoo Python team easier, at least. Cheers, If distutils gets new features I think it's killing the packaging effort. Maybe these new features could be implemented in packaging, then bridged in Distutils ? the Compilation feature is isolated enough to do this. In any case, I guess we should have some kind of policy in place where we list the exceptions when distutils can be changed. Maybe in the packaging PEP ? Cheers Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:04 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 15:00:52 +1000 Nick Coghlanncogh...@gmail.com wrote: On Wed, Jun 20, 2012 at 11:23 AM, Antoine Pitrousolip...@pitrou.net wrote: The question is what will happen after 3.3. There doesn't seem to be a lot of activity around the project, does it? I think the desire is there, What makes you think that, exactly? Maybe because the packaging fatigue occurs around 3 years after you start fighting that best, and we do have fresh blood working on it ? :) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:12 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 11:05:43 +0200 Dirkjan Ochtmandirk...@ochtman.nl wrote: On Wed, Jun 20, 2012 at 10:55 AM, Tarek Ziadéta...@ziade.org wrote: So I prefer to hold it and have a solid implementation in the stldib. The only thing I am asking is to retain ourselves to do *anything* in distutils and continue to declare it frozen, because I know it will be tempting to do stuff there... That policy has been a bit annoying. Gentoo has been carrying patches forever to improve compilation with C++ stuff (mostly about correctly passing on environment variables), and forward-porting them on every release gets tedious, but the packaging/distutils2 effort has made it harder to get them included in plain distutils. I understand there shouldn't be crazy patching in distutils, but allowing it to inch forward a little would make the lives of the Gentoo Python team easier, at least. I think the whole idea that distutils should be frozen and improvements should only go in distutils2 has been misled. Had distutils been improved instead, many of those enhancements would already have been available in 3.2 (and others would soon be released in 3.3). I tried to improve Distutils and I was stopped and told to start distutils2, because distutils is so rotten, any *real* change/improvment potentially brakes the outside world. This has not changed. Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. So what are your suggesting, since you seem to know what's a mistake and what's not ? (time-travel machine not allowed) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:49 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 11:22:07 +0200 Tarek Ziadéta...@ziade.org wrote: I tried to improve Distutils and I was stopped and told to start distutils2, because distutils is so rotten, any *real* change/improvment potentially brakes the outside world. If distutils was so rotten, distutils2 would not reuse much of its structure and concepts (and test suite), would it? 'much' is pretty vague here. distutils2 is a fork of distutils, that has evolved a *lot* if you look at the code, beside the compilation part and some commands, most things are different. distutils is rotten because when you change its internals, you might break some software that rely on them. Most of the distutils2 improvements (new PEPs, setup.cfg, etc.) were totally possible in distutils, weren't they? I started there, remember ? And we ended up saying it was impossible to continue without breaking the packaging world. Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. So what are your suggesting, since you seem to know what's a mistake and what's not ? I don't have any suggestion apart from keeping packaging in 3.3. But I also think it would be better for the community if people were not delusional when making decisions. Removing packaging from 3.3 is a big risk: users and potential contributors will be even less interested than they already are. That's a good point. But if no one works on its polishing *now*, it's going to be the same effect on people: they'll likely to be very annoyed if the replacer is not rock solid. Here's a datapoint: distribute (*) is downloaded 100x more times than distutils2 (**). (*) http://pypi.python.org/pypi/distribute/ (**) http://pypi.python.org/pypi/Distutils2/ why would you expect a different datapoint ? - Distutils2 was released as a beta software, and not really promoted yet - Distribute is downloaded automatically by many stacks out there, and PyPI does not make a difference whether the hit was from a human behind pip, or from a stack like zc.buildout Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:54 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 09:51:03 + (UTC) Vinay Sajipvinay_sa...@yahoo.co.uk wrote: Antoine Pitrousolipsisat pitrou.net writes: Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. What's the rationale for leaving it in, when it's known to be incomplete/unfinished? As an incentive for users to start using the features that are finished enough, and exercise the migration path from distutils. The module can be marked provisional so as to allow further API variations. It's true that some modules are quite mature and already useful: - packaging.version (PEP 386) - packaging.pypi - packaging.metadata (PEP 345) - packaging.database (PEP 386) the part that is not ready is the installer and some setuptools bridging Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 12:39 PM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 12:30:51 +0200 Tarek Ziadéta...@ziade.org wrote: Most of the distutils2 improvements (new PEPs, setup.cfg, etc.) were totally possible in distutils, weren't they? I started there, remember ? And we ended up saying it was impossible to continue without breaking the packaging world. we were only certain people, AFAIR. That was the BDFL decision after a language summit. Having tried to innovate in Distutils in the past, I think it's a very good decision, ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 1:19 PM, Paul Moore wrote: On 20 June 2012 11:34, Tarek Ziadéta...@ziade.org wrote: On 6/20/12 11:54 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 09:51:03 + (UTC) Vinay Sajipvinay_sa...@yahoo.co.ukwrote: Antoine Pitrousolipsisatpitrou.netwrites: Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. What's the rationale for leaving it in, when it's known to be incomplete/unfinished? As an incentive for users to start using the features that are finished enough, and exercise the migration path from distutils. The module can be marked provisional so as to allow further API variations. It's true that some modules are quite mature and already useful: - packaging.version (PEP 386) - packaging.pypi - packaging.metadata (PEP 345) - packaging.database (PEP 386) the part that is not ready is the installer and some setuptools bridging I've never seen that information mentioned before. So that's (good) news. A question, then. Why is it not an option to: 1. Rip out all bar those 4 modules. 2. Make sure they are documented and tested solidly (they may already be, I don't know). 3. Declare that to be what packaging *is* for Python 3.3. Whether any of those modules are of any use in isolation, is a slightly more complex question. As is whether the APIs are guaranteed to be sufficient for further development on the rest of packaging, given that by doing this we commit to API stability and backward compatibility. Your comment quite mature and already useful is not quite firm enough to reassure me that we're ready to set those modules in stone (although presumably the 3 relating to the PEPs are, simply because they implement what the PEPs say). The last time I checked: packaging.version is the implementation of PEP 386, and stable. It's one building block that would be helpful as-is in the stdlib. it's completely standalone. packaging.metadata is the implementation of all metadata versions. standalone too. packaging.pypi is the PyPI crawler, and has fairly advanced features. I defer to Alexis to tell us is it's completely stable packaging.database is where PEP 376 is. It has the most innovations, implements PEP 376 packaging.config is the setup.cfg reader. Ity's awseome because together with packaging.database and packaging.markers, it gives you OS-independant data files. see http://docs.python.org/dev/packaging/setupcfg.html#resources Yeah maybe this subset could be left in 3.3 and we'd remove packaging-the-installer part (pysetup, commands, compilers) I think it's a good idea ! Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 5:44 PM, Georg Brandl wrote: Am 20.06.2012 17:34, schrieb Éric Araujo: Hi all, Sorry I can’t take the time to reply to all messages, this week I’m fully busy with work and moving out. To answer or correct a few things: - I am lacking time these months, but that’s because I’m still getting used to having a full-time job and being settled into a new country. With the feedback we’ve been getting from people recently, I am motivated, not burned out. - I have started building a group of distutils2 contributors here in Montreal. People are motivated, but it takes time to learn the codebase and tackle on the big things. - The four modules identified as minimal, standalone, good subset all have big problems (the PEPs have open issues, and the modules' APIs need improvements). Tarek seems to think otherwise... looks like in the end, this subset could only be included as provisional. I defer to Eric -- My answers are probably missing recent changes he knows Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASED] Distutils2 1.0a4
Thanks a lot for your hard work and dedication on packaging ! On 3/13/12 9:37 AM, Éric Araujo wrote: Hello, On behalf of the distutils2 contributors, I am thrilled to announce the release of Distutils2 1.0a4. Distutils2 is the packaging library that supersedes Distutils. It supports distributing, uploading, downloading, installing and removing projects, and is also a support library for other packaging tools like pip and buildout. It will be provided as part of Python 3.3; this release is a backport compatible with Python 2.5 to 2.7. Distutils2 1.0a4 contains a number of known bugs, limitations and missing features, but we have released it to let users and developers download and try it. This means you! If you want to report new issues, request features or contribute, please read DEVNOTES.txt in the source distribution or http://wiki.python.org/Distutils2/Contributing More alpha releases will be cut when important bugs get fixed during the next few months, like Windows or PyPy compatibility. The first beta is planned for June, and 1.0 final for August (to follow Python 3.3.0). Until beta, the API is subject to drastic changes and code can get removed. Basic documentation is at http://docs.python.org/dev/packaging ; it will get updated, expanded and improved in the coming months. Enjoy! Repository: http://hg.python.org/distutils2 Bug tracker: http://bugs.python.org/ (component Distutils2) Mailing list: http://mail.python.org/mailman/listinfo/distutils-sig/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On Sun, Oct 9, 2011 at 9:31 PM, PJ Eby p...@telecommunity.com wrote: ... What we can do however is to see what bdist_egg does and define a new bdist command inspired by it, but without zipping, pkg_resource calls, etc. Why? If you just want a dumb bdist format, there's already bdist_dumb. Conversely, if you want a smarter format, why reinvent wheels? Just to make sure we're on the same page here. PEP 376 provide the installation format for the 'future' -- http://www.python.org/dev/peps/pep-0376/ Introducing back another *installation* format would be against the goal we've initially had with PEP 376 : have a single installation format all tools out there would support, for the sake of standardization of interoperability. (and for consumers in other communities) So, while 'eggs' are interesting as plugins for a given application (that was the initial use case right ?), please do not consider them as an installation format for Python. Now for a binary archive, that would get installed ala PEP 376, why not ? I'd just be curious to have someone list the advantage of having a project released that way besides the importable as-is feature. Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging in Python 2 anyone ?
On Thu, Aug 18, 2011 at 5:17 AM, Fred Drake fdr...@acm.org wrote: On Wed, Aug 17, 2011 at 11:00 PM, Nick Coghlan ncogh...@gmail.com wrote: It's actually for the same reason that unittest changes are backported under the unittest2 name - the distutils2 name can be used in the future to get Python 3.4 packaging features in Python 3.3, but that would be difficult if the backport shadowed the standard library name. Ah, yes... the old too bad we stuck it in the standard library problem. For some things, an easy lament, but for foundational packaging-related things, it's hard to get around. Yeah exactly. And the good thing about packaging and distutils2 is that for the regular usage (package your project) you don't type any code, just define options in setup.cfg. IOW there's no import packaging or import distutils2. Cheers Tarek -Fred -- Fred L. Drake, Jr. fdrake at acm.org A person who won't read has no advantage over one who can't read. --Samuel Langhorne Clemens ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging in Python 2 anyone ?
On Thu, Aug 18, 2011 at 12:30 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: ... Okay, I had a bit of spare time today, and here's as far as I've got: Awesome, thanks a lot ! Step 1 - done. Step 2 - done. Step 3 - On Python 2.6 most of the tests pass: Ran 322 tests in 12.148s FAILED (failures=3, errors=4, skipped=39) See the detailed test results (for Linux) at https://gist.github.com/1152791 The code is at https://bitbucket.org/vinay.sajip/du2/ stdlib dependency code is either moved to util.py or test/support.py as appropriate. You need to test in a virtualenv with unittest2 installed. No work has been done on packaging the project. I'm not sure if I'll have much more time to spend on this, but it's there in case someone else can look at the remaining test failures, plus Steps 4 and 5; hopefully I've broken the back of it :-) Thank you very much ! Ideally, if you could push this to hg.python.org/distutils2 (overwriting the existing stuff). Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging in Python 2 anyone ?
On Thu, Aug 18, 2011 at 11:26 AM, Antoine Pitrou solip...@pitrou.net wrote: On Thu, 18 Aug 2011 09:16:21 + (UTC) Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Tarek Ziadé ziade.tarek at gmail.com writes: Ideally, if you could push this to hg.python.org/distutils2 (overwriting the existing stuff). Okay, done. I've overwritten existing files and added new ones, only removing/renaming things like index - pypi and mkcfg - create. I haven't touched existing code e.g. the top-level test scripts or the _backport directory. The added test_distutils2.py is what I used to run the tests. Impressive work! That said, I'm not sure it was the best moment to backport, since test_packaging currently fails under Windows (I think Éric is supposed to look at it). Frankly, I think there's no best moment for this. We'll need to backport everything we do in packaging/ in distutils2/ (Yeah, painful...) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging in Python 2 anyone ?
On Thu, Aug 18, 2011 at 11:16 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Tarek Ziadé ziade.tarek at gmail.com writes: Ideally, if you could push this to hg.python.org/distutils2 (overwriting the existing stuff). Okay, done. I've overwritten existing files and added new ones, only removing/renaming things like index - pypi and mkcfg - create. I haven't touched existing code e.g. the top-level test scripts or the _backport directory. The added test_distutils2.py is what I used to run the tests. Thanks again Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging in Python 2 anyone ?
On Thu, Aug 18, 2011 at 8:27 PM, Éric Araujo mer...@netwok.org wrote: Hi Tarek, Doing an automated conversion turned out to be a nightmare, and I was about to go ahead and maintain a fork of the packaging package, with the few modules that are needed (sysconfig, etc) within a standalone release. Can you give us more info? Do you have a repo somewhere, or notes? I tried using relative imports, but that made the whole thing complicated and not working under older 2.x then there are a lot of spots where the word 'packaging' is used for other things than modules. then there are spots when we needed to change the bytes/str behavior depending on the py version, making everything complex to maintain I guess it's the addition of the three that made it too complex : transparent renaming + 3to2 + 3.xto3.x A related question: what is the minimum 2.x version that we should support? 2.6 would be a dream, thanks to bytes literal and all that, but I’m sure it’s not realistic; 2.5 would be nice for the with statement and hashlib, otherwise 2.4 is okay. 2.5 sounds good. I am sold on dropping 2.4 frankly. Maybe we can drop 2.5 in a few months ;) When I talked with Łukasz in private email about backports and 3to2, we agreed that there were some serious bugs in 3to2 and we wanted to work on patches. I also wanted to make the command-line driver more flexible, so that it would be easy to run a command to apply only 3.3→3.2 fixes, then another for 3.2→2.7, etc. Maybe your problems were caused by the state of the packaging codebase. The conversion to 3.x was a little messy: in some cases there were parallel code branches for 2.x and 3.x, on other cases 2to3 was run, and in many cases the conversion had to be cleaned up (esp. bytes/str madness). Even now that the code runs and the tests pass, there may still be things in need of a cleanup in the codebase, and maybe they trip up 3to2. I think that's not worth the effort frankly. keeping a clean fully py3 code without worrying about making it 3to2 friendly, make all contributors life easier ihmo. The tradeoff is that we will have to backport to distutils2 changes. That's what was done for a while between the Python trunk and the Py3k branch, so I guess it's doable -- if all packaging contributors agree to do this backport work. I am looking for someone that has some free time and that is willing to lead this work. Well, free time is scarce with all these distutils bugs on my plate, but I am definitely interested in heading the backport, as I stated earlier. I think the key point is to avoid making the same work over and over again, and I see a few ways of managing that. The first way is to start with a 2.x-converted codebase (thanks Vinay!) and manually port all cpython/packaging changesets to distutils2, like I used to do. This is just as annoying as backporting to 2.7, and just as simple. The second way is to work on a conversion tool instead of working on changesets. The idea is to make a robust tool based on 3to2 that copies code and converts it. This would not be the easiest way, as shown by your experience, but surely the less cumbersome in the long term. The third way is to use a new Mercurial repo converted from the cpython repo, so that we can run “hg convert” again to pull new changesets. Convert, test and commit. The advantage is that it’s not required to port each changeset: the convert-merge dance can be done once a month, or just for new releases. The fourth way is hybrid: start from a 2.x-converted codebase, and each month, make a diff for cpython/Lib/packaging and apply to distutils2. I fear that such diffs would be painful to apply, and consist mostly of rejects. With idea #3, we get to use a merge tool, which is much better. After writing out these ideas, I think the first one is certainly the simplest thing that could work with minimum pain. I think so too. The automatic conversion sounded like a great thing, but the nature of the project makes it too hard, Cheers -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging in Python 2 anyone ?
On Mon, Aug 15, 2011 at 1:20 AM, Nick Coghlan ncogh...@gmail.com wrote: On Sun, Aug 14, 2011 at 7:41 PM, Tarek Ziadé ziade.ta...@gmail.com wrote: Hi all, I am lacking of time right now to finish an important task before 3.2 final is out: If anyone else got at all confused by Tarek's email, s/3.x/3.x+1/ and it will all make sense (the mentioned release numbers in the 3.x series are all one lower than they should be - packaging is planned for 3.3, but a standalone library will allow feedback to be gathered from 2.x and 3.2 users before the API is 'locked in' for 3.3). Oh yeah sorry for the version mess up :) How far has packaging diverged from distutils2, though? Wasn't that the planned venue for any backports in order to avoid name conflicts? The plan is to provide under earlier versions of Python a standalone project that does not use the packaging namespace, but the distutils2 namespace. IOW, the task to do is: 1/ copy packaging and all its stdlib dependencies in a standalone project 2/ rename packaging to distutils2 3/ make it work under older 2.x and 3.x (2.x would be the priority) 4/ release it, promote its usage 5/ consolidate the API with the feedback received I realize it's by far the less interesting task to do in packaging, but it's by far one of the most important Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Packaging in Python 2 anyone ?
Hi all, I am lacking of time right now to finish an important task before 3.2 final is out: we need to release packaging as a standalone release under Python 2.x and Python 3.1, to gather as much feedback as we can from more people. Doing an automated conversion turned out to be a nightmare, and I was about to go ahead and maintain a fork of the packaging package, with the few modules that are needed (sysconfig, etc) within a standalone release. I am looking for someone that has some free time and that is willing to lead this work. 3.2 can go out without this work of course, but it would be *much* better to have that feedback If you are interested, please let me know. Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] packaging backport
Hello I am starting the backport of Packaging into a standalone version named Distutils2 -- this is very important to speed up the development of packaging itself since it'll give of more feedback from 2.x projects To do this I need to create a script that does a mass renaming of 'packaging' into 'distutils2', then I can start to play with 3to2 and ...3.xto3.x :) . I tried to script rope but the py3k version seem not compatible with our current AST -- and it's a bit overkill for this task I guess. Before I start to write my own refactoring tool, I was wondering if anyone here had some experience in this, and could give me some hints. Thanks Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging backport
On Tue, Jun 21, 2011 at 2:44 PM, Nick Coghlan ncogh...@gmail.com wrote: 2to3 or Brett's mnfy are likely to be reasonable starting points. Will look at mnfy, thanks -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging backport
On Tue, Jun 21, 2011 at 3:12 PM, R. David Murray rdmur...@bitdance.com wrote: On Tue, 21 Jun 2011 13:42:05 +0200, =?ISO-8859-1?Q?Tarek_Ziad=E9?= ziade.ta...@gmail.com wrote: Before I start to write my own refactoring tool, I was wondering if anyone here had some experience in this, and could give me some hints. Coul you could just write a 3to2 fixer? I don't know how hard it is to run just a selected set of fixers (so that you could use it to generate python3 code), but it seems to me that renaming modules is something that 3to2 (and 2to3, of course) should be good at. The one thing rope is good at is to find where a given variable name is used, and rename all occurrences recursively. So basically, when you rename an import, it renames all the code that uses it. I don't really know how 2to3/3to2 work but I assumed that it does not do this, but simply give you a hook for every visited node. IOW that looking for dependencies is to be done Cheers Tarek -- R. David Murray http://www.bitdance.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging backport
On Tue, Jun 21, 2011 at 3:37 PM, Joe Amenta ament...@msu.edu wrote: ... Yes, 2to3/3to2 does not do anything fancy like that. The tools are purely concerned with syntax, whereas renaming imports is semantic. The good news is that this line: import packaging as _myPackaging can be replaced by: import distutils2 as _myPackaging and code that uses _myPackaging will work. I've attached a fixer that can go into your lib3to2/fixes folder. You should also edit fix_imports.py and add the line: packaging : distutils2, to the MAPPING dictionary near the top, then you can run 3to2 like this: 3to2 -fpackaging -fimports files_to_fix.py (-w option to write changes to the files modified) Hope this helps, It does, thanks a lot ! Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sniffing passwords from PyPI using insecure connection
On Fri, Jun 3, 2011 at 11:40 PM, Martin v. Löwis mar...@v.loewis.de wrote: I followed up on the tracker. I'm +0 on adding this to 2.6, but not until after the 2.6.7 release on Friday. How well has this change been tested? Are there people for whom this could break things? As others have pointed out: it would break systems that don't have the _ssl module built. yeah, we would need to fallback to http in that case. while using https by default is a nice addition, maybe we should also look at adding a scp-like upload/register command, since the server has now this ability. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Docs for the packaging module
I do care :) Looks fine Please push, as we can always change and fix things afterwards in the tip before 3.3 is out. Le 1 juin 2011 19:38, Éric Araujo mer...@netwok.org a écrit : Hi, Looks like nobody cares enough about the packaging docs :) If there is no feedback, here’s what I propose to do: - Add new Doc/library/packaging* files (library reference for developers of packaging tools) - Add new packaging-based “Installing Python Projects” to Doc/install, replacing old distutils docs - Add new packaging-based “Distributing Python Projects” Doc/packaging, replacing old distutils docs (+ link to it from the main HTML page instead of linking to Doc/distutils) For users needing the legacy distutils docs in 3.3, I would move the older distutils Doc/install/index.rst to Doc/distutils/install.rst, and add a link to Doc/distutils in Doc/library/distutils (because the main page would no longer link to Doc/distutils). Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pysetup as a top script
On Tue, May 31, 2011 at 7:13 AM, Nick Coghlan ncogh...@gmail.com wrote: On Tue, May 31, 2011 at 2:44 AM, Tarek Ziadé ziade.ta...@gmail.com wrote: Hello If no one objects, I'll promote Tools/scripts/pysetup3 to a top level script that gets installed in scripts/ like 2to3, pydoc etc.. That way, people will be able to use it directly when installing, removing projects, or studying what's installed Cool. Now I'm trying to remember if it was a list discussion or the language summit where you got the initial consensus on that approach... The thread starts here: http://mail.python.org/pipermail/python-dev/2010-October/104535.html The pysetup top-level script was mentioned here: http://mail.python.org/pipermail/python-dev/2010-October/104581.html Cheers Tarek Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] pysetup as a top script
Hello If no one objects, I'll promote Tools/scripts/pysetup3 to a top level script that gets installed in scripts/ like 2to3, pydoc etc.. That way, people will be able to use it directly when installing, removing projects, or studying what's installed Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: removed spurious output
On Mon, May 30, 2011 at 7:04 PM, Georg Brandl g.bra...@gmx.net wrote:
On 30.05.2011 17:54, Terry Reedy wrote:
On 5/30/2011 6:25 AM, tarek.ziade wrote:
Should not old_out be sys.stderr, since that is what you over-write and
'restore'?
+ old_out = sys.stdout
+ sys.stderr = StringIO()
+ try:
+ dist = self.run_setup('install_dist', '--prefix=' +
self.root_dir)
+ finally:
+ sys.sterr = old_out
And even more importantly, shouldn't this be sys.stderr instead of
sys.sterr?
Yes,
Really, what happened to testing before you push?
I did test it, before and after my push, sir.
This was not to fix a test bug, but to avoid a spurious output in the tests.
Georg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
--
Tarek Ziadé | http://ziade.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots update
On Mon, May 23, 2011 at 3:00 AM, Bill Janssen jans...@parc.com wrote: Tarek Ziadé ziade.ta...@gmail.com wrote: Yes, I am aware of this. I have fixed today most remaining issues, and fixing the final ones right now. Just FYI: the AMD64 Snow Leopard buildbot and PPC Leopard buildbots are now green, but the PPC Tiger buildbot is still failing for all branches because of packaging errors: All the linux and windows stable slaves are now green, and I have a few issues left to be fixed for all solaris flavors and the two you are showing, that are also failing under Free BSD. Thanks Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] the distutils2 repo and 3to2
Hey, Now that packaging has landed, the distutils2 repo is going to be re-seted and will be the python 2.x / 3.1 / 3.2 backport of packaging. In theory, we want to automate the extraction of packaging from the stdlib and a few other modules, and run 3to2 at install time. Or should I say 3.3tosomething. I want to do this to avoid maintaining yet another code base. In practice, I don't really know the current state of 3to2 so we'll see.. Any help/hint in this project would be appreciated. Thanks Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] the distutils2 repo and 3to2
2011/5/23 Łukasz Langa luk...@langa.pl: .. I'm maintaining a configparser 3.2+ backport for 2.6-2.7 using 3to2. Do you backport to 3.1 ? .. There's some documentation there on the conversion process I came up with. Awesome, will look up, thanks As for distutils2, I was already contacted by Éric Araujo and will help him improve 3to2. We are yet to contact its authors to see if they believe merging our changes upstream will be possible. Great, anything was started already ? If so, we should sync to see how we can initiate the d2 repo Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] the distutils2 repo and 3to2
2011/5/23 Łukasz Langa luk...@langa.pl: ... I've heard you're targetting 2.4 compatibility so be prepared that this is not going to be easy. yeah well, we might raise the bar to 2.5 and use some __future__ statements. I am not sure that keeping 2.4 support is that useful anymore. Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots update
On Mon, May 23, 2011 at 8:48 AM, Tarek Ziadé ziade.ta...@gmail.com wrote: On Mon, May 23, 2011 at 3:00 AM, Bill Janssen jans...@parc.com wrote: Tarek Ziadé ziade.ta...@gmail.com wrote: Yes, I am aware of this. I have fixed today most remaining issues, and fixing the final ones right now. Just FYI: the AMD64 Snow Leopard buildbot and PPC Leopard buildbots are now green, but the PPC Tiger buildbot is still failing for all branches because of packaging errors: All the linux and windows stable slaves are now green, and I have a few issues left to be fixed for all solaris flavors and the two you are showing, that are also failing under Free BSD. I have now completed the cleanup and we're back on green-land for the stable bots. The red slaves should get green when they catch up with the latest rev (they are slow). If they're not and they are failing in packaging or sysconfig let me know. Sorry again if it has taken so long. Setting up Solaris and BSD VMs took some time ;) Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots update
On Sat, May 21, 2011 at 4:37 PM, Antoine Pitrou solip...@pitrou.net wrote: Hello, We recently got a couple of new stable buildbots: - R. David Murray's x86 Gentoo machine, which builds in non-debug mode and therefore checks that release Pythons work fine - Stefan Krah's AMD64 FreeBSD 8.2 machine - Bill Janssen's AMD64 Snow Leopard machine Many stable buildbots on the default branch (*) are currently red because of test_packaging issues. (*) http://www.python.org/dev/buildbot/all/waterfall?category=3.x.stable Yes, I am aware of this. I have fixed today most remaining issues, and fixing the final ones right now. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging landed in stdlib
On Thu, May 19, 2011 at 8:30 PM, Georg Brandl g.bra...@gmx.net wrote: On 19.05.2011 13:35, Tarek Ziadé wrote: Hey I've pushed packaging in stdlib. There are a few buildbots errors we're fixing right now. We will continue our work in their directly for now on. Rock on! Thanks :) Still working on some issues under windows and solaris bbots today, but we're getting there. Sorry for the inconvenience Tarek Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] packaging landed in stdlib
Hey I've pushed packaging in stdlib. There are a few buildbots errors we're fixing right now. We will continue our work in their directly for now on. The next big commit will be for the documentation, Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging landed in stdlib
On Thu, May 19, 2011 at 1:35 PM, Tarek Ziadé ziade.ta...@gmail.com wrote: Hey I've pushed packaging in stdlib. There are a few buildbots errors we're fixing right now. FYI. there are still some failures we're fixing. Thanks for your patience and thanks to the folks that are helping me on this :) I expect the bbots to be back on track later today Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] packaging merge imminent
Hello I am about to merge packaging in the stdlib, and we will continue our work there :) The impact is: - addition of Lib/packaging - addition of test/test_packaging.py - changes in Lib/sysconfig.py - addition of Lib/sysconfig.cfg For the last one, I would like to make sure again that everyone is ok with having a .cfg file added in the Lib/ directory. If not, we need to discuss how to do this differently. == purpose of sysconfig.cfg == The sysconfig.cfg file is a ini-like file that sysconfig.py reads to get the installation paths. We currently have these paths harcoded in the python module. The next change I have planned is to allow several levels of configuration, like distutils.cfg does. sysconfig.py will look for a sysconfig.cfg file in these places: 1. the current working directory -- so can be potentially included in a project source release 2. the user home (specific location be defined, maybe in ~/local) [inherits from the previous one] 3. the global [inherits from the previous one] I have decided to make it a .cfg file instead of a .py file for various reasons: - easier for people to edit, without the danger of ending-up with an over-engineered python module (that's the problem we have with setup.py files) - the override logic is easier to implement and understand: if I want to change a single path, I add a ini file in my home with this single path. If I have no complains, the merge will happen tomorrow of my time == next moves == - make sysconfig.py stop reading Makefile and pyconfig.h, this will be done by adding a _sysconfig.py file created by the Makefile - continue our work in packaging for 3.3 - we're planning to merge the doc in Doc/packaging very soon (still working on it) Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging merge imminent
On Tue, May 17, 2011 at 7:21 PM, Ronald Oussoren ronaldousso...@mac.com wrote: ... 1. the current working directory -- so can be potentially included in a project source release Does this mean that python behaves differently when there happens to be a sysconfig.cfg file in the current working directory? That's a potentional security risk. The use case is to have it there at install time so packaging can have alternative locations if needed. We could also drop the working dir scanning and have it: 1- passed explicitly to the pysetup script via an option. 2- used only if found in a root of a project at installation time, and only then 2. the user home (specific location be defined, maybe in ~/local) [inherits from the previous one] How hard would it be to disable this behavior for tools like virtualenv and py2app? Not hard at all, just an option. And the goal is also to allow virtualenv to have its own copy, like it does for distutils.cfg Ronald -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging merge imminent
On Tue, May 17, 2011 at 10:40 PM, Victor Stinner victor.stin...@haypocalc.com wrote: Le mardi 17 mai 2011 à 17:36 +0200, Tarek Ziadé a écrit : - addition of Lib/packaging - addition of test/test_packaging.py - changes in Lib/sysconfig.py - addition of Lib/sysconfig.cfg Does setup.py continue to use the old distutils module? Yes. The plan is to keep distutils support, so projects with setup.py still work. For the new packaging, people will have to provide new sections in setup.cfg. The pysetup script will detect at installation time if the project has the required bits in the cfg, and if not will fallback to executing setup.py I fixed recently some bugs in distutils. Should I also fix them in the packaging module, or are both modules already synchronized? They need to be backported yes. We did some, but we'll need to double check distutils timeline to make sure it's synced Victor -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging merge imminent
On Tue, May 17, 2011 at 6:42 PM, Christian Heimes li...@cheimes.de wrote: Am 17.05.2011 17:36, schrieb Tarek Ziadé: The next change I have planned is to allow several levels of configuration, like distutils.cfg does. sysconfig.py will look for a sysconfig.cfg file in these places: 1. the current working directory -- so can be potentially included in a project source release 2. the user home (specific location be defined, maybe in ~/local) [inherits from the previous one] 3. the global You may want to study my site package PEP [1] regarding possible security implications. I recommend that you ignore the current working directory and user's home directory under conditions like different effective user or the -E option. Sounds good, thanks A good place for a local sysconfig.cfg could be the user's stdlib directory (e.g. ~/.local/lib/python3.2/sysconfig.cfg). Yes, so, part of the packaging imcoming work will be to relocate all user .cfg files in the same, python-specific place. That includes pydistutils.cfg, and pypirc. I remember we did talk about that a few months ago, and will restart this discussion asap Christian [1] http://www.python.org/dev/peps/pep-0370 -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging merge imminent
On Wed, May 18, 2011 at 1:25 AM, Ned Deily n...@acm.org wrote: ... Just to be clear: what about for the build of the interpreter itself, i.e. its setup.py for the standard library extension modules? Will the existing distutils code continue to be used for that? Or is it being replaced by code in packaging? If so, have Python builds been tested yet on the various platforms? It will remain distutils-based for now. Moving it to packaging is not our top priority. Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 386 and dev repository versions workflow
On Mon, May 2, 2011 at 7:14 PM, anatoly techtonik techto...@gmail.com wrote: http://guide.python-distribute.org/quickstart.html proposes suffixing version of a module in repository with 'dev' in a way that after release of '1.0' version, the repository version is changed to '2.0dev'. This makes sense, but it is not compatible with PEP 386, which suggests using 2.0.devN, where N is a repository revision number. I'd expand PEP 386 to include 2.0dev use case. This is a typo I'll fix, thanks for noticing -- anatoly t. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] the role of assert in the standard library ?
Hello
I removed some assert calls in distutils some time ago because the
package was not behaving correctly when people were using Python with
the --optimize flag. In other words, assert became a full part of the
code logic and removing them via -O was changing the behavior.
In my opinion assert should be avoided completely anywhere else than
in the tests. If this is a wrong statement, please let me know why :)
So, I grepped the stdlib for assert calls, and I have found 177 of
them and many of them are making Python acts differently depending on
the -O flag,
Here's an example on a randomly picked assert in the threading module:
import threading
class test(threading.Thread):
def __init__(self):
self.bla = 1
def run(self):
print('running')
t = test()
print(t)
The __repr__ method is not behaving the same way depending on the O flag:
$ python3 -O test.py
Traceback (most recent call last):
File test.py, line 12, in module
print(t)
File /usr/local/lib/python3.2/threading.py, line 652, in __repr__
if self._started.is_set():
AttributeError: 'test' object has no attribute '_started'
$ python3 test.py
Traceback (most recent call last):
File test.py, line 12, in module
print(t)
File /usr/local/lib/python3.2/threading.py, line 650, in __repr__
assert self._initialized, Thread.__init__() was not called
AttributeError: 'test' object has no attribute '_initialized'
$ python test.py
Traceback (most recent call last):
File test.py, line 12, in module
print(t)
File /usr/lib/python2.6/threading.py, line 451, in __repr__
assert self.__initialized, Thread.__init__() was not called
AssertionError: Thread.__init__() was not called
--- oops different error
$ python -O test.py
Traceback (most recent call last):
File test.py, line 12, in module
print(t)
File /usr/lib/python2.6/threading.py, line 453, in __repr__
if self.__started.is_set():
AttributeError: 'test' object has no attribute '_Thread__started'
I have seen some other places where thing would simply break with -O.
Am I right thinking we should do a pass on those and remove them or
turn them into exception that are triggered with -O as well ?
This flag is meant to optimize generated bytecode slightly, but I am
not sure this involves also slightly changing the way the code behaves
Cheers
Tarek
--
Tarek Ziadé | http://ziade.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] the role of assert in the standard library ?
On Thu, Apr 28, 2011 at 12:27 PM, Michael Foord fuzzy...@voidspace.org.uk wrote: On 28/04/2011 09:34, Terry Reedy wrote: On 4/28/2011 3:54 AM, Tarek Ziadé wrote: Hello I removed some assert calls in distutils some time ago because the package was not behaving correctly when people were using Python with the --optimize flag. In other words, assert became a full part of the code logic and removing them via -O was changing the behavior. In my opinion assert should be avoided completely anywhere else than in the tests. If this is a wrong statement, please let me know why :) My understanding is that assert can be used in production code but only to catch logic errors by testing supposed invariants or postconditions. It should not be used to test usage errors, including preconditions. In other words, assert presence or absence should not affect behavior unless the code has a bug. Agreed. We should ideally have buildbots doing test runs with -O and -OO. R. David Murray did a lot of work a year ago (or so) to ensure the test run passes with -OO but it easily degrades.. There are a couple of asserts in unittest (for test discovery) but I only use them to provide failure messages early. The functionality is unchanged (and tests still pass) with -OO. All the best, I'll try to add a useful report on bad asserts in the bug tracker. I am replying again to this on Python-ideas because I want to debate on assert :) Cheers Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] the role of assert in the standard library ?
On Thu, Apr 28, 2011 at 5:26 PM, Barry Warsaw ba...@python.org wrote: On Apr 28, 2011, at 10:22 AM, s...@pobox.com wrote: Barry I would agree. Use asserts for this can't possibly happen Barry wink conditions. Without looking, I suspect that's probably what the author thought he was doing. BTW, I think it always helps to have a really good assert message, and/or a leading comment to explain *why* that condition can't possibly happen. why bother, it can't happen ;) -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Core Mentorship program
On Fri, Mar 25, 2011 at 2:12 PM, s...@pobox.com wrote: Boggle. Jesse I assume that means your in, or you hate that idea? Or that he just really likes to play Boggle. :-) Or that he's confused ? https://secure.wikimedia.org/wiktionary/en/wiki/mindboggling S ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging
On Tue, Mar 15, 2011 at 7:50 AM, Nick Coghlan ncogh...@gmail.com wrote: On Tue, Mar 15, 2011 at 10:27 AM, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Maybe this will be the killer app for the or enhancement to the import statement. :-) Except that won't help, since even if it were added right now, pre-3.3 compatible code couldn't use it :) or if you backport it, we could add a new fallback ;) try: from __future__ import or_importer except ImportError: XXX -- previous proposal else: or based proposal Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] packaging
Hey, I just wanted to summarize what we've started at the sprint (and hopefully finish 1 to 7 this week): 1/ distutils2 is merged as the packaging Python package in the standard library 2/ distutils2 will provide a pysetup script in Python to run all packaging tools (pysetup is a wrapper that will run packaging.run.main) 3/ pkgutil gets the new API (PEP 376) we implemented in distutils2._backport 4/ sysconfig installations paths are moved to a sysconfig.cfg file. 5/ the sysconfig,cfg file will be located besides sysconfig.py in the standard library 6/ sysconfig will lookup for sysconfig.cfg in several places and merge sections from down to bottom: 1/ current directory 2/ per-user site-packages 3/ global site-packages 7/ a backport for 2.4 to 3.2 will be provided for packaging using the distutils2 name 8/ we will release distutils2 in the next 18 months 9/ once 3.3 is out, the backport will just get bug fixes Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging
On Mon, Mar 14, 2011 at 4:46 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 14 Mar 2011 15:39:55 -0400 Tarek Ziadé ziade.ta...@gmail.com wrote: Hey, I just wanted to summarize what we've started at the sprint (and hopefully finish 1 to 7 this week): 1/ distutils2 is merged as the packaging Python package in the standard library Why does it get yet another name? We already have distutils, setuptools, distribute, distutils2... now packaging? For it makes sense to have a packaging namespace in the standard library, where we'll be able to add more modules, features etc. Distutils becomes a misnomer now that we'll have features like: - display the list of installed projects - search projects at PyPI - ... And it's also a good way to prevent any conflict with 3.3 : the standalone version for 2.4 to 3.2 is distutils2, and people won't have to deal with the same package being in the stdlib and at PyPI. (like json vs simplejson, unittest vs unittest2...) Cheers Tarek Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] packaging
Setup.py is gone in distutils2 and therefore in packaging Le 14 mars 2011 18:27, Antoine Pitrou solip...@pitrou.net a écrit : On Mon, 14 Mar 2011 18:00:50 -0400 Tarek Ziadé ziade.ta...@gmail.com wrote: And it's also a good way to prevent any conflict with 3.3 : the standalone version for 2.4 to 3.2 is distutils2, and people won't have to deal with the same package being in the stdlib and at PyPI. (like json vs simplejson, unittest vs unittest2...) But doesn't it also mean many setup.py scripts will have very tedious import sequences, such as: try: from packaging.compiler import FooCompiler from packaging.commands import BarCommand except ImportError: try: from distutils2.compiler import FooCompiler from distutils2.commands import BarCommand except ImportError: try: from setuptools.compiler import FooCompiler from setuptools.commands import OtherNameForBarCommand as \ BarCommand except ImportError: from distutils.compiler import FooCompiler from distutils.commands import OtherNameForBarCommand as \ BarCommand (I'm still remembering the import dances which were necessary to get cElementTree/ElementTree in the 2.4-2.5 days) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Fwd: Re: packaging
Sorry i am not used to my new phone .. fwding to python-dev -- Message transféré -- De : Tarek Ziadé ziade.ta...@gmail.com Date : 14 mars 2011 19:12 Objet : Re: [Python-Dev] packaging À : Paul Moore p.f.mo...@gmail.com That document is from the latest release at pypi. You can find a fresher doc in docs in the hg repo. There's also a rendering at http://distutils2.notmyidea.org which is more recent. One task during the sprint is to consolidate all the doc ;) Le 14 mars 2011 19:03, Paul Moore p.f.mo...@gmail.com a écrit : On 14 March 2011 22:34, Tarek Ziadé ziade.ta...@gmail.com wrote: Setup.py is gone in distutils2 and therefore in packaging Where can I find the documentation? The distutils2 docs (A simple example) still use setup.py. See http://packages.python.org/Distutils2/distutils/introduction.html#a-simple-example Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 next steps
On Mon, Feb 21, 2011 at 8:48 AM, Georg Brandl g.bra...@gmx.net wrote: On 21.02.2011 08:41, Tarek Ziadé wrote: Hello Now that Python 3.2 is out, I am planning to do the following with Distutils2: 1 - release a new alpha before Pycon for community feedback 2 - add distutils2 back in the trunk, along with the changes in pkgutil and sysconfig 3 - continue the ongoing work in Distutils2 to prepare the first Python 3.3 release If you want me to give more details here on what is going to be done precisely in the various stdlib parts, let me know. I think I'm also speaking for the prospective release manager of 3.3 by saying yes, please, details would be nice, but not necessarily *right* now. :) It's easy enough to give right now: - pkgutil will gain all the API that are implementing PEP 376 -- a py2 version of this module that has them can be seen at http://hg.python.org/distutils2/file/tip/distutils2/_backport/pkgutil.py for the moment - sysconfig will need to have two things: - create via the Makefile a _sysconfig module, so the API stop to read data dynamically in pyconfig.h and Makefile at runtime - [to be validated-discussed] an easier way to configure the installation paths. the current plan is to use a sysconfig.cfg file that may be changed. This file will be global to Python, but also possibly local to a user or a project -- http://hg.python.org/distutils2/file/tip/distutils2/_backport/sysconfig.cfg - distutils2 will be converted to Py3 using 2to3, then included in Lib/ -- the code base is ready for this, besides a few spots. - distutils2 will continue to be released as a standalone project from 2.4 to 3.2. Probably by using 3to2, but I have not tried the tool yet. Cheers Tarek Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 next steps
On Mon, Feb 21, 2011 at 7:22 PM, Brett Cannon br...@python.org wrote: ... - distutils2 will continue to be released as a standalone project from 2.4 to 3.2. Probably by using 3to2, but I have not tried the tool yet. So does this mean that primary development will move to py3k and then you will simply push changes downstream to the distutils2 project for separate releases? Or are you going to go the reverse route? I will backport from py3k to the standalone version, yes. I think it will be better because the changes will be exposed to more people in the Python trunk, and maybe get more feedback before pushing downstream Cheers Tarek -Brett Cheers Tarek Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Distutils2 next steps
Hello Now that Python 3.2 is out, I am planning to do the following with Distutils2: 1 - release a new alpha before Pycon for community feedback 2 - add distutils2 back in the trunk, along with the changes in pkgutil and sysconfig 3 - continue the ongoing work in Distutils2 to prepare the first Python 3.3 release If you want me to give more details here on what is going to be done precisely in the various stdlib parts, let me know. I plan to do 2. as a sprint task, and will be looking for help from other core devs, for reviews, advices etc. So if anyone is interested and present at Pycon, please add your name at http://us.pycon.org/2011/sprints/projects under the Disttuils2 project. Thanks ! Cheers Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
On Sun, Dec 12, 2010 at 3:05 PM, Barry Warsaw ba...@python.org wrote: On Dec 12, 2010, at 02:42 PM, Antoine Pitrou wrote: On Sun, 12 Dec 2010 13:01:42 +0100 Łukasz Langa luk...@langa.pl wrote: Wiadomość napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18: *(I sometimes lose track of which changes were made in both branches pre-2.7, which ones were mode post-2.7 release, and which ones went in pre-2.7, but were 3.x only regardless) Right. I missed that it was already in 2.7. So, now we're stuck with it, forever. Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere anyway, even in 2.7) and introduce sys.sysconfig. We already had a lot of churn around these APIs (distutils friends). I don't think it's a good idea to introduce even more churn. Oh and by the way it's too late to add or remove features from 3.2. I really like that much better than Java-like accessor functions. Do you actually use sysconfig yourself? It's quite a specialized module, and I don't think API elegance arguments have a great weight here. I have used them and I do think they're fairly ugly and unwieldy. However, I agree that we should not rush into a different design. Since sysconfig was essentially refactored out of distutils (and now we have two ways to do it!), and we're getting distutils2 for 3.3, I think it would be better to work out a more natural design for sysconfig for 3.3. Then the sysconfig module can go through the natural deprecation cycle. I don't think any API refactoring worth the pain here. I don't see the proposed changes make the caller's code that much better. The existing one is good enough in my opinion. Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
Sorry to get late in the discussion I am travelling. Nick resumes well the motivations behind sysconfig. I'll emphase that this module could hold more functions in the future that could be useful to other python implementations to abstract what is a python installation. E.g. more than paths and variables. On Dec 11, 2010 3:53 AM, Nick Coghlan ncogh...@gmail.com wrote: On Sat, Dec 11, 2010 at 7:59 AM, R. David Murray rdmur...@bitdance.com wrote: On Thu, 09 Dec 201... ISTM it mostly could have been reduced to single call returning a nested dictionary. If what ... We've reverted from beta to pseudo-alpha after discovering sufficiently egregious mistakes in the past. (we didn't *actually* revert the build naming because release IDs that don't increase monotonically can confuse matters). However, I don't think this is sufficiently egregious to qualify. My own reasons for advocating retention of the separate module: 1. Something equivalent to the current sysconfig module is going to be needed to actually work out the contents of the data structure. Rewriting all that in C would be silly, so the most that could happen is to rename sysconfig to _sysconfig. 2. Invoking Python code from sys module functions is possible but untidy (generally involving implicit module imports). Since we need a module anyway for implementation purposes, why not make it public? 3. The sysconfig module docs provide a nicer space to talk about the purpose of the provided information rather than squirreling it away in a corner of the sys module docs 4. python -m sysconfig provides a useful dump of the system status 5. The API design did not get much *new* thought, as the idea was to allow users to do anything that the old distutils.sysconfig could do, but without requiring them to depend on distutils to get that functionality. The easiest way to ensure that is to just copy the existing API rather than trying to get creative. Don't think of it as adding a whole new module, thinking of it as decoupling a relatively small existing module (i.e. distutils.sysconfig) from a larger package that isn't installed by default on all systems (i.e. distutils). A concrete problem with Raymond's idea in particular is that there is actually a full set of paths defined for *each scheme*. get_path() and get_paths() let you ignore this most of the time, since there is an appropriate default scheme defined for each platform, so they just use that if you don't specify one explicitly. A named tuple could obviously expose the paths for all of the schemes, with the paths for the default scheme duplicated at the top level, but the data structure starts looking very messy at that point. The named tuple idea also conflates meta-data about the sysconfig data (the list of scheme names and path names) with the actual data for the current platform (platform, version, paths, config_vars). So I think reusing the tried and tested API is easily justified in this case. What may be desirable in 3.3 (particularly with the distutils2 reintegration taking place), is to revisit the sysconfig API to see if there are better ways to expose some of the data (such as a function returning a named tuple containing the raw data underlying the output of python -m sysconfig). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ... Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Uns... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
On Sat, Dec 4, 2010 at 12:27 PM, Antoine Pitrou solip...@pitrou.net wrote: On Fri, 3 Dec 2010 18:11:57 -0500 James Y Knight f...@fuhm.net wrote: On Dec 3, 2010, at 5:52 PM, Martin v. Löwis wrote: Am 03.12.2010 23:48, schrieb Éric Araujo: But I'm not interested at all in having it in distutils2. I want the Python build itself to use it, and alas, I can't because of the freeze. You can’t in 3.2, true. Neither can you in 3.1, or any previous version. If you implement it in distutils2, you have very good chances to get it for 3.3. Isn’t that a win? It is, unfortunately, a very weak promise. Until distutils2 is integrated in Python, I probably won't spend any time on it. It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then. +1 from me. +1 too. Just to clarify my position in a few sentences: - I was told not to progressively change distutils - the PEPs for my changes were accepted under the condition that the changes would be made in distutils2 - PEP 384 is yet another accepted PEP, why the rule would change - because it's a small feature ? - I have tons of small features I had to revert and push back in distutils2 If Martin changes make it into Distutils1, I am going to ask that a collection of small features that I have should go there too, because they should not break things Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Change to the Distutils / Distutils2 workflow
Hey We discussed with Eric about the debugging workflow and we agreed that our life would be easier if every bug fix would land first in Distutils2 when it makes sense, then get backported to Distutils1. For other core-devs that would mean that your patches should be done against hg.python.org/distutils2, which uses unittest2. Then Eric and I would take care of the backporting. I am planning to set up a wiki page with the workflow as soon as I get a chance. Thanks Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
On Thu, Dec 2, 2010 at 9:24 PM, Martin v. Löwis mar...@v.loewis.de wrote: Since discussion has trailed off without any blocking objections, I'm accepting PEP 384. Martin, you may mark the PEP accepted and proceed with merging the implementation for the beta on Saturday. Thanks! will do (I'll also take into consideration the proposed changes). I did not get an answer to my last mail about distutils / distutils2 Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
2010/12/2 Martin v. Löwis mar...@v.loewis.de: Am 02.12.2010 21:48, schrieb Tarek Ziadé: On Thu, Dec 2, 2010 at 9:24 PM, Martin v. Löwis mar...@v.loewis.de wrote: Since discussion has trailed off without any blocking objections, I'm accepting PEP 384. Martin, you may mark the PEP accepted and proceed with merging the implementation for the beta on Saturday. Thanks! will do (I'll also take into consideration the proposed changes). I did not get an answer to my last mail about distutils / distutils2 What was the question again, and whom did you want an answer from? You can read it in the archives here: http://mail.python.org/pipermail/python-dev/2010-November/106138.html tldr: The question was Why not implementing this in Distutils2 ? Your answer was No, PEP 3149 was accepted, I will do this in Distutils1 My answer was Having an accepted PEP does not imply your code lands in the sdtlib (like PEP 376 and 345) So the question still stands: Why not implementing this in Distutils2 ? Regards Tarek Regards, Martin -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
2010/12/2 Martin v. Löwis mar...@v.loewis.de: So the question still stands: Why not implementing this in Distutils2 ? Because it then wouldn't be available in Python 3.2, which is the target release of the PEP. The exact feature I am mentioning is the ability to compile extensions with new options, so I am not sure which PEP is involved since distutils changes refer to PEP 384 in the other PEP. I was told not to touch to Distutils code to avoid any regression since it's patched to the bones in third party products. So we decided to freeze distutils and add all new features in Distutils2, which is at alpha stage now. So this move seems contradictory to me. Grouping all new features in the new version and keep Distutils1 in maintenance mode seems to make more sense to me, if we want to make Distutils die and push forward Distutils2 for its new features etc. Or we might get back into backward hell again :) So, I am +1 on a patch on distutils2 and -1 on de-freezing Distutils for any new feature. If that really causes too much pain, I'll refrain from making any changes to distutils; PEP 384 doesn't specify any changes, anyway. That would be awesome, and we can work on a patch for distutils2 to provide that abi option. Regards, Martin -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
2010/12/2 Martin v. Löwis mar...@v.loewis.de: I was told not to touch to Distutils code to avoid any regression since it's patched to the bones in third party products. So we decided to freeze distutils and add all new features in Distutils2, which is at alpha stage now. So this move seems contradictory to me. I think it was a bad decision to freeze distutils, and we certainly didn't make that (not any we that includes me, that is). We is the people at the last language summit. Sorry if I used such a vague word. This freeze made the situation worse. Can you extend on this and explains why it makes it worse ? If we (as you included) don't agree it's the best solution, I would not want to be pushed back to square one at the next summit.. I happily reverted all my changes last year when asked, and started to work on Distutils2. But I'll get out of steam if the direction changes again, with you stating that it makes the situation worse. IIRC, it was really the incompatible changes that made people ask you to stop changing distutils. Who is people ? Are you suggesting that we could have added all the new features in Distutils in the stdlib ? The decision was because we had a mix of: - incompatible changes in private parts -- and some packages where patching distutils internals - changes on public APIs behavior, whith a behavior that was not clearly documented and suggest to interpretation - some mistakes I made as well But that's what you would expect for a project that needs to evolve a lot. Thus the freezing. So how would you make the situation better, if not by doing the work in distutils2 ? Regards, Martin -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
2010/12/2 Martin v. Löwis mar...@v.loewis.de: Am 02.12.2010 22:54, schrieb Michael Foord: On 02/12/2010 21:39, Martin v. Löwis wrote: I was told not to touch to Distutils code to avoid any regression since it's patched to the bones in third party products. So we decided to freeze distutils and add all new features in Distutils2, which is at alpha stage now. So this move seems contradictory to me. I think it was a bad decision to freeze distutils, and we certainly didn't make that (not any we that includes me, that is). This freeze made the situation worse. What situation worse? The distutils is unmaintained situation. It's not only unmaintained now, but proposed improvements are rejected without consideration, on the grounds that they are changes. I welcome those changes in Distutils2. That's the whole point. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
2010/12/2 Martin v. Löwis mar...@v.loewis.de: This freeze made the situation worse. Can you extend on this and explains why it makes it worse ? Before the freeze, distutils was unmaintained (i.e. before you started maintaining it), but people who want to improve it gradually atleast could. Now gradual improvements are also banned, so it's not only unmaintained, but I can't even provide support for the PEP in Python that was just accepted. IIRC, it was really the incompatible changes that made people ask you to stop changing distutils. Who is people ? Are you suggesting that we could have added all the new features in Distutils in the stdlib ? No, only the ones that didn't cause backwards incompatibilities, and broke existing packages. This is impossible. I can point you to some third party project that can break if you touch some distutils internals, like setuptools. Setuptools also uses some privates global variables in some other modules in the stdlib FYI. The right answer was maybe back then: make setuptools and other projects evolve with distutils. But it did not happen. So we left the status quo and moved forward in distutils2. Because we knew distutils needed deeper changes anyways, and we knew setuptools was used everywhere and unfortunately not evolving at the same pace. (note: I am not blaming PJE or anyone when I say this -- the way distutils worked and was poorly maintained was the main reason) But that's what you would expect for a project that needs to evolve a lot. Thus the freezing. Instead of evolving a lot, and instead of freezing, I would have preferred evolve a little. So how would you make the situation better, if not by doing the work in distutils2 ? Lift the freeze. I'm all for replacing distutils with distutils2, but I'm not sure whether you will declare distutils2 ready tomorrow, next year, or ten years from now. Depends on what ready means. If by ready you mean it can be used to replace Distutils1 in a project, I declare Distutils2 ready for usage NOW. It's in alpha stage. I want a solid beta before Pycon. I would even remove Distutils from 3.x altogether at some point since setuptools is not Python 3 compatible, and just put distutils2. 3.3 sounds like a good target. Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
2010/12/2 Martin v. Löwis mar...@v.loewis.de: The distutils is unmaintained situation. It's not only unmaintained now, but proposed improvements are rejected without consideration, on the grounds that they are changes. I welcome those changes in Distutils2. That's the whole point. That would be useful if there was a clear vision of when distutils2 will be released. Please understand that I'm not blaming you for not releasing it (it *is* too much for a single person), but please understand that it's also not helpful to submit changes to a codebase that is not going to be released in a foreseeable future. I know you're not blaming me. Distutils 2 alpha3 is currently released and available at PyPI. I use it in some of my professional projects FWIW. alpha4 was postponed but should be out this month. It contains major features, people from the GSOC worked on. The initial roadmap was to have a final by the time 3.2 final is out, but that'll be too short. So the target is to have a beta release for Pycon, and to sync the final release with 3.3, with lots of feedback in the meantime hopefully, and people using it from 2.4 onward. Regards, Martin -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
2010/12/2 Martin v. Löwis mar...@v.loewis.de: No, only the ones that didn't cause backwards incompatibilities, and broke existing packages. This is impossible. I can point you to some third party project that can break if you touch some distutils internals, like setuptools. Setuptools also uses some privates global variables in some other modules in the stdlib FYI. So what would break if Extension accepted an abi= keyword parameter? I suppose you have code behind this, that will be in build_ext and in the compilers. So you will need to try out ALL projects out there that customize build_ext, like numpy or setuptools, etc, But you won't be able to try out all projects because they are not listed somewhere. For starters, the Extension class is replaced by another one in setuptools, that patches the constructor if Pyrex is installed, which is unlikely I guess, so no big deal. But you will also get a replaced version of the Distribution class that uses a private method from distutils, and another version of build_ext with custom compiling flags. Now depending on how you do your thing it could work if you are careful at doing things on the top of setuptools. And then, if numpy.distutils is installed, it relies on distutils build_ext and tries to rely on setuptools one's too, so it gets in the mix of the patched classes, and you get an horrible mix and possible bad interactions. So I am not saying it's impossible to add the feature, but it is impossible to be sure nothing gets broken in third party. So the freeze seems wise indeed Lift the freeze. I'm all for replacing distutils with distutils2, but I'm not sure whether you will declare distutils2 ready tomorrow, next year, or ten years from now. Depends on what ready means. Included in Python, so that changes become possible again. If by ready you mean it can be used to replace Distutils1 in a project, I declare Distutils2 ready for usage NOW. It's in alpha stage. I want a solid beta before Pycon. I would even remove Distutils from 3.x altogether at some point since setuptools is not Python 3 compatible, and just put distutils2. 3.3 sounds like a good target. So will distuils2 be released before that? If so, when? An alpha is already released. A beta will be released for Pycon (I need it for my talk :) ) Then hopefully the final before 3.2 Regards, Martin -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
On Fri, Dec 3, 2010 at 12:01 AM, Martin v. Löwis mar...@v.loewis.de wrote: I think distutils is simply a bugfix branch for distutils2. Similarly as how we don't commit improvements in e.g. 2.7 or 3.1, neither do we commit improvements to distutils. It's different, though, in the sense that Python has a release schedule and multiple committers working on it, and that it normally gets released even if some changes don't get included in a specific release yet. All this seems not to be true for distutils2. We have 3 or 4 regular contributors. That's not a lot for sure. So my motivation to contribute changes to it is *much* lower than my desire to contribute to distutils, and it is also provably lower than my motivation to contribute to distribute (say). I'm just getting tired having to talk to five projects just to make a single change to the build infrastructure available to the Python community. I am not trying to motivate you to contribute to Distutils2. I am trying to make sure we are all on the same page for what's good for Python. So if we work in Distutils2 and you work in Distutils saying publicly that you don't want to contribute to Distutils2, that's a total nonsense. We took some decisions, and you want to go against them. So I want to have a consensus here for the packaging eco-system and make sure we are still on track. I am sorry if you get tired of it, but I don't want to be told at the next summit: sorry Tarek, now we need to do changes little by little in distutils1 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 291 versus Python 3
On Tue, Nov 30, 2010 at 7:33 AM, Éric Araujo mer...@netwok.org wrote: Good morning python-dev, PEP 291 (Backward Compatibility for Standard Library) does not seem to take Python 3 into account. Is this PEP only relevant for the 2.7 branch?* If it’s supposed to apply to 3.x too, despite the view that 3.0 was a clean break, what does it mean to have a module that is developed in the py3k branch and should retain compatibility with 2.3 or 1.5.2? * Tarek’s interpretation: “The 2.x needs to stay 2.3 compatible so we should keep the 3.x as similar as possible for bugfixes.” In the particular case of distutils (should be compatible with 2.3), we (including I) have been lax. Our tests for example use modern unittest features like skips, which makes them not runnable on old Pythons. I am very uncomfortable with code that seems to run fine but which tests (however few) cannot be run, so I think I’ll have to trade the skips for old-style “return” statements. You shouldn't be uncomfortable with the current state of distutils and try to improve its tests (or improve any other nasty stuff you'll find in that code) Distutils is dead code. All we have to do is the bare minimum maintenance. Everything else is a waste of time. The other way of solving that is to change the compat policy. If I remember correctly, the rationale for code compat in distutils is that people may copy distutils from Python x.y to their install of x.y-n; I don’t know if this is still an active practice, and if it is, I don’t know if it should be supported, considering that distutils2 (compatible with 2.4+ and available from PyPI) is coming. Again, don't worry about these rules in Distutils now. The only rule that now apply to Distutils is that we do only bug fixing, and we should not waste our precious time to do other stuff in there. Plain python tests are fine for what we want to do and simplify our forward ports and backports. One thing we should do though, is fix those bugs in Distutils2 first when they exist there too. I really appreciate all the hard work your are doing in triaging the issues and bug fixing by the way ! Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 final review
On Mon, Nov 29, 2010 at 11:24 AM, Matthias Klose d...@ubuntu.com wrote: On 29.11.2010 00:40, Martin v. Löwis wrote: I have now completed http://www.python.org/dev/peps/pep-0384/ Benjamin has volunteered to rule on this PEP. Please comment with any changes you want to see, or speak in favor or against this PEP. I looked at a diff with r84330 from the py3k branch. Extensions built with Py_LIMITED_API have the python version encoded in it's name. Which abi name should be used for these extensions? .. - Should the distutils support for LIMITED_API be part of the pep, or be implemented later? In any case, it has to be implemented in Distutils2, not in Distutils. Distutils is frozen and just in maintenance mode. Once Distutils2 final is released (it's currently in alpha), it will be installable from 2.4 to 3.x and can provide this feature. For Python itself we can backport the feature in its setup.py, until Distutils2 is back to the sdtlib In favour of the pep. +1 Matthias -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 final review
2010/11/29 Martin v. Löwis mar...@v.loewis.de: - Should the distutils support for LIMITED_API be part of the pep, or be implemented later? In any case, it has to be implemented in Distutils2, not in Distutils. Distutils is frozen and just in maintenance mode. I think it's too late for that. PEP 3149 is accepted, and it does specify a change to distutils (namely, the abi= parameter). ISTM that an approved PEP will override the distutils code freeze. Having an accepted PEP does not imply that it should be implemented in the standard library. For instance PEP 345 and PEP 376 are accepted but implemented in Distutils2. it's also a: - good opportunity to boost Distutils2 adoption - way to get feedback from people for that abi= option and have the chance to correct any design issue before d2 is added in the sdtlib For Python itself we can backport the feature in its setup.py, until Distutils2 is back to the sdtlib This won't be for python itself, but for extension modules. ok. Regards, Martin -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Reminder: Distutils vs Distutils2
Hello, I have seen some efforts recently to improve Distutils in the standard library, Just a quick reminder of the status of Distutils: it's frozen and is just being bug fixed at this time. The work I done last year was reverted and pushed to Distutils2. A lot of work has been done since then, and we had 4 GSOC students working this summer on Distutils2. It's backward-incompatible, so we can remove the things we don't like and add new things w/o suffering from backward compatibility pains. So if you want to improve the tool, or if you have some pending changes to Distutils, I would encourage you to join the Distutils2 effort and not to waste time on Distutils anymore. The patches that did not make it to Distutils can still be added in Distutils2, for most of them. The workflow we currently use to change the code is as follow and make it easy for everyone to contribute: 1. clone http://bitbucket.org/tarek/distutils2 2. discuss / propose a patch on IRC (#distutils - Freenode) or on the dedicated mailing list (http://groups.google.com/group/the-fellowship-of-the-packaging) 3. I review and merge all changes at bitbucket, then push them on http://hg,python.org/distutils2 Crazy ideas are welcome. setup.py is gone in d2 for instance ;) Thanks ! Regards. Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reminder: Distutils vs Distutils2
On Sun, Nov 21, 2010 at 12:05 AM, Tarek Ziadé ziade.ta...@gmail.com wrote: .. Crazy ideas are welcome. setup.py is gone in d2 for instance ;) But you can still use a similar form if you want - just to mention ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
On Tue, Oct 12, 2010 at 1:55 PM, Paul Moore p.f.mo...@gmail.com wrote: ... I would assume (am I wrong?) that the canonical way of installing modules on Windows for non-advanced users under distutils2 would still be to download and run a binary installer. Yes this won't change. Regards Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Distutils2 scripts
Hello, In the Distutils2 project, we'll have quite a few scripts that can be called via -m $ python -m distutils2.depgraph : shows a dependency graph $ python -m distutils2.install : installs a project $ python -m distutils2.run command : runs a distutils2 command etc.. The feedback I received for this is pretty clear: people want a single script that can be called directly. e.g. $ distutils2 depgraph $ distutils2 install $ distutils2 run command Fair enough, I can add that script in the project and get it installed when people install the project. I just wanted to make sure that once distutils2 is back in the stdlib, it's OK for us to add that script in the distribution. I remember that people did not want yet another script in there, so I just wanted to double-check before I take this path Thanks ! Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
On Fri, Oct 8, 2010 at 9:25 AM, Jeroen Ruigrok van der Werven
asmo...@in-nomine.org wrote:
-On [20101008 09:07], Tarek Ziadé (ziade.ta...@gmail.com) wrote:
I just wanted to make sure that once distutils2 is back in the stdlib,
it's OK for us to add that script in the distribution.
Ah, one thing that came to mind:
is this script supposed to be installed in /usr{/local}/bin? If so, how
would it look like for multiple Python distributions that are installed?
This script will be installed besides the Python executable, so it
should follow the same naming rules depending on the OSes.
e.g. a -MINOR.MINOR suffix is appended *or* each python version has
its own bin/ location etc.
--
Tarek Ziadé | http://ziade.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
On Fri, Oct 8, 2010 at 9:50 AM, Tarek Ziadé ziade.ta...@gmail.com wrote: ... e.g. a -MINOR.MINOR MAJOR.MINOR ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
On Fri, Oct 8, 2010 at 12:24 PM, Georg Brandl g.bra...@gmx.net wrote: Am 08.10.2010 09:05, schrieb Tarek Ziadé: Hello, In the Distutils2 project, we'll have quite a few scripts that can be called via -m $ python -m distutils2.depgraph : shows a dependency graph $ python -m distutils2.install : installs a project $ python -m distutils2.run command : runs a distutils2 command etc.. What happened to python setup.py action? Or is this a step towards not requiring setup.py at all? Yes, setup.py is gone and everything is driven by scripts, that read setup.cfg. There are some configurable hooks to call code when wanted bu the idea is that those are restricted to the build part of the process, so a release made at PyPI provides static metadata. Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
2010/10/8 Fred Drake fdr...@acm.org: On Fri, Oct 8, 2010 at 7:24 AM, Dirkjan Ochtman dirk...@ochtman.nl wrote: It doesn't seem very nice to have a version in the script. Can we just call it distutils? Or py-dist? If we go this route, then - make altinstall should include the version number in the name of *any* scripts it installs. - make install would then add links without the version numbers. This mirrors the name of the Python executable, so is likely as right as it's going to get (*if* we install separate scripts). Yes that what I was thinking about -- I am not too worried about this, since every Linux deals with the 'more than one python installed' case. Georg: What happened to python setup.py action? Or is this a step towards not requiring setup.py at all? I'm in favor of add a top-level setup module that can be invoked using python -m setup There will be three cases: Nice idea ! I wouldn't call it setup though, since it does many other things. I can't think of a good name yet, but I'd like such a script to express the idea that it can be used to: - query pypi - browse what's installed - install/remove projects - create releases and upload them pkg_manager ? - d2 projects without a setup.py, where this will invoke the module from the standard library, Yes ! - d2 projects with a setup.py, where the local setup.py will be invoked, and Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. Some project might want to provide both setups for backward compatibility: - a setup.py (d1) - a setup,cfg (d2 and optionally some d1 options) It's easy enough in that case to detect if the .cfg has what d2 requires (like the [metadata] section) - d1 projects, which always have a setup.py, which will be invoked. Yes This would provide the most consistent story for package users, where the only command they'll need to remember (for Python versions with the setup module) will be python -m setup ... Nice ! And I'd like to extend this idea with what we said at the last Summit. What if a project like Pip is able to replace d1 and or d2 when it comes to install a package. As long as we agree on the same command line interface, we could provide a way for pip to be called by this global script. Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
On Fri, Oct 8, 2010 at 3:28 PM, Dirkjan Ochtman dirk...@ochtman.nl wrote: On Fri, Oct 8, 2010 at 15:22, Tarek Ziadé ziade.ta...@gmail.com wrote: Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. So, sorry for backing up to this, but isn't it true that many projects do custom stuff in their setup.py that they wouldn't be able to do in setup.cfg? They will still be able to do it by using hooks. Is the goal really to make that impossible/unnecessary? In Mercurial, for example, we have a fairly large setup.py that helps us deal with many kinds of issues: incomplete Python installations on different platforms, adding packages depending on the platform, do custom stuff to compile gettext files, etc... No the goal is to stop having setup.py as the standard for getting metadata from a project. There will always be ways to call some code in the various steps. setup.py is currently called for everything : building, installing, providing metadata etc. We want to break that and make it possible for people to express their metadata into a static file, and offer ways to express them differently depending on the platforms. See PEP 345. So you should have what you need. Now, we are still in early stages in providing these features, and I'd be happy to work on the Mercurial setup.py conversion, if it's a complex use case. So we can see how things go. Tarek Cheers, Dirkjan -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
2010/10/8 Michael Foord fuzzy...@voidspace.org.uk: On 08/10/2010 14:28, Dirkjan Ochtman wrote: On Fri, Oct 8, 2010 at 15:22, Tarek Ziadéziade.ta...@gmail.com wrote: Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. So, sorry for backing up to this, but isn't it true that many projects do custom stuff in their setup.py that they wouldn't be able to do in setup.cfg? Is the goal really to make that impossible/unnecessary? The goal is to make it unnecessary. My understanding is that it will still be possible to use a setup.py, just unnecessary for the vast majority of cases. Yes. And to make it possible to keep a d1 setup.py, which works with all installers out there, we decided that d2 would ignore that file, and provide other ways to hook code if necessary. IOW one project may provide a d1 setup.py and a d2 configuration in the same release. All the best, Michael In Mercurial, for example, we have a fairly large setup.py that helps us deal with many kinds of issues: incomplete Python installations on different platforms, adding packages depending on the platform, do custom stuff to compile gettext files, etc... Cheers, Dirkjan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
On Fri, Oct 8, 2010 at 4:54 PM, Georg Brandl g.bra...@gmx.net wrote: Am 08.10.2010 16:26, schrieb Barry Warsaw: - query pypi - browse what's installed - install/remove projects - create releases and upload them pkg_manager ? No underscores, please. :) Actually, a decent wrapper script could just be called 'setup'. My command-not-found on Ubuntu doesn't find a collision, or even close similarities. No generic name, *please*. easy_install was bad enough, no need to repeat that mistake. egg would be better, but weren't we phasing out the egg format? -1 on anything containing the word 'egg'. It'll add confusion with egg-the-format Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
On Fri, Oct 8, 2010 at 5:53 PM, Toshio Kuratomi a.bad...@gmail.com wrote: On Fri, Oct 08, 2010 at 05:12:44PM +0200, Antoine Pitrou wrote: ... pysetup is shorter Let's use pysetup ! ... I won't bikeshed as long as we stay away from conflicting names. +1. So. Let's add pysetup in distutils2, that will be installed as a classical script. Once we move distutils2 back in the stdlib, it will be provided in Python's bin dir, so people will have the same pysetup name everywhere, Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
