Re: [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
Grig Gheorghiu schrieb: Titus and I are thinking about mentoring a Google Summer of Code project that would use the 'buildbot try' feature: set up a bunch of buildbot slaves and set them up so sending them a patch will trigger a checkout of the latest Python svn, followed by the application of the patch, followed by building and running unit tests for Python, followed by running test suites for various projects (similar to how it's being done in the community buildbot farm). This will hopefully give us a better grasp about how good a patch is, and will make the process of accepting patches more smooth. What do people think? Sounds great! It would be even better if it could automatically pull patches off the roundup tracker, try to apply them, and post a message into roundup whether the patch applied and tested successfully. To prevent spamming, it may be necessary to explicitly release patches to buildbot (i.e. classify attachments as non-patch, released, or verified), but that could be done later. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
At 08:57 AM 3/9/2007 +0100, Martin v. Löwis wrote: In the case that triggered the discussion, the change implemented was not an incompatible change, because the new implementation still met the old specification (which, of course, was underspecified). No, it wasn't, actually. Read the doc strings, which state exactly what the code does. Windows: import os help(os.path.splitext) Help on function splitext in module ntpath: splitext(p) Split the extension from a pathname. Extension is everything from the last dot to the end. Return (root, ext), either part may be empty. Posix: import os help(os.path.splitext) Help on function splitext in module posixpath: splitext(p) Split the extension from a pathname. Extension is everything from the last dot to the end. Returns (root, ext), either part may be empty. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
Phillip J. Eby schrieb: At 08:57 AM 3/9/2007 +0100, Martin v. Löwis wrote: In the case that triggered the discussion, the change implemented was not an incompatible change, because the new implementation still met the old specification (which, of course, was underspecified). No, it wasn't, actually. Read the doc strings, which state exactly what the code does. The doc strings were precise, yes. The documentation (Doc/lib) was underspecified and allowed for both interpretations: splitext(path) Split the pathname path into a pair (root, ext) such that root + ext == path, and ext is empty or begins with a period and contains at most one period. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
At 07:18 PM 3/9/2007 +0100, Martin v. Löwis wrote: Phillip J. Eby schrieb: At 08:57 AM 3/9/2007 +0100, Martin v. Löwis wrote: In the case that triggered the discussion, the change implemented was not an incompatible change, because the new implementation still met the old specification (which, of course, was underspecified). No, it wasn't, actually. Read the doc strings, which state exactly what the code does. The doc strings were precise, yes. The documentation (Doc/lib) was underspecified and allowed for both interpretations: splitext(path) Split the pathname path into a pair (root, ext) such that root + ext == path, and ext is empty or begins with a period and contains at most one period. I'm just pointing out that anyone who reads *all* the documentation would reasonably conclude that the more specific of the two pieces is a reliable description of the behavior. Therefore, this is a change to documented behavior, and can't be considered a bug fix. I agree 100% with the people who think splitext() should arguably have had the new proposed behavior from the beginning. However, I also agree 100% with Glyph that it's ridiculous that we should even have to be having this argument about changing something that's been like this for about a *decade* and can reasonably be considered to not be a bug, given that its detailed behavior is clearly documented in the docstrings. Where were all these 10 Unix programmers for the last ten years? Obviously they've been writing their own replacement for splitext(), if and when they needed one -- or else the normal use cases for splitext either aren't dotfile-sensitive, or are *conducive to the existing implementation*. In other words, one of four cases apply for existing code: 1. Old code uses its own fucntion - change has no effect 2. Old code uses splitext, but never has and never will touch dotfiles - change has no effect 3. Old code uses splitext, and has never touched dotfiles (otherwise it would have been changed to not use splitext!) but *might*, *possibly*, touch one at some point in the future - change *might* fix a latent bug in such code, but clearly it's unlikely 4. Old code uses splitext, does touch dotfiles, *likes it that way* - change breaks code. (A small, but definitely non-empty group.) So, ignoring the cases where the change makes no difference, this change will: 1. DEFINITELY break some code 2. MAYBE fix some code that hasn't actually broken before I don't see how a hypothetical fix for one small group justifies definitely breaking the code of some other small group. The only group the change benefits is people writing *new* code -- so give them a new function. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
Martin v. Löwis wrote: splitext(path) Split the pathname path into a pair (root, ext) such that root + ext == path, and ext is empty or begins with a period and contains at most one period. Actually, that spec could be satisfied by a function that *always* returned an empty string for ext... -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
[EMAIL PROTECTED] wrote: That assumes there is a need for the old functionality. I really don't see it (pje claimed he needed it once, but I remain unconvinced, not having seen an actual fragment where the old behavior is helpful). This passage is symptomatic of the thing that really bothers me here. I have rarely used splitext before (since what I really want is determineMIMEType(), and that is easier to implement with one's own string-munging) and certainly won't use it again after following this discussion. The real issue I have here is one of process. Why is it that PJE (or any python user who wishes their code to keep working against new versions of Python) must frequent python-dev and convince you (or whatever Python developer might be committing a patch) of each use-case for old functionality? I would think that the goal here would be to keep all the old Python code running which it is reasonably possible to, regardless of whether the motivating use-cases are understood or not. It is the nature of infrastructure code to be used in bizarre and surprising ways. My understanding of the current backwards-compatibility policy for Python, the one that Twisted has been trying to emulate strictly, is that, for each potentially incompatible change, there will be: * at least one release with a pending deprecation warning and new, better API * at least one release with a deprecation warning * some number of releases later, the deprecated functionality is removed I was under the impression that this was documented in a PEP somewhere, but scanning the ones about backwards compatibility doesn't yield anything. I can't even figure out why I had this impression. *Is* there actually such a policy? If there isn't, there really should be. Deciding each one of these things on a case-by-case basis leaves a lot of wiggle room for a lot of old code to break in each release. For example, if you put me in charge of Python networking libraries and I simply used my judgment about what usages make sense and which don't, you might find that all the synchronous socket APIs were mysteriously gone within a few releases... ;-) Perhaps policy isn't the right way to solve the problem, but neither is asking every python application developer to meticulously follow and participate in every discussion on python-dev which *might* affect their code. As recently as last week, Guido commented that people build mental models of performance which have no relation to reality and we must rely on empirical data to see how things *really* perform. This is a common theme both here and anywhere good software development practices are discussed. These broken mental models are not limited to performance. In particular, people who develop software have inaccurate ideas about how it's really used. I am avoiding being specific to Python here because I've had a similarly broken idea of how people use Twisted, heard extremely strange ideas from kernel developers about the way programs in userland behave, and don't get me started on how C compiler writers think people write C code. If Python is not going to have an extremely conservative (and comprehensive, and strictly enforced) backwards-compatibility policy, we can't rely on those mental models as a way of determining what changes to allow. One way to deal with the question of how do people really use python in the wild is to popularize the community buildbots and make it easy to submit projects so that at least we have a picture of what Python developers are really doing. Buildbot has a build this branch feature which could be used to settle these discussions more rapidly, except for the fact that the community builders are currently in pretty sad shape: http://www.python.org/dev/buildbot/community/all/ By my count, currently only 9 out of 22 builders are passing. The severity of these failures varies (many of the builders are simply offline, not failing) but they should all be passing. If they were, rather than debating use-cases, we could at *least* have someone check this patch into a branch, and then build that branch across all these projects to see if any of them failed. Unfortunately open source code is quite often better tested and understood than the wider body of in-house and ad-hoc Python code out there, so it will be an easier target for changes like this than reality. I don't really have a problem with that though, because it creates a strong incentive for Python projects to both (A) be open source and (B) write comprehensive test suites, both of which are useful goals for many other reasons. In the past I've begged off of actually writing PEPs because I don't have the time, but if there is interest in codifying this I think I don't have the time *not* to write it. I'd prefer to document the pending/deprecate/remove policy first, but I can write up something more complicated about
Re: [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
On 3/8/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Buildbot has a build this branch feature which could be used to settle these discussions more rapidly, except for the fact that the community builders are currently in pretty sad shape: http://www.python.org/dev/buildbot/community/all/ By my count, currently only 9 out of 22 builders are passing. The severity of these failures varies (many of the builders are simply offline, not failing) but they should all be passing. If they were, rather than debating use-cases, we could at *least* have someone check this patch into a branch, and then build that branch across all these projects to see if any of them failed. Titus and I are thinking about mentoring a Google Summer of Code project that would use the 'buildbot try' feature: set up a bunch of buildbot slaves and set them up so sending them a patch will trigger a checkout of the latest Python svn, followed by the application of the patch, followed by building and running unit tests for Python, followed by running test suites for various projects (similar to how it's being done in the community buildbot farm). This will hopefully give us a better grasp about how good a patch is, and will make the process of accepting patches more smooth. What do people think? Grig ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
[EMAIL PROTECTED] wrote: In the past I've begged off of actually writing PEPs because I don't have the time, but if there is interest in codifying this I think I don't have the time *not* to write it. I'd prefer to document the pending/deprecate/remove policy first, but I can write up something more complicated about community buildbots and resolving disputes around potential breakages if there is actually no consensus about that. Please do write this up. It's a good policy and should be codified. # Steve ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
[EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] [EMAIL PROTECTED] wrote: My understanding of the current backwards-compatibility policy for Python, the one that Twisted has been trying to emulate strictly, is that, for each potentially incompatible change, there will be: * at least one release with a pending deprecation warning and new, better API * at least one release with a deprecation warning * some number of releases later, the deprecated functionality is removed = There is not and hardly can be any such policy for bug fixes. Unfortunately, * Honest people can disagree about whether a change is a bug fix or feature addition or replacement. And there will be borderline cases regardless of one's definition, unless one makes either category empty. (Having only one category and hence only one change policy would simplify life but at some cost.) * Fixing bugs can break code just as it fixes other code. I believe the current policy is that depending on buggy behavior, as opposed to working around it, is proceed at your own risk. Bugs can be fixed without notice, at the next bug fix release. On the other hand, bug fixes can and sometime are delayed or modified in recognition of degrees of bugginess and differences of opinions and cost-benefit analyses. But deviations from a rigid procedure require case-by-case judgments, to me preferably based on information-gathering and discussion and a chance to influence judgment. For new features that don't break code (that doesn't depend on the features absence), the current (and somewhat new) policy is to wait until the next minor .x feature release, skipping any micro .x.y releases (now called bug-fix releases). For replacement features, the policy is what you give above. Example: f(*args) instead of apply(f, args). At one time, the expectation was that removal could be and possibly would be one release after deprecation. It is currently to wait until 3.0. What it will be after that I have not seen discussed. Semantic changes are trickier. We don't want a different API when we want a different meaning for the currently API. The integer division change is a mixture. Int//int replaces int/int and is already available. But int/int will change meaning to float(int)/float(int) rather than being removed. So I see five categories of code change -- clear bug, possibly fuzzy bug, semantic change, replacement, and addition -- each with different appropriate policies. And there are mixtures that belong together in a package. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc'))
[EMAIL PROTECTED] schrieb: The real issue I have here is one of process. Why is it that PJE (or any python user who wishes their code to keep working against new versions of Python) must frequent python-dev and convince you (or whatever Python developer might be committing a patch) of each use-case for old functionality? Because this is the way free software works. I'm the one contributing (actually, merely integrating changes proposed by others, Larry Hastings in particular), so I get to make decisions. This patch had been open for three years. If pje had any problems with it, he could have rejected it. If you had problems with it, you could have commented that it should not be applied. Nobody did, so I decide. I would think that the goal here would be to keep all the old Python code running which it is reasonably possible to, regardless of whether the motivating use-cases are understood or not. Exactly: which it is reasonably possible to. In this case, I found no reasonable way to keep it running while still fixing the bug, and not breaking too much other stuff in order to keep that particular code running. So the only alternative would have been to reject the change and live with the bug forever, something that many people here considered bad. My understanding of the current backwards-compatibility policy for Python, the one that Twisted has been trying to emulate strictly, is that, for each potentially incompatible change, there will be: * at least one release with a pending deprecation warning and new, better API * at least one release with a deprecation warning * some number of releases later, the deprecated functionality is removed If you want to contribute a patch that changes splitext to follow this policy, please go ahead and submit one. I personally consider a deprecation warning unacceptable in this specific case. I was under the impression that this was documented in a PEP somewhere, but scanning the ones about backwards compatibility doesn't yield anything. I can't even figure out why I had this impression. *Is* there actually such a policy? There is PEP 5, which is not followed, since it requires backwards-incompatible behavior to go through the PEP process. If there isn't, there really should be. So submit a PEP. Or better, take over maintenance of PEP 5 (or better, cooperate with the author of the PEP on maintaining it). Perhaps policy isn't the right way to solve the problem, but neither is asking every python application developer to meticulously follow and participate in every discussion on python-dev which *might* affect their code. And there is no need to. If they find that their code breaks, they can submit bug reports, or, better, contribute patches. This is what alpha releases are for, and the trunk is buildable all the time, so you can test your code against the new release even before it happens. If you want to study the changes, you can read Misc/NEWS from time to time. If Python is not going to have an extremely conservative (and comprehensive, and strictly enforced) backwards-compatibility policy, we can't rely on those mental models as a way of determining what changes to allow. One way to deal with the question of how do people really use python in the wild is to popularize the community buildbots and make it easy to submit projects so that at least we have a picture of what Python developers are really doing. I was going to say that. Now, if the community doesn't pick up the concept of community buildbots, the community has no right to complain (IMO, and I put that deliberately into quotes, knowing that the community doesn't exist as such, but is made up of individuals, some interested in the buildbots, stability, and so on, and others being less conservative). In the past I've begged off of actually writing PEPs because I don't have the time, but if there is interest in codifying this I think I don't have the time *not* to write it. If you think it needs codification, I think you will need to write it yourself - nobody else will do it for you. Depending on what it will say, I might disagree. I do disagree with the requirement that every incompatible go through the PEP, and I understand incompatible change as programs only relying on document behavior might break. In the case that triggered the discussion, the change implemented was not an incompatible change, because the new implementation still met the old specification (which, of course, was underspecified). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
