Re: [Python-Dev] Rounding float to int directly ...
Greg Ewing writes: > [I propose for Py3k that] builtin round() > to return an int, and have something somewhere > else, such as math.fround(), for round-to-float. +1. The need to convert float to int is pervasive. The need to round (binary) floats to integer (or other decimal) floats is _much_ less common. Furthermore, the need for "round to nearest int" is much more common than the need for "integer floor" (although as pointed out, each could be defined in terms of the other except for certain rules for rounding exact halves). And most telling of all, if I were to stop 10 random programmers on the street, most of them not familiar with Python, and ask them to guess what the function named "round" did, I'm guessing the majority would think of float-to-int. It's a nice feature of Python that "it just works". Marc-Andre Lemburg writes: > You often have a need for controlled rounding when doing > financial calculations or [other reason snipped] Hmm. Not in the banks _I_ have worked at! We *never* use binary floating point for money. The decimal class is fairly useful in that regard. Nick Maclaren write: > Decimal doesn't even help people who care about accuracy. Not true! The float class is incapable of maintaining 700 digits of precision, but Decimal handles it just fine. (Why you would WANT more than around 53 bits of precision is a different question, but Decimal handles it.) > People who care about the accuracy of calculations prefer binary, > as it is a more accurate model. Huh? That doesn't even make sense! A model is not inherently accurate or inaccurate, it is only an accurate or inaccurate representation of some "real" system. Neither binary nor decimal is a better representation of either rational or real numbers, the first candidates for "real" system I thought of. Financial accounting rules tend to be based on paper-and-pencil calculations for which decimal is usually a far better representation. If you had said that binary floats squeeze out more digits of precision per bit of storage than decimal floats, or that binary floats are faster because they are supported by specialized hardware, then I'd go along, but they're not a "better model". -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly ...
Nick Maclaren: > The "wobbling precision" effect may be overstated, > but is real, and gets worse the larger the base is. To the best of my > knowledge, that is almost the only way in which binary is more accurate > than decimal, in absolute terms, and it is a marginal difference. Okay, that makes sense. However, if I understand "wobbling precision" correctly (and it's a BIG if) then it refers to the fact that interpolating from the last digit of precision doesn't work so well (intentionally vague phrase). So I would think (but could be wrong) that it is always trumped by adding another digit of precision to your numbers / lookup tables / whatever. If so, then even more credit needs to be given to the system that supports adjustable precision (the Decimal module). -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys
I'm changing the subject line because I want to convince everyone that
the problem being discussed in the "unicode hell" thread has nothing
to do with unicode and strings. It's all about dicts.
I have not observed real breakage in my own code, but I will present
a sample of made-up-but-completely-reasonable code that works in
2.4, fails in 2.5, and arguably ought to work fine. I think we should
restore the behavior of dicts that when they compare keys for
equality they suppress exceptions (treating the objects as unequal),
or at LEAST retain the behavior for one more release making it a
warning this time.
Here is my sample code:
--- problem_with_dicts.py --
# A sample program to demonstrate that the proposed behavior
# of dicts in Python 2.5 generates bugs. This is not code from
# an actual program, but it is completely reasonable.
# First import from
# http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413486
# the only 5-star recipe in the Python Cookbook for implementing
# enums.
import cookbookenum
# Then set up some reasonable enums. We'll say we're writing
# a program for dealing with dates.
DaysOfWeek = cookbookenum.Enum(
'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun')
Months = cookbookenum.Enum(
'Jan','Feb','Mar','Apr','May','Jun','Jul','Aug',
'Sep','Oct','Nov','Dec')
# Let's say we also do some translations. Here is a
# useful dictionary:
translations = {}
# which we populate with values
translations[ DaysOfWeek.Mon ] = {
'en': 'Monday',
'es': 'Lunes',
'fr': 'Lundi',
}
# and assume we do the other days
translations[ Months.Jan ] = {
'en': 'January',
'es': 'Enero',
'fr': 'Janvier',
}
# and assume we do the other months
# ...then later in the code we could do things like this:
language = 'en'
dayOfWeek = DaysOfWeek.Mon
month = Months.Jan
dayOfMonth = 3
print '%s, %s %s' % (
translations[dayOfWeek][language],
translations[month][language],
dayOfMonth)
# this works in 2.4 but fails in 2.5
- end problem_with_dicts.py
Please reconsider.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys
Marc-Andre Lemburg writes: > How about generating a warning instead and then go for the exception > in 2.6 ? Agreed. Michael Hudson's explanation convinced me. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dict suppressing exceptions
Michael Urman writes: > I strongly believe that unicode vs str here is the symptom and not the > actual problem. The comparison between two non-us-ascii str/unicode > instances is but one of many ways to raise an exception during > comparison. [... example ...] > Yes this is made up code. But I'm not arguing for a feature; I'm > arguing for backwards compatibility. Because we do not know where > these legitimate uses are, we cannot evaluate their likelihood to > exist nor the level of breakage they will cause. If we make this a > warning for any exception, we can satisfy both imagined camps. Those > in Armin's position can make that warning raise an exception while > debugging, and those using it on purpose can squash it. See also my example from the beginning of this thread (http://mail.python.org/pipermail/python-dev/2006-August/067978.html). The example wasn't from real code, but it WAS quite plausible -- straightforward use of a popular Python Cookbook recipe. But we seem to have reached a rough consensus: Later, Guido writes: > How about we change unicode-vs-str __eq__ to > issue a warning (and return False) instead of raising > UnicodeException? [... Marc-Andre Lemburg agrees ...] > Great! Now we need someone to volunteer to write a patch (which should > include doc and NEWS updates) in time for beta 3 (Aug 18). I don't *strongly* object to this consensus, but if you haven't glanced at my original example, take a look - it might convince you. The proposed solution will not help with my example. I'm not sure the motivation for breaking code like that example -- the bug-hunting motivation is satisfied by issuing a warning as Michael Urman proposes, then use an exception after one more release when people have had time to fix their code. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dict suppressing exceptions
Guido writes: > Alas, I have no idea what it does. Can you come up with an example > that doesn't require enums and localization? Sorry. Here's the short version: Fact 1: Sometimes people create objects that raise exceptions when compared for equality. Maybe it's a bad idea to do this, and objects should never raise exceptions when compared except to signal internal inconsistancy -- but the practice is common enough to show up in a 5-star recipe for enums in the Cookbook. Fact 2: Unrelated objects are often put in dictionarys together. I used a "localization dictionary" as an example. I feel this is a legitimate practice. If dictionarys stop suppressing exceptions raised by equality tests then those two facts aren't compatible. Programs that used to work fine will break with 2.4. I'm OK with your proposed solution (fix the case of str-unicode but let everyone else using objects that raise exceptions suffer), but I think it would be more friendly to use warnings for now and exceptions after another full release cycle. The warnings should solve the underlying issue (hard-to-debug problems). -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Elementtree and Namespaces in 2.5
Chris Spencer writes: > there's an issue > regarding [ElementTree's] namespace parsing that should be addressed. [... it performs namespace rewriting ...] > while most users and the w3 spec > (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting) > agree this feature is actually a bug, Fredrik Lundh has refused to fix > this problem. Of course, this is his right. Unfortunately, > Elementtree's design makes a work-around rather awkward. Therefore, we > might want to rethink inclusion of Elementtree in the stdlib, or at > least patch the stdlib's version of Elementtree to produce an output > more in line with the w3 standard. The good news for you is that: (1) Including ElementTree in the stdlib does not (immediately) take away from any of the other XML librarys out there. (2) Including ElementTree in the stdlib *increases* the chance that requests from users will lead to a change in the library. (3) Changing ElementTree to preserve namespace prefixes would be backward compatible with a version that didn't preserve it, so nothing happening now forecloses fixing this in future releases. Now, in my opinion, the W3 flubbed badly in the way they designed namespaces, prefixes, and allowing namespace prefixes to appear within element content and attributes. Creating a universal namespace with local aliases to prevent name clashes is nice, making the local aliases significant so that we haven't prevented name clashes after all but have merely introduced vast complexity (and *encouraged* clashes by shortening the names) is... not so nice. But that's beside the point. Even if we presume 100% agreement on the need to change ElementTree, the best thing to do is still to release ElementTree in the stdlib exactly as it is in 2.5 and change it for 2.6. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dict suppressing exceptions
Martin v. Löwis writes: > Now I looked at it, and think that the recipe is broken. Not broken, but perhaps wrongheaded. The recipe went out of its way to ensure that it would raise an exception of enum values from different enumerations were compared. There's nothing out there saying that this is a bad thing to do. I propose that we institute a new policy. The policy should state: __eq__ methods should always return True or False. They should only raise an exception if there is some internal error within one of the objects being compared -- they should never raise an exception because the other object is of an unexpected type. On the other hand, __lt__, __gt__ and friends SHOULD raise an exception when the object being compared is of some type for which the ordering does not make sense (e.g.: unicode vs byte-string or complex vs anything). I think we should note this policy someplace official -- perhaps in the Language Reference where __eq__ and __lt__ are defined. But I do not think that these changes should be made until Py3K. What do others think? Is this the "right" approach? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Jim Jewett writes: > This change [in docs] looks wrong: > > PyDoc_STRVAR(rpartition__doc__, > -"S.rpartition(sep) -> (head, sep, tail)\n\ > +"S.rpartition(sep) -> (tail, sep, head)\n\ Raymond Hettinger replies: > It is correct. There may be some confusion in terminology. Head > and tail do not mean left-side or right-side. Instead, they refer to > the "small part chopped-off" and "the rest that is still choppable". > Think of head and tail in the sense of car and cdr. It is incorrect. The purpose of documentation is to explain things to users, and documentation which fails to achieve this is not "correct". The level of confusion generated by using "head" to refer to the last part of the string and "tail" to refer to the beginning, is quite significant. How about something like this: S.partition(sep) -> (head, sep, tail) S.rpartition(sep) -> (tail, sep, rest) Perhaps someone else can find something clearer than my suggestion, but in my own head, the terms "head" and "tail" are tighly bound with the idea of beginning and end (respectively) rather than with the idea of "small part chopped off" and "big part that is still choppable". -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
ome encouragement in order to continue to invest energy in > pushing this forward. Please keep up the good work! Some day I'd like to see NumPy built in to the standard Python distribution. The incremental, PEP by PEP approach you are taking is the best route to getting there. But there may be some changes along the way -- convergence with ctypes may be one of those. - Look, my advice is to try to make ctypes work for you. Not having any easy way to construct or to interrogate ctypes objects from C is a legitimate complaint... and if you can define your requirements, it should be relatively easy to add a C interface to meet those needs. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] re: 2.4 news reaches interesting places
Carlos Riberio writes: > One possible marketing strategy is to use the adjective "fast" in a > broader sense. The Python slogan could be something like: "Programming > has never been any faster" -- this changes the playing ground, from > raw performance to *programming* performance. I think Carlos is onto something here. Guido's original question was how to fight this meme... in other words, people think of Python as slow, whether they have measured it or not. Just like they think of Java as being more portable. Talking about "fast enough" is just another way of reminding people that we're really quite slow (even if that's not true). So how about a slogan like "Code it Fast, with Python", or "Python: Code Fast" -- one which emphasizes the (easily defended) claim that development time is shorter with Python, but which at the same time manages to associate the word "fast" with "Python". -- Michael Chermside ___ Python-Dev mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: [Csv] Minor change to behaviour of csv module
Andrew explains that in the CSV module, escape characters are not properly removed. Magnus writes: > IMO this is the *only* reasonable behaviour. I don't understand why > the escape character should be left in; this is one of the reason why > UNIX-style colon-separated values don't work with the current module. Andrew writes back later: > Thinking about this further, I suspect we have to retain the current > behaviour, as broken as it is, as the default: it's conceivable that > someone somewhere is post-processing the result to remove the backslashes, > and if we fix the csv module, we'll break their code. I'm with Magnus on this. No one has 4 year old code using the CSV module. The existing behavior is just simply WRONG. Sure, of course we should try to maintain backward compatibility, but surely SOME cases don't require it, right? Can't we treat this misbehavior as an outright bug? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: Subscribing to PEP updates
Barry writes: > As an experiment, I just added a PEP topic to the python-checkins > mailing list. You could subscribe to this list and just select the PEP > topic (which matches the regex "PEP" in the Subject header or first few > lines of the body). > > Give it a shot and let's see if that does the trick. I just got notification of the change to PEP 246 (and I haven't received other checkin notifications), so I guess I can report that this is working. Thanks, Barry. Should we now mention this on c.l.py for others who may be interested? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] PEP 246, redux
Phillip: I think you must inhabit a far more perfect world than I do. You say, for instance, that: > ...-1 if this introduces a performance penalty [...] just to > support people who want to create deliberate Liskov violations. > I personally don't think that we should pander to Liskov > violators ... but in my world, people violate Liskov all the time, even in languages that attempt (unsuccessfully) to enforce it. [1] You say that: > I think one should adapt primarily to interfaces, and > interface-to-interface adaptation should be reserved for > non-lossy, non-noisy adapters. ... but in my world, half the time I'm using adaptation to correct for the fact that someone else's poorly-written code requests some class where it should have just used an interface. You seem to inhabit a world in which transitivity of adaptation can be enforced. But in my world, people occasionally misuse adaptation because they think they know what they're doing or because they're in a big hurry and it's the most convenient tool at hand. I wish I lived in your world, but I don't. -- Michael Chermside [1] - Except for Eiffel. Eiffel seems to do a pretty good job of enforcing it. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] PEP 246, redux
I wrote: > >... but in my world, half the time I'm using adaptation to > >correct for the fact that someone else's poorly-written > >code requests some class where it should have just used > >an interface. Phillip replies: > PEP 246 adaptation? Or are you talking about some other language? > (I ask out of curiosity.) Well, it's partly just a rhetorical device here. I mean PEP 246 adaption, but (unlike you!) I'm not actually using it yet (aside from playing around to try things out), really I'm just guessing how I WOULD be using it if it were part of core python. > I agree that if it's possible to adapt to concrete types, people will do > so. However, I think we all agree that this isn't a great idea and should > still be considered bad style. I'd agree except for the case where I am trying to pass an object into code which is misbehaving. If we do add type declarations that trigger an adapt() call, then people WILL write poor code which declares concrete types, and I will find myself writing __conform__ methods to work around it. In this case, I'm the one making use of adaption (the original author was just expecting a TypeError), but what I'm doing isn't (IMO) bad style. > >You seem to inhabit a world in which transitivity of adaptation > >can be enforced. But in my world, people occasionally misuse > >adaptation because they think they know what they're doing > >or because they're in a big hurry and it's the most convenient > >tool at hand. > > How is this different from abuse of *any* language feature that you're > then forced to work around? Are you saying we should not provide a > feature because *some* people will abuse the feature? I don't > understand. If we're just recomending that people design for transitivity, then I don't have a problem (although see Alex's fairly good point illustrated with LotsOfInfo, PersonName, and FullName -- I found it convincing). But I was under the impression that the point of transitivity was to make it "required", then automatically walk chains of adaptions. Then I fear one case of mis-used adaption could "poison" my entire adaption mechanism. The N^2 explosion of pairwise-only adapters scares me less, because I think in most real situations N will be small. > If you allow interface inheritance, you're just as susceptible to an > invalid adaptation path, and in my experience this is more likely to > bite you unintentionally, mainly because interface inheritance works > differently than class inheritance (which of course is used more > often). Do you want to prohibit interface inheritance, too? Hmm. Sounds like you're making a point here that's important, but which I don't quite get. Can you elaborate? I certainly hadn't intended to prohibit interface inheritance... how exactly does it "bite" one? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
David Ascher writes: > Terminology point: I know that LiskovViolation is technically correct, > but I'd really prefer it if exception names (which are sometimes all > users get to see) were more informative for people w/o deep technical > background. Would that be possible? I don't see how. Googling on Liskov immediately brings up clear and understandable descriptions of the principle that's being violated. I can't imagine summarizing the issue more concisely than that! What would you suggest? Including better explanations in the documentation is a must, but "LiskovViolation" in the exception name seems unbeatably clear and concise. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] PEP 246, redux
This is a collection of responses to various things that don't appear to have been resolved yet: Phillip writes: > if a target protocol has optional aspects, then lossy adaptation to it is > okay by definition. Conversely, if the aspect is *not* optional, then > lossy adaptation to it is not acceptable. I don't think there can really > be a middle ground; you have to decide whether the information is required > or not. I disagree. To belabor Alex's example, suppose LotsOfInfo has first, middle, and last names; PersonName has first and last, and FullName has first, middle initial and last. FullName's __doc__ specifically states that if the middle name is not available or the individual does not have a middle name, then "None" is acceptable. Converting LotsOfInfo to FullName via PersonName results in None for the middle name. But that's just not right... it _should_ be filling in the middle initial because that information IS available. It's technically correct in a theoretical sort of a way (FullName never PROMISES that middle_initial will be available), but it's wrong in a my-program-doesn't- work-right sort of way, because it HAS the information available yet doesn't use it. You're probably going to say "okay, then register a LotsOfInfo->FullName converter", and I agree. But if no such converter is registered, I would rather have a TypeError then an automatic conversion which produces incorrect results. I can explicitly silence it by registering a trivial converter: def adapt_LotsOfInfo_to_FullName(lots_of_info): person_name = adapt(lots_of_info, PersonName) return adapt(person_name, FullName) but if it just worked, I could only discover that by being clever enough to think of writing unit test for middle name. -- Elsewhere, Phillip writes: > If you derive an interface from another interface, this is supposed to mean > that your derived interface promises to uphold all the promises of the base > interface. That is, your derived interface is always usable where the base > interface is required. > > However, oftentimes one mistakenly derives an interface from another while > meaning that the base interface is *required* by the derived interface, > which is similar in meaning but subtly different. Here, you mean to say, > "IDerived has all of the requirements of IBase", but you have instead said, > "You can use IDerived wherever IBase is desired". Okay, that's beginning to make sense to me. > it's difficult because intuitively an interface defines a *requirement*, so > it seems logical to inherit from an interface in order to add requirements! Yes... I would fall into this trap as well until I'd been burned a few times. -- Alex summarizes nicely: > Personally, I disagree with having transitivity at all, unless perhaps > it be restricted to adaptations specifically and explicitly stated to > be "perfect and lossless"; PJE claims that ALL adaptations MUST, > ALWAYS, be "perfect and lossless" -- essentially, it seems to me, he > _has_ to claim that, to defend transitivity being applied > automatically, relentlessly, NON-optionally, NON-selectively [...] > Much the same applies to inheritance, BTW, which as PJE has pointed out > a few times also induces transitivity in adaptation, and, according to > him, is a more likely cause of bugs than chains of adapters But Alex goes on to say that perhaps we need two grades of adaptations (perfect and real-world) and two grades of interface inheritance (perfect and otherwise) so that the transitivity can be (automatically) invoked only for the perfect ones. That feels to me like excessive complexity: why not just prohibit transitivity? What, after all, is the COST of prohibiting transitivity? For the first case (adapter chains) the cost is a N^2 explosion in the number of adapters needed. I said I thought that N would be small, but Phillip (who knows what he's talking about, don't get me wrong here) says that it's big enough to be mildly annoying at times to Twisted and Eclipse developers. For the second case (interface inheritance), I haven't yet thought through clearly how at affects things... in fact, it sort of seems like there's no good way to _prevent_ "transitivity" in this case short of prohibiting interface inheritance entirely. And, as Phillip points out to me (see above) this is a more common type of error. Gee... I'm understanding the problem a little better, but elegant solutions are still escaping me. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: PEP 246: LiskovViolation as a name
I wrote: > I don't see how [LiskovViolation could have a more descriptive name]. > Googling on Liskov immediately brings up clear and understandable > descriptions of the principle David writes: > Clearly, I disagree. [...] Skip writes: > I had never heard the term before and consulted the Google oracle as well. This must be one of those cases where I am mislead by my background... I thought of Liskov substitution principle as a piece of basic CS background that everyone learned in school (or from the net, or wherever they learned programming). Clearly, that's not true. Guido writes: > How about SubstitutabilityError? It would be less precise and informative to ME but apparently more so to a beginner. Obviously, we should support the beginner! -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Python Interpreter Thread Safety?
Martin v. Löwis writes: > Due to some > unfortunate historical reasons, there is code which enters free() > without holding the GIL - and that is what the allocator specifically > deals with. Except for this single case, all callers of the allocator > are required to hold the GIL. Donovan Baarda writes: > Just curious; is that "one case" a bug that needs fixing, or is the some > reason this case can't be changed to use the GIL? Surely making it > mandatory for all free() calls to hold the GIL is easier than making the > allocator deal with the one case where this isn't done. What Martin is trying to say here is that it _IS_ mandatory to hold the GIL when calling free(). However, there is some very old code in existance (written by other people) which calls free() without holding the GIL. We work very hard to provide backward compatibility, so we are jumping through hoops to ensure that even this old code which is violating the rules doesn't get broken. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: Moving towards Python 3.0 (was Re: [Python-Dev] Speed up functioncalls)
Evan Jones writes: > My knowledge about garbage collection is weak, but I have read a little > bit of Hans Boehm's work on garbage collection. [...] The biggest > disadvantage mentioned is that simple pointer assignments end up > becoming "increment ref count" operations as well... Hans Boehm certainly has some excellent points. I believe a little searching through the Python dev archives will reveal that attempts have been made in the past to use his GC tools with CPython, and that the results have been disapointing. That may be because other parts of CPython are optimized for reference counting, or it may be just because this stuff is so bloody difficult! However, remember that changing away from reference counting is a change to the semantics of CPython. Right now, people can (and often do) assume that objects which don't participate in a reference loop are collected as soon as they go out of scope. They write code that depends on this... idioms like: >>> text_of_file = open(file_name, 'r').read() Perhaps such idioms aren't a good practice (they'd fail in Jython or in IronPython), but they ARE common. So we shouldn't stop using reference counting unless we can demonstrate that the alternative is clearly better. Of course, we'd also need to devise a way for extensions to cooperate (which is a problem Jython, at least, doesn't face). So it's NOT an obvious call, and so far numerous attempts to review other GC strategies have failed. I wouldn't be so quick to dismiss reference counting. > My only argument for making Python capable of leveraging multiple > processor environments is that multithreading seems to be where the big > performance increases will be in the next few years. I am currently > using Python for some relatively large simulations, so performance is > important to me. CPython CAN leverage such environments, and it IS used that way. However, this requires using multiple Python processes and inter-process communication of some sort (there are lots of choices, take your pick). It's a technique which is more trouble for the programmer, but in my experience usually has less likelihood of containing subtle parallel processing bugs. Sure, it'd be great if Python threads could make use of separate CPUs, but if the cost of that were that Python dictionaries performed as poorly as a Java HashTable or synchronized HashMap, then it wouldn't be worth the cost. There's a reason why Java moved away from HashTable (the threadsafe data structure) to HashMap (not threadsafe). Perhaps the REAL solution is just a really good IPC library that makes it easier to write programs that launch "threads" as separate processes and communicate with them. No change to the internals, just a new library to encourage people to use the technique that already works. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] license issues with profiler.py and md5.h/md5c.c
Jeremy writes: > Unfortunately a license that says it is in the public domain is > unacceptable (and should be for Debian, too). That is to say, it's > not possible for someone to claim that something they produce is in > the public domain. See http://www.linuxjournal.com/article/6225 Not quite true. It would be a bit off-topic to discuss on this list so I will simply point you to: http://creativecommons.org/license/publicdomain-2 ...which is specifically designed for the US legal system. It _IS_ possible for someone to produce something in the public domain, it just isn't as easy as some people think (just saying it doesn't necessarily make it so (at least under US law)) and it may not be a good idea. I would expect that if something truly WERE in the public domain, then it would be acceptable for Python (and for Debian too, for that matter). I can't comment on whether this applies to libmd. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: [Python Dev] PEP 309
Nick Coghlan writes: > I'm actually half-tempted to suggest that [map, filter and reduce] should be > aliased in the functional module for 2.5 (in addition to being in builtins - ala > the contents of the exceptions module). Well, I think it's a good idea, so I'll formally propose it! Let's alias map, filter, and reduce into the functional module now that it exists. This doesn't create any need or pressure to remove these as builtins, but it does contemplate a day when we might choose to make such a choice. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: Useful thread project for 2.5?
Greg writes:
> This is one of my top two Java features [1].
...
> [1] The other is compiler recognition of "@deprecated" in doc comments.
Hmm... what a good idea! Let's see... what exactly does "@deprecated" DO in
Java? Why it causes the compiler to emit a warning if you use the function in
question. Not a bad idea. Amusingly enough, the syntax is even the same in
Python:
= code =
import warnings
def deprecated(func):
"""This is a decorator which can be used to mark functions
as deprecated. It will result in a warning being emmitted
when the function is used."""
def newFunc(*args, **kwargs):
warnings.warn("Call to deprecated function.")
return func(*args, **kwargs)
return newFunc
= example =
>>> @deprecated
def func(x,y):
return x + y
>>> func(3,4)
Warning (from warnings module):
File "g:/Documents/Personal/Python/deprecated.py", line 10
warnings.warn("Call to deprecated function.")
UserWarning: Call to deprecated function.
7
So... shall I go add this to the cookbook?
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: @deprecated
[I followed Greg's suggestion and proposed a "deprecated" decorator.] Raymond writes: > Decorators like this should preserve information about the underlying > function Of course! The version I posted was intended to illustrate the idea, not to be a clean implementation. A long time ago, I proposed a decorator-maker-decorator (see "Creating Well-Behaved Decorators" in http://www.python.org/moin/PythonDecoratorLibrary), and I still think it's probably a wise approach since it's easy for people to be careless and forget to preserve these sorts of features. Jim Jewett writes: > I agree that it should go in the cookbook, but I think you should > set the category to a DeprecationWarning and give the offending > function's name. I had wondered about that, but wasn't familiar with how people actually use categories. DeprecationWarning certainly sounds right, or is there some reason why I should use a custom subclass of DeprecationWarning? Michael Hudson and Irmen point out: > One difference: I imagine (hope!) the java compiler would complain if > the deprecated function is referenced, whereas the python version only > complains if it is called... True enough. And java doesn't complain at all if the deprecated function is invoked via reflection. It's a fundamental difference in style between the two languages: Python barely even HAS a "compile phase", and Python programs tend to be far more dynamic than Java. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RELEASED Python 2.4.1, release candidate 1
[Martin v. Löwis] > I'd like to encourage feedback on whether the Windows installer works > for people. It replaces the VBScript part in the MSI package with native > code, which ought to drop the dependency on VBScript, but might > introduce new incompatibilities. [Tim Peters] > Worked fine here. Did an all-default "all users" install, WinXP Pro > SP2, from local disk, and under an account with Admin rights. I > uninstalled 2.4 first. I suppose that's the least stressful set of > choices I could possibly have made, but at least it confirms a happy > baseline. I tried several stranger things, like installing over 2.4.0 but in a different directory. Everything worked like clockwork. I did NOT try anything that would have involved a system with various things missing (like lack of VBScript), but I did play around with alternate install locations, repairs, uninstalls, single-user and all-user installs, and I found no problems anywhere. Nice work! -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] (no subject)
Nick Coghlan writes: > Patch against current CVS added to SF with the behaviour: > >def update_meta(self, other): > self.__name__ = other.__name__ > self.__doc__ = other.__doc__ > self.__dict__.update(other.__dict__) Nice... thanks. But I have to ask: is this really the right set of metadata to be updating? Here are a few things that perhaps ought be copied by update_meta: f.__name__ (already included) f.__doc__ (already included) f.__dict__ (already included) f.__module__ (probably should include) f.func_code.co_filename (to match f.__name__, but I'd leave it alone) there's also the annoying fact that in IDLE (and in some other python-aware IDEs) one can see the argument signature for a function as a "tool tip" or other hint. Very handy that, but if a decorator is applied then all you will see is "func(*args, **kwargs)" which is less than helpful. I'm not sure whether this CAN be duplicated... I believe it is generated by examining the following: f.func_code.co_argcount f.func_code.co_varnames f.func_code.co_flags & 0x4 f.func_code.co_flags & 0x8 ...and I suspect (experimentation seems to confirm this) that if you mangle these then the code object won't work correctly. If anyone's got a suggestion for fixing this, I'd love to hear it. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Rationale for sum()'s design?
Tim writes: > I'd personally be delighted if sum() never worked on anything other > than numbers. Guido writes: > I think the conclusion should be that sum() is sufficiently > constrained by backwards compatibility to make "fixing" it impossible > before 3.0. But in 3.0 I'd like to fix it so that the 2nd argument is > only used for empty lists. Guido, I find myself far more convinced by Tim's suggestion than yours. This all started with your claim that sum() was a replacement for most reduce() uses. We all agree that it works fine for summing up numbers. We all agree it DOESN'T work for any case where the operator isn't __add__. So trying to make it work well for non-numbers is just arguing over whether it replaces 75% of all reduce() uses, or 85% of them. Who cares? Remember, *ANY* use of sum() can be replaced by a simple, readable, three line for loop: total = for i in total += i And for that matter, other uses of reduce are similarly easy to write as a loop. I'm not saying reduce() is useless, just that it's not worthwhile to expend effort trying to cover every last use case with things like sum() or product() -- one can always write the loop instead. What I think is MORE important is that sum() have predictable behavior in the degenerate cases. And by "predictable", I mean predictable by a newbie user or one someone who hasn't looked at the docs for sum() for the past year and mostly remembers what it does only because the name is so blatantly obvious. _IF_ we follow Tim's advice and think of sum() as designed to add up numbers (as the name implies), then the corner cases are obvious: summing a list of 1 item should return the item, and summing a list of 0 items should return 0. If we think of sum as a form of reduce() that operates only on the __add__ operator, then we need to make other choices for these corner cases, and the people using it to add numbers will suffer the small indignity of having to recall how the corner case works. Honestly, the only case I even remotely care about is concatenating strings -- as far as _I_ am concerned, everyone else can just write out the loop or provide their own "sum_matrix()" method. And if I understand correctly, there's no plan to make sum() "just work" for concatenating strings (besides which, we'd still need to handle the empty list). So I'm in favor of REMOVING the second argument of sum() for python 3000, unless it was kept purely for "backward compatibility" reasons, which would be defeated by changing it's behavior. (And if this doesn't convince you, then so be it... this isn't a particularly important issue.) -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] @decoration of classes
Josiah Carlson writes: [... stuff about reST and TeX ...] > While I have not used it often, I have done the equivalent of decorating > classes; it is as natural (though perhaps not quite as useful initially) > as decorating functions, [... stuff about ice cream and sprinkles ...] Hmm... the only bit that I found particularly interesting there was the bit where you mention that you've used class decorators (without the syntax) before. What did you use them for? After all, the current state of things is "don't bother implementing class decorators because there is no compelling use case". If you've got some sample use cases, what were they? For my own part, I observe the following. Because a function decorator acts after the function object is created, there are limits to what it can do to the function. It can add some markup (eg: set properties or doc strings). It can "hook" either before or after the function is called. Or it can "veto" the function call and do something else instead. In the case of function calls, these are pretty much the interesting things you would want to do. Similarly, because a class decorator acts after the class is created there are limits on what it can do. It can modify the class object (replacing methods and such). It can add markup. It can replace the class with another (perhaps a proxy or some such). But most of these are things which are more easily done by a metaclass... and all of them *can* be done by metaclasses. The only noticable advantage that I see to class decorators over metaclasses is that there's a more straightforward way to combine them. And I'm not sure that combining metaclasses (or class decorators) is something I want to encourage. So I'm inclined to use different tools for modifying functions and modifying classes because the ways you want to modify them are different, and decorators are "tuned" to what people normally want to do with functions (like simple wrapping) while metaclasses are "tuned" to what people normally want to do with classes (like support for inheritance. But a few *good* use cases would change my mind. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: switch statement
Andrew McGregor writes: > I can post an alternative, inspired by this bit of Haskell [...] > The intent is that within the case, the bit before each : is a boolean > expression, they're evaluated in order, and the following block is > executed for the first one that evaluates to be True. If we're going to be evaluating a series of booleans, then the One Proper Format in Python is: if : elif : elif : else: When people speak of introducing a "switch" statement they are speaking of a construct in which the decision of which branch to take requires time proportional to something LESS than a linear function of the number of branches (it's not O(n) in the number of branches). Now the pattern matching is more interesting, but again, I'd need to see a proposed syntax for Python before I could begin to consider it. If I understand it properly, pattern matching in Haskell relies primarily on Haskell's excellent typing system, which is absent in Python. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: switch statement
I wrote:
> Now the pattern matching is more interesting, but again, I'd need to
> see a proposed syntax for Python before I could begin to consider it.
> If I understand it properly, pattern matching in Haskell relies
> primarily on Haskell's excellent typing system, which is absent in
> Python.
Nick Coghlan replies:
> There's no real need for special syntax in Python - an appropriate tuple
> subclass will do the trick quite nicely:
[... sample code matching tuples ...]
Aha, but now you've answered my question about syntax, and I can see
that your syntax lacks most of the power of Haskell's pattern matching.
First of all, it can only match tuples ... most things in Python are
NOT tuples. Secondly (as Michael Walter explained) it doesn't allow
name binding to parts of the pattern.
Honestly, while I understand that pattern matching is extremely powerful,
I don't see how to apply it in Python. We have powerful introspective
abilities, which seems to be helpful, but on the other hand we lack
types, which are typically a key feature of such matching. And then
there's the fact that many of the elegent uses of pattern matching use
recursion to traverse data structures... a no-no in a CPython that
lacks tail-recursion elimination.
There is one exception... matching strings. There we have a powerful
means of specifying patterns (regular expressions), and a multi-way
branch based on the content of a string is a common situation. A new
way to write this:
s = get_some_string_value()
if s == '':
continue;
elif re.match('#.*$', s):
handle_comment()
elif s == 'DEFINE':
handle_define()
elif s == 'UNDEF':
handle_undefine()
elif re.match('[A-Za-z][A-Za-z0-9]*$', s):
handle_identifier()
else:
syntax_error()
would be might be nice, but I can't figure out how to make it work
more efficiently than the simple if-elif-else structure, nor an
elegent syntax.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] defmacro (was: Anonymous blocks)
Jim Jewett writes: > As best I can tell, the anonymous blocks are used to take > care of boilerplate code without changing the scope -- exactly > what macros are used for. Folks, I think that Jim is onto something here. I've been following this conversation, and it sounds to me as if we are stumbling about in the dark, trying to feel our way toward something very useful and powerful. I think Jim is right, what we're feeling our way toward is macros. The problem, of course, is that Guido (and others!) are on record as being opposed to adding macros to Python. (Even "good" macros... think lisp, not cpp.) I am not quite sure that I am convinced by the argument, but let me see if I can present it: Allowing macros in Python would enable individual programmers or groups to easily invent their own "syntax". Eventually, there would develop a large number of different Python "dialects" (as some claim has happened in the Lisp community) each dependent on macros the others lack. The most important casualty would be Python's great *readability*. (If this is a strawman argument, i.e. if you know of a better reason for keeping macros OUT of Python please speak up. Like I said, I've never been completely convinced of it myself.) I think it would be useful if we approached it like this: either what we want is the full power of macros (in which case the syntax we choose should be guided by that choice), or we want LESS than the full power of macros. If we want less, then HOW less? In other words, rather than hearing what we'd like to be able to DO with blocks, I'd like to hear what we want to PROHIBIT DOING with blocks. I think this might be a fruitful way of thinking about the problem which might make it easier to evaluate syntax suggestions. And if the answer is that we want to prohibit nothing, then the right solution is macros. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] defmacro (was: Anonymous blocks)
Guido writes: > My problem with macros is actually more practical: Python's compiler > is too dumb. I am assuming that we want to be able to import macros > from other modules, and I am assuming that macros are expanded by the > compiler, not at run time; but the compiler doesn't follow imports > (that happens at run time) so there's no mechanism to tell the > compiler about the new syntax. And macros that don't introduce new > syntax don't seem very interesting (compared to what we can do > already). That's good to hear. It expresses fairly clearly what the challenges are in implementing macros for Python, and expressing the challenges makes it easier to attack the problem. My interest comes because some recent syntax changes (generators, generator expressions) have seemed to me like true language changes, but others (decorators, anonymous-blocks) to me just cry out "this would be easy as a macro!". I wrote: > I think it would be useful if we approached it like this: either what > we want is the full power of macros (in which case the syntax we choose > should be guided by that choice), or we want LESS than the full power > of macros. If we want less, then HOW less? > > In other words, rather than hearing what we'd like to be able to DO > with blocks, I'd like to hear what we want to PROHIBIT DOING with > blocks. I think this might be a fruitful way of thinking about the > problem which might make it easier to evaluate syntax suggestions. And > if the answer is that we want to prohibit nothing, then the right > solution is macros. Guido replied: > I'm personally at a loss understanding your question here. Perhaps you > could try answering it for yourself? You guys just think too fast for me. When I started this email, I replied "Fair enough. One possibility is...". But while I was trying to condense my thoughts down from 1.5 pages to something short and coherent (it takes time to write it short) everything I was thinking became obscelete as both Paul Moore and Jim Jewett did exactly the kind of thinking I was hoping to inspire: Paul: > What I feel is the key concept here is that of "injecting" code into a > template form (try...finally, or try..except..else, or whatever) [...] > Specifically, cases where functions aren't enough. If I try to > characterise precisely what those cases are, all I can come up with is > "when the code being injected needs to run in the current scope, not > in the scope of a template function". Is that right? Jim: > Why not just introduce macros? If the answer is "We should, it is just > hard to code", then use a good syntax for macros. If the answer is > "We don't want > xx sss (S\ to ever be meaningful", then we need to figure out exactly what to > prohibit. [...] > Do we want to limit the changing part (the "anonymous block") to > only a single suite? That does work well with the "yield" syntax, but it > seems like an arbitrary restriction unless *all* we want are resource > wrappers. > > Or do we really just want a way to say that a function should share its > local namespace with it's caller or callee? In that case, maybe the answer > is a "lexical" or "same_namespace" keyword. My own opinion is that we DO want macros. I prefer a language have a few, powerful constructs rather than lots of specialized ones. (Yet I still believe that "doing different things should look different"... which is why I prefer Python to Lisp.) I think that macros could solve a LOT of problems. There are lots of things one might want to replace within macros, from identifiers to punctuation, but I'd be willing to live with just two of them: expressions, and "series-of-statements" (that's almost the same as a block). There are only two places I'd want to be able to USE a macro: where an expression is called for, and where a series-of-statements is called for. In both cases, I'd be happy with a function-call like syntax for including the macro. Well, that's a lot of "wanting"... now I all I need to do is invent a clever syntax that allows these in an elegant fashion while also solving Guido's point about imports (hint: the answer is that it ALL happens at runtime). I'll go think some while you guys zoom past me again. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: switch statement
Guido writes: > You mean like this? > > if x > 0: >...normal case... > elif y > 0: > abnormal case... > else: > ...edge case... > > You have guts to call that bad style! :-) Well, maybe, but this: if x == 1: do_number_1() elif x == 2: do_number_2() elif x == 3: do_number_3() elif y == 4: do_number_4() elif x == 5: do_number_5() else: raise ValueError is clearly bad style. (Even knowing what I did here, how long does it take you to find the problem? Hint: line 7.) I've seen Jim's recipe in the cookbook, and as I said there, I'm impressed by the clever implementation, but I think it's unwise. PEP 275 proposes an O(1) solution... either by compiler optimization of certain if-elif-else structures, or via a new syntax with 'switch' and 'case' keywords. (I prefer the keywords version myself... that optimization seems awefully messy, and wouldn't help with the problem above.) Jim's recipe fixes the problem given above, but it's a O(n) solution, and to me the words 'switch' and 'case' just *scream* "O(1)". But perhaps it's worthwhile, just because it avoids repeating "x ==". Really, this seems like a direct analog of another frequently-heard Python gripe: the lack of a conditional expression. After all, the problems with these two code snippets: if x == 1:|if condition_1: do_1() |y = 1 elif x == 2: |elif condition_2: do_2() |y = 2 elif x == 3: |elif condition_3: do_3() |y = 3 else: |else: default() |y = 4 is the repetition of "x ==" and of "y =". As my earlier example demonstrates, a structure like this in which the "x ==" or the "y =" VARIES has a totally different *meaning* to the programmer than one in which the "x ==" or "y =" is the same for every single branch. But let's not start discussing conditional expressions now, because there's already more traffic on the list than I can read. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Example for PEP 343
In PEP 343 Guido writes: >8. Another use for this feature is the Decimal context. It's left > as an exercise for the reader. (Mail it to me if you'd like to > see it here.) Here are two such examples. Pick your favorite for the PEP. PS: Writing this helped convince me that allowing the use of generators instead of classes with the "do_template" decorator is quite nice in practice, even though it gets confusing (for beginners anyhow) if you start to think about it too much. -- Michael Chermside # = SAMPLE #1: increasing precision during a sub-calculation = import decimal @do_template def with_extra_precision(places=2): "Performs nested computation with extra digits of precision." decimal.getcontext().prec += 2 yield None decimal.getcontext().prec -= 2 # == SAMPLE USE of #1 == # (Sample taken from the Python Library Reference) def sin(x): "Return the sine of x as measured in radians." do with_extra_precision(): i, lasts, s, fact, num, sign = 1, 0, x, 1, x, 1 while s != lasts: lasts = s i += 2 fact *= i * (i-1) num *= x * x sign *= -1 s += num / fact * sign return +s # = SAMPLE #2: insisting on exact calculations only = import decimal @do_template def assert_exact(): "Raises exception if nested computation is not exact." ctx = decimal.getcontext() was_inexact = ctx.traps[decimal.Inexact] ctx.traps[decimal.Inexact] = True yield None ctx.traps[decimal.Inexact] = was_inexact # == SAMPLE USE of #2 == # Lemma 2 ensures us that each fraction will divide evenly do assert_exact(): total = decimal.Decimal(0) for n, d in zip(numerators, denominators): total += n / d ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Example for PEP 343
Phillip writes: > >@do_template > >def with_extra_precision(places=2): > > "Performs nested computation with extra digits of precision." > > decimal.getcontext().prec += 2 > > yield None > > decimal.getcontext().prec -= 2 > > Won't this do the wrong thing if something within the block alters > the precision? Depends on what behavior you want. I wrote it this way partly because that's how the example in the manual was written, and partly because I was thinking "if someone increased the precision by 3 within the block, then we probably want to leave it increased by 3 on block exit". On careful re-consideration, I think it'd be better to require that the block NOT make unbalanced changes to the precision... we could verify it using an assert statement. I avoided caching the context and restoring it, because I WANTED to allow code in the block to make OTHER alterations to the context and not clobber them after the block exits (eg: setting or clearing some trap). It's debatable whether that was a good design choice... there's a great deal of elegence to Guido's version used like this: Guido: > do with_decimal_context() as ctx: > ctx.prec += 2 > # change other settings > # algorithm goes here However, all of these are minor details compared to the bug that Raymond points out: Raymond: > The final "return +s" should be unindented. It should > be at the same level as the "do with_extra_precision()". The purpose of > the "+s" is to force the result to be rounded back to the *original* > precision. In effect, the "with_extra_precision" wrapper causes the calculations to be done with higher precision, AND causes any variables set during the block will retain their higher precision. (It's because context controls OPERATIONS but changing context never affects individual Decimal OBJECTS.) So I fear that the whole with_extra_precision() idea is just likely to tempt people into introducing subtle bugs, and it should probably be scrapped anyhow. Guido's approach to save-and- restore context works fine. -- Michael Chermside (PS: Another reason that I avoided a basic save-and-restore is that we have plenty of examples already of 'do' statements that save-and-restore, I was trying to do something different. It's interesting that what I tried may have turned out to be a poor idea.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 -Abstract Block Redux)
[I apologize in advance if this sounds a bit disjointed... I started to argue one thing, but by the end had convinced myself of the opposite, and I re-wrote the email to match my final conclusion.] Guido writes: > About deleting VAR I have mixed feelings. [...] > I think that, given that we let the for-loop variable survive, we > should treat the with-statement variable the same way. We said the same thing about the variable in list comprehensions and it's now obvious that it should NEVER have been allowed to escape it's scope. But the key difference is that list comprehensions are EXPRESSIONS, and for and 'with' are STATEMENTS. Expressions shouldn't modify the local environment, statements often do. Of course, that argument _permits_ not deleting VAR, but doesn't recomend in favor of it. My first thought for ideal behavior was that if VAR was previously defined (eg: a global, an existing attribute of some object, etc), then it should not be 'del''ed afterward. But VAR was newly created by the 'with' statement then we WOULD del it to keep the namespace "neat". Trouble is, that's FAR too complex, and depends on a distinction Python has not used before (although it's nearly the same as the property that controls the meaning of globally declared variables). My next thought was to just allow 'with' statements to introduce their own "scope"... the meaning that the VAR variable takes on within a 'with' statement is not propogated outside the scope of the statement. But imagine trying to implement this in CPython... don't forget details like supporting locals(). If it's too hard to do, then it's probably not the right solution. So then I thought "Well, what's the harm in letting the variable survive the 'with' statement?" I'm a big fan of keeping namespaces "clean", but it's just not important enough to incurr other penalties. So in this case, I (reluctantly, after giving myself quite a talking-to) favor having the 'with' statement with VAR create said variable in the appropriate scope as a side-effect, much like 'for'. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
Guido writes: [a rather silly objection to Phillip's proposal that 'with x:' is a no-op when x lacks __enter__ and __exit__] > I know this is not a very strong argument, but my gut tells me this > generalization of the with-statement is wrong, so I'll stick to it > regardless of the strength of the argument. The real reason will come > to me. Perhaps the real reason is that it allows errors to pass silently. If I write with foo: BLOCK where I should have written with locked(foo): BLOCK ...it silently "succeeds" by doing nothing. I CLEARLY intended to do the appropriate cleanup (or locking, or whatever), but it doesn't happen. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
[Tim and Raymond are slugging it out about whether Decimal constructors should respect context precision] Tim, I find Raymond's arguments to be much more persuasive. (And that's even BEFORE I read his 11-point missive.) I understood the concept that *operations* are contex-dependent, but decimal *objects* are not, and thus it made sense to me that *constructors* were not context-dependent. On the other hand, I am NOT a floating-point expert. Can you educate me some? What is an example of a case where users would get "wrong" results because constructors failed to respect context precision? (By the way... even if other constructors begin to respect context precision, the constructor from tuple should NOT -- it exists to provide low-level access to the implementation. I'll express no opinion on the constructor from Decimal, because I don't understand the issues.) -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
Tim Peters writes:
> Other important
> implementations of the standard didn't make this mistake; for example,
> Java's BigDecimal|(java.lang.String) constructor follows the rules
> here:
[...]
> Hmm -- or maybe it doesn't! The text says:
[...]
Here is the actual behavior:
Jython 2.1 on java1.5.0_03 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> import java.math.BigDecimal as BigDecimal
>>> import java.math.MathContext as MathContext
>>> BigDecimal("0.333")
0.333
>>> BigDecimal("0.333", MathContext.DECIMAL32)
0.333
>>>
In other words, Java's behavior is much closer to the current behavior
of Python, at least in terms of features that are user-visible. The
default behavior in Java is to have infinite precision unless a context
is supplied that says otherwise. So the constructor that takes a string
converts it faithfully, while the constructor that takes a context
obeys the context.
One could argue that they are "following the rules" appropriately and it
just happens that their default context has infinite precision. But from
the point of view of the typical, non-floating-point-aware user, Java's
constructor gives "all the digits you told it to", and so does Python's
current string constructor. Using "+decimal.Decimal(s)" today "rounds off
the constant" (in the nieve user's viewpoint).
Of course, the user is going to be surprised in the NEXT step since the
Python *operations* respect context while the Java ones use infinite
precision for +, -, and * (and require you to specify the behavior for /).
(PS: No, I don't think we should design decimal.Decimal to match the
behavior of Java... but I don't think that the Java example really helps
make your point.)
Elsewhere, Tim writes:
> Sorry, I can't make more time for this now.
I understand!
> The short course is that
> a module purporting to implement an external standard should not
> deviate from that standard without very good reasons
Yes, but should we think of the constructor-from-string as an
implementation-specific means of creating Decimal objects, which is
separate from string-to-Decimal converter that the standard requires
(and which is provided by the Context objects)?
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
Guido writes:
> It looks like if you pass in a context, the Decimal constructor still
> ignores that context
No, you just need to use the right syntax. The correct syntax for
converting a string to a Decimal using a context object is to use
the create_decimal() method of the context object:
>>> import decimal
>>> decimal.getcontext().prec = 4
>>> decimal.getcontext().create_decimal("1.234567890")
Decimal("1.235")
Frankly, I have no idea WHAT purpose is served by passing a context
to the decimal constructor... I didn't even realize it was allowed!
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
I'd like to respond to a few people, I'll start with Greg Ewing:
Greg writes:
> I don't see how it
> helps significantly to have just the very first
> step -- turning the input into numbers -- be
> exempt from this behaviour. If anything, people
> are going to be even more confused. "But it
> can obviously cope with 1.101,
> so why does it give the wrong answer when I add
> something to it?"
As I see it, there is a meaningful distinction between constructing
Decimal instances and performing arithmatic with them. I even think
this distinction is easy to explain to users, even beginners. See,
it's all about the program "doing what you tell it to".
If you type in this:
x = decimal.Decimal("1.13")
as a literal in your program, then you clearly intended for that
last decimal place to mean something. By contrast, if you were to
try passing a float to the Decimal constructor, it would raise an
exception expressly to protect users from "accidently" entering
something slightly off from what they meant.
On the other hand, in Python, if you type this:
z = x + y
then what it does is completely dependent on the types of x and y.
In the case of Decimal objects, it performs a "perfect" arithmetic
operation then rounds to the current precision.
The simple explanation for users is "Context affects *operations*,
but not *instances*." This explains the behavior of operations, of
constructors, and also explains the fact that changing precision
doesn't affect the precision of existing instances. And it's only
6 words long.
> But I also found it interesting that, while the spec
> requires the existence of a context for each operation,
> it apparently *doesn't* mandate that it must be kept
> in a global variable, which is the part that makes me
> uncomfortable.
>
> Was there any debate about this choice when the Decimal
> module was being designed?
It shouldn't make you uncomfortable. Storing something in a global
variable is a BAD idea... it is just begging for threads to mess
each other up. The decimal module avoided this by storing a SEPARATE
context for each thread, so different threads won't interfere with
each other. And there *is* a means for easy access to the context
objects... decimal.getcontext().
Yes, it was debated, and the debate led to changing from a global
variable to the existing arrangement.
--
As long as I'm writing, let me echo Nick Coghlan's point:
> The fact that the BDFL (and others, me included) were at least temporarily
> confused by the ability to pass a context in to the constructor suggests there
> is an interface problem here.
>
> The thing that appears to be confusing is that you *can* pass a context in to
> the Decimal constructor, but that context is then almost completely ignored.
Yeah... I agree. If you provide a Context, it should be used. I favor changing
the behavior of the constructor as follows:
def Decimal(data, context=None):
result = Existing_Version_Of_Decimal(data)
if context is None:
result = +result
return result
In other words, make FULL use of the context in the constructor if a context
is provided, but make NO use of the thread context when no context is
provided.
--
One final point... Thanks to Mike Cowlishaw for chiming in with a detailed
and well-considered explanation of his thoughts on the matter.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344: Explicit vs. Implicit Chaining
James Knight writes: > I still don't see why people think the python interpreter should be > automatically providing __context__. To me it seems like it'll just > clutter things up for no good reason. If you really want the other > exception, you can access it via the local variable in the frame > where it was first caught. No you can't, because you didn't know the second exception was going to happen! I write something like this: db_connection = get_db_connection() try: do_some_stuff(db_connection) except DatabaseException, err: log_the_problem(err) cleanup(db_connection) If something goes wrong inside of do_some_stuff, I enter the exception handler. But then if an error occurs within log_the_problem() or cleanup(), then I lose the original exception. It's just GONE. I didn't expect log_the_problem() or cleanup() to fail, but sometimes things DO fail. An example of this happens to me in in Java (which has the same problem. I have code like this: db_connection = get_db_connection() try: do_some_stuff(db_connection) finally: db_connection.close() For instance, when I want to do unit testing, I create a mock database connection that raises an exception if you don't use it as the test expects. So I get exceptions like this all the time: Error: did not expect call to "close()" Of course, what REALLY happened was that we tried to update a row that didn't exist, which created an exception: Error: tried to update row with key "100", but it does not exist. But then it entered the finally clause, and tried to close the connection. That wasn't expected either, and the new exception replaces the old one... and we lose information about what REALLY caused the problem. In Java, I had to fix this by making my mock objects very smart. They have to keep track of whether any problem has occurred during this test (in any of the cooperating mock objects) and if so, then they have to re-report the original problem whenever something new goes wrong. This is the only way I've found to work around the problem in Java. Wouldn't it be nice if Python could do better? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Closing old bugs
Raymond writes: > Is there anyone else on python-dev who thinks it's a bad idea to automatically > close reports without checking whether the issue is real? Raymond: I'm speaking up with some trepidation here, since I am NOT one of those who frequently closes bugs. But I think that at least sometimes there IS an advantage to closing old bugs. I can envision a world in which there would be only 5-10 open bug reports at any given time, with enough developer volunteers available to respond to new reports as they come in. And I realize that we are nowhere near such a world and we're not going to be getting there anytime soon. But it is still the case that MOST people are intimidated by the enormous stack of open bugs. Perhaps "intimidated" is the wrong word... what I want to convey is that most people have no idea what's important or where to start -- it's just too big a pile. Perhaps all of the people who would actually work to close Python bugs are perfectly happy with the existing system. Or perhaps we scare off some potential volunteers because they have no idea where to start. I've seen some systems that solve this problem by allowing users to "vote" for favorite bugs... then you can tell the "important" bugs because they are more likely to have lots of votes. As I see it, Facundo is using a variant of that system. He is asking whether there is *ONE PERSON* out there who cares enough about a bug to subscribe to it and then to respond to his inquiry. If there's not even one such person, then he's closing the bug (but if one such person comes along later, they can re-report it). Sure, we may be throwing away some useful information by closing a bug report without proper investigation. But we're not in a situation where we are short of information... we've got plenty of information (bug reports), what we lack is developer time. If throwing away information helps to better focus the developer time we have, then it may be a useful process! And someday when nirvana arrives and there are only 5-10 open bugs, we can send intrepid volunteers digging through the archives to examine bugs that got closed without proper investigation. I'm not holding my breath. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple expression eval in compound if statement?
Skip writes: > Even an apparently simple variable can have side effects [...] Consider > the builtin help object. Type it at the prompt with no parens. I tried. As far as I can tell, the result (in the interactive interpreter) is the same as with any other object except None: the _ variable gets set, the interpreter prints out the __str__ of the object, sets the _ variable, and prints a new '>>>' prompt. It happens that the __str__ of the help object is: 'Type help() for interactive help, or help(object for help about object.' ... but if there's some side effect going on here, I don't see it. What am I missing? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple expression eval in compound if statement?
A few thought's on Skip's idea here: Skip: > I can't imagine anyone relying on a side-effect when evaluating x in this > context, and if someone did, their code would be truly fragile. Yeah... people like that really annoy me. If people just wouldn't write code that depends on the FULL dynamics possible, then it'd be a lot easier for the VM to optimize. Unfortunately, we have to support such people. Raymond: I think it unwise to allow x to be any expression. Besides altering existing semantics, it leads to code redundancy and to a fragile construct (where the slightest alteration of any of the expressions triggers a silent reversion to O(n) behavior). Skip's proposal isn't the problem here. The syntax of the if-elif-else statement requires that when it is used in lieu of C's switch() statement that the switch() argument be repeated in each elif clause. Repeating it DOES violate once-and-only-once, but we've decided that that's an acceptable price to pay to avoid needing an extra kind of flow control statement for which there wasn't a really elegent syntax anyhow. A reasonably tradeoff, I agree, but the cost is that we DO violate once-and-only-once. The REAL danger from the "slightest alteration of any of the expressions" is not that it will alter from O(1) behavior to O(n) behavior, the REAL danger is that you may have MEANT it to be the same everywhere and the alteration is a typo and a bug! Guido: > I think it would be wiser if you only did this when x is a simple > local variable. I agree. I think of Skip's proposal as just an optimization, of the sort that the Python interpreter is allowed to make when there's no possible semantic effect. It would then restrict the use to situations where the 'switch() argument' was a local variable and the 'case values' were string literals, integer literals, float literals (but that shouldn't count because it's a really bad idea anyhow), None, and perhaps (I wouldn't bother, myself) tuples of the above. It sounds pretty restricted, which raises the question of whether it's worth it given the harsh restrictions on when the optimization would kick in. I write such things with local variables and literal strings quite often. On the other hand, I don't usually have more than 4-10 values though, so O(1) and O(n) may not be THAT different. It's one of those cases where the only thing I'd really believe was experiments done on real code. But it's a cool optimization if it actually pays off. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Wishlist: dowhile
Jp Calderone writes: > for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''): > f2.write(chunk) Phillip J. Eby responds: > Showoff. ;) > > More seriously, I think your translation makes an excellent argument in > *favor* of having a do/while statement for greater clarity. :) Interesting... I had the opposite reaction. I often see someone do something "far too clever", which looks cute and impresses me, but makes me think that one would be wiser to avoid clever tricks and just write straightforward (if dull) code. But when I read JP's code I immediately thought that it WAS the straightforward way to write that code, and that I was just not smart enough to have realized it until he showed me. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Wishlist: dowhile
Jp Calderone writes: > Anything can be a for loop. > > for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''): > f2.write(chunk) Raymond responds: > It would be nice to have this encapsulated in a file method: > > for chunk in f1.iterblocks(CHUNK_SIZE): > f2.write(chunk) What ever happened to "Not every 3 line function needs to be a builtin"? It's a common pattern. It's easy to do. Where's the problem? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
Martin v. Löwis writes: > You are missing the rationale of the PEP process. The point is > *not* documentation. The point of the PEP process is to channel > and collect discussion, so that the BDFL can make a decision. > The BDFL is not bound at all to the PEP process. > > To document things, we use (or should use) documentation. You are oversimplifying significantly. The purpose of the PEP process is to lay out and document the reasoning that went into forming the decision. The BDFL is *allowed* to be capricious, but he's sensible enough to choose not to: in cases where it matters, he tries to document the reasoning behind his decisions. In fact, he does better than that... he gets the PEP author to document it for him! The PEP (whether accepted, rejected, or in-progress) serves as the official documentation of how the decision was made (or of what option it is that is still undecided). If a _trivial_ decision is already made, there's no need for a PEP, but if a difficult decision has been made, then documenting it in a PEP saves years of having to justify it to newbies. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict and on_missing()
A minor related point about on_missing(): Haven't we learned from regrets over the .next() method of iterators that all "magically" invoked methods should be named using the __xxx__ pattern? Shouldn't it be named __on_missing__() instead? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict and on_missing()
Walter Dörwald writes: > I always thought that __magic__ method calls are done by Python on > objects it doesn't know about. The special method name ensures that it > is indeed the protocol Python is talking about, not some random method > (with next() being the exception). In the defaultdict case this isn't a > problem, because defaultdict is calling its own method. I, instead, felt that the __xxx__ convention served a few purposes. First, it indicates that the method will be called in some means OTHER than by name (generally, the interpreter invokes it directly, although in this case it's a built-in method of dict that would invoke it). Secondly, it serves to flag the method as being special -- true newbies can safely ignore nearly all special methods aside from __init__(). And it serves to create a separate namespace... writers of Python code know that names beginning and ending with double-underscores are "reserved for the language". Of these, I always felt that special invocation was the most important feature. The next() method of iterators was an interesting object lesson. The original reasoning (I think) for using next() not __next__() was that *sometimes* the method was called directly by name (when stepping an iterator manually, which one frequently does for perfectly good reasons). Since it was sometimes invoked by name and sometimes by special mechanism, the choice was to use the unadorned name, but later experience showed that it would have been better the other way. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes.from_hex()
I wrote: > ... I will say that if there were no legacy I'd prefer the tounicode() > and tostring() (but shouldn't itbe 'tobytes()' instead?) names for Python 3.0. Scott Daniels replied: > Wouldn't 'tobytes' and 'totext' be better for 3.0 where text == unicode? Um... yes. Sorry, I'm not completely used to 3.0 yet. I'll need to borrow the time machine for a little longer before my fingers really pick up on the 3.0 names and idioms. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] iterator API in Py3.0
Raymond writes: > The double underscore convention is appropriate where the method is always > invoked magically in normal code and not called directly. The next() method is > differenct because it is a mixed case, sometimes called magically and sometimes > called directly. In the latter case, the name is highly visible and therefore > should not have double underscores. I think it's clear that if a method is invoked magically, it ought to have underscores; if it is invoked directly then it ought not to. next() is invoked both ways, so the question is which of the following invariants we would rather maintain: * Any method that gets invoked 'magically' (by syntax) will have underscores. * Any method with underscores should be invoked directly only if you are practicing deep magic. What I would *really* like is to keep both invariants... but the only way to do that is to have a __next__() method paired with a next() builtin. (The pattern we use for things like len().) This is great for everyone except for (1) backward compatibility, (2) performance (but does the indirection really need to hurt, and how much?), and (3) proliferation of builtins (adding one more for next() won't hurt, but if we make a practice of it we'll eventually have too many). All told, I prefer using the underscores. I think that the first invariant is important. But it's a judgement call, and a close one, so I'll be happy with either decision. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] .len() instead of __len__() (was: iterator API inPy3.0)
Oleg Broytmann writes that he thinks methods are inherently "better" than
methods. He asks for advantages of a function over a method, after first
presenting his arguments for methods over functions:
> -- a method is a more natural way to query or manipulate objects;
> -- a direct method call is faster than two-level call (len() => .__len__());
> -- unnecessary cluttering of builtin namespace; the namespace should(1)
>contains only really global functions (__import__(), eval(), etc.)
Responding to your first point, "a more natural way" is not an argument...
it's an expression of your personal preference.
Your second point (performance) has some merit -- although one would hope
the difference is small in a good implementation. And your third point
(cluttering the builtin namespace) is a valid and real reason to prefer
methods.
In defense of functions, my first exhibit is the following snippet of Java
code:
/** Returns b^2 - 4ac */
public BigDecimal determinant(BigDecimal a, BigDecimal b, BigDecimal c) {
return b.multiply(b).subtract(new BigDecimal(4).multiply(a).multiply(c));
}
In other words, some things are less clear when written using the notation
used for method invocation. That is particularly true when there are
multiple objects involved and no clear reason to prefer one over another
as the "object performing the action". In languages with multiple dispatch,
like CLOS, everything looks like a function for exactly this reason.
My second exhibit is also some Java code:
public int numItems(Object o) {
if (o instanceof Collection) { // includes List, Set
return ((Collection) o).size();
} else if (o instanceof Dictionary) {
return ((Dictionary) o).size();
} else if (o instanceof CharSequence) { // includes String, StringBuffer
return ((CharSequence) o).length();
} else if (o instanceof Object[]) {
return ((Object[]) o).length;
} else {
throw new IllegalArgumentException();
}
}
The inconsistancy here is amazing... length, length(), size(). You could
put it down to poor interface design by the Java team, but you must
admit, individual objects are designed in isolation, but when there
exists a common lanugage protocol like Python's builtin len() and __len__,
that developers of individual objects go out of their way to make them
conform with the common protocol... and in the end that makes them more
usable.
I'm sure there are additional arguments, but these are the first few I
came up with. Of course, in nearly all cases, I prefer using methods...
but len() is such a powerful concept applicable across MANY diverse
kinds of objects, that I have no objection to granting it space in the
builtin namespace.
By the way... you write:
> (1) I am a perfectionist and I partially disagree with "practicality beats
> purity"; I would really love a small uncluttered builtin namespace in
> Python.
I just wanted to point out that one of the things I *LOVE* about Python
is that the design of name resolution in Python ensures that no
programmer need suffer from the size of the builtin namespace. If you
never use certain builtins, perhaps "hex" or "super", then go ahead
and use those as identifiers in your code. Avoid only that portion of
the builtin namespace that you feel is worth keeping.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] (no subject)
Jim Jewet writes:
> I can't imagine [a conditional] expression small enough that [requiring]
> parentheses really hurt.
>
> var = t if c else f
> var = (t if c else f)
This is a good example. I'm now +1 on adding this to PEP 8. I'm +0 on
adding it to the grammer... in the long run it's unlikely to make any
difference.
Steve Holden writes:
> Contrast with the bleeding obvious:
>
> level = 0
> if "absolute_import" in self.futures:
> level = -1
Comparing this to the alternative:
level = (0 if ("absolute_import" in self.futures) else -1)
In the latter, level clearly takes on values of 0 or -1. In the
former, level has a default value of 0, which is followed by what
appears to be some branching logic.
> or even, if a certain obscurity is desirable:
>
> level = - ("absolute_import" in self.futures)
The only USE I have ever seen for code obscurity was in writing
certain vote-counting software where the true behavior needed
to remain obscure even in the face of source code publication.[1]
-- Michael Chermside
[1] http://graphics.stanford.edu/~danielrh/vote/scores.html
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Coverity Open Source Defect Scan of Python
Dennis Allison writes: > I'd also encourage Coventry to explain their business model a bit more > clearly. Ben Chelf writes: > Of course it's no surprise that I see open > source projects everywhere -- as part of infrastructure or part of code > bases that people are developing. So from a Coverity perspective, > clearly we want to provide source code analysis for the projects that > our customers care about [...] > I really just want every developer to use source code analysis > while they write code [...] > We got a lot of the > good publicity in the research lab because there existed this big open > source OS that we could test our theories on. [...] > I think it makes sense for Coverity to have a strong relationship with > the open source community since that community has been helping us > pretty much since day 1 I work for a business... and we pay full price for the tools that we use in our work. I am aware of and follow the work that I and my colleagues do -- when someone has a good idea, I tend to learn from that and introduce the same idea in future projects. I also am aware of and follow the work of quite a few open source projects (to different degrees depending on the project). In fact, I see far more open source projects than I do other projects. I learn a lot of good ideas from these projects also, and I use them in my paid work. For example, it was my experience with open source projects that convinced me that extensive unit tests that are expected to always pass was a feasable and useful idea. Many of the ways that open source projects are managed are also beginning to work their way into my professional life also. It's just that I see far more open source projects than others, and frankly the open source projects are almost always better run, with higher standards. I doubt I'm the only one... I think open source will be leading the way in software development standards for some time now. So I think offering a software development tool FREE to open source projects in hopes of selling it for money to comercial projects is a WONDERFUL business model. Good luck! -- Michael Chermside (PS: too bad I can't buy stock in Coverity. How come all the GOOD companies are private? I had to wait around 6 years before I could buy stock in Google.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Threading idea -- exposing a global thread lock
Josiah Carlson writes: > It would be nice if Jython or IronPython could (and would) implement > these 'critical sections'. Whether they can or not, I think that it > would be a useful feature in the CPython runtime. The issue is not whether Jython and IronPython "will", it's whether they "can". Essentially, Jython and IronPython both use locking within their fundamental data structures. This then allows them to freely allow threads to run on multiple processors. Meanwhile, CPython lacks locks in its fundamental data structures, so it uses the GIL to ensure that code which might touch Python data structures executes on only one CPU at a time. The concept of a "critical section" makes great sense when there is effectively only one CPU: just stop switching threads. But if code is using multiple CPUs, what does it mean? Shut down the other CPUs? To do such a thing cooperatively would require checking some master lock at every step... (a price which CPython pays, but which the implementations built on good thread-supporting VMs don't have to). To do such a thing non-cooperatively is not supported in either VM. Since I doubt we're going to convince Sun or Microsoft to change their approach to threading, I think it is unwise to build such a feature into the Python language. Supporting it in CPython only requires (I think) no more than a very simple C extension. I think it should stay as an extension and not become part of the language. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GeneratorExit inheriting from Exception
Barry writes: > I still believe in this, and I'm thankful for the support I've seen. It > won't happen for Python 2.x, but I do plan on addressing this for Py3K. When you do, I'd like you to consider one change to the names. You are proposing this: > Exception > +- KeyboardInterrupt > +- GeneratorExit > +- SystemExit > +- StopIteration > +- Error > | +- ImportError > | +- (etc.) [...] If we use that structure, I still strongly feel that what you call "Error" ought to be named "Exception" (and what you call "Exception" should be named "BaseException" or perhaps "Raisable"). The reason is this: Most people coming to Python today are coming from languages more modern than Fortran and C, and (if they have any programming background at all) they are familiar with the idea of an exception. When chatting around the water cooler, they probably call it "an exception", rather than "a dingus", "one of those thingys that unwinds the stack until reaching a handler", or even "an error". (The term "an error" is usually used around the water cooler for any kind of mistake, whether human or programatic.) This is encouraged in Python by the fact that the keyword for handling exceptions in Python is "except". But we don't want people using the dingus at the top of the hierarchy -- only a few experts who know what they're doing should care about that one. We want them using the one that you call "Error". So by naming that one "Exception", we take advantage of the meaning of the word, the experience from other languages, and the keywords of the language to encourage correct usage. Naming the top level dingus something slightly odd is a small price to pay (although both "BaseException" and "Raisable" seem instantly comprehensible to me). I present Java as an example. Given it's position as the new Cobol, there are literally hundreds of thousands of not-very-skillful Java programmers out there. Yet it is rare to see them write "catch Throwable" when they should use "catch Exception". That mistake is far more common among beginning Python users. Surely we can do at least as well as Java! -- Michael Chermside PS: I've intentionally ignored the question Guido raised about whether Warning belongs under your "Error" or your "Exception". ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pysqlite for 2.5?
Gerhard Häring writes: > db.sql.sqlite is another possibility, if adding something like Durus or > ZODB in the same top-level namespace could be considered for 2.6. Flat is better than nested. I see no reason why we couldn't have all of this: database.sqllite database.zodb database.duras database.oracle there's no need to group the SQL databases. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
In the discussion over class decorators, Jim Jewett writes: > I have often started with a function, and ended up replacing it with a > callable object so that I could save state without resorting to > "defalt args" or worse. > > I would prefer to decorate these exactly like the functions they replace. I have observed the entire discussion about class decorators with absolutely no opinion, until I read Jim's brief post quoted above. I am now completely convinced that class decorators ought to exist and behave exactly like function decorators. Thanks, Jim for pointing out what should have been obvious to me from the start. The ability to use callable objects as functions is a powerful tool in Python, and ought not be broken by decorator inconsistencies. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3101 Update
One small comment:
> The conversion specifier consists of a sequence of zero or more
> characters, each of which can consist of any printable character
> except for a non-escaped '}'.
"Escaped"? How are they escaped? (with '\'?) If so, how are backslashes
escaped (with '\\'?) And does the mechanism un-escape these for you
before passing them to __format__?
Later...
> - Variable field width specifiers use a nested version of the {}
> syntax, allowing the width specifier to be either a positional
> or keyword argument:
>
> "{0:{1}.{2}d}".format(a, b, c)
This violates the specification given above... it has non-escaped '}'
characters. Make up one rule and be consistant.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3101 Update
I wrote:
> > - Variable field width specifiers use a nested version of the {}
> > syntax, allowing the width specifier to be either a positional
> > or keyword argument:
> >
> > "{0:{1}.{2}d}".format(a, b, c)
>
> This violates [the rule that '}' must be escaped]
Talin writes:
> What would you suggest? I'd be interested in hearing what kinds of
> ideas people have for fixing these problems.
Honestly? I propose that the PEP not provide any means to pass
format specifiers as arguments. After all, one PEP need not support
everything anyone would ever want to do with string interpolation.
Fixed format attributes are powerful enough for nearly all
applications, and those needing a more powerful tool can either use
multiple interpolation passes (simple and obvious, but a bit ugly)
or define a custom formatting class.
In fact, if _I_ were writing the PEP, I'd probably have opted for
simplicity in a few more areas. For one thing, where you have:
>string.format() will format each field using the following steps:
>
> 1) See if the value to be formatted has a __format__ method. If
>it does, then call it.
>
> 2) Otherwise, check the internal formatter within string.format
>that contains knowledge of certain builtin types.
>
> 3) Otherwise, call str() or unicode() as appropriate.
I would have left out (2) by adding __format__ methods to those
built-in types just to keep the description (and the implementation)
as simple as possible.
And probably I would have ommitted the whole ability to support custom
formatting classes. Not that they're not a nice idea, but they are a
separate feature which can be built on top of the basic format() method
and I would have tried to take one small step at a time.
But please don't read this as saying I object to the above ideas...
I wouldn't have said anything if you hadn't asked what my approach
would be.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] feature request: inspect.isgenerator
Phillip J. Eby writes: > Yes, I think the whole concept of inspecting for this is broken. > *Any* function can return a generator-iterator. A generator > function is just a function that happens to always return one. Just following up on Phillip's comments, consider the following functions: def foo(x): while still_going(x): yield some_func(x) def bar(x): while still_going(x): yield other_func(x) def foo_or_bar(x): if some_condition(x): return foo(x) else: return bar(x) I presume that Michele's proposal is that inspect.isgenerator() (or perhaps "inspect.isgenfunc()") would return True for "foo" and "bar" but false for "foo_or_bar". Can you give a single use case for which that distinction is desirable? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Benchmarks
Marc-Andre Lemburg writes: > Using the minimum looks like the way to go for calibration. > > I wonder whether the same is true for the actual tests; since > you're looking for the expected run-time, the minimum may > not necessarily be the choice. No, you're not looking for the expected run-time. The expected run-time is a function of the speed of the CPU, the architechure of same, what else is running simultaneously -- perhaps even what music you choose to listen to that day. It is NOT a constant for a given piece of code, and is NOT what you are looking for. What you really want to do in benchmarking is to *compare* the performance of two (or more) different pieces of code. You do, of course, care about the real-world performance. So if you had two algorithms and one ran twice as fast when there were no context switches and 10 times slower when there was background activity on the machine, then you'd want prefer the algorithm that supports context switches. But that's not a realistic situation. What is far more common is that you run one test while listening to the Grateful Dead and another test while listening to Bach, and that (plus other random factors and the phase of the moon) causes one test to run faster than the other. Taking the minimum time clearly subtracts some noise, which is a good thing when comparing performance for two or more pieces of code. It fails to account for the distribution of times, so if one piece of code occasionally gets lucky and takes far less time then minimum time won't be a good choice... but it would be tricky to design code that would be affected by the scheduler in this fashion even if you were explicitly trying! Later he continues: > Tim thinks that it's better to use short running tests and > an accurate timer, accepting the added noise and counting > on the user making sure that the noise level is at a > minimum. > > Since I like to give users the option of choosing for > themselves, I'm going to make the choice of timer an > option. I'm generally a fan of giving programmers choices. However, this is an area where we have demonstrated that even very competent programmers often have misunderstandings (read this thread for evidence!). So be very careful about giving such a choice: the default behavior should be chosen by people who think carefully about such things, and the documentation on the option should give a good explanation of the tradeoffs or at least a link to such an explanation. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)
Guido writes: [discussion of how to fix the can't-bind-outer-scope-vars wart] > I think we have to continue to search for a solution that extends the > idea of global declarations. > > I've proposed extending its meaning to refer to the nearest outer > scope where the variable is set; if there is no such scope it's an > error. This will break a small number of program but probably not very > many; still, it'll require a future statement or waiting until Python > 3.0. The downside is that "global" is not a very intuitive word for > this new meaning. I disagree with your last statement -- I think "global" _is_ a very intuitive word for this. As I understand it, in programming "global" has two meanings, closely intertwined. One is "universal, same throughout the system". For instance, "The singleton pattern is used to create a single, global instance of a type." The second meaning is the term "global variable". This term developed (I believe) in languages that had only two scopes: local-to-current-function and global-to-entire-program. But the term "global variable" refers to any variable whose assignment is a "side effect", regardless of whether that variable is global-to-entire-program, global-to-module, or even global-to-enclosing-function. I have even heard the term "global variable" (mis)used to refer to any kind of side effect. Anyhow, in Python only builtins is _really_ global -- even today's global keyword only refers to module scope. So I believe that it would be a very reasonable interpretation of "global" to mean "not local", and implement as "search enclosing scopes in order to find the binding". -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)
Phillip Eby writes: > I don't see a problem with requiring '.x' to be used for both > reading and writing of outer-scope names; it just shouldn't be > required for an outer-scope name that you don't rebind in the > current scope. > > def counter(num): > def inc(): > .num += 1 > return .num > return inc I am reminded of Tim Peter's declaration in response to a similar proposal some time ago: Syntax should not look like grit on my monitor. (Sorry, no reference... but I swear it's word-for-word accurate because the quote burned itself into my memory.) -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)
Phillip Eby writes: > The big problem that comes to mind with that idea is that it makes > it impossible to have argument names that are the same as attribute > names, unless the 'whee'/'.whee' prohibition were relaxed. :( But > it's an intriguing thought, nonetheless. My three-year-old has been working on that 'whee'/'.whee' prohibition, but he hasn't mastered it yet. Gotta-go-wash-another-load-of-underpants -lly yours, Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
Ka-Ping Yee writes: > If you mean getting from a trusted interpreter to an untrusted > interpreter -- then how is a resource going to travel between > interpreters? Brett Cannon responds: > Beats me, but I am always scared of Armin and Samuele. =) Okay, those two scare me also, but I would still rather not spread FUD. Your proposal contains lots of details about how to address the danger that Python objects can cross from one interpreter to another. Could we instead attack that straight-on and try to find a convincing proof that objects cannot possibly cross the interpreter barrier? If so, it would simplify a bit of your proposal, and make me feel a little less worried. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] In defense of Capabilities [was: doc for new restricted execution design for Python]
oing the security checks
directly, it could make sure that the objects returned were
contained within the correct secure wrappers. That's OK so far
as it goes, but the C checks are per interpreter instance, so
how do we get them to apply the correct wrappers? Maybe the
interpreter maintains (in C) a stack of wrappers per-thread and
provides a means for stack frames (functions) to register
wrappers? We would wind up wrapping the wrappers, but that's
just the nature of capabilities. Perhaps it would look
something like this:
def invoke_user_function():
PyXXX.set_file_wrapper(ReadOnlyRestriction)
PyXXX.set_socket_wrapper(
SingleDomainRestriction('example.com'))
untrusted_object.do_stuff()
...
To sum up: I agree that you cannot rely on prevent all the
possible "python tricks", but I still think that capabilities
are a superior solution. I'd like to find a way to achieve
the user-customizability of capabilities without throwing
out the entire standard library -- maybe some hybrid of
"resource hiding" and "resource crippling". I can almost see
a way to achieve it, but there are still a few flaws.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
I wrote: > I would still rather not spread FUD. Brett Cannon responds: > I don't consider it FUD. Armin in an email said that he thought it > was a losing battle to try to hide 'file' from an interpreter. That > is what I am worried about, period. Everythign else can be > protected through resource hiding. I never intended to say that "we can't hide 'file' from the user" was FUD. Only objects crossing between interpreters. And it *might* not be FUD, but so far I haven't heard anyone say that they thought objects *could* cross between interpreters. I think your proposal would read better with a categorical statement that objects cannot cross between interpreters (right along with your categorical statement that Python code cannot directly access dangerous resources without help from the C level), so I present the following challenge: can anyone think of a way that an object could "cross" interpreters? For instance, are certain basic objects shared (str and int for instance) and objects could be passed between interpreters by attaching them as attributes of these core objects? I just don't know enough about multiple interpreters to be sure -- but somehow I thought they had separate object pools. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted execution: what's the threat model?
Ka-Ping Yee writes: > i'm starting to think > that it would be good to clarify what kinds of threats we are > trying to defend against, and specify what invariants we are > intending to preserve. Yes! > So here are a couple of questions for clarification (some with my > guesses as to their answers): Okay, I'll throw in my thoughts also. > 1. When we say "restricted/untrusted/ interpreter" we > don't really mean that the *interpreter* is untrusted, right? > We mean that the Python code that runs in that interpreter is > untrusted (i.e. to be prevented from doing harm), right? Agreed. My interpretation of the proposal was that interpreters were either "sandboxed" or "trusted". "Sandboxed" means that there are security restrictions imposed at some level (perhaps even NO restrictions). "Trusted" means that the interpreter implements no security restrictions (beyond what CPython already implements, which isn't much) and thus runs faster. > 2. I'm assuming that the implementation of the Python interpreter > is always trusted Sure... it's got to be. > What do > we take the Trusted Computing Base to include? The Python VM > implementation -- plus all the builtin objects and C modules? > Plus the whole standard library? My interpretation of Brett's proposal is that the CPython developers would try to ensure that Python VM had no "security holes" when running in sandboxed mode. Of course, we also "try" to ensure no crashes are possible also, and while we're quite good, we're not perfect. Beyond that, all pure-python modules with source available (whether in the stdlib or not) can be "trusted" because they run in a sandboxed VM. All C modules are *up to the user*. Brett proposes to provide a default list of useful-but-believed-to-be-safe modules in the stdlib, but the user can configure the C-module whitelist to whatever she desires. > 3. Is it part of the plan that we want to protect Python code from > other Python code? For example, should a Python program/function > X be able to say "i want to launch/call program/function Y with > *these* parameters and have it run under *these* limitations?" > This has a big impact on the model. Now *that* is a good question. I would say the answer is a partial "no", because there are pieces of Brett's security model that are tied to the interpreter instance. Python code cannot launch another interpreter (but perhaps it *should* be able to?), so it cannot modify those restrictions for new Python code it launches. However, I would rather like to allow Python code to execute other code with greater restrictions, although I would accept all kinds of limitations and performance penalties to do so. I would be satisfied if the caller could restrict certain things (like web and file access) but not others (like memory limits or use of stdout). I would satisfied if the caller paid huge overhead costs of launching a separate interpreter -- heck, even a separate process. And if it is willing to launch a separate process, then Brett's model works just fine: allow the calling code to start a new (restricted) Python VM. > We want to be able to guarantee that... > > A. The interpreter will not crash no matter what Python code > it is given to execute. Agreed. We already want to guarantee that, with the caveat that the guarantee doesn't apply to a few special modules (like ctypes). > B. Python programs running in different interpreters embedded > in the same process cannot communicate with each other. I don't want to guarantee this, does someone else? It's astonishingly hard... there are all kinds of clever "knock on the walls" tricks. For instance, communicate by varying your CPU utilization up and down in regular patterns. I'd be satisfied if they could pass information (perhaps even someday provide a library making it *easy* to do so), but could not pass unforgable items like Python object references, open file descriptors, and so forth. > C. Python programs running in different interpreters embedded > in the same process cannot access each other's Python objects. I strengthen that slightly to all "unforgable" items, not just object references. > D. A given piece of Python code cannot access or communicate > with certain Python objects in the same interpreter. > > E. A given piece of Python code can access only a limited set > of Python objects in the same interpreter. Hmmm. I'm not sure. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what can we do to hide the 'file' type?
Armin Rigo writes: > I don't think I can "sign off" [on hiding the file type]. Really hiding > Python objects is quite hard IMHO. I agree. But we don't have to give up yet. How about instead of hiding file, we cripple it. Completely. Modify the file type so that when executing on a sandboxed interpreter, all of the dangerous methods and attributes of file throw exceptions. Then we create a separate thing (in C) called a "SecureFileWrapper". It has methods that are passed a reference to a file object and can invoke the methods without error. We provide a means for obtaining a SecureFileWrapper bound to a given file (perhaps open()). Essentially, we give up on hiding file, which is a frequently-used type, and very hard to hide, and instead we rely on our ability to write a reliably secure "SecureFileWrapper" class (in C). -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)
I quoted this unwritten bit of Python Zen, attributing it to Tim: > Syntax should not look like grit on my monitor. mwh writes: > I think it was Anthony: > > http://mail.python.org/pipermail/python-dev/2005-July/054581.html But that's not the original. Turns out, it WAS Anthony, and I had misquoted. The correct source appears to be: http://mail.python.org/pipermail/python-dev/2004-August/047179.html And the correct quote (for the record) is: Syntax Shall Not Look Like Grit On Tim's Monitor. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what can we do to hide the 'file' type?
Me: > I agree. But we don't have to give up yet. How about instead of hiding > file, we cripple it. Completely. Modify the file type so that when > executing on a sandboxed interpreter, all of the dangerous methods > and attributes of file throw exceptions. Brett: > This is basically what I proposed in the first place! in circles, pulling at his hair like a crazy man> Not quite. Your original proposal had the file type throwing exceptions when the user did something they weren't allowed to (access a file not on the list, etc). This version proposes that it *always* fails, and creates a separate beast (the SecureFileWrapper) for applying the restrictions. Why? Because if the C code in file enforces the rules, then all possible rules need to be written in advance, and you have to hold long arguments about whether to allow subdirectories, restrict file sizes, etc. Whereas SecureFileWrapper could delegate its restrictions to Python functions provided by the USER and then USERS could design whatever level of restriction they wanted. Imagine we want some code to be able to open files only if they contain the letter 'w': # get my own wrapper from __builtins__ myFileWrapper = file # define a function to constrain my callers def filenameContainsLetterW(path, mode): filename = os.path.basename(path) if 'w' not in filename and 'W' not in filename: raise PyXXX_SecurityException # create more restrictive wrapper class MustHaveW_FileWrapper: __metaclass__ = SecureFileWrapperMeta wrapped_file = init_condition = filenameContainsLetterW # register the wrapper so it applies to any code # in this stack frame or lower. The restriction is # automatically lifted when the current stack # frame exits. PyXXX_RegisterFileWrapper( MustHaveW_FileWrapper ) # Invoke less-trusted code with restrictions in place less_trusted_code() If the code fragment shown above ALREADY received a wrapped form of file which was restricted to read-only access, then less_trusted_code() would be restricted to read-only access to files containing 'w'. Okay, the syntax needs work, but the idea is that I can defin restrictions *in python* and apply them to other code. Using the stack to enforce wrappers is something Python code cannot get around (although it does prevent passing more-powerful callbacks to later stackframes). It all depends on whether we think that a simple set of restrictions implementable in C (such as 4 lists: read-only files, read-write files, read-only dirs, write-only dirs) will be sufficient, or if it is valuable to allow end users to fine-tune the restrictions. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple expression eval in compound if statement?
Gerrit Holl writes: > What would happen if... Raymond replies: > Every molecule in your body would simultaneously implode at the speed of > light. So you're saying it triggers C-language "undefined behavior"? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicitly declaring expected exceptions for ablock
Dmitry Dvoinikov writes:
> The reason for that being self-tests with lots and lots of
> little code snippets like this:
>
> try:
> c().foo()
> except TypeError:
> pass
Paul Du Boise already responded explaining that PEP 343 probably handles
the task you want. I just wanted to mention that you may need to
reconsider the task. The above snippet is almost certainly incorrect.
I suspect that you wanted either:
try:
c().foo()
fail('Should have raised TypeError')
except TypeError:
pass # expected
or perhaps this:
try:
c().foo()
except TypeError:
fail('Should not have raised TypeError')
There ARE situations when you want to allow an exception (but not
necessarily expect it) and do nothing when it occurs, but I don't
find them all that common, and I certainly don't find them arising
in unit tests.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess.call() and stdin
Stuart Bishop writes: > When I invoke subprocess.call(), I often want to ensure that the subprocess' > stdin is closed. This ensures it will die if the subprocess attempts to read > from stdin rather than block. > > This could be done if the subprocess.call() helper closes the input if > stdin=subprocess.PIPE, which may be the only sane way to handle this > argument (I can't think of any use cases for spawning a subprocess with an > input stream that nothing can write too). +0.5. I agree that if you pass "stdin=subprocess.PIPE" to subprocess.call() that the current behavior of having the child process block forever is totally useless. I have little reason to prefer "assume an empty input" over "let subprocess.call() raise an exception if stdin==subprocess.PIPE" -- but if I take your word for it that this is a common need, then that's one good reason. > It could also be done by adding a subprocess.CLOSED constant, which if > passed to Popen causes a new closed file descriptor to be given to the > subprocess. -1. It is easy enough to create a closed FD to read from... why complicate the API? -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal floats as default (was: discussion aboutPEP239 and 240)
Fredrik Johansson writes: > In either case, compatibility can be ensured by allowing both n-digit > decimal and hardware binary precision for floats, settable via a float > context. Perhaps you can show me a design (or working code) that proves me wrong, but I don't believe that such a design could be made compatible with the existing Decimal module... certainly while continuing to maintain compatibility with the Cowlinshaw specification. > There is the alternative of providing decimal literals by using > separate decimal and binary float base types If, by this, you mean adding a "binary float context" modeled after the Decimal float context and providing access to the underlying FP flags and traps and generally enhancing the use of binary FP, then I think it's a great idea. It's probably impossible to write in a cross-platform manner (because C supplies support for binary FP but does not offer access to the flags and traps), but this is one of those few cases where it's worth using platform-and-compiler specific code. Of course, someone still has to step forward and offer to code it. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Terminology for PEP 343
Raymond writes: > On a separate note, I also propose that __exit__() be renamed to > __leave__(). The enter/leave word pairing are a better fit in standard > English: I don't feel strongly about it, but I tend to disagree. Certainly you will admit that enter-exit IS a gramatically correct pairing in English. Raymond argues that: > Google has no shortage of hits for the pairing: Yes, but google has about half as many hits for the enter-exit pairing. I think a factor of 2 is not really that big for this sort of thing. I would take that to mean that enter-leave and enter-exit are roughly equally frequent pairs with enter-leave being somewhat more prevelant. > FWIW, there is precedent in x86 assembly language: > Likewise, GUI programmers use Enter and Leave for mouse events. These precedents stand. I don't find them compelling, but they are certainly valid. In favor of "exit" over "leave" I have only two arguments. First, I just like "exit" better. Hmm... didn't find that compelling? I was afraid of that. The second (also not a very strong argument) is that "exit" is somewhat more distinct in meaning than "leave". Somehow for me "exit" has a faint connotation of a distinct boundary being crossed, while "leave" is somewhat less crisp. I would use "She left the ballpark." for someone walking into the dugout, climbing up into the stands, or walking out of the stadium, and might refer to the entire process. Using "She exited the ballpark." to me suggests the exact moment that she went through the door dividing outside from inside. If you accept this connotation, then it's clear that "exit" is a closer match for the with statement's behavior than "leave". Anyway, this stuff is always very subjective and, as I said, I'm just expressing an opinion. So take it for what it's worth. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'With' context documentation draft (was Re: Terminology for PEP 343
Nick writes sample documentation: > For example, the following context manager allows prompt closure of > any resource with a 'close' method (e.g. a generator or file): > > @context > def closing(resource): > try: > yield resource > finally: > resource.close() Reading this I get the feeling that perhaps the decorator should be named "context_manager" not "context": @context_manager def closing(resource): try: yield resource finally: resource.close() Does anyone else agree? Paul Moore writes: > I also like the fact that it offers a neat 1-word name for the > generator decorator, "@context". Well, ok... does anyone *else* agree? I too saw this and thought "neat! a simple one-word name!". But then I started worrying that it's not defining the *context*, but rather the *context manager*. While "context manager" is a term I could easily imagine associating only with 'with' statements, "context" is too general a term... even after Python supports 'with' statements I will continue to use "context" to mean lots of different things (eg: decimal.context). By the way, great job Nick... these docs read quite nicely. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible context managers in stdlib
James Y Knight writes: > It is a really bad idea to codify the practice of modifying non- > threadlocal global state like sys.std[in|out|err] and current > directory with a context manager. Barry Warsaw responds: > Thinking about the types of code I write over and over again, I think I > disagree (slightly) with James about the global state thing. While I > agree that 'with redirected_stdio' isn't too useful in the face of > print>>, I very often have to write try/finally protections around > temporary settings of the cwd, the umask, and other global process > values. I'd love to see cm's for those constructs in the stdlib. I agree with Barry. Not only should they be in the stdlib, but they should have very clear warnings in their docstrings and other documentation that state that they are ONLY safe to use in single-threaded programs. This achieves two things: it makes them available to those who need them (not everyone uses threads!), and it rather forcefully makes the point that it's NOT usually a good idea to modify global state info in a context manager because doing so is not generally threadsafe. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible context managers in stdlib
I wrote: > I agree with Barry. Not only should they be in the stdlib, but they > should have very clear warnings in their docstrings and other documentation > that state that they are ONLY safe to use in single-threaded programs. > > This achieves two things: it makes them available to those who need > them (not everyone uses threads!), and it rather forcefully makes the > point that it's NOT usually a good idea to modify global state info in > a context manager because doing so is not generally threadsafe. Nick Coghlan replies: > Wouldn't they be able to actually emit a warning at run-time if they're used > in a multi-threaded program? That would be even better motivation for > including them, IMO. I don't think that would be desirable. These things CAN be useful in a multi-threaded program if you know what you're doing. One common example would be to use them only from the main thread. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFEfor review)
M A Lemburg writes: > we should use strings and Unicode > like they are supposed to be used (in the context of text > data): > > * strings are fine for text data that is encoded using > the default encoding > > * Unicode should be used for all text data that is not > or cannot be encoded in the default encoding > > Later on in Py3k, all text data should be stored in Unicode > and all binary data in some new binary type. Wow. That is the most succinct and clear explanation of how to use unicode in Python that I think I've ever heard. It might even be simple enough for _me_ to understand it! I think I'm going to go frame this and have it posted in my cubical. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Revised PEP 349: Allow str() to return unicodestrings
Neil Schemenauer wrote:
> The PEP has been rewritten based on a suggestion by Guido to change
> str() rather than adding a new built-in function. Based on my testing, I
> believe the idea is feasible.
Fredrik Lundh replies:
> note that this breaks chapter 3 of the tutorial:
>
> http://docs.python.org/tut/node5.html#SECTION00513
>
> where str() is first introduced.
It's hardly "introduced"... the only bit I found reads:
... When a Unicode string is printed, written to a file, or converted
with str(), conversion takes place using this default encoding.
>>> u"abc"
u'abc'
>>> str(u"abc")
'abc'
>>> u"äöü"
u'\xe4\xf6\xfc'
>>> str(u"äöü")
Traceback (most recent call last):
File "", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode characters in position
0-2: ordinal not in range(128)
To convert a Unicode string into an 8-bit string using a specific encoding,
Unicode objects provide an encode() method that takes one argument, the
name of the encoding. Lowercase names for encodings are preferred.
>>> u"äöü".encode('utf-8')
'\xc3\xa4\xc3\xb6\xc3\xbc'
I think that if we just took out the example of str() usage and replaced
it with a sentence or two that DID introduce the (revised) str() function,
it ought to work. In particular, it could mention that you can call str()
on any object, which isn't stated here at all.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bare except clauses in PEP 348
Raymond Hettinger writes: > The latest version of PEP 348 still proposes that a bare except clause > will default to Exception instead of BaseException. Initially, I had > thought that might be a good idea but now think it is doomed and needs > to be removed from the PEP. Guido writes: > If we syntactically enforce that the bare except, if present, must be > last, would that remove your objection? I agree that a bare except in > the middle is an anomaly, but that doesn't mean we can't keep bare > except: as a shorthand for except Exception:. Explicit is better than Implicit. I think that in newly written code "except Exception:" is better (more explicit and easier to understand) than "except:" Legacy code that uses "except:" can remain unchanged *IF* the meaning of "except:" is unchanged... but I think we all agree that this is unwise because the existing meaning is a tempting trap for the unwary. So I don't see any advantage to keeping bare "except:" in the long run. What we do to ease the transition is a different question, but one more easily resolved. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FW: Bare except clauses in PEP 348
Raymond writes: > Hey guys, don't give up your bare except clauses so easily. [...] Raymond: I agree that when comparing: // Code snippet A try: ... except SomeException: ... except BaseException: ... with // Code snippet B try: ... except SomeException: ... except: ... that B is nicer than A. Slightly nicer. It's a minor esthetic point. But consider these: // Code snippet C try: ... except Exception: ... // Code snippet D try: ... except: ... Esthetically I'd say that D is nicer than A for the same reasons. It's a minor esthetic point. But you see, this case is different. You and I would likely never bother to compare C and D because they do different things! (D is equivalent to catching BaseException, not Exception). But we know that people who are not so careful or not so knowlegable WILL make this mistake... they make it all the time today! Since situation C (catching an exception) is hundreds of times more common than situation A (needing default processing for exceptions that don't get caught, but doing it with try-except instead of try-finally because the nothing-was-thrown case is different), I would FAR rather protect beginners from erroniously confusing C and D than I would provide a marginally more elegent syntax for the experts using A or B. And that elegence is arguable... there's something to be said for simplicity, and having only one kind of "except" clause for try statements is clearly simpler than having both "except :" and also bare "except:". -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bare except clauses in PEP 348
Raymond writes: > Efforts to improve Py3.0 have spilled > over into breaking Py2.x code with no compensating benefits. [...] > We don't have to wreck 2.x in order to make 3.0 better. I think you're overstating things a bit here. > Remember, the ONLY benefit from the whole PEP is that in 3.0, it will no > longer be necessary to write "except (KeyError, SystemExit): raise". > [...] IOW, there's nothing worth inflicting destruction on tons of > 2.x code. And now I *KNOW* you're overstating things. There are LOTS of benefits to the PEP in 3.0. My own personal favorite is that users can be guaranteed that all exceptions thrown will share a particular common ancestor and type. And no one is proposing "destruction" of 2.x code. On the other hand, I thought these were very good points: > Bare > except clauses appear in almost every Python book that has ever been > written and occur at least once in most major Python applications. [...] > I had thought the plan was to introduce Py3.0 capabilities into 2.x as > they become possible but not to break anything. [...] > I propose that the transition plan be as simple as introducing > BaseException. This allows people to write code that will work on both > 2.x and 3.0. I think the situation is both better than and worse than you describe. The PEP is now proposing that bare "except:" be removed in Python 3.0. If I understand Guido correctly, he is proposing that in 2.5 the use of bare "except:" generate a PendingDeprecationWarning so that conscientious developers who want to write code now that will continue to work in Python 3.0 can avoid using bare "except:". Perhaps I'm misreading him here, but I presume this was intended as a PENDINGDeprecationWarning so that it's easy to ignore. But it's a bit worse than it might seem, because conscientious users aren't ABLE to write safe 2.5 code that will run in 3.0. The problem arises when you need to write code that calls someone else's library but then unconditionally recovers from errors in it. Correct 2.4 syntax for this reads as follows: try: my_result = call_some_library(my_data) except (KeyboardInterrupt, MemoryError, SystemError): raise except: report_error() Correct 3.0 syntax will read like this: try: my_result = call_some_library(my_data) except (KeyboardInterrupt, MemoryError, SystemError): raise except BaseException: report_error() But no syntax will work in BOTH 2.5 and 3.0. The 2.4 syntax is illegal in 3.0, and the 3.0 syntax fails to catch exceptions that do not inherit from BaseException. Such exceptions are deprecated (by documentation, if not by code) so our conscientious programmer will never raise them and the standard library avoids doing so. But "call_some_library()" was written by some less careful developer, and may well contain these atavisims. The only complete solution that comes to mind immediately is for the raising of anything not extending BaseException to raise a PendingDeprecationWarning as well. Then the conscientious developer can feel confident again so long as her unit tests are reasonably exhaustive. If we cannot produce a warning for these, then I'd rather not produce the warning for the use of bare "except:". After all, as it's been pointed out, if the use of bare "except:" is all you are interested in it is quite easy to grep the code to find all uses. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bare except clauses in PEP 348
Guido: > But how about the following compromise: make it a silent deprecation > in 2.5, and a full deprecation in 2.6. Reinhold Birkenfeld: > That said, I think that unless it is a new feature (like with statements) > transitions to Python 3.0 shouldn't be enforced in the 2.x series. With 3.0, > everyone expects a clear cut and a compatibility breach. Raymond: > I'd love to compromise but it's your language. If you're going to > deprecate, just do it. Pulling the band-aid off slowly doesn't lessen > the total pain. There are actually THREE possible levels of deprecation available. In order of severity, they are: 1. Modifying the documentation to advise people to avoid this feature. No one gets alerted. 2. Using a PendingDeprecationWarning so people who explicitly request it can have the compiler alert them when they use it. 3. Using a DeprecationWarning so people using it are alerted unless they explicitly request NOT to be alerted. I think 3 is unwarrented in this case. For reasons I explained in a previous posting, I would be in favor of 2 if we can *also* have a PendingDeprecationWarning for use of string exceptions and arbitrary-object exceptions (those not derived from BaseException). I am in favor of 3 in any case. Of course, that's just one person's opinion... Raymond also raised this excellent point: > There is a reason that over 120 bare except clauses remain in the > standard library despite a number of attempts to get rid of them. [...] > If the proponents don't have time to fix the standard library, how can > they in good conscience mandate change for the rest of the world. That seems like a fair criticism to me. As we've already noted, it is impossible to replace ALL uses of bare "except:" in 2.5 (particularly the ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bare except clauses in PEP 348
[PLEASE IGNORE PREVIOUS EMAIL... I HIT [Send] BY MISTAKE] Guido: > But how about the following compromise: make it a silent deprecation > in 2.5, and a full deprecation in 2.6. Reinhold Birkenfeld: > That said, I think that unless it is a new feature (like with statements) > transitions to Python 3.0 shouldn't be enforced in the 2.x series. With 3.0, > everyone expects a clear cut and a compatibility breach. Raymond: > I'd love to compromise but it's your language. If you're going to > deprecate, just do it. Pulling the band-aid off slowly doesn't lessen > the total pain. There are actually THREE possible levels of deprecation available. In order of severity, they are: 1. Modifying the documentation to advise people to avoid this feature. No one gets alerted. 2. Using a PendingDeprecationWarning so people who explicitly request it can have the compiler alert them when they use it. 3. Using a DeprecationWarning so people using it are alerted unless they explicitly request NOT to be alerted. I think 3 is unwarrented in this case. For reasons I explained in a previous posting, I would be in favor of 2 if we can *also* have a PendingDeprecationWarning for use of string exceptions and arbitrary-object exceptions (those not derived from BaseException). I am in favor of 3 in any case. Of course, that's just one person's opinion... Raymond also raised this excellent point: > There is a reason that over 120 bare except clauses remain in the > standard library despite a number of attempts to get rid of them. [...] > If the proponents don't have time to fix the standard library, how can > they in good conscience mandate change for the rest of the world. That seems like a fair criticism to me. As we've already noted, it is impossible to replace ALL uses of bare "except:" in 2.5 (particularly the use in code.py that Guido referred to). But we ought to make an extra effort to remove unnecessary uses of bare "except:" from the standard library if we intend to deprecate it. -- Michael Chermisde ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Style for raising exceptions (python-dev Summary for 2005-08-01 through 2005-08-15 [draft])
Marc-Andre Lemburg writes: > This is from a comment in ceval.c: > > /* We support the following forms of raise: > raise , > raise , > raise , None > raise , > raise , None > raise , > raise , None > > An omitted second argument is the same as None. > > In addition, raise , is the same as > raising the tuple's first item (and it better have one!); > this rule is applied recursively. > > Finally, an optional third argument can be supplied, which > gives the traceback to be substituted (useful when > re-raising an exception after examining it). */ > > That's quite a list of combinations that will all break > in Python 3.0 if we only allow "raise ". Oh my GOD! Are you saying that in order to correctly read Python code that a programmer must know all of THAT! I would be entirely unsurprised to learn that NO ONE on this list... in fact, no one in the whole world could have reproduced that specification from memory accurately. I have never seen a more convincing argument for why we should allow only limited forms in Python 3.0. And next time that I find myself in need of an obfuscated python entry, I've got a great trick up my sleeve. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove str.find in 3.0?
Raymond writes: > That suggests that we need a variant of split() that has been customized > for typical find/index use cases. Perhaps introduce a new pair of > methods, partition() and rpartition() +1 My only suggestion is that when you're about to make a truly inspired suggestion like this one, that you use a new subject header. It will make it easier for the Python-Dev summary authors and for the people who look back in 20 years to ask "That str.partition() function is really swiggy! It's everywhere now, but I wonder what language had it first and who came up with it?" -- Michael Chermside [PS: To explain what "swiggy" means I'd probably have to borrow the time machine.] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() (was: Remove str.find in 3.0?)
Michael Hoffman writes: > Dare I ask whether the uncompiled versions [of re object methods] should > be considered for removal in Python 3.0? > > *puts on his asbestos jacket* No flames here, but I'd rather leave them. The docs make it clear that the two sets of functions/methods are equivalent, so the conceptual overhead is small (at least it doesn't scale with the number of methods in re). The docs make it clear that the compiled versions are faster, so serious users should prefer them. But the uncompiled versions are preferable in one special situation: short simple scripts -- the kind of thing often done with shell scriping except that Python is Better (TM). For these uses, performance is irrelevent and it turns a 2-line construct into a single line. Of course the uncompiled versions can be written as little 2-line functions but that's even WORSE for short simple scripts. Nearly everything I write these days is larger and more complex, but I retain a soft spot for short simple scripts and want Python to continue to be the best tool available for these tasks. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Revising RE docs (was: partition() (was: Remove str.find in 3.0?))
Barry Warsaw writes: > Although it's mildly annoying that the docs describe the compiled method > names in terms of the uncompiled functions. I always find myself > looking up the regexp object's API only to be shuffled off to the > module's API and then having to do the argument remapping myself. An excellent point. Obviously, EITHER (1) the module functions ought to be documented by reference to the RE object methods, or vice versa: (2) document the RE object methods by reference to the module functions. (2) is what we have today, but I would prefer (1) to gently encourage people to use the precompiled objects (which are distinctly faster when re-used). Does anyone else think we ought to swap that around in the documentation? I'm not trying to assign more work to Fred... but if there were a python-dev consensus that this would be desirable, then perhaps someone would be encouraged to supply a patch. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proof of the pudding: str.partition()
Raymond's original definition for partition() did NOT support any of the following: (*) Regular Expressions (*) Ways to generate just 1 or 2 of the 3 values if some are not going to be used (*) Clever use of indices to avoid copying strings (*) Behind-the-scenes tricks to allow repeated re-partitioning to be faster than O(n^2) The absence of these "features" is a GOOD thing. It makes the behavior of partition() so simple and straightforward that it is easily documented and can be instantly understood by a competent programmer. I *like* keeping it simple. In fact, if anything, I'd give UP the one fancy feature he chose to include: (*) An rpartition() function that searches from the right ...except that I understand why he included it and am convinced by the arguments (use cases can be demonstrated and people would expect it to be there and complain if it weren't). Simplicity and elegence are two of the reasons that this is such an excellent proposal, let's not lose them. We have existing tools (like split() and the re module) to handle the tricky problems. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] String views
Tim Delaney writes: > One of the big disadvantages of string views is that they need to keep > the original object around, no matter how big it is. But in the case of > partition, much of the time the original string survives for at least a > similar period to the partitions. Why do you say that? Didn't several of Raymond's examples use the idiom: part_1, _, s = s.partition(first_sep) part_2, _, s = s.partition(second_sep) part_3, _, s = s.partition(third_sep) --- Skip writes: > I'm skeptical about performance as well, but not for that reason. A string > object can have a referent field. If not NULL, it refers to another string > object which is INCREFed in the usual way. At string deallocation, if the > referent is not NULL, the referent is DECREFed. If the referent is NULL, > ob_sval is freed. Won't work. A string may have multiple referrents, so a single referent field isn't sufficient. --- My own contribution: I know that the Java string class has support for this. The String class contains a reference to a char array along with start and length indices. The substring() method constructs what we're calling "string views". I wonder whether there is a way to instrument a JVM to record how often the underlying buffers are shared, then run some common Java apps. Since the feature is exactly analogous to what is being proposed here, I would find such such analysis very helpful. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Replacement for print in Python 3.0
Please bear with me if this has already been stated, or if I ought to be directing this to the wiki instead of to python-dev at this point. I've been trying to follow this whole discussion but have only gotten as far as last Saturday. Two things: First of all, I wanted to encourage Guido. There have been lots of people objecting to "fixing" the print statement, but I for one agree that it's a wart which should be addressed if it can be done elegantly. I'm just not speaking up because others (particularly Guido) have already said most of what I am thinking and I don't want to clutter the discussion with "me too!"'s. And one thing which (as far as I've read) _IS_ a new suggestion. I agree that a new built-in function ought to be named 'print'. This poses problems for those who want to write code NOW that runs in Python 2.x (for large values of x) which will also run in 3.0. We could satisfy both people if in Python 2.x we introduced a built-in function named "print30" (for Python 3.0) with the intended new behavior. People could start coding now using the "print30" builtin. When Python 3.0 was released, 'print' would no longer be a keyword, and both 'print' and 'print30' would be built-ins that both refered to the same function. Sure, it's a tiny bit of backward compatibility cruft to have a second name for the builtin, but it may be worth it because the ability to write in the "Python 3.0 style" (all new-style classes, only raise proper exceptions, etc) in the 2.x series is a VERY useful feature. We want to handle the transition better than Perl. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hacking print (was: Replacement for print in Python3.0)
Bill Hanssen writes: > I think the "-ln" > variants made familiar by Pascal and Java were a bad idea, on a par > with the notion of a split between "text" and "binary" file opens. It's a bit off topic, but it wasn't the languages that introduced the difference between "text" and "binary" files. Pascal defined a difference between "text" and "record" files because the operating systems of the time had two distinct file types. Java initially had only one type (binary files which got automagically converted to a stream of unicode characters) and later modified things to allow manual control of the encoding because "modern" operating systems (like Windows) have two distinct file types. Don't blame the language designers, blame the OS folks. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Replacement for print in Python 3.0
Guido writes:
> Is it worth doing this and completely dropping the %-based formats in
> Py3k? (Just asking -- it might be if we can get people to get over the
> shock of $ becoming first class ;-).
In my opinion, YES -- it's worth seriously considering it. A single,
well-designed solution for string interpolation (with syntactic support
if needed to make it very easy to use) is FAR better than having one
good solution and another legacy solution. Just the awkwardness of the
trailing s in "%(foo)s" is enough to motivate a search for something
better.
But this presuposes that there IS a single well-designed solution. PEP 292
templates are an excellent start, but they are not that solution. The
largest problem is the lack of a means for formatting numbers. People
should think hard about good solutions.
He continues:
> I proposed ${varname%fmt} earlier but it prevents you to extend the
> varname syntax to arbitrary expressions, which I think is an extension
> that will get lots of requests.
I certainly agree that we should keep open the syntactic possibility
to allow arbitrary Python expressions between "${" and "}" in an
interpolation string even though it may not be supported today.
I favor idea (Barry's?) of using "${::}"
where is an identifier (but someday might allow expressions),
and and behave like the % interpolation modifiers
today. I would have suggested it myself, but somehow I failed to realize
that slice literals are allowed only within subscripts and thus do not
conflict with this use.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3000 and new style classes
Lisandro Dalcín proposes:
> Any possibility to add something like
>
> from __future__ import new_style_classes
Tristan Seligmann writes:
> Why does it matter if the single statement you insert is spelled
> " metaclass = type" instead of "from future import whatever"?
Russell Owen responds:
> It matters because "metaclass = type" is completely obscure. How would
> any non-expert have a clue what it means?
Holger asks:
> How would this non-expert have a clue what
> "from __future__ import new_style_classes" means?
Mon-expert users can safely assume that any "from __future__ import"
statements are there to future-proof a program or make use of advanced
features early. Non-expert users cannot safely assume anything about
assignments to __metaclass__ and, in fact, may well break into a cold
sweat any time they hear the word "metaclass".
I'm not saying that it's necessary, but if it were my call (and it isn't)
I'd probably not bother to code "from __future__ import new_style_classes"
but I'd probably accept a patch if someone wrote one. I think it would
provide a REALLY nice migration path if it were possible to write
Python 3.0 code in Python 2.x (for large values of x) so long as you
included an appropriate preamble of "from __future__ import" statements.
I don't believe we'll ever get it perfect because there would be a few
minor incompatibilities no matter how hard we try, but just imagine how
the Perl5 users today would feel if they were told that they could use
Perl6 code in the Perl5 interpreter by using the "@ .fture. <<" command.
I love making Perl users jealous, so I certainly wouldn't vote less than
-0 ("I don't care so why bother") on a proposal like this one.
-- Michael Chermside
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify the file-like-object interface
Andrew Durdin writes: > Another area where I think this approach can help is with the > text/binary file distinction. file() could always open files as > binary, and there could be a convenience function textfile(name, mode) > which would simply return textstream(file(name, mode)). This would > remove the need for "a" or "b" in the mode parameter, and make it > easier to keep text- and binary-file access separate in one's mind: I think you are suffering from the (rather common) misconception that all files are binary, and the definition of "text file" is a binary file which should be interpreted as containing characters in some encoding. In unix, the above is true. One of the fundamental decisions in Unix was to treat all files (and lots of other vaguely file-like things) as undiferentiated streams of bytes. But this is NOT true on many other operating systems. It is not, for example, true on Windows. Many operating systems make a distinction between two basic types of files... "text files" which are line-oriented and contain "text", and "binary files" which contain streams of bytes. This distinction is supported by the basic file operations in the C library. To open a text file in binary mode is technically an error (although in many OSs you'll get away with it). -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] list splicing
Karl Chen writes: > Hi, has anybody considered adding something like this: > a = [1, 2] > [ 'x', *a, 'y'] > > as syntactic sugar for > a = [1, 2] > [ 'x' ] + a + [ 'y' ]. A bit later in the thread, Josiah Carlson replies: > I don't think the parser would get measureably more complex, but I > believe that any work to support this uncommon (from my experience) > operation, rather than the dozens of other usable RFEs in the tracker, > would be misspent time. I'd just like to point out that this is a FRF (Frequently Requested Feature). I'm not arguing in favor of it, just pointing out that using "star unpacking" in tuple and list literals is an idea that I'm sure I've seen proposed at least a couple of times before. This doesn't necessarily make it a good idea, nor a bad one. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
