Re: [Python-Dev] The docs, reloaded
On Tue, 22 May 2007, Steve Holden wrote: But doesn't *everyone* now know that documentation contributions don't have to be marked up? It's certainly been said enough. Maybe that fact should be more prominent in the documentation? Then we'll just have to worry about getting people to read it ... I think the issue is instant gratification. You don't get the satisfaction of seeing your changes unless you're willing to write them in LaTeX, and that's a pretty big barrier -- a lot of what motivates open source volunteers is the sense of fulfillment. (Hence, by the same nature, Wiki-like editing with instant changes visible online will probably greatly increase contributions.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 30XZ: Simplified Parsing
I fully support the removal of implicit string concatenation (explicit is better than implicit; there's only one way to do it). I also fully support the removal of backslashes for line continuation of statements (same reasons). (I mean this as distinct from line continuation within a string; that's a separate issue.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime module enhancements
On Sat, 10 Mar 2007, Greg Ewing wrote: Collin Winter wrote: Treat dates as if they have a time-part of midnight. This is my preferred solution, and it is already what the datetime module does, for example, when subtracting two dates. Does it really? Seems to me you can pick any time of day to be the representative time and get the same result when subtracting two dates, as long as you pick the same one for both dates. So it's not really assuming anything here. It certainly gives that appearance: from datetime import datetime datetime(2007, 1, 2) - datetime(2007, 1, 1, 0, 0, 0) datetime.timedelta(1) str(_) '1 day, 0:00:00' The behaviour to the end user exposes the midnight assumption, regardless whether it's explained by saying the constructor makes the assumption or the subtraction makes the assumption. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pydoc Improvements / Rewrite
On Thu, 4 Jan 2007, Talin wrote: One issue that needs to be worked out, however, is the division of responsibility between markup processor and output formatter. Does a __markup__ plugin do both jobs, or does it just do parsing, and leave the formatting of output to the appropriate HTML / text output module? How does the HTML output module know how to handle non-standard metadata? [...] I guess the markup processor has to deliver some kind of DOM tree, which can be rendered either into text or into HTML. CSS can take over from that point on. If the markup processor is going to deliver a tree, let me just point out that it would be a pretty major project to define the format of that tree -- about as large as inventing ReST or any other markup language, except that the design of such an intermediate format has to foresee future changes to the input and be flexible enough to target multiple output formats. The design would also have to tackle the question of whether the intermediate format should contain semantic information (what about cross-references?) and what types of such information should be allowed (e.g. names of modules, arguments, exceptions, Python expressions, etc.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pydoc Improvements / Rewrite
Hi Ron and Laurent, I welcome attempts to improve pydoc (especially since I don't have much time to work on improving it myself). I definitely agree that moving to CSS is long overdue, though I would like some input on the style of the produced pages. It's probably a good idea to explain how pydoc got to be the way that it is. The module boundary between inspect and pydoc is a pretty clear one, intended to isolate pydoc from future changes to Python's introspection features (such as attributes on internal types like frames and functions). On the other hand, I've often seen the question of why pydoc does both text and HTML generation instead of generating some intermediate data structure from which both kinds of output are produced. The answer is: I tried it. The result turned out to be longer than I expected and needlessly more complicated than what we have now. It may be that a better job could have been done, but I think there is a rational basis for why it turned out that way. The Python objects themselves already are a data structure containing all of the information we need. I discovered that translating this data structure into another data structure and then producing text or HTML was more work than simply producing text or HTML. With CSS, the last step gets even easier and so the intermediate stage becomes even less necessary. Also, the intermediate step required me to essentially invent an API, and I decided that I trusted the stability of Python's API more than that of some API I invented just for this. This is not to say that text and HTML generation can't be separated; it's just a caution against attempting to overgeneralize by creating an intermediate format. I'm glad you backed away from XML (or I'd have warned you that processing the XML would be a lot of extra work). The inspect module was intended to pull out as much as possible of the extraction functionality that's shared by the text and HTML documentation generators. But pydoc is still big. At the time I was proposing pydoc for addition to the standard library, I didn't want to pollute the top-level module namespace with too many names, so I tried hard to minimize the number of modules. And of course it has grown since then with bits of new functionality and support for new language features in Python. But now if a package is being considered, it makes sense to split out some of the pieces (as you have done), such as the web server, the search function, and the interactive interpreter help prompt. It may even enable pydoc to provide search from the interactive help prompt, which would be a nice feature! The package could contain several modules for ease of maintenance, while still providing a single, convenient command for running pydoc from the Unix prompt. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] a feature i'd like to see in python #2: indexing of match objects
On Sun, 3 Dec 2006, Fredrik Lundh wrote: Martin v. Löwis wrote: Well, the proposal was to interpret m[i] as m.group(i), for all values of i. I can't see anything confusing with that. it can quickly become rather confusing if you also interpret m[:] as m.groups(), not to mention if you add len() and arbitrary slicing to the mix. what about m[] and m[i,j,k], btw? I'd say, don't pretend m is a sequence. Pretend it's a mapping. Then the conceptual issues go away. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python in first-year MIT core curriculum
Wow. Did you catch this news? http://www-tech.mit.edu/V125/N65/coursevi.html The first four weeks of C1 will be a lot like the first four weeks of 6.001, Abelson said. The difference is that programming will be done in Python and not Scheme. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Importing .pyc in -O mode and vice versa
On Mon, 6 Nov 2006, Armin Rigo wrote: I know it's a discussion that comes up and dies out regularly. My two cents is that it would be saner to have two separate concepts: cache files used internally by the interpreter for speed reasons only, and bytecode files that can be shipped and imported. I like this approach. Bringing source code and program behaviour closer together makes debugging easier, and if someone wants to run Python programs without source code, then EIBTI. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
Hi Brett, Here are some comments on your proposal. Sorry this took so long. I apologize if any of these comments are out of date (but also look forward to your answers to some of the questions, as they'll help me understand some more of the details of your proposal). Thanks! Introduction /// [...] Throughout this document several terms are going to be used. A sandboxed interpreter is one where the built-in namespace is not the same as that of an interpreter whose built-ins were unaltered, which is called an unprotected interpreter. Is this a definition or an implementation choice? As in, are you defining sandboxed to mean with altered built-ins or just restricted in some way, and does the above mean to imply that altering the built-ins is what triggers other kinds of restrictions (as it did in Python's old restricted execution mode)? A bare interpreter is one where the built-in namespace has been stripped down the bare minimum needed to run any form of basic Python program. This means that all atomic types (i.e., syntactically supported types), ``object``, and the exceptions provided by the ``exceptions`` module are considered in the built-in namespace. There have also been no imports executed in the interpreter. Is a bare interpreter just one example of a sandboxed interpreter, or are all sandboxed interpreters in your design initially bare (i.e. sandboxed = bare + zero or more granted authorities)? The security domain is the boundary at which security is cared about. For this dicussion, it is the interpreter. It might be clearer to say (if i understand correctly) Each interpreter is a separate security domain. Many interpreters can run within a single operating system process, right? Could you say a bit about what sort of concurrency model you have in mind? How would this interact (if at all) with use of the existing threading functionality? The powerbox is the thing that possesses the ultimate power in the system. In our case it is the Python process. This could also be the application process, right? Rationale /// [...] For instance, think of an application that supports a plug-in system with Python as the language used for writing plug-ins. You do not want to have to examine every plug-in you download to make sure that it does not alter your filesystem if you can help it. With a proper security model and implementation in place this hinderance of having to examine all code you execute should be alleviated. I'm glad to have this use case set out early in the document, so the reader can keep it in mind as an example while reading about the model. Approaches to Security /// There are essentially two types of security: who-I-am (permissions-based) security and what-I-have (authority-based) security. As Mark Miller mentioned in another message, your descriptions of who-I-am security and what-I-have security make sense, but they don't correspond to permission vs. authority. They correspond to identity-based vs. authority-based security. Difficulties in Python for Object-Capabilities // [...] Three key requirements for providing a proper perimeter defence is private namespaces, immutable shared state across domains, and unforgeable references. Nice summary. Problem of No Private Namespace === [...] The Python language has no such thing as a private namespace. Don't local scopes count as private namespaces? It seems clear that they aren't designed with the intention of being exposed, unlike other namespaces in Python. It also makes providing security at the object level using object-capabilities non-existent in pure Python code. I don't think this is necessarily the case. No Python code i've ever seen expects to be able to invade the local scopes of other functions, so you could use them as private namespaces. There are two ways i've seen to invade local scopes: (a) Use gc.get_referents to get back from a cell object to its contents. (b) Compare the cell object to another cell object, thereby causing __eq__ to be invoked to compare the contents of the cells. So you could protect local scopes by prohibiting these or by simply turning off access to func_closure. It's clear that hardly any code depends on these introspection featuresl, so it would be reasonble to turn them off in a sandboxed interpreter. (It seems you would have to turn off some introspection features anyway in order to have reliable import guards.) Problem of Mutable Shared State === [...] Regardless, sharing of state that can be influenced by another interpreter is not safe for object-capabilities. Yup. Threat Model /// Good to see this specified here. I like the way you've broken
Re: [Python-Dev] uuid module - byte order issue
On Fri, 4 Aug 2006, Oren Tirosh wrote: Compatibility with Windows GUIDs may be one of the most important use cases for the UUID module. It's important to resolve this or users will have unpleasant surprises. I did. [...] alternatives: 1. Default is big endian byte order. Little endian is explicit. 2. Default is native byte order. Little endian and big endian are explicit. 3. No default. Little endian and big endian are both explicit. I implemented and committed the first alternative above (as a new attribute, 'bytes_le'), shortly before the freeze. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] uuid module - byte order issue
On Thu, 3 Aug 2006, Oren Tirosh wrote: The UUID module uses network byte order, regardless of the platform byte order. On little-endian platforms like Windows the .bytes property of UUID objects is not compatible with the memory layout RFC 4122 says: In the absence of explicit application or presentation protocol specification to the contrary, a UUID is encoded as a 128-bit object, as follows: The fields are encoded as 16 octets, with the sizes and order of the fields defined above, and with each field encoded with the Most Significant Byte first (known as network byte order). Ka-Ping Yee writes* that the Windows UUID generation calls are not RFC 4122 compliant and have an illegal version field. [...] If the correct byte order is used the UUIDs generated by Windows XP are valid version 4 UUIDs: I see. I stand corrected, then. My interpretation of RFC 4122 would be that the uuid.py module currently completely implements RFC 4122; while Windows XP can't be said to violate RFC 4122, supporting this alternative byte order would be an optional feature. The bytes property and bytes argument to the constructor should use the platform byte order. I disagree; i think the main interface to uuid.py should behave consistently independent of the platform, and should conform to the default encoding in RFC 4122. However, we could certainly consider adding a 'bytes_le' field to support the functionality you describe. I'll ask Anthony whether adding such a field would be permitted at this stage of the release process. There is another issue with version 1 uuid generation: len(set(uuid.uuid1() for i in range(1000))) 992 The problem is that the random clock_seq field is only 14 bits long. If enough UUIDs are generated within the same system clock tick there will be collisions. The clock field of the UUID has enough resolution to avoid collisions; the problem you're describing is a limitation of the platform's clock, not the UUID module. It doesn't happen on Mac OS X, for example. Suggested solution: use the high-resolution of the time field (100ns) to generate a monotonically increasing timestamp that advances at least by 1 for each call, when time.time() returns the same value on subsequent calls. That sounds like a fine solution to me. I'll look into it. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)
On Thu, 3 Aug 2006, Greg Ewing wrote: The time isn't the main issue. The main issue is that almost all the use cases for round() involve doing an int() on it afterwards. That's my experience as well. In my opinion, the purpose of round() is most commonly described as to make an integer. So it should yield an integer. As other comments in this discussion imply, whether you want round() to return an integer really depends on your intended audience and their pattern of usage. It seems to me that for round() to produce an integer is about as intuitive as for 1/2 to produce 0.5. That is, if you bring basic mathematical intuition to the table instead of a background in how C behaves, then 1/2 suggests one-half and rounding suggests convert to an integer. If you have a background in programming languages and you understand the value of type consistency, then you would see why dividing integers should yield integers and rounding floats should yield floats. Since Python was indeed changed to make 1/2 evaluate to 0.5, that suggests that it is Pythonic to target the first audience rather than the second. And that implies that it would also be Pythonic to make round() return an int. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] uuid test suite failing
On Fri, 28 Jul 2006, Neal Norwitz wrote: This only fixes 1 of the 2 failures in test_uuid. The other one is due to _unixdll_getnode() failing. This is because _uuid_generate_time is None because we couldn't find it in the uuid library. This is just broken, not sure if it's the code or the test though. We should handle the case if _uuid_generate_time and the others are None better. I don't know what to do in this case. The design intention is this: each of the various *_getnode() functions is supposed to work on the platform for which it was written. For example, _windll_getnode() is supposed to work on Windows, and will raise an exception on other platforms; if it raises an exception on Windows, something is wrong (the code's expectations of the OS are not met). When uuid_generate_time is unavailable, _unixdll_getnode() is supposed to fail. The getnode() function is just supposed to get an available MAC address; that's why it catches any exceptions raised by the *_getnode() functions. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] test_uuid
On Sat, 29 Jul 2006, Neal Norwitz wrote: I just checked in a change to disable testing 2 uuid functions (_ifconfig_get_node and unixdll_getnode) that fail on many platforms. Here's the message: Disable these tests until they are reliable across platforms. These problems may mask more important, real problems. One or both methods are known to fail on: Solaris, OpenBSD, Debian, Ubuntu. They pass on Windows and some Linux boxes. _ifconfig_get_node() should work on all Linuxes. (Thanks for fixing it to work on more types of Unix.) It's okay for unixdll_getnode to fail when the necessary shared library is not available. Ideally, test_uuid should serve as a record of which platforms we expect these routines to work on. The uuid module as a whole isn't broken if one of these routines fails; it just means that we don't have complete platform coverage and/or test_uuid has inaccurate expectations of which routines work on which platforms. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
On Wed, 19 Jul 2006, Brett Cannon wrote: I have decided to go with object-capabilities for securing interpreters. I have rewritten my design doc from scratch and deleted the old one. The new doc is named securing_python.txt and can be found through the svn web interface at http://svn.python.org/view/python/branches/bcannon-sandboxing/securing_python.txt?rev=50717view=log. This is amazing news!! I'm going off to read your document right now. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] User's complaints
On Thu, 13 Jul 2006, Wolfgang Langner wrote: On 7/13/06, Jeroen Ruigrok van der Werven [EMAIL PROTECTED] wrote: Things that struck me as peculiar is the old: if __name__ == __main__: whatever() This is so out of tune with the rest of python it becomes a nuisance. It is not beautiful but very useful. In Python 3000 we can replace it with: @main def whatever(): ... to mark this function as main function if module executed directly. Why not simply: def __main__(): ... or even pass in the command-line arguments: def __main__(*args): ... Having to 'import sys' to get at the command-line arguments always seemed awkward to me. 'import sys' feels like it should be a privileged operation (access to interpreter internals), and getting the command-line args isn't privileged. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Handling of sys.args (Re: User's complaints)
On Thu, 13 Jul 2006, Greg Ewing wrote: Would it help if sys were pre-imported into the builtins? Or do you think that args shouldn't live in sys at all? I feel like the command-line arguments don't really belong in sys, and i'd rather not have 'sys' pre-imported into the builtins. I think of 'sys' as the place for sensitive interpreter internals and the builtins as the place for harmless functions. They're at opposite ends of the scale -- 'sys' is the most privileged; builtins are the least privileged. From a security point of view, 'import' is the main way to get dangerous new functionality. I realize 'open' breaks this concept, but it's nice to be able to say, to a rough approximation, that preventing 'import' gives you an interpreter that can do computations but isn't a security risk. These are just my feelings -- i'm not sure how consistent this is with the original intent of 'sys'. I think it would be a quite useful distinction to have, though. And while we're on the subject, anyone think it would be a good idea to drop the silly feature of having the program name as args[0]? You almost *never* want to treat it the same way as the rest of the args, so the first thing you always do is args[1:]. Starting the args array with the first argument (and putting the program name somewhere else) seems Pythonic to me. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)
On Mon, 10 Jul 2006 [EMAIL PROTECTED] wrote: I think Talin's got a point though. It seems hard to find one short English word that captures the essence of the desired behavior. None of the words in his list seem strongly suggestive of the meaning to me. I suspect that means one's ultimately as good (or as bad) as the rest. What's wrong with nonlocal? I don't think i've seen an argument against that one so far (from Talin or others). -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)
On 7/10/06, Ka-Ping Yee [EMAIL PROTECTED] wrote: What's wrong with nonlocal? I don't think i've seen an argument against that one so far (from Talin or others). On Mon, 10 Jul 2006, Jeremy Hylton wrote: It's a made-up word. You won't find it in the dictionary and the google define: query sends me to a wikipedia page about quantum mechanics. Two million Google hits for nonlocal seems like plenty. The explanation of nonlocal is pretty straightforward -- If the definition of f() contains an assignment to x, then x is a local variable in f, unless x is declared nonlocal. To express this email in the positive form: 1. Reserved words should be real words. 2. The meaning of the word should be clear. 3. Put statements in positive form. (Strunk White) 4. The word should sound good. global meets all of these requirements. But it's the wrong word. Purple also meets all of these requirements. I'd rather accurately express the concept in the negative (the true meaning really is in the negative: don't make a new binding), than choose a simple-sounding word that is essentially a lie. x = 1 def f(x): print x def g(): global x x = 3 print x The x used in g is not global at all -- it belongs to f. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)
On Sun, 9 Jul 2006, Andrew Koenig wrote: Sounds reasonable to me. If we're talking py3k I'd chuck global as a keyword though and replace it with something like outer. I must say that I don't like outer any more than I like global. The problem is that in both cases we are selecting the *inner*most definition that isn't in the current scope. That's why nonlocal is a better choice in my opinion. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Musings on concurrency and scoping (replacing Javascript)
On Fri, 7 Jul 2006, Guido van Rossum wrote: On 7/7/06, Ka-Ping Yee [EMAIL PROTECTED] wrote: I've been doing a bunch of Firefox extension programming in Javascript and suddenly a few of the recent topics here came together in my head in a silent kapow of thoughts. This is kind of a side note to the security discussion, but they're all interconnected: network programming, concurrency, lexical scoping, security. Hm... I wonder if this style has become so popular in JS because it's all they have? I find callback-style programming pretty inscrutable pretty soon. Yes, i would love to be able to use generators instead. Maybe this will be a strong selling point for Python when it's ready for browser scripting. (There may be a fair amount of glue required to cope with the existing callback-based Firefox APIs though.) How can you ever keep track of when a '}' must be followed by a ';' ? Hee hee -- dizzying, eh? Actually there are no occurrences of }; in that example, just );. In JavaScript the semicolons are optional anyway; my personal style is to use semicolons after statements but not after function declarations. 1. Variables are more appropriately scoped; they exist only where they are meaningful. (In a class, all the variables would be mixed into one namespace, where some of them would be invalid some of the time.) This doesn't strike me as very important. The same can happen in a local scope and doesn't seem to bother anyone there. That's true, but then we're talking about existing coding styles where people aren't concerned about minimizing security risks. Scopes are a very powerful tool for limiting access, but you might not notice what you're missing if you haven't been faced with a security problem and tried using them to solve it. (It's a bit like generators before we had them and got used to how handy they were: substitute scopes = generators, limiting access = flow control, security problem = concurrency problem.) 2. Local variables are private. (Class instances don't get their own private namespaces to play in.) Hmm... I wouldn't be so sure of that. Exception handlers have access to locals of stack frames above and below the current frame. True. It would be more accurate to say that local variables are usually thought of as private, and therefore could be enforced as private without breaking (hardly) anything. Making instance members private would break tons of stuff. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In defense of Capabilities [was: doc for new restricted execution design for Python]
On Fri, 7 Jul 2006, Talin wrote: While I was typing this, I did realize a drawback to poisoned objects, which I will illustrate by the following example: Suppose we want to grant to the sandboxed program permission to read and write cofiguration properties. We don't want to give them arbitrary write access to the file, instead we want to force the sandbox code to only access that file by setting and getting properties. This is an example where a subsystem would require elevated privileges compared to the main program - the config file reader / writer needs to be able to read write the file as a text stream, but we don't want to allow the sandboxed program to just write arbitrary data to it. The situation you're describing here is a classic case of one component keeping a closely held authority while using it to provide some limited capability to some other component. This comes up quite often when you're trying to write secure code. If you want to be able to write that subsystem in Python, then we will need a way to create airtight Python objects (i.e. objects that only leak what they explicitly choose to leak). So this goes back to the big question of goals: Do we want to be able to protect one piece of Python code from another piece of Python code? I'd like the answer to be yes. It sounded for a while like this was not part of Brett's plan, though. Now i'm not so sure. It sounds like you're also interested in having the answer be yes? Let's keep talking about and playing with more examples -- i think they'll help us understand what goals we should aim for and what pitfalls to anticipate before we nail down too many details. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Restricted execution: what's the threat model?
After reading the messages on this thread i'm starting to think that it would be good to clarify what kinds of threats we are trying to defend against, and specify what invariants we are intending to preserve. For example, here are a few things Brett mentioned: Right. I am thinking more of an implementation screw up that somehow provides access to an object that has escalated rights. But you are correct, I am only concerned with preventing a crash of a sandboxed interperter. [on what is meant by it getting out] Out of a trusted interpreter and ending up in a sandboxed interpreter some how. So here are a couple of questions for clarification (some with my guesses as to their answers): 1. When we say restricted/untrusted/whatever interpreter we don't really mean that the *interpreter* is untrusted, right? We mean that the Python code that runs in that interpreter is untrusted (i.e. to be prevented from doing harm), right? 2. I'm assuming that the implementation of the Python interpreter is always trusted. As a starting point it seems to me we have to draw the line somewhere -- around at least the C code that implements the Python interpreter, and possibly more. What do we take the Trusted Computing Base to include? The Python VM implementation -- plus all the builtin objects and C modules? Plus the whole standard library? (trusted = behaves safely because it's our job to write it correctly, not due to something else imposing restrictions upon it; untrusted = we wish to be able to impose restrictions on it) 3. Is it part of the plan that we want to protect Python code from other Python code? For example, should a Python program/function X be able to say i want to launch/call program/function Y with *these* parameters and have it run under *these* limitations? This has a big impact on the model. And here are some possible goals or invariants to consider. It will be helpful to decide on some of these so that, when someone points to what they think is a flaw in the security implementation, we can say yes, that is our responsibility or no, it isn't. We want to be able to guarantee that... A. The interpreter will not crash no matter what Python code it is given to execute. B. Python programs running in different interpreters embedded in the same process cannot communicate with each other. C. Python programs running in different interpreters embedded in the same process cannot access each other's Python objects. D. A given piece of Python code cannot access or communicate with certain Python objects in the same interpreter. E. A given piece of Python code can access only a limited set of Python objects in the same interpreter. I think in order to get truly useful restricted interpreters we will end up wanting to make guarantees of all of these kinds. There may be others i haven't thought of -- feel free to edit or add others. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Musings on concurrency and scoping (replacing Javascript)
On Wed, 21 Jun 2006, Brett Cannon wrote:
I have been working on a design doc for restricted execution of Python
as part of my dissertation for getting Python into Firefox to replace
JavaScript on the web.
I've been doing a bunch of Firefox extension programming in Javascript
and suddenly a few of the recent topics here came together in my head
in a silent kapow of thoughts. This is kind of a side note to the
security discussion, but they're all interconnected: network
programming, concurrency, lexical scoping, security.
Client-side web scripting tends to have a callback/continuation-ish
concurrency style because it has to deal with network transactions
(which can stall for long periods of time) in a user interface that
is expected to stay always responsive. The Firefox API is full of
listeners/observers, events, and continuation-like things. So one
thing to consider is that, when Python is used for these purposes,
it may be written in a specialized style.
As i write JavaScript in this style i find i use nested functions
a lot. When i want to set up a callback that uses variables in the
current context, the natural thing to do is to define a new function
in the local namespace. And if that function has to also provide a
callback, then it has another function nested within it and so on.
function spam() {
var local_A = do_work();
do_network_transaction(
new function(result_1) {
var local_B = do_work(result_1);
do_network_transaction(
new function(result_2) {
do_work(local_A, local_B, result_1, result_2);
...
}
);
}
);
}
So it is a possible consequence of embedding Python in Firefox that
people will be using nested functions and lexical scoping in Python
more often, which makes the recent discussion about access to
enclosing scopes more significant.
This is even related to security as well. Namespaces and lexical
scoping are a natural and visually apparent way to limit access.
If, for example, result_1 and result_2 in the above example are
security-sensitive objects like secret keys or networking functions,
you can see just by inspection that they cannot leak outside of
spam() except by directly being passed in an outgoing function call.
The standard Pythonic response to nested functions is to translate
them into classes. But the nested function style has two advantages:
1. Variables are more appropriately scoped; they exist
only where they are meaningful. (In a class, all the
variables would be mixed into one namespace, where
some of them would be invalid some of the time.)
2. Local variables are private. (Class instances don't
get their own private namespaces to play in.)
The first becomes more significant when in a more continuation-y
style because it helps keep the various continuations from
interfering with each other. The second becomes more significant
if you care about restricting untrusted Python code.
-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Musings on concurrency and scoping (replacing Javascript)
On Thu, 6 Jul 2006, Phillip J. Eby wrote: As much as I'd love to have the nested scope feature, I think it's only right to point out that the above can be rewritten as something like this in Python 2.5: def spam(): local_A = do_work() result_1 = yield do_network_transaction() local_B = do_work(result_1) result_2 = yield do_network_transaction() do_work(local_A, local_B, result_1, result_2) ... All you need is an appropriate trampoline (possibly just a decorator) that takes the objects yielded by the function, and uses them up to set up callbacks that resume the generator with the returned result. Clever! Could you help me understand what goes on in do_network_transaction() when you write it this way? In the Firefox/JavaScript world, the network transaction is fired off in another thread, and when it's done it posts an event back to the JavaScript thread, which triggers the callback. And what happens if you want to supply more than one continuation? In my JavaScript code i'm setting up two continuations per step -- one for success and one for failure, since with a network you never know what might happen. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)
On Wed, 5 Jul 2006, Guido van Rossum wrote: On 7/5/06, Phillip J. Eby [EMAIL PROTECTED] wrote: Using the classic nonsense example: def counter(num): def inc(): .num += 1 return .num return inc Would this also use ..num to refer to num in an outer scope two levels removed? I don't think there's any need for that. I see '.num' as just another way of saying num, but don't make a new binding. I agree with Guido that the best proposals so far are converging on the idea that it's more Pythonic to say don't make a new binding when a variable is used, than to declare this is the scope for this binding ahead of time. Of those there are two kinds: (a) State once (anywhere in a scope, but preferably at the beginning) that a variable is non-local. This is like the global keyword works now, and this category includes: - Change the meaning of 'global'. - Add a new keyword 'outer' or 'nonlocal', etc. (b) Indicate, when mentioning a variable, that the variable is non-local. This category includes: - Say 'global.x' or 'outer.x' instead of 'x'. - Say '.x' instead of 'x'. My favourite so far is to use a new keyword -- i think changing the meaning of 'global' would be misleading. '.x' is probably my next favourite, though i share Guido's concern about allowing both 'x' and '.x' to refer to the same thing. I see that 'outer' is used as an identifier in hmac.py in the standard library and also as a variable in test_set*.py. On the other hand 'nonlocal' does not appear anywhere in the standard library. Thus, 'nonlocal' is the best option that i've seen so far; it's less likely to break anything and it says exactly what it means. I can't think of a more accurate keyword. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
On Wed, 5 Jul 2006, Brett Cannon wrote: On 7/4/06, Ka-Ping Yee [EMAIL PROTECTED] wrote: In response to Guido's comment about confusing the words trusted and untrusted, how about empowered and restricted? Maybe. I am really starting to lean towards trusted and sandboxed. It can be risky to use words of the form '*trust*' when talking about security because 'trust' can have different meanings in different contexts. (Examples: 'X is trusted' might mean 'X is something i feel safe running without restrictions' or 'X *is* in fact allowed to run without restrictions' or 'i need to run X without restrictions in order to accomplish my task' or 'X is something i am relying on for the security of my system'.) The most common context for 'trusted' that i seem to come across is in the phrase 'trusted computing base' (TCB), which refers to 'the thing that is supposed to be enforcing security restrictions' as opposed to 'something i'm willing to let run unrestricted'. So when i read 'trusted code' what comes to mind is the C implementation of the Python interpreter, and it may be a good idea to reserve that phrase for that purpose, if it's to be used at all. In any case, let's try to nail down clear names for the different pieces we're talking about here, and i gently advise avoiding '*trust*' words or using them with very limited meaning. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
Hi Brett,
Here are some comments on the description of the restricted execution
model that you posted.
When referring to the state of an interpreter, it is either trusted or
untrusted. A trusted interpreter has no restrictions imposed upon any
resource. An untrusted interpreter has at least one, possibly more, resource
with a restriction placed upon it.
In response to Guido's comment about confusing the words trusted and
untrusted, how about empowered and restricted?
When the Interpreter Is Embedded
Single Untrusted Interpreter
This use case is when an application embeds the interpreter and never has more
than one interpreter running.
The main security issue to watch out for is not having default abilities be
provided to the interpreter by accident.
I'd rather rephrase this in the opposite direction. The onus shouldn't
be on the application to hunt down each possible dangerous authority and
deactivate them all one by one. The main security issue is to let the
application choose which abilities it wants the restricted interpreter
to have, and then ensure that the restricted interpreter gets only those
abilities.
Multiple Untrusted Interpreters
---
When multiple interpreters, all untrusted at varying levels, need to be
running within a single application. This is the key use case that this
proposed design is targetted for.
On top of the security issues from a single untrusted interpreter,
there is one additional worry. Resources cannot end up being leaked
into other interpreters where they are given escalated rights.
What is your model here for communication between interpreters? If two
interpreters can communicate, any attempt to prevent leakage of
resources is meaningless. When you say leaked into other interpreters
are you talking about a Python object leaking or something else at a
lower level?
Suppose for example that the application wants to embed two interpreters,
P and Q, and that the application wants P to be able to write files but
Q to be restricted against writing files. When you say leaked above,
that suggests to me that you want to prevent something like
# code running in P
import spam
f = open('/home/doofus/.ssh/authorized_keys', 'a')
spam.f = f
# code running in Q
import spam
spam.f.write('blargh')
The above example supposes that P and Q can communicate through a
shared module, spam, where they can pass Python objects.
But notice that even if you prevent them from passing Python objects
like open files, any form of communication is sufficient to leak
resources:
# code running in P
def add_key(key):
f = open('/home/doofus/.ssh/authorized_keys', 'a')
f.write(key + '\n')
f.close()
import socket
s = socket.socket()
s.bind(('', ))
s.listen(1)
ns, addr = s.accept()
add_key(ns.recv(100))
# code running in Q
import webbrowser
webbrowser.open('http://localhost:/zebra')
As long as P can listen for instructions from Q, it can give Q
the power to write to the filesystem.
Filesystem
===
The most obvious facet of a filesystem to protect is reading from it.
One does not want what is stored in ``/etc/passwd`` to get out. And
one also does not want writing to the disk unless explicitly allowed
for basically the same reason; if someone can write ``/etc/passwd``
then they can set the password for the root account.
There's a big difference between modifying (or erasing) an existing file
and writing a new file (e.g. for temporary storage). If i give you a
little filesystem of your own to play in, and it starts out empty, you
can put whatever you want in it without violating my secrecy or the
integrity of my files.
I think you should be talking about this in terms of specifically
what abilities you want to be able to allow, based on examples of
real-life applications.
Physical Resources
===
Memory should be protected. It is a limited resource on the system
that can have an impact on other running programs if it is exhausted.
Being able to restrict the use of memory would help alleviate issues
from denial-of-service (DoS) attacks.
Networking
===
Networking is somewhat like the filesystem in terms of wanting similar
protections. You do not want to let untrusted code make tons of socket
connections or accept them to do possibly nefarious things (e.g., acting
as a zombie).
You also want to prevent finding out information about the network you are
connected to. This includes doing DNS resolution since that allows one
to find out what addresses your intranet has or what subnets you use.
Again, it's risky to describe only individual cases of things to
prevent. What networking abilities are safe or necessary for the
kinds of applications you have in mind? Start from nothing and
work up from
Re: [Python-Dev] Lexical scoping in Python 3k
On Sat, 1 Jul 2006, Andrew Koenig wrote: I'd rather see a simpler rule: = never defines a variable in a surrounding scope. If you want to affect the binding of such a variable, you have to define it explicitly in the scope in which you want it. Example: x = 42 def f(): x = 123 # rebinds x as defined above y = 123 # defines local variable f() print x # prints 123 print y # error -- y not defined Yes, I know that rule is too simplistic. But I think I'd still prefer it to the way things are now. I agree with you that this is a nicer and more consistent rule. What do you think of the proposal for a keyword to say don't rebind? It would achieve the same distinction you're aiming for above, but without the drastic incompatibility with today's Python. This has been previously discussed as change the meaning of 'global' to mean 'not local': http://mail.python.org/pipermail/python-dev/2006-February/061568.html http://mail.python.org/pipermail/python-dev/2006-July/066908.html I support this proposal, though i would prefer a clearer keyword such as outer x or nonlocal x. If we can't agree on another keyword (or can't afford to spend one more keyword), i'm willing to support global for this purpose. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Lexical scoping in Python 3k
On Sat, 1 Jul 2006, Greg Ewing wrote: I don't disagree with anything you said, but I think it would be a good idea to avoid using phrases like proper lexical scopes, which is likely to set people off on a tangent. The issue isn't lexicality, it's writeability. Fully functional lexical scopes, then? Python's scopes are lexical (except for builtins) but currently somewhat hamstrung. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
On Fri, 30 Jun 2006, Andrew Koenig wrote: I saw messages out of sequence and did not realize that this would be a change in behavior from 2.4. Sigh. Yes, this is not a good time to change it. I hope Py3000 has lexical scoping a la Scheme... Me too -- that would be really nice. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] For sandboxing: alternative to crippling file()
On Fri, 30 Jun 2006, Brett Cannon wrote: On 6/30/06, Armin Rigo [EMAIL PROTECTED] wrote: object.__subclasses__() [..., type 'file'] Maybe this one won't work if __subclasses__ is forbidden, but in general I think there *will* be a way to find this object. Yeah, that's been my (what I thought was paranoid) feeling. Glad I am not the only one who thinks that hiding file() is near impossible. If you want to do this right, it should be about *making* hiding possible. If you can't hide things, it will be hard to get very far. I realize that may be difficult for Python 2.x, but hiding is pretty essential for security. It would be really good to keep this in mind for the design of Python 3k. (It doesn't mean we can't have introspection, just that we need to agree on some discipline for how to do it.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Lexical scoping in Python 3k
On Fri, 30 Jun 2006, Guido van Rossum wrote: On 6/30/06, Ka-Ping Yee [EMAIL PROTECTED] wrote: On Fri, 30 Jun 2006, Andrew Koenig wrote: I hope Py3000 has lexical scoping a la Scheme... Me too -- that would be really nice. That's not a very constructive proposal (especially since I don't know Scheme). Perhaps you could elaborate on what needs to change? Sorry! I should have been more clear. Right now, Python supports the existence of nested scopes: a = 3 def f(): a = 4 def g(): a = 5 print a # prints 5 g() print a # prints 4 f() print a # prints 3 The above example shows that there are three distinct scopes, and that each one has a distinct binding named 'a' -- assigning to one doesn't affect the others. a = 3 def f(): b = 4 def g(): c = 5 print a, b, c # i can see all three g() f() The above example shows that all of the scopes can be *read*. But in today's Python, not all of the scopes can be *written*. a = 3 def f(): b = 4 def g(): c = 5 a, b, c = 0, 1, 2 # changes local c, not outer a and b g() f() The code in g() can affect its own local, 'c', and it can affect the global variable 'a' if it declares 'global a', but no matter what you write in g(), it cannot assign to 'b' (or to any other intermediate scope). This is a strange limitation and it would be nice to remove it. The asymmetry comes from Python having one foot in the new paradigm of nested lexical scopes and one foot still in the older paradigm of only two scopes, local and global. Most other languages that support lexical scoping (including Scheme, JavaScript, Ruby, Perl, E, Java, Smalltalk) provide a uniform way to read and write to scopes at all levels. This is done by letting programmers specify the scope in which they want a variable bound (usually with a keyword like var in JavaScript, my in Perl, or define in E). So here are some thoughts on how Python might be adjusted to support this. I'm not saying these would be the only ways, but at least they're some ideas to start with. In JavaScript, the var keyword is required whenever you want to declare a local variable. Anything without var is assumed to be a global name. The cleanest and most consistent solution that comes to mind would be to adopt exactly this for Python. Without var: a = 3 # global def f(): b = 4 # global def g(): c = 5 # global a, b, c = 0, 1, 2 # changes all three globals g() f() print a, b, c # prints 0, 1, 2 With var: var a = 3 # global def f(): var b = 4 # local to f def g(): var c = 5 # local to g a, b, c = 0, 1, 2 # changes outer a, outer b, and c print c # prints 2 g() print b # prints 1 f() print a # prints 0 print b # no such name print c # no such name But that is a big change. Perhaps it would be too unpythonic to suddenly require declarations for all local variables. So an alternative would be to retain the default assumption that undeclared variables are local. Here's what we'd get: Without var: a = 3 def f(): b = 4 def g(): c = 5 a, b, c = 0, 1, 2 # changes local c, not outer a and b g() f() With var: var a = 3 def f(): var b = 4 def g(): var c = 5 a, b, c = 0, 1, 2 # changes outer a, outer b, and c g() f() Now i think this is a little bit weird, because the statement var b = 4 in an outer scope changes the meaning of b in an inner scope. But it does have the virtue of retaining behaviour compatible with today's Python, while offering a way to get proper lexical scopes for those who want to use them. Thoughts? Other ideas? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
On Fri, 30 Jun 2006, Andrew Koenig wrote: The fundamental principle is that the binding of every name is determined during compilation, not during execution. This property does not quite apply to Python at present. I think this property does apply. In your example: def g(): return x x = 42 g() the result is 42. It is already known at compile time that the return x in g() refers to an 'x' in the outer scope. 'x' cannot be a local variable to g() because there are no statements in g() that bind 'x'. Regardless of whether the binding itself exists yet, you (the reader) and the compiler can know which scope to look in for the binding at runtime. Have i understood your desired property correctly? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement - handling errors
On Tue, 27 Jun 2006, Jim Jewett wrote: (Almost) everyone agrees that the case expressions SHOULD be run-time constants. The disagreements are largely over what to do when this gets violated. I like your summary and understood most of it (options 1, 2, 3, 5, 6). The only part i didn't understand was this: Bad Case Option (4) -- Ignore problems -- This is for optimization; go ahead and ignore any problems you can. Maybe that branch will never be taken... Ironically, this is also largely school I, since it matches the if semantics. Could you elaborate on what this means? Does ignore any problems mean even if a case value changes, pretend it didn't change? But that wouldn't match the 'if' semantics, so i'm not sure what you had in mind. Bad Case Option (6) -- Undefined [...] The down side is that people may start to count on the actual behavior anyhow; then (in practice) we might just have Bad Case Option (5) without documentation. I agree with this last paragraph. Option 6 seems the most risky of all. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
On Mon, 26 Jun 2006, Guido van Rossum wrote: I've written a new PEP, summarizing (my reaction to) the recent discussion on adding a switch statement. While I have my preferences, I'm trying to do various alternatives justice in the descriptions. Thanks for writing this up! The section that most draws my attention is Semantics, and i guess it isn't a surprise to either of us that you had the most to say from the perspective you currently support (School II). :) Let me suggest a couple of points to add: - School I sees trouble in the approach of pre-freezing a dispatch dictionary because it places a new and unusual burden on the programmer to understand exactly what kinds of case values are allowed to be frozen and when the case values will be frozen. - In the School II paragraph you say Worse, the hash function might have a bug or a side effect; if we generate code that believes the hash, a buggy hash might generate an incorrect match -- but that is primarily a criticism of the School II approach, not of the School I approach as you have framed it. It's School II that mandates that the hash be the truth. (It looks to me like what you're actually criticizing here is based on some assumptions about how you think School I might be implemented, and having taken School I a number of steps down that (unexplained) road you then see problems with it.) Also, why is the discussion of School II mostly an argument against School I? What about describing the advantages of each school? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
On Mon, 26 Jun 2006, Guido van Rossum wrote: Can you please edit the PEP yourself to add this? That will be most efficient. I've done so, and tried to clarify the next line to match (see below). With school I, if you want to optimize using a hash table (as in PEP 275 Solution 1) you have to catch and discard exceptions in hash(), and a bug in hash() can still lead this optimization astray Right. As written, the problem a buggy hash might generate an incorrect match is not specific to School I; it's a problem with any approach that is implemented by a hash lookup. School II is necessarily implemented this way; School I might or might not be. So i think the part that says: the hash function might have a bug or a side effect; if we generate code that believes the hash, a buggy hash might generate an incorrect match doesn't belong there, and i'd like your consent to remove it. On the other hand, this criticism: if we generate code that catches errors in the hash to fall back on an if/elif chain, we might hide genuine bugs is indeed specific to School I + hashing. Right. School I appears just as keen as school II to use hashing to optimize things, but isn't prepared to pay the price in semantics; Ok. Then there's an inconsistency with the definition of School I: School I wants to define the switch statement in term of an equivalent if/elif chain To clear this up, i've edited the first line of the School II paragraph, which previously said: School II sees nothing but trouble in that approach It seems clear that by that approach you meant trying to achieve if/elif semantics while using hash optimization rather than the more general definition of School I that was given. I believe there are a few voices here (and i count myself among them) that consider the semantics more important than the speed and are in School I but aren't treating hash optimization as the quintessence of 'switch', and we shouldn't leave them out. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
On Mon, 26 Jun 2006, Guido van Rossum wrote: Most school I proponents (perhaps you're the only exception) have claimed that optimization is desirable, but added that it would be easy to add hash-based optimization. IMO it's not so easy in the light of various failure modes of hash(). (A possible solution would be to only use hashing if the expression's type is one of a small set of trusted builtins, and not a subclass; we can trust int.__hash__, str.__hash__ and a few others.) That's a good idea! At first glance, it seems like that could lead to a plausible compromise. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] School IIb?
Here's a possible adjustment to the School-II approach that i think avoids the issues i've been raising, while giving the desired O(n)-to-O(1) speedup in common situations. It's basically School-II dispatch, plus a check: On compilation, freeze any cases that meet the School-II conditions and have a trustworthy __hash__ method into a dictionary. At runtime, when the dictionary yields a hit, check if the case expression yields a different value. If the value has changed, use if/elif processing. In most cases the case-equality check will be cheap (e.g. an attribute lookup), but it would allow us to establish for sure that the switch value really matches the case value when we branch to a particular case, so we'd not be so vulnerable to __hash__ misbehaving, which seems to be your main source of discomfort with if/elif semantics. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simple Switch statement
On Sat, 24 Jun 2006, Raymond Hettinger wrote: The main points of contention are 1) a non-ambiguous syntax for assigning multiple cases to a single block of code, 2) how to compile variables as constants in a case statement, and 3) handling overlapping cases. Here's a simple approach that will provide most of the benefit without trying to overdo it: [...] The result of f(x) should be hashable or an exception is raised. Cases values must be ints, strings, or tuples of ints or strings. No expressions are allowed in cases. I like this proposal. It eliminates all of the surprises that have been hiding in the other switch proposals so far, except for throwing an exception when the switch expression isn't hashable. But i can easily live with that restriction, since the cases are all literals that must be hashable, so if the switch expression comes out to some other type then it probably really is an error that should be caught. I agree with the general feeling that it would be nice to have a bit more flexibility, but so far i haven't thought of any way to get more flexibility without more surprises, so until a better idea comes along i'm happy with this one. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simple Switch statement
On Sat, 24 Jun 2006, Phillip J. Eby wrote: At 03:49 PM 6/24/2006 -0700, Raymond Hettinger wrote: Cases values must be ints, strings, or tuples of ints or strings. -1. There is no reason to restrict the types in this fashion. Even if you were trying to ensure marshallability, you could still include unicode and longs. When he said ints i assumed that included longs. The distinction is so nearly gone by now (from the Python programmer's point of view). I don't see any particular problem with allowing all basestrings rather than just 8-bit strings. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simple Switch statement
On Sun, 25 Jun 2006, Guido van Rossum wrote: In your eagerness to rule out surprises, you're creating the biggest surprise of all: the restriction to literals is certainly a surprise! I disagree. Perhaps what we mean by surprise is different. In Raymond's design, there is a simple rule for what's allowed in a case. The whole statement can be described in very few words: Each case is a literal integer, string, or tuple of integers or strings. Execution jumps to the first case that matches the value of the switch expression (or to 'default' if no match). That's it. The simpler the rule, the less surprising it is. It would take a lot more words to explain the behaviour of something like Nick's 'once'-based proposal. Here's an attempt: Each case consists of an arbitrary expression, but the expression may not refer to any local variables or arguments of the immediately surrounding function definition. The value of each case is computed when the surrounding function definition is compiled. If any two cases have the same value, an exception is thrown at compile time. At runtime, execution jumps to the case whose previously fixed value matches the value of the switch expression (or to 'default' if no match). Not only is that longer to describe, it's also more difficult for a beginning programmer to understand, since it requires knowing when different parts of a program are compiled (and what happens if the same part is compiled more than once). -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simple Switch statementZ
On Mon, 26 Jun 2006, Georg Brandl wrote: Raymond Hettinger wrote: five = 5 eight = [8] def f(x, six=6): seven = 7 a = static(five + 4)# this is legal b = static(six + 4) # this is illegal c = static(seven + 4) # this is illegal d = static(eight + [4]) # this is illegal Why would the last line be illegal? I believe Raymond is assuming it would be illegal because it's mutable. I don't think much has been said about whether static(EXPR) should be allowed to yield a mutable value, but if we did allow that, it might open up an avenue to much confusion. (I join the chorus of voices that dislike the name 'static' for this feature.) Whether or not 'eight + [4]' is allowed in 'static', it certainly wouldn't be allowed after 'switch' or 'case' since it's unhashable. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simple Switch statementZ
On Sun, 25 Jun 2006, Guido van Rossum wrote: What do you think of Nick C's 'once'? It's a bit closer to the right meaning... but what about: def f(x): def g(y): return y + once x return g Does once mean not really once here, but once for each new function object that's created for g? Right. But there are all sorts of objects that are compared by object identity (e.g. classes, modules, even functions) which may contain mutable components but are nevertheless constant for the purpose of switch or optimization. Let's not confuse this concept of constness with immutability. That's a good point. We need a concept like stable for equality separate from constant, since constant and immutable will mislead those who are used to the meanings of these words in other languages. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement
On Fri, 23 Jun 2006, Josiah Carlson wrote: This is a good thing, because if switch/case ends up functionally identical to if/elif/else, then it has no purpose as a construct. This doesn't make sense as a rule. Consider: If x.y ends up functionally identical to getattr(x, 'y'), then it has no purpose as a construct. If print x ends up functionally identical to import sys; sys.stdout.write(str(x) + '\n'), then it has no purpose as a construct. What matters is not whether it's *functionally* identical. What matters is whether it makes more sense to the reader and has a meaning that is likely to be what the writer wanted. Evaluate the switch expression just once is a semantic win. Evaluate the switch expression just once, but throw an exception if the result is not hashable is a weaker semantic win. (How often is that what the writer is thinking about?) Throw an exception at compile time if the cases overlap is also a weaker semantic win. (How often is this an actual mistake that the writer wants to be caught at compile time?) Use the case values computed at compile time, not at runtime doesn't seem like much of a win. (How often will this be what the writer intended, as opposed to a surprise hiding in the bushes?) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement
On Wed, 21 Jun 2006, Guido van Rossum wrote: I worry (a bit) about this case: y = 12 def foo(x, y): switch x: case y: print something which to the untrained observer (I care about untrained readers much more than about untrained writers!) looks like it would print something if x equals y, the argument, while in fact it prints something if x equals 12. I am quite concerned about this case too. I think if Python were to behave this way, it would be a new pitfall for people learning the language -- like other pitfalls such as using unbound locals, mutable default arguments, or the historical non-nested scopes. I'm not saying the other pitfalls don't have good reasons -- some are outweighed by other design advantages (unbound locals are a consequence of having no variable declarations) and some have since been fixed (like nested scopes). But i'd be wary of adding a new pitfall to that list without a very substantial win. Me too. I guess I was just pointing out that just evaluating it in the global scope would not give an error, just like this is valid (but confusing): y = 12 def foo(y=y): print y y = 13 foo() # prints 12 I see how frozen-cases and default-arguments could have comparable semantics, but i do think frozen-cases are more confusing. In this default-arguments example, there is at least a hint from the syntax that we're introducing a new local variable, so there is a landmark where the reader can hang the mental note that a new thing is being introduced. Also, it is easier to see that default arguments are being fixed at function-definition time because their value expressions are localized in the source code in the def line, a line that makes sense to be evaluating at definition time. For frozen-cases, you don't have this kind of landmark, and the bits that are evaluated at function-definition time are scattered and mixed with the rest of the function evaluated at function-call time. That's pretty subtle; i can't think of any other Python construct off the top of my head that mixes evaluation times like that. (Yes, the compiler does optimize literals, but it's done in a way that doesn't affect semantics.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement
On Wed, 21 Jun 2006, Guido van Rossum wrote: (Note how I've switched to the switch-for-efficiency camp, since it seems better to have clear semantics and a clear reason for the syntax to be different from if/elif chains.) I don't think switch-for-efficiency (at least if efficiency is the primary design motivator) makes sense without some strong evidence that the use of if/elif constructs often causes a severe speed problem in many Python programs. (Premature optimization and all that.) Long if/elif chains probably don't occur often enough or slow down programs enough to invent syntax *just* for speed; and even if they did, i don't think there's any precedent for a Python statement invented primarily as a speed optimization. I'm hoping we can talk more about the question: How can a new statement help programmers express their intent more clearly? So far i've seen two possible answers to that question: 1. The switched-on expression is written and evaluated just once. 2. The cases could help you unpack things into local variables. (There was some discussion about unpacking earlier: http://mail.python.org/pipermail/python-dev/2005-April/052780.html which petered out, though there may still be the possibility of designing something quite useful and readable.) Any others? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement
On Wed, 21 Jun 2006, Phillip J. Eby wrote: Well, EIBTI and all that: switch x: case == 1: foo(x) case in S: bar(x) It even lines up nicely. :) Hmm, this is rather nice. I can imagine possible use cases for switch x: case 3: foo(x) case is y: spam(x) case == z: eggs(x) An interesting use case for which this offers no corresponding syntax is case instanceof ClassA: ham(x) which doesn't work because Python spells a type test as isinstance(a, b) rather than with an operator. (I suppose whether we want it to be an operator might be another question to think about for Python 3000.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement
On Mon, 19 Jun 2006, Josiah Carlson wrote: I personally don't find switch/case statements to be significantly (if at all) easier to read than if/elif/else chains, but that is subjective, and I note that Ka-Ping finds switch/case to be significantly easier to read. Uh, i didn't mean to say that. I said readability should be the primary motivator for new syntax (in general). Whether switch/case provides a significant readability improvement, and how often it turns out to be useful -- these things depend on the semantics we choose. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement
On Mon, 19 Jun 2006, Guido van Rossum wrote: Um, is this dogma? Wouldn't a switch statement also be a welcome addition to the readability? I haven't had the time to follow this thread (still catching up on my Google 50%) but I'm not sure I agree with the idea that a switch should only exist for speedup. I feel quite strongly that readability should be the primary motivator for just about any new syntax. Choosing an implementation that runs at a reasonable speed is also worthwhile consideration, but readability is where it starts, IMHO. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] uuid backward compatibility
On 6/18/06, Martin v. Löwis [EMAIL PROTECTED] wrote: As for the comment: It apparently *is* misleading, George mistakenly took it as a requirement for future changes, rather than a factual statement about the present (even though it uses the tense of simple present). Anybody breaking 2.3 compatibility will have to remember to remove the comment, which he likely won't. This sentiment is puzzling to me. It seems you assume that we can trust future developers to change the code but we can't trust them to update the documentation. So we can't have documentation even if it's factually true just because someone might forget to update it? Why is the mere possibility of incorrect documentation in the future more significant than actual correct documentation in the present? Couldn't the same argument be used to support removing all documentation from all code? If you see a better way to word the comment to reduce the possibility of misunderstanding, that's cool with me. I'd just like people who get their hands on the module to know that they can use it with 2.3. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] uuid backward compatibility
On Mon, 19 Jun 2006, Guido van Rossum wrote: If you want to encourage people to use your module with older versions, the right path is to have a distribution (can be very light-weight) on your own website and add it to PyPI Okay, i've removed the comment and submitted the package to PyPI. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] uuid backward compatibility
On Sun, 18 Jun 2006, George Yoshida wrote: uuid.py says in its docstring: This module works with Python 2.3 or higher. And my question is: Do we plan to make it 2.3 compatible in future releases? If so, uuid needs to be listed in PEP 291. Otherwise, the comment is misleading. The comment isn't misleading, because the module actually does work with Python 2.3. It would only become misleading if it were later changed to break compatibility with Python 2.3 without updating the comment. I intentionally avoided breaking compatibility with Python 2.3 so that there would be just one current version of uuid.py, both in the svn repository and available for use with existing installations of Python, since Python 2.3 is so widely deployed right now. Anyway, it looks like someone has added this module to the list of backward-compatible modules in PEP 291. Regarding whether we want it to be on that list (i.e. whether or not this backward-compatibility should be retained as Python moves forward), i'm happy to have it either way. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] UUID module
On Sun, 11 Jun 2006, Giovanni Bajo wrote: Some comments on the code: for dir in ['', r'c:\windows\system32', r'c:\winnt\system32']: Can we get rid of these absolute paths? Something like this should suffice: from ctypes import * buf = create_string_buffer(4096) windll.kernel32.GetSystemDirectoryA(buf, 4096) 17 buf.value.decode(mbcs) u'C:\\WINNT\\system32' I'd like to, but i don't want to use a method for finding the system directory that depends on ctypes. Is there a more general way? for function in functions: try: _node = function() except: continue This also hides typos and whatnot. The intended semantics of getnode() are that it cannot fail. The individual *_getnode() functions do throw exceptions if something goes wrong, and so they can be tested individually on platforms where they are expected to work. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] UUID module
On Sun, 11 Jun 2006, Phillip J. Eby wrote: At 07:24 PM 6/11/2006 -0500, Ka-Ping Yee wrote: I've added code to make uuid1() use uuid_generate_time() if available and uuid4() use uuid_generate_random() if available. These functions are provided on Mac OS X (in libc) and on Linux (in libuuid). Does that work for you? Sure - but actually my main point was to have a uuid() call you could use to just get whatever the platform's preferred form of GUID is, without having to pick what *type* you want. I'm reluctant to do that, because there's a privacy question here. I think the person using the module should have control over whether the UUID is going to leak the host ID or not (rather than leaving it up to whatever the platform prefers or which call the implementor of uuid.py happened to choose for a given platform). Perhaps that isn't feasible, or is a bad idea for some other reason, but my main point was to have a call that means get me a good unique ID. :) Couldn't we just recommend uuid.uuid4() for that? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] UUID module
On Mon, 12 Jun 2006, Giovanni Bajo wrote: GetSystemDirectory() is the official way to find the system directory. You're right. I've added a call to GetSystemDirectory, with a fallback to the usual locations if it doesn't work. Another thing that you might do is to drop those absolute system directories altogether. After all, ipconfig should always be in the path. Yup, that's why '' is the first item in the list of directories to try. As a last note, you are parsing ipconfig output assuming an English Windows installation. My Italian Windows 2000 has localized output. Thanks for catching this. I've fixed it in the latest version, which is now checked in to the trunk. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] UUID module
Thomas Heller wrote: I don't know if this is the uuidgen you're talking about, but on linux there is libuuid: Thanks! Okay, that's in there now. Have a look at http://zesty.ca/python/uuid.py . Phillip J. Eby wrote: By the way, I'd love to see a uuid.uuid() constructor that simply calls the platform-specific default UUID constructor (CoCreateGuid or uuidgen(2)), I've added code to make uuid1() use uuid_generate_time() if available and uuid4() use uuid_generate_random() if available. These functions are provided on Mac OS X (in libc) and on Linux (in libuuid). Does that work for you? I'm using the Windows UUID generation calls (UuidCreate and UuidCreateSequential in rpcrt4) only to get the hardware address, not to make UUIDs, because they yield results that aren't compliant with RFC 4122. Even worse, they actually have the variant bits set to say that they are RFC 4122, but they can have an illegal version number. If there are better alternatives on Windows, i'm happy to use them. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Should hex() yield 'L' suffix for long numbers?
I did this earlier: hex(9) '0x9184e729fffL' and found it a little jarring, because i feel there's been a general trend toward getting rid of the 'L' suffix in Python. Literal long integers don't need an L anymore; they're automatically made into longs if the number is too big. And while the repr() of a long retains the L on the end, the str() of a long does not, and i rather like that. So i kind of expected that hex() would not include the L either. I see its main job as just giving me the hex digits (in fact, for Python 3000 i'd prefer even to drop the '0x' as well), and the L seems superfluous and distracting. What do you think? Is Python 2.5 a reasonable time to drop this L? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pre-PEP: Allow Empty Subscript List Without Parentheses
On Sat, 10 Jun 2006, Greg Ewing wrote: I'm having trouble seeing a real use for a 0D array as something distinct from a scalar, as opposed to them just being an oddity that happens to arise as a side effect of the way Numeric/Numpy are implemented. I think the whole discussion about the concept and meaning of zero-dimensional arrays is mostly irrelevant to the original issue. The original issue is a *syntax* question: should x[()] be written as x[]? And from a syntax perspective, it's a bad idea. x[] is much more often a typo than an intentional attempt to index a zero-dimensional array. The compiler should continue to treat it as a syntax error. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] UUID module
Okay. I have done as Fredrik suggested: 6. Combine 2 and 3: require the user to pass in a MAC argument to uuid1, but provide a SlowGetMacAddress helper that wraps the existing code. I agree with Thomas Wouters: That sounds like the right thing to do, although I wouldn't call it slow; just let it be documented as 'might not always work and might be inefficient', The method for getting the node ID is called getnode() and is documented as possibly slow. If a hardware address cannot be obtained, we use a random 48-bit number with the highest bit set to 1, as recommended by RFC 4122. I have done as Skip proposed here: Or make the MAC address an optional arg to uuid1. If given, use it. If not, use the slow lookup (then cache the result). I've made the address an optional argument, not a required one, because i think Explicit better than implicit applies to changes in output. It makes sense to require an explicit argument if it's actually going to produce a different result, but i don't think it is necessary to require an explicit argument just because the routine might be a bit slow. Letting the address be an optional argument means that in future, we can change the implementation of getnode() to make it faster or more reliable, and users of the module will benefit without having to change any of their code. Finally, Phillip brought up PEAK: PEAK's uuid module does this such that if win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.: ...so i looked at PEAK's getnodeid48() routine and borrowed the Win32 calls from there, with a comment giving attribution to PEAK. Oh, and there is now a test suite that should cover all the code paths. This is all posted at http://zesty.ca/python/uuid.py now, documentation page at http://zesty.ca/python/uuid.html, tests at http://zesty.ca/python/test_uuid.py . I'll sleep now (6 am), commit tomorrow unless there are objections. Thanks for your input, everyone! -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] UUID module
On Sat, 10 Jun 2006, Mike Brown wrote: I have a couple of suggestions for improving that implementation: 1. You're currently using os.urandom, which can raise a NotImplementedError. You should be prepared to fall back on a different PRNG... The latest version (http://zesty.ca/python/uuid.py) does this. 2. random.randrange is a method on a default random.Random instance that, although seeded by urandom (if available), may not be the user's preferred PRNG. I recommend making it possible for the user to supply their own random.Random instance for use by the module. I decided not to add more code to do this, because the UUID constructor is now designed in such a way that it's very simple to convert your own randomness into a UUID. If you want to use another source of randomness, you'd just get 16 random bytes and then call UUID(bytes=random_stuff, version=4). That seems simpler to me than adding an extra argument and requiring a randrange() method on it. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] beta1 coming real soon
On Thu, 8 Jun 2006, Neal Norwitz wrote: If you plan to make a checkin adding a feature (even a simple one), you oughta let people know by responding to this message. Please get the bug fixes in ASAP. Remember to add tests! Just to make this appear on this thread: i'm planning to check in the uuid.py module at http://zesty.ca/python/uuid.py (with tests). -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] UUID module
Hi Thomas, This does not work, for several reasons. 1. (pythoncom|pywintypes).CreateGuid() return a PyIID instance, which you cannot slice: You're right. The PEAK code must have been based on an earlier version of the CreateGuid() routine. I've fixed this, and added detection of the UUID version so that the MAC address will only be picked up if the UUID version is 1. (BTW: Why does it first try pywintypes, the pythoncom?) Because PEAK does this, and i see the CreateGuid routine imported from both modules in Google searches for code, i assumed that it used to live in one module and moved to the other. I've also figured out how to get the MAC address using NetBIOS calls, and added that to the repertoire of methods. I've tested this and it works fast. I think this may be better than UuidCreateSequential, because it should work on both Win98 and XP. The updated version is posted at http://zesty.ca/python/uuid.py . -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] UUID module
On Sat, 10 Jun 2006, Thomas Heller wrote: I have not tested the speed, but extending my snippet to also work on 98 should be nearly trivial: try: _func = windll.rpcrt4.UuidCreateSequential except AttributeError: _func = windll.rpcrt4.UuidCreate def CreateGuid(): uuid = UUID() if 0 == _func(byref(uuid)): return str(buffer(uuid)) Okay. Thanks for this. I'll give it a shot. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] beta1 coming real soon
On Sat, 10 Jun 2006, Paul Moore wrote: Actually, the code uses ifconfig, which doesn't exist on Windows. You want the command ipconfig /all. I fixed that before you posted this message. :) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] UUID module
Quite a few people have expressed interest in having UUID functionality in the standard library, and previously on this list some suggested possibly using the uuid.py module i wrote: http://zesty.ca/python/uuid.py This module provides a UUID class, functions for generating version 1, 3, 4, and 5 UUIDs, and can convert UUIDs to and from strings of 32 hex digits and strings of 16 bytes. The thread of messages posted on python-dev begins here: http://mail.python.org/pipermail/python-dev/2006-March/062119.html My reading of this is that there was a pretty good consensus on the issues with this module that need to be addressed: (a) UUIDs should be immutable (and usable as dict keys). (b) The str() of a UUID should not contain curly braces. (c) The uuid1() function, which generates a version-1 UUID, uses a very slow method of getting the MAC address. (d) The retrieved MAC address should be cached. (e) Tests need to be written. The big question seems to be item (c); all the other items are easy to take care of. Assuming (a), (b), (d), and (e) are done, i see a few options for how to proceed from there: 1. Remove the uuid1() function, then put uuid.py in the standard library so at least we'll have the rest of the UUID functionality in 2.5b1. Fill in uuid1() later. 2. Remove the MAC-address functionality from uuid.py; instead let the caller give the MAC address as an argument to uuid1(). Put that in the standard library for 2.5b1 and fill in the function for retrieving the MAC address later. 3. Add uuid.py to the standard library with its current slow method of finding the MAC address (parsing the output of ifconfig or ipconfig), but cache the output so it's only slow the first time. 4. Figure out how to use ctypes to retrieve the MAC address. This would only work on certain platforms, but we could probably cover the major ones. On the other hand, it seems unlikely that this would be fully hammered out before the code freeze. 5. Don't include any UUID functionality in 2.5b1; put it off until 2.6. What are your thoughts on this? Thanks! -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Missing PyCon 2006
Hi folks. I had been planning to attend PyCon this year and was really looking forward to it, but i need to cancel. I am sorry that i won't be getting to see you all in a couple of weeks. If you know anyone who hasn't yet registered but wants to go, please contact me -- we can transfer my registration. Thanks, and sorry for using python-dev for this. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python icon
On Sun, 15 Jan 2006, Georg Brandl wrote: does Python have an official icon? Not py.ico from PC/, that's a bit ugly and does not scale. Has no designerhead ever done such a thing? There have been a couple of proposed logos -- i found some images at http://www.pythonology.com/logos -- but i don't know what their official status is. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Naming conventions in Py3K
On Fri, 30 Dec 2005, Reinhold Birkenfeld wrote: Ka-Ping Yee wrote: Constants in all caps: NONE, TRUE, FALSE, ELLIPSIS That's ugly. I know it looks ugly to you now. But there's a good reason why we use capitalization for class names -- anyone reading code who comes across a CapitalizedName can be reasonably certain that it refers to a class. It's a helpful way to express the intended usage. And, like it or not, None, True, False, and Ellipsis aren't classes. In fact, I like it that the basic Python functions I didn't say anything about renaming functions. Functions in lowercase are one of the naming conventions that Python does follow consistently. and most of the types are all-lowercase. That's just not true, though. (Or at least it depends on what you mean by most and by types.) The types in the built-in module are in lowercase, and the vast majority of the other types aren't. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Naming conventions in Py3K
Brett Cannon wrote: I am fine with changing the built-in types, but changing the built-in singletons just looks *really* odd to me. So odd that I just don't want to see them changed. I mean when I think of constants, I think of a variable that references an object and that reference never changes. The built-ins you are referencing, though, are singletons: named objects that are not variables. In behaviour and intended usage, they are much more like constants than like classes, though. So keeping their naming scheme as-is does not feel like a breaking of the rules to me since they are not treated the same (especially at the C level). I don't know if we share the same philosophy on this, but i don't think the C level should play a big role here -- the names in Python programs should express how something works (and more importantly how it is intended to be used) at the Python level, not details under the hood. On Fri, 30 Dec 2005, Martin v. Löwis wrote: Actually, I thought some of them would become reserved words in P3k, anyway. That would be cool. If so, it would make sense for them to be all in lowercase. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Naming conventions in Py3K
On Fri, 30 Dec 2005, Martin v. Löwis wrote: Ka-Ping Yee wrote: That would be cool. If so, it would make sense for them to be all in lowercase. And rename None to null in the process :-) Is there a really good reason to do that? It's not obvious to me. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Naming conventions in Py3K
On Fri, 30 Dec 2005, Fred L. Drake, Jr. wrote: On Friday 30 December 2005 06:31, Ka-Ping Yee wrote: Is there a really good reason to do that? It's not obvious to me. No more than there is to rename None to none or NONE. For that, there is a reason: None is not a class. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Naming conventions in Py3K
In a fair number of cases, Python doesn't follow its own recommended naming conventions. Changing these things would break backward compatibility, so they are out of the question for Python 2.*, but it would be nice to keep these in mind for Python 3K. Constants in all caps: NONE, TRUE, FALSE, ELLIPSIS Classes in initial-caps: Object, Int, Float, Str, Unicode, Set, List, Tuple, Dict, and lots of classes in the standard library, e.g. anydbm.error, csv.excel, imaplib.error, mutex.mutex... I know these probably look a little funny now to most of us, as we're used to looking at today's Python (they even look a little funny to me) but i'm pretty convinced that consistency will be better in the long run. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] a quit that actually quits
It sounds to me like what is being proposed amounts to essentially promote sys.exit to a builtin. So why not do that? I see two options. Either: (a) Simply let __builtins__.exit = sys.exit. Then we get: exit built-in function exit which may be enough of a clue that you type exit() to exit. (b) If more handholding seems like a good idea, then: class ExitHatch: def __call__(self): sys.exit() def __repr__(self): return 'Type exit() to exit Python.' __builtins__.exit = __builtins__.quit = ExitHatch() -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] a quit that actually quits
On Wed, 28 Dec 2005, Fredrik Lundh wrote:
Ka-Ping Yee wrote
It sounds to me like what is being proposed amounts to essentially
promote sys.exit to a builtin.
no, what's being proposed is what the subject says: a quit/exit command
that actually quits, instead of printing a you didn't say please! message.
Okay, that would be fine. It's just that the solutions so far that
work without parentheses are a bit too magical for me.
Fredrik's NameError-based proposal (exit when there's an exception
with no tb_next that says name 'exit' is not defined) causes the
interpreter to quit when you enter any expression involving 'exit'.
print exit # seems surprising
[3, 4, 5, exit] # seems surprising
'a' in 'xyz' or exit# desirable or undesirable?
del exit# actually happened to me
x = exit# seems surprising
Reinhold's proposal (exit when sys._getframe(1).f_code.co_names
is (exit,)) causes the interpreter to quit whenever any function
attempts to print out or represent 'exit', as long as it doesn't
mention any other variables.
print exit # seems surprising
[3, 4, 5, exit] # seems surprising
'a' in 'xyz' or exit# desirable or undesirable?
def foo():
print exit
foo() # seems surprising
I'd be happy with having Python exit when the user types just plain
'exit' without parentheses, but only in that case, not others.
However, i'm starting to think that may be impossible to implement.
I can't think of any way to make 'print exit' not exit, for example.
('exit' is a shorthand for 'exit' or 'quit' above.)
-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] AST manipulation and source code generation
Would there be any interest in extending the compiler package with tools for AST transformations and for emitting Python source code from ASTs? I was experimenting with possible translations for exception chaining and wanted to run some automated tests, so i started playing around with the compiler package to do source-to-source transformations. Then i started working on a way to do template-based substitution of ASTs and a way to spit source code back out, and i'm wondering if that might be good for experimenting with future Python features. (If there's already stuff out there for doing this, let me know -- i don't intend to duplicate existing work.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344: Explicit vs. Implicit Chaining
On Sat, 21 May 2005, James Y Knight wrote: On May 20, 2005, at 6:37 PM, Phillip J. Eby wrote: This only helps if you can get to a debugger. What if you're reading your web server's error log? Then you're in trouble anyways because you need the contents of some local to figure out what's going on, also. Ever used cgitb? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 344: Explicit vs. Implicit Chaining
Guido van Rossum wrote: Do we really need both __context__ and __cause__? Well, it depends whose needs we're trying to meet. If we want to satisfy those who have been asking for chaining of unexpected secondary exceptions, then we have to provide that on some attribute. If we also want to provide the facility that Java and C# provide with initCause/InnerException, then we need a separate attribute devoted to explicit chaining. The Java and C# documentation is clear that the cause/inner exception is to be set only on an exception that is caused or a direct result of the primary exception, which i've taken as a sign that this is an important distinction. I wanted to give a shot at making both camps happy. If the two were unified, we'd still be better off than we are now, but we should be aware that we would not be providing the functionality that Java and C# provide. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding content to exception messages
On Thu, 19 May 2005, Nick Coghlan wrote: With PEP 344, this could simply be: try: parser.parseFile(file) exeption Exception, exception: raise type(exception)(Error at line %s in file %s % (x,y)) Only if we also made all exceptions new-style classes. That's just a minor nit, though. The more important question to me is: Do you care about the difference between a secondary exception that was raised intentionally (as in your example) and a secondary exception due to a problem in exception handling? (For me, the answer is yes.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 344: Exception Chaining and Embedded Tracebacks
This PEP is a concrete proposal for exception chaining, to follow up on its mention here on Python-Dev last week as well as earlier discussions in the past year or two. http://www.python.org/peps/pep-0344.html I've tried to summarize the applications for chaining mentioned in these discussions, survey what's available in other languages, and come up with a precise specification. PEP 344 proposes three standard attributes on traceback objects: __context__ for implicit chaining (an unexpected exception occurred during 'except' or 'finally' processing) __cause__ for explicit chaining (intentional translation or augmenting of exceptions, set by raise EXC from CAUSE) __traceback__ to point to the traceback Hope this is useful. I'd like your feedback. Thanks! -- Ping ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
On Sun, 15 May 2005, Shane Hathaway wrote:
You might add to the PEP the following example, which could really
improve the process of building GUIs in Python:
class MyFrame(Frame):
def __init__(self):
with Panel():
with VerticalBoxSizer():
self.text = TextEntry()
self.ok = Button('Ok')
I don't understand how this would be implemented. Would a widget
function like 'TextEntry' set the parent of the widget according to
some global 'parent' variable? If so, how would 'Panel' know that
its parent is supposed to be the 'MyFrame' object?
-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merging PEP 310 and PEP 340-redux?
On Wed, 11 May 2005, Guido van Rossum wrote: [Steven Bethard] exc = () try: try: BLOCK1 except: exc = sys.exc_info() finally: stmt_exit(*exc) would this make any of the examples impossible to write? All you have to do to suppress an exception is to not reraise it in __exit__. But this use case would contain a trap for the unwary user who is writing an __exit__ method -- they have to remember to re-raise an exception if it was passed in, but that's easy to forget (and slightly tricky since you have to check the arg count or whether the first argument is not None). Then wouldn't it be simplest to separate normal exit from exceptional exit? That is, split __exit__ into __except__ and __finally__. If __except__ is defined, then it handles the exception, otherwise the exception is raised normally. class locking: def __init__(self, lock): self.lock = lock def __enter__(self): self.lock.acquire() def __exit__(self, *args): self.lock.release() Having __exit__ take varargs is a signal to me that it mashes together what really are two different methods. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] with use case: exception chaining
- Ka-ping Yee thinks we need separate entry points for the exceptional and the normal termination case. I disagree; this would end up in unnecessary duplication of code (or boilerplate to equate the two methods) in most cases. The whole *point* is that finally gets to do its clean-up act regardless of whether an exception is being processed or not. Okay, let me back up for a second. My suggestion responded to your reply to Steve Bethard's example about exception handling. The point of the suggestion is that *if* we are going to let with do exception handling, it should be done in a separate method. I didn't mean to imply that __finally__ should be skipped. This brings us back to the question of whether with should be able to handle exceptions. On this, you wrote: For try/finally we have a large body of use cases that just scream for abstraction. I'm not convinced that we have the same for try/except. So let's look at some use cases. I've thought of two; the first one is nice and simple, and the second one is messier so i'll discuss it in a separate message. Example 1: Exception Chaining. As has been previously discussed, the information from an exception can be lost when the handling of the exception runs into a problem. It is often helpful to preserve the original reason for the problem. Suppose, by convention, that the reason attribute on exception objects is designated for this purpose. The assignment of this attribute can be conveniently abstracted using a with clause as follows: try: # ... risky operation ... except: with reason(sys.exc_info()): # ... cleanup ... The with reason construct would be implemented like this: class reason: def __init__(self, etype, evalue, etb): self.reason = etype, evalue, etb def __except__(self, etype, evalue, etb): evalue.reason = self.reason raise etype, evalue, etb (Other possible names for reason might be cause or context.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] with use case: replacing a file
Here's another use case to think about. Example 2: Replacing a File. Suppose we want to reliably replace a file. We require that either: (a) The file is completely replaced with the new contents; or (b) the filesystem is unchanged and a meaningful exception is thrown. We'd like to be able to write this conveniently as: with replace(filename) as file: ... file.write(spam) ... file.write(eggs) ... To make sure the file is never only partially written, we rely on the filesystem to rename files atomically, so the basic steps (without error handling) look like this: tempname = filename + '.tmp' file = open(tempname, 'w') ... file.write(spam) ... file.close() os.rename(tempname, filename) We would like to make sure the temporary file is cleaned up and no filehandles are left open. Here's my analysis of all the execution paths we need to cover: 1. +open +write +close +rename 2. +open +write +close -rename ?remove 3. +open +write -close ?remove 4. +open -write +close ?remove 5. +open -write -close ?remove 6. -open (In this list, + means success, - means failure, ? means don't care.) When i add error handling, i get this: tempname = filename + '.tmp' file = open(tempname, 'w') # okay to let exceptions escape problem = None try: try: ... file.write(spam) ... except: problem = sys.exc_info() raise problem finally: try: file.close() except Exception, exc: problem, problem.reason = exc, problem if not problem: try: os.rename(tempname, filename) except Exception, exc: problem, problem.reason = exc, problem if problem: try: os.remove(tempname) except Exception, exc: problem, problem.reason = exc, problem raise problem In this case, the implementation of replace() doesn't require a separate __except__ method: class replace: def __init__(self, filename): self.filename = filename self.tempname = '%s.%d.%d' % (self.filename, os.getpid(), id(self)) def __enter__(self): self.file = open(self.tempname, 'w') return self def write(self, data): self.file.write(data) def __exit__(self, *problem): try: self.file.close() except Exception, exc: problem, problem.reason = exc, problem if not problem: # commit try: os.rename(tempname, filename) except Exception, exc: problem, problem.reason = exc, problem if problem: # rollback try: os.remove(tempname) except Exception, exc: problem, problem.reason = exc, problem raise problem This is all so intricate i'm not sure if i got it right. Somebody let me know if this looks right or not. (In any case, i look forward to the day when i can rely on someone else to get it right, and they only have to write it once!) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] with use case: exception chaining
On Thu, 12 May 2005, Guido van Rossum wrote: [Ka-Ping Yee] Example 1: Exception Chaining. As has been previously discussed, the information from an exception can be lost when the handling of the exception runs into a problem. It is often helpful to preserve the original reason for the problem. [example deleted] This problem is universal -- every except clause (in theory) can have this problem. I'd much rather deal with this in a systematic way in the Python VM's exception handling machinery. That's reasonable. Unless another use case comes up, i withdraw my suggestion for a separate __except__ method. I hope the examples were interesting, anyhow. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Tidier Exceptions
It occurred to me as i was messing around with handling and re-raising exceptions that tossing around these (type, value, traceback) triples is irritating and error-prone. How about just passing around a single value? All we'd have to do is put the traceback in value.traceback. Implementation: - raise Class and raise Class, string automatically set the .traceback attribute on the new instance of Class. - raise instance automatically sets the .traceback attribute on the instance unless it already has one. The behaviour of except and sys.exc_* could remain unchanged. raise t, v, tb would eventually be deprecated in favour of raise v. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Chained Exceptions
Suppose exceptions have an optional context attribute, which is
set when the exception is raised in the context of handling another
exception. Thus:
def a():
try:
raise AError
except:
raise BError
yields an exception which is an instance of BError. This instance
would have as its context attribute an instance of AError.
Or, in a more complex case:
def compute():
try:
1/0
except Exception, exc:
log(exc)
def log(exc):
try:
file = open('error.log') # oops, forgot 'w'
print file, exc
file.close()
except:
display(exc)
def display(exc):
print 'Aaaack!', ex# oops, misspelled 'exc'
Today, this just gives you a NameError about 'ex'.
With the suggested change, you would still get a NameError about 'ex';
its 'context' attribute would show that it occurred while handling an
IOError on error.log; and this IOError would have a 'context' attribute
containing the original ZeroDivisionError that started it all.
What do you think?
-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Chained Exceptions
On Thu, 12 May 2005, Guido van Rossum wrote:
Define raise. Does that involve a raise statement?
Not necessarily; it could be a raise statement or an inadvertently
triggered exception, such as in the example code i posted.
What about 1/0?
That counts.
What if you call a method that executes 1/0?
That counts too.
What if that method catches that exception?
Did you mean something like this?
def handle():
try:
open('spamspamspam')
except:
catchit()
# point A
...
def catchit():
try:
1/0
except:
pass
Then there's no exception to propagate, so it doesn't matter.
Once we're get to point A, the division by zero is long forgotten.
What about the StopIteration conceptually
raised by next() called by the for-loop implementation?
It's caught by the for-loop, so to speak, so it never gets out.
Conceptually, the for-loop expands to:
while 1:
try:
item = it.next()
except StopIteration:
break
# body of loop goes here
The 'break' can't possibly cause an exception, so the StopIteration
exception isn't retained.
I believe there are (at least) two use cases:
(1) I catch some low-level exception (e.g. socket.error) and turn it
into a high-level exception (e.g. an HTTPRequestFailed exception).
(2) I write some exception handling code and somehow a bug in the
handler (or an uncooperative environment, e.g. a full disk) causes the
exception handling code to trip over an exception.
I'm fairly certain (but not 100%) that Ping meant to include both use cases.
Yes, though i did not expect to provide any mechanism for distinguishing
the two cases. Do you think such a mechanism would be necessary?
-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tidier Exceptions
On Thu, 12 May 2005, Brett C. wrote: whether bare 'except' statements should go or only catch certain exceptions. Maybe bare 'except' should be spelled 'except *'. I don't think it can be removed altogether because sometimes you just need to be able to do magic, but it can be made a little more explicit. With the asterisk, it's greppable, and editors can find it or highlight it. I like the parallel to 'import *' (frowned upon, but sometimes useful if you really know what you are doing). -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tidier Exceptions
On Thu, 12 May 2005, Guido van Rossum wrote: How is except: less greppable? Duh. I'm slow today. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking off Enhanced Iterators PEP from PEP 340
On Fri, 6 May 2005, Guido van Rossum wrote: There's one alternative possible (still orthogonal to PEP 340): instead of __next__(), we could add an optional argument to the next() method, and forget about the next() built-in. I prefer your original proposal. I think this is a good time to switch to next(). If we are going to change the protocol, let's do it right. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Breaking out.
On Thu, 5 May 2005, Delaney, Timothy C (Timothy) wrote: Aahz wrote: My standard workaround is using exceptions, but I'm not sure how that interacts with a block: try: for name in filenames: with opened(name) as f: if f.read(2) == 0xFEB0: raise Found except Found: pass For any sane block iterator, it will work as expected. However, an evil block iterator could suppress the `Found` exception. I was thinking about more use cases for the block statement, and one of the ideas was an exception-logging block: def logged_exceptions(file): try: yield except Exception, value: file.write(repr(value) + '\n') block logged_exceptions(file): do stuff do stuff do stuff ...but then i wasn't sure whether this was supposed to be possible in the proposed scheme. Currently, generators do not catch exceptions raised in the code that they yield values to, because the target of the yield is in a higher stack frame. This makes sense from a language design perspective, since there is no try...finally construct lexically wrapping the thing that raises the exception. In current Python, for example, this says 'caught outside generator': def spam_generator(): try: yield 'spam' except ValueError, value: print 'caught inside generator' try: g = spam_generator() i = g.next() raise ValueError(5) except ValueError, value: print 'caught outside generator' But now i'm confused. Tim's words above seem to suggest that the interior generator could actually catch exceptions raised outside, by whatever is on the other end of the yield. So, could a block statement really catch that exception inside? I think it might be too confusing if it were possible. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Breaking out.
On Wed, 4 May 2005, Shane Hathaway wrote: I'd like to suggest a small language enhancement that would fix this example. Allow the break and continue statements to use a keyword, either for or while, to state that the code should break out of both the block statement and the innermost for or while statement. The example above would change to: for name in filenames: opening(name) as f: if f.read(2) == 0xFEB0: break for This is very elegant. It works beautifully with break, though at first that natural analogs continue for, continue while appear to conflict with Guido's proposed extension to continue. But if we choose the keyword with to introduce an anonymous block, it comes out rather nicely: continue with 2 That's easier to read than continue 2, in my opinion. (If it's not too cute for you.) Anyway, i like the general idea of letting the programmer specify exactly which block to break/continue, instead of leaving it looking ambiguous. Explicit is better than implicit, right? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Reinhold Birkenfeld wrote: There is one problem with using no keyword: You cannot use arbitrary expressions in the new statement. [...] resource = opening(file.txt) resource: (...) The latter would have to be forbidden. Noam Raphael wrote: Can you explain why it would have to be forbidden please? Reinhold Birkenfeld wrote: Well, with it you could create suites with _any_ introducing identifier. Consider: with: (...) synchronized: (...) try: (...) transaction: (...) Do you understand my concern? It would be very, very hard to discern these user-defined statements from real language constructs. I think part of the debate is about whether that's good or bad. I happen to agree with you -- i think a keyword is necessary -- but i believe some people see an advantage in having the flexibility to make a real-looking construct. As i see it the argument boils down to: Python is not Lisp. There are good reasons why the language has keywords, why it distinguishes statements from expressions, uses indentation, and so on. All of these properties cause Python programs to be made of familiar and easily recognizable patterns instead of degenerating into a homogeneous pile of syntax. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Keyword for block statements
On Fri, 29 Apr 2005, Phillip J. Eby wrote: At 08:21 PM 4/29/05 -0500, Ka-Ping Yee wrote: All the statements in Python are associated with keywords, except for assignment, which is simple and extremely common. I don't think the block statement is simple enough or common enough for that; its semantics are much too significant to be flagged only by a little punctuation mark like a colon. Don't forget the 'as' clause. It's optional, and you have to skip an arbitrarily long expression to get to it. if there is a distinctive keyword, a Python programmer who comes across this unfamiliar construct will be able to ask someone What does this 'spam' keyword mean? or can search on Google for Python spam to find out what it means. Without a keyword, they're out of luck. Names are power. help(synchronized) or help(retry) would doubtless display useful information. The programmer who writes the function used to introduce a block can hardly be relied upon to explain the language semantics. We don't expect the docstring of every class to repeat an explanation of Python classes, for example. The language reference manual is for that; it's a different level of documentation. Conversely, try Googling for Python's for or if keywords, and see if you get anything useful -- I didn't. I tried some of my favourite Python keywords :) and found that the following searches all successfully turn up information on the associated kinds of Python statements in the first couple of hits: python if python else python del python while python assert python yield python break python continue python pass python raise python try python finally python class python for statement python return statement python print statement -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: syntax suggestion - try opening(filename) as f:
On Fri, 29 Apr 2005, Guido van Rossum wrote: The more I think about it the more I like having no keyword at all (see other messages). I hope you'll reconsider this. I really think introducing a new statement requires a keyword, for pedagogical reasons as well as readability and consistency. Here's my pitch: All the statements in Python are associated with keywords, except for assignment, which is simple and extremely common. I don't think the block statement is simple enough or common enough for that; its semantics are much too significant to be flagged only by a little punctuation mark like a colon. I can empathize with wanting to avoid a keyword in order to avoid an endless debate about what the keyword will be. But that debate can't be avoided anyway -- we still have to agree on what to call this thing when talking about it and teaching it. The keyword gives us a name, a conceptual tag from which to hang our knowledge and discussions. Once we have a keyword, there can be no confusion about what to call the construct. And if there is a distinctive keyword, a Python programmer who comes across this unfamiliar construct will be able to ask someone What does this 'spam' keyword mean? or can search on Google for Python spam to find out what it means. Without a keyword, they're out of luck. Names are power. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
On Thu, 28 Apr 2005, Greg Ewing wrote: If such an explanation can't be found, I strongly suspect that this doesn't correspond to a cohesive enough concept to be made into a built-in language feature. If you can't give a short, understandable explanation of it, then it's probably a bad idea. In general, i agree with the sentiment of this -- though it's also okay if there is a way to break the concept down into concepts that *are* simple enough to have short, understandable explanations. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On Thu, 21 Apr 2005, Guido van Rossum wrote:
Perhaps it could be even simpler:
[assignment_target '=']* expr ':' suite
This would just be an extension of the regular assignment statement.
It sounds like you are very close to simply translating
expression... function_call(args):
suite
into
expression... function_call(args)(suitefunc)
If i understand what you proposed above, you're using assignment
as a special case to pass arguments to the inner suite, right? So:
inner_args = function_call(outer_args):
suite
becomes:
def suitefunc(inner_args):
suite
function_call(outer_args)(suitefunc)
?
This could get a little hard to understand if the right-hand side
of the assignment is more complex than a single function call.
I think the meaning would be unambiguous, just non-obvious. The
only interpretation i see for this:
x = spam('foo') + eggs('bar'):
suite
is this:
def suitefunc(x):
suite
spam('foo') + eggs('bar')(suitefunc)
but that could seem a little too mysterious. Or you could (in a
later compiler pass) forbid more complex expressions on the RHS.
On another note, would there be any difference between
x = spam():
suite
and
x = spam:
suite
?
-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Check out a new way to read threaded conversations.
I hope you will not mind too much if I ask a small favor. Sorry for this off-topic post. I am working on a new design for displaying online conversations. (Some of you saw this at PyCon.) I'm conducting a short survey to gather some opinions on the current design. If you have just a few minutes to spare, would you please visit: http://zesty.ca/threadmap/pydev.cgi You'll see a new way of looking at this discussion list that you may find pretty interesting. I look forward to learning what you think of it. I am very grateful for your time and assistance. (If you reply to this message, please reply to me only -- I don't want to clutter up python-dev with lots of off-topic messages.) -- Ping ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Security capabilities in Python
On Sun, 10 Apr 2005, Eyal Lotem wrote: It may be really hard to get it right, unless we are overlooking some simple solution. To get it right, you at least need to know exactly what your operators mean. I messed up because i failed to realize that '==' can be redefined, and 'in' depends on '==' to work properly. What about implementing the facet in C? This could avoid the class of problems you have just mentioned. I don't think that's a good solution. A facet is just one basic programming pattern that you can build in a capability system; it would be silly to have to go back to C every time you wanted to build some other construct. A better way would be to start with capabilities that behave simply and correctly; then you can build whatever you want. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
