Re: [Python-Dev] readd u'' literal support in 3.3?
On 12/07/2011 11:31 PM, Chris McDonough wrote: All I can really offer is my own experience here based on writing code that needs to straddle Python 2.5, 2.6, 2.7 and 3.2 without use of 2to3. Having u'' work across all of these would mean porting would not require as much eyeballing as code modified via from future import unicode_literals, it would let more code work on 2.5 unchanged, and the resulting code would execute faster than code that required us to use a u() function. Could you elaborate on why from __future__ import unicode_literals is inadequate (other than the Python 2.6 requirement)? Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Implementing restricted Python in Zope2
ranjith kannikara wrote: As a student I am not familiar with Restricted Python and python AST implementation.And in need of help to start the Restricted Python implementation. Here is some context for Python-Dev. RestrictedPython is a custom Python compiler that, when combined with a restricted environment, provides a sandbox safe enough to allow partly-trusted people to write and execute scripts on a Zope server. It has been used in Zope 2 for a long time and will have a future in Zope 3. The sandbox is more extensive than what the rexec module provides. The safety of RestrictedPython has been validated in a somewhat formal process with Python 2.4. Ranjith is working to validate it with Python 2.5. He is first working to discover all changes between Python 2.4 and 2.5 that might have affected the safety of a RestrictedPython sandbox. Any changes to the AST, builtin functions, methods of builtin types, etc., need to be evaluated for safety. So, in general, he is looking for detailed lists of changes between Python 2.4 and 2.5--more than the What's New doc. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Test, please ignore
... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Tail Calls
Shane Hathaway wrote: Shane Hathaway wrote: I'm interested in seeing a good way to write tail calls in Python. Some algorithms are more readable when expressed using tail recursion. About ten seconds after I wrote the previous message, I realized two things: - It's easy to write return Return instead of raise Return. So raise TailCall is probably better. - I can write a complete implementation of this idea with nothing but a simple decorator. Check it out! With yet another 10 seconds, I realized my quick implementation actually does nothing to optimize tail calls. Working on a fix. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Tail Calls
Shane Hathaway wrote: I'm interested in seeing a good way to write tail calls in Python. Some algorithms are more readable when expressed using tail recursion. About ten seconds after I wrote the previous message, I realized two things: - It's easy to write return Return instead of raise Return. So raise TailCall is probably better. - I can write a complete implementation of this idea with nothing but a simple decorator. Check it out! Shane class TailCall(Exception): def __init__(self, f, *args, **kwargs): self.f = f self.args = args self.kwargs = kwargs def has_tail_calls(f): def tail_call_wrapper(*args, **kwargs): try: return f(*args, **kwargs) except TailCall, e: return e.f(*e.args, **e.kwargs) tail_call_wrapper.__doc__ = f.__doc__ return tail_call_wrapper @has_tail_calls def fact2(n, v=1): fact2(1) 1 fact2(2) 2 fact2(3) 6 fact2(4) 24 fact2(20) 243290200817664L if not n: return v else: raise TailCall(fact2, n - 1, v * n) if __name__ == '__main__': import doctest doctest.testmod() ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Explicit Tail Calls
Hello, I'm interested in seeing a good way to write tail calls in Python. Some algorithms are more readable when expressed using tail recursion. I know tail call optimization has been discussed before [1], but I would like to consider a different approach. The previous discussion centered on implicit tail call optimization, which incurs the risk of changing the behavior of currently working code. (For example, is it safe to optimize tail calls within try...finally blocks? Probably not. And I generally want all stack frames to appear in tracebacks, unless I say otherwise.) I would like to suggest an explicit form of tail calls. A new built-in exception type called Return will be added, and it will be used like this: def fact2(n, v): if n: raise Return(fact2, n-1, v*n) else: return v The interpreter will catch Return exceptions and use them to call something else. The caller of a function that uses raise Return will see the result of the tail call as the returned value, rather than the Return exception. I am not yet considering implementation details. Not all algorithms are good candidates for this. I used the fact2 example only because it's readily available. I know there are other people interested in tail call optimization in Python [2] [3]; perhaps some of them are watching and can provide better examples. Furthermore, there might be some explicit syntax that converts return f(...) statements to raise Return(f, ...), such as a decorator. However, I'm less interested in the syntax and more interested in the basic capability. Shane [1] http://mail.python.org/pipermail/python-dev/2004-July/046150.html [2] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/474088 [3] http://www.voidspace.org.uk/python/weblog/arch_d7_2007_09_22.shtml#e833 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Explicit Tail Calls
Adam Olsen wrote: Since they're not going to be accepted into python anyway, the implementation is off-topic for python-dev. Please take them up elsewhere (such as my offer to discuss in private.) I was aware of Guido's earlier rejections, but I figured the rejection was due to the risky implicit optimization proposed by others. Perhaps the objection is deeper than that; I suspect Guido specifically objects to excessive use of recursion. Anyway, I've posted a recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/534104 Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
Anthony Baxter wrote: On Saturday 01 July 2006 12:55, Guido van Rossum wrote: It's up to the release manager now to decide whether the pitchforks at Google or the pitchforks in the larger Python community are sharper. ;-) At this point, I think removing the warning code is the prudent course. If someone wanted to find an easy and safe way to make it only be triggered when the import fails, it could stay in. I created a patch (#1515361) intended to do just that. https://sourceforge.net/tracker/index.php?func=detailaid=1515361group_id=5470atid=305470 Unfortunately, something appears to be misconfigured in my ISP's mail server, and as a result, my messages aren't making it to this list. So you may not have noticed Guido's re-posting of my message announcing the patch. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pre-PEP: Allow Empty Subscript List Without Parentheses
Noam Raphael wrote:
2006/6/17, Martin v. Löwis [EMAIL PROTECTED]:
Noam Raphael wrote:
I meant the extra code for writing a special class to handle scalars,
if I decide that the x[()] syntax is too ugly or too hard to type,
so I write a special class which will allow the syntax x.value.
What I cannot understand is why you use a zero-dimensional array to
represent a scalar. Scalars are directly supported in Python:
x = 5
I need a zero-dimensional array as a single cell - an object that
holds a value that can change over time. It works just like a cell in
a spreadsheet: For example, say that if you change the value of cell
A1 to 0.18, cell A2 changes to 5. When using the library I design, you
would write sheet1[0, 0] = 0.18, and, magically, sheet1[0, 1] will
become 5. But in my library, everything is meaningful and doesn't have
to be two-dimensional. So, if in the spreadsheet example, A1 meant the
income tax rate, you would write income_tax[] = 0.18, and,
magically, profit['Jerusalem', 2005] will become 5.
Try to think more about how users will use your API. You haven't
specified where those names (sheet1, income_tax, and profit) are coming
from. What do you expect users of your library to do to bring those
names into their namespace?
Let me take a wild guess so you can see what I'm asking:
import spreadsheetlib
sheet1 = spreadsheetlib.sheet('sheet1')
income_tax = spreadsheetlib.cell('income_tax')
profit = spreadsheetlib.cell('profit')
So far, that's a mess! What are you really going to do? Will it be
better? This could be a much greater concern than optimizing away
parentheses.
A possible way to solve the namespace problem is to make all names an
attribute of some object.
from spreadsheetlib import sp
sp.sheet1[0, 0] = 0.18
assert sp.sheet1[0, 1] == 5
sp.income_tax = 0.18
assert sp.profit['Jerusalem', 2005] == 5
That would be a pretty usable API, IMHO, and you'd be able to write it
now without any changes to Python.
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Logging enhancements
I'd like to create a patch for the logging package, but before I do, I need to know whether someone else is working on the package and whether my patch is likely to be accepted. Is there another group I should talk to? Specifically, I want to change the following: - The logging.config.fileConfig function has bare excepts that hide configuration errors. I'd like to remove the bare excepts and break up fileConfig for clarity. - I'd like to make it possible to configure the format of exception tracebacks. Some logs want compressed tracebacks like Medusa; others want extra information like Zope. - There is no support for external log rotation. There is plenty of support for Python rotating logs directly, but permission settings often make that impossible. Long-running Python processes need to accept a signal which notifies them when an external tool rotates the logs. - The logging documentation is lacking somewhat. For example, it fails to mention that the logging handlers are in the 'handlers' module, which I only learned by searching the source code. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Logging enhancements
Neal Norwitz wrote: On 1/9/06, Shane Hathaway [EMAIL PROTECTED] wrote: I'd like to create a patch for the logging package, but before I do, I need to know whether someone else is working on the package and whether my patch is likely to be accepted. Is there another group I should talk to? Vinay (copied) maintains it separately. He's the best person to talk to AFAIK. He maintains compatability with older pythons, possibly 1.5, I don't recall. Check PEP 291. Specifically, I want to change the following: There are several bugs (any maybe patches) on SF related to logging. AFAIK, they are all assigned to Vinay. It would be great if you could fix those as well. What is the time frame for Python 2.5? If I finish soon and the changes are not disruptive, do I have enough time for my changes to make it into 2.5? Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Logging enhancements
Tim Peters wrote: [Shane Hathaway] What is the time frame for Python 2.5? If I finish soon and the changes are not disruptive, do I have enough time for my changes to make it into 2.5? Yup! 2.5 hasn't even had an alpha release yet, and I don't expect 2.5a1 before March: http://mail.python.org/pipermail/python-dev/2005-August/055342.html That's good news. Thanks. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Logging enhancements
Vinay Sajip wrote: On 1/9/06, Shane Hathaway [EMAIL PROTECTED] wrote: - The logging.config.fileConfig function has bare excepts that hide configuration errors. I'd like to remove the bare excepts and break up fileConfig for clarity. I've no problem with this in principle, though there have been cases where ConfigParser behaviour is not what you would expect (e.g. a non-existent file leads to a missing section exception). I'd be grateful for more specifics as to where you have encountered problems. I wrote a config file with an invalid handler section. Because of the bare except, not only was my error not reported anywhere, but the daemon started with no logging at all. It took me a while to realize what had happened. The logging package needs to report erroneous configurations. - I'd like to make it possible to configure the format of exception tracebacks. Some logs want compressed tracebacks like Medusa; others want extra information like Zope. Again, no problem in principle, but it's probably an unusual use case so any change should not require extra work for users without specialised requirements. Agreed. - There is no support for external log rotation. There is plenty of support for Python rotating logs directly, but permission settings often make that impossible. Long-running Python processes need to accept a signal which notifies them when an external tool rotates the logs. I'd be interested in your approach to solve this. To be compatible with an external tool like logrotate, all a daemon needs to do is close and reopen log files at about the same time log rotation happens. To handle this use case, I suggest the logging module needs a function named reopen(), which daemons can call upon receipt of a signal. The reopen function will find all handlers with open files and ask them to reopen. - The logging documentation is lacking somewhat. For example, it fails to mention that the logging handlers are in the 'handlers' module, which I only learned by searching the source code. Not true for this specific case - for example, http://docs.python.org/lib/module-logging.html has a list of 12 handlers, followed by a statement to the effect that all but StreamHandler and FileHandler are defined in the handlers module. Oops. I guess I skipped over that because I was using the documentation as a reference rather than as a tutorial. I suggest the documentation needs to become a better reference. In general, patches are gratefully received! If you don't want to invest too much time up front, send me a mail indicating your general approach and we'll discuss. I'll get started soon. I just wanted to be sure I'm not duplicating someone else's efforts. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency
Bruce Eckel wrote: But. I do happen to have contact with a lot of people who are at the forefront of the threading world, and *none* of them (many of whom have written the concurrency libraries for Java 5, for example) ever imply that threading is easy. In fact, they generally go out of their way to say that it's insanely difficult. What's insanely difficult is really locking, and locking is driven by concurrency in general, not just threads. It's hard to reason about locks. There are only general rules about how to apply locking correctly, efficiently, and without deadlocks. Personally, to be absolutely certain I've applied locks correctly, I have to think for hours. Even then, it's hard to express my conclusions, so it's hard to be sure future maintainers will keep the locking correct. Java uses locks very liberally, which is to be expected of a language that provides locking using a keyword. This forces Java programmers to deal with the burden of locking everywhere. It also forces the developers of the language and its core libraries to make locking extremely fast yet safe. Java threads would be easy if there wasn't so much locking going on. Zope, OTOH, is far more conservative with locks. There is some code that dispatches HTTP requests to a worker thread, and other code that reads and writes an object database, but most Zope code isn't aware of concurrency. Thus locking is hardly an issue in Zope, and as a result, threading is quite easy in Zope. Recently, I've been simulating high concurrency on a PostgreSQL database, and I've discovered that the way you reason about row and table locks is very similar to the way you reason about locking among threads. The big difference is the consequence of incorrect locking: in PostgreSQL, using the serializable mode, incorrect locking generally only leads to aborted transactions; while in Python and most programming languages, incorrect locking instantly causes corruption and chaos. That's what hurts developers. I want a concurrency model in Python that acknowledges the need for locking while punishing incorrect locking with an exception rather than corruption. *That* would be cool, IMHO. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency
Antoine Pitrou wrote: I'd be happy to explain how ZODB solves those problems, if you're interested. Well, yes, I'm interested :) (I don't anything about Zope internals though, and I've never even used it) Ok. Quoting your list: To apply the same thing to Python you would at least need : 1. a way to define a subset of the current bag of reachable objects which has to stay consistent w.r.t. transactions that are applied to it (of course, you would have several such subsets in any non-trivial application) ZODB holds a tree of objects. When you add an attribute to an object managed by ZODB, you're expanding the tree. Consistency comes from several features: - Each thread has its own lazy copy of the object tree. - The application doesn't see changes to the object tree except at transaction boundaries. - The ZODB store keeps old revisions, and the new MVCC feature lets the application see the object system as it was at the beginning of the transaction. - If you make a change to the object tree that conflicts with a concurrent change, all changes to that copy of the object tree are aborted. 2. a way to start and end a transaction on a bag of objects (begin / commit / rollback) ZODB includes a transaction module that does just that. In fact, the module is so useful that I think it belongs in the standard library. 3. a precise definition of the semantics of consistency here : for example, only one thread could modify a bag of objects at any given time, and other threads would continue to see the frozen, stable version of that bag until the next version is committed by the writing thread As mentioned above, the key is that ZODB maintains a copy of the objects per thread. A fair amount of RAM is lost that way, but the benefit in simplicity is tremendous. You also talked about the risk that applications would accidentally pull a lot of objects into the tree just by setting an attribute. That can and does happen, but the most common case is already solved by the pickle machinery: if you pickle something global like a class, the pickle stores the name and location of the class instead of the class itself. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency
Bruce Eckel wrote:
I'd like to restart this discussion; I didn't mean to put forth active
objects as the solution, only that it seems to be one of the better,
more OO solutions that I've seen so far.
What I'd really like to figure out is the pythonic solution for
concurrency. Guido and I got as far as agreeing that it wasn't
threads.
I've pondered this problem. Python deals programmers a double whammy
when it comes to threads: not only is threading unsafe like it is in
other languages, but the GIL also prevents you from using multiple
processors. Thus there's more pressure to improve concurrency in Python
than there is elsewhere.
I like to use fork(), but fork has its own set of surprises. In
particular, in the programmer's view, forking creates a disassociated
copy of every object except files. Also, there's no Pythonic way for
the two processes to communicate once the child has started.
It's tempting to create a library around fork() that solves the
communication problem, but the copied objects are still a major source
of bugs. Imagine what would happen if you forked a Zope process with an
open ZODB. If both the parent and child change ZODB objects, ZODB is
likely to corrupt itself, since the processes share file descriptors.
Thus forking can just as dangerous as threading.
Therefore, I think a better Python concurrency model would be a lot like
the subprocess module, but designed for calling Python code. I can
already think of several ways I would use such a module. Something like
the following would solve problems I've encountered with threads,
forking, and the subprocess module:
import pyprocess
proc = pyprocess.start('mypackage.mymodule', 'myfunc', arg1, arg2=5)
while proc.running():
# do something else
res = proc.result()
This code doesn't specify whether the subprocess should continue to
exist after the function completes (or throws an exception). I can
think of two ways to deal with that:
1) Provide two APIs. The first API stops the subprocess upon function
completion. The second API allows the parent to call other functions in
the subprocess, but never more than one function at a time.
2) Always leave subprocesses running, but use a 'with' statement to
guarantee the subprocess will be closed quickly. I prefer this option.
I think my suggestion fits most of your objectives.
1) It works by default, so that novices can use it without falling
into the deep well of threading. That is, a program that you write
using threading is broken by default, and the tool you have to fix it
is inspection. I want something that allows me to say this is a
task. Go. and have it work without the python programmer having to
study and understand several tomes on the subject.
Done, IMHO.
2) Tasks can be automatically distributed among processors, so it
solves the problems of (a) making python run faster (b) how to utilize
multiprocessor systems.
Done. The OS automatically maps subprocesses to other processors.
3) Tasks are cheap enough that I can make thousands of them, to solve
modeling problems (in which I also lump games). This is really a
solution to a cerain type of program complexity -- if I can just
assign a task to each logical modeling unit, it makes such a system
much easier to program.
Perhaps the suggested module should have a queue-oriented API. Usage
would look like this:
import pyprocess
queue = pyprocess.ProcessQueue(max_processes=4)
task = queue.put('mypackage.mymodule', 'myfunc', arg1, arg2=5)
Then, you can create as many tasks as you like; parallelism will be
limited to 4 concurrent tasks. A variation of ProcessQueue might manage
the concurrency limit automatically.
4) Tasks are self-guarding, so they prevent other tasks from
interfering with them. The only way tasks can communicate with each
other is through some kind of formal mechanism (something queue-ish,
I'd imagine).
Done. Subprocesses have their own Python namespace. Subprocesses
receive messages through function calls and send messages by returning
from functions.
5) Deadlock is prevented by default. I suspect livelock could still
happen; I don't know if it's possible to eliminate that.
No locking is done at all. (That makes me uneasy, though; have I just
moved locking problems to the application developer?)
6) It's natural to make an object that is actor-ish. That is, this
concurrency approach works intuitively with objects.
Anything pickleable is legal.
7) Complexity should be eliminated as much as possible. If it requires
greater limitations on what you can do in exchange for a clear,
simple, and safe programming model, that sounds pythonic to me. The
way I see it, if we can't easily use tasks without getting into
trouble, people won't use them. But if we have a model that allows
people to (for example) make easy use of multiple processors, they
will use that approach and the (possible) extra
Re: [Python-Dev] Replacement for print in Python 3.0
Guido van Rossum wrote:
On 9/1/05, Raymond Hettinger [EMAIL PROTECTED] wrote:
string.Template is a bit too simplified. But perhaps it can be adapted.
We still want some way to express %r, %6.2f, etc.Since string
formatting has been around since Tim was in diapers, we should probably
start by looking at the solutions used by other languages. With Py3.0,
we have a real opportunity to break-away from doing things the way C
does it.
Hrm. Most other languages these days do floating point formatting the
way C does it. I'm happy to look for other ways to invoke the thing,
but I think that we shouldn't tinker with %6.2f. (In fact, the major
complaint is about the one place where I *did* tinker with it --
%(boo)s.)
Maybe the ${boo} form can be extended to allow ${boo%6.2f} ???
Unfortunately that would prevent a different extension of ${boo}: %{boo+far}.
May I also suggest the following shortcut for creating and evaluating a
string template. (Ever since I thought of this, I've actually used this
in code without thinking... it's just too natural):
message = $Hello, $name!
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proof of the pudding: str.partition()
Greg Ewing wrote: LD Gus Landis wrote: .piece() can be both a verb and a noun Er, pardon? I don't think I've ever heard 'piece' used as a verb in English. Can you supply an example sentence? After Java splintered in 20XX, diehard fans desperately pieced together the remaining fragments. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() (was: Remove str.find in 3.0?)
Eric Nieuwland wrote: Pierre Barbier de Reuille wrote: Or you want to have some partition method which accept regular expressions: head, sep, tail = some_str.partition(re.compile(sep+'.'*offset)) Neat! +1 on regexps as an argument to partition(). Are you sure? I would instead expect to find a .partition method on a regexp object: head, sep, tail = re.compile(sep+'.'*offset).partition(some_str) Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() (was: Remove str.find in 3.0?)
Pierre Barbier de Reuille wrote: Shane Hathaway a écrit : Are you sure? I would instead expect to find a .partition method on a regexp object: head, sep, tail = re.compile(sep+'.'*offset).partition(some_str) Well, to be consistent with current re module, it would be better to follow Antoine's suggestion : head, sep, tail = re.partition(re.compile(sep+'.'*offset), some_str) Actually, consistency with the current re module requires new methods to be added in *both* places. Apparently Python believes TMTOWTDI is the right practice here. ;-) See search, match, split, findall, finditer, sub, and subn: http://docs.python.org/lib/node114.html http://docs.python.org/lib/re-objects.html Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proof of the pudding: str.partition()
Terry Reedy wrote: Shane Hathaway [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] You can do both: make partition() return a sequence with attributes, similar to os.stat(). However, I would call the attributes before, sep, and after. One could see that as a special-case back-compatibility kludge that maybe should disappear in 3.0. My impression is that the attributes were added precisely because unpacking several related attributes into several disconnected vars was found to be often awkward. The sequencing is arbitrary and one often needs less that all attributes. Good point. Unlike os.stat(), it's very easy to remember the order of the return values from partition(). I'll add my +1 vote for part() and +0.9 for partition(). As for the regex version of partition(), I wonder if a little cleanup effort is in order so that new regex features don't have to be added in two places. I suggest a builtin for compiling regular expressions, perhaps called regex. It would be easier to use the builtin than to import the re module, so there would no longer be a reason for the re module to have functions that duplicate the regular expression methods. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() (was: Remove str.find in 3.0?)
Delaney, Timothy (Tim) wrote: I think that one of the things I have against it is that most times I type it, I get a typo. If this function is accepted, I think it will (and should!) become one of the most used string functions around. As such, the name should be *very* easy to type. FWIW, the analogy with quicksort convinced me that partition is a good name, even though I'm a terirlbe tpyist. I'm a pretty good proofreader, though. ;-) Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bare except clauses in PEP 348
Barry Warsaw wrote: I agree about bare except, but there is a very valid use case for an except clause that catches every possible exception. We need to make sure we don't overlook this use case. As an example, say I'm building a transaction-aware system, I'm going to want to write code like this: txn = new_transaction() try: txn.begin() rtn = do_work() except AllPossibleExceptions: txn.abort() raise else: txn.commit() return rtn I'm fine with not spelling that except statement as except: but I don't want there to be any exceptions that can sneak past that middle suite, including non-errors like SystemExit or KeyboardInterrupt. I can't remember ever writing a bare except with a suite that didn't contain (end in?) a bare raise. Maybe we can allow bare except, but constrain things so that the only way out of its suite is via a bare raise. I also use this idiom quite frequently, but I wonder if a finally clause would be a better way to write it: txn = new_transaction() try: txn.begin() rtn = do_work() finally: if exception_occurred(): txn.abort() else: txn.commit() return rtn Since this doesn't use try/except/else, it's not affected by changes to the meaning of except clauses. However, it forces more indentation and expects a new builtin, and the name exception_occurred is probably too long for a builtin. Now for a weird idea. txn = new_transaction() try: txn.begin() rtn = do_work() finally except: txn.abort() finally else: txn.commit() return rtn This is what I would call qualified finally clauses. The interpreter chooses exactly one of the finally clauses. If a finally except clause is chosen, the exception is re-raised before execution continues. Most code that currently uses bare raise inside bare except could just prefix the except and else keywords with finally. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bare except clauses in PEP 348
Niko Matsakis wrote: txn = new_transaction() try: txn.begin() rtn = do_work() finally: if exception_occurred(): txn.abort() else: txn.commit() return rtn Couldn't you just do: txn = new_transaction () try: complete = 0 txn.begin () rtn = do_work () complete = 1 finally: if not complete: txn.abort () else: txn.commit () and then not need new builtins or anything fancy? That would work, though it's less readable. If I were looking over code like that written by someone else, I'd have verify that the complete variable is handled correctly in all cases. (As Martin noted, your code already has a bug.) The nice try/except/else idiom we have today, with a bare except and bare raise, is much easier to verify. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 309: Partial method application
Martin v. Löwis wrote: So I would propose the syntax lst.sort(key=virtual.lower) # where virtual is functional.virtual Ooh, may I say that idea is interesting! It's easy to implement, too: class virtual: def __getattr__(self, name): return lambda obj: getattr(obj, name)() virtual = virtual() Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers
Keith Dart wrote:
I guess I just clarify this more. My unsigned type really is an object
that represents a type of number from the external system. Previously,
there was a nice, clean mapping between external types and Python types.
Now there is not so clean a mapping. Not that that makes it a problem
with Python itself.
However, since it is sometimes necessary to interface to other systems
with Python, I see no reason why Python should not have a full set of
built in numeric types corresponding to the machine types and, in turn,
other system types. Then it would be easier (and probaby a bit faster)
to interface to them. Perhaps Python could have an integer type for
long/int unified types, and just int type as normal integers?
For your purposes, would it work to use the struct module to detect
overflows early?
import struct
struct.pack('i', 2 ** 33)
Traceback (most recent call last):
File stdin, line 1, in ?
OverflowError: long int too large to convert to int
Another possibility would be to add to the struct module a full set of
integer types with a fixed width: int8, uint8, int16, uint16, int32,
uint32, int64, and uint64. Code that's focused on integration with
other languages might benefit.
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Summer of Code: Developing complete SSL support for Python
Florencio Cano Gabarda wrote:
I would like to do the new SSL module as good as possible. A piece of
art and efficiency if possible and obviusly having in mind all
programming standards.
Guido and much of the community would certainly be appreciative of a new
SSL module, especially if you can overcome the problems that plague
M2Crypto.
http://www.artima.com/weblogs/viewpost.jsp?thread=95863
I would say that the criteria for success would be:
1) A module, expected to be included in the standard library, that makes
it easy to create both client and server SSL sockets.
2) No leaks or segfaults.
3) An API that any programmer can use without knowing much about
cryptography.
I want to be able to write code that's as simple as this:
import socket
import ssl
def open_ssl_socket(address):
base = socket.socket()
base.connect(address)
sock = ssl.client(base)
return sock
def run_server(port, handler, pki_files):
keys = ssl.load_keys(pki_files)
s = socket.socket()
s.bind(('', port))
s.listen(5)
while True:
base, address = s.accept()
sock = ssl.server(base, keys)
handler(sock)
sock.close()
pki_filenames in the example is a list of key files, certificate
files, certificiate signing requests, and perhaps other PKI files. I
want the ssl module to figure out for itself what each file means, so
that I as a mere human can forget about those details. :-) However, if
there's any ambiguity in the set of files provided, the SSL module
should throw an exception rather than try to guess the intent.
If you're ambitious, you could also figure out how to make this work
with non-blocking sockets. I believe Twisted has made progress there.
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Greg Ewing wrote: Brett C. wrote: Nick's was obviously directly against looping, but, with no offense to Nick, how many other people were against it looping? It never felt like it was a screaming mass with pitchforks but more of a I don't love it, but I can deal crowd. My problem with looping was that, with it, the semantics of a block statement would be almost, but not quite, exactly like those of a for-loop, which seems to be flying in the face of TOOWTDI. And if it weren't for the can't-finalise-generators-in-a-for-loop backward compatibility problem, the difference would be even smaller. I wonder if we should reconsider PEP 340, with one change: the block iterator is required to iterate exactly once. If it iterates more than once or not at all, the interpreter raises a RuntimeError, indicating the iterator can not be used as a block template. With that change, 'break' and 'continue' will obviously affect 'for' and 'while' loops rather than block statements. Advantages of PEP 340, with this change, over PEP 343: - we reuse a protocol rather than invent a new protocol. - decorators aren't necessary. - it's a step toward more general flow control macros. At first I wasn't sure people would like the idea of requiring iterators to iterate exactly once, but I just realized the other PEPs have the same requirement. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python's Unicode width default (New Py_UNICODE doc)
M.-A. Lemburg wrote: It is important to be able to rely on a default that is used when no special options are given. The decision to use UCS2 or UCS4 is much too important to be left to a configure script. Should the choice be a runtime decision? I think it should be. That could mean two unicode types, a call similar to sys.setdefaultencoding(), a new unicode extension module, or something else. BTW, thanks for discussing these issues. I tried to write a patch to the unicode API documentation, but it's hard to know just what to write. I think I can say this: sometimes your strings are UTF-16, so you're working with code units that are not necessarily complete code points; sometimes your strings are UCS4, so you're working with code units that are also complete code points. The choice between UTF-16 and UCS4 is made at the time the Python interpreter is compiled and the default choice varies by operating system and configuration. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote:
[Nick Coghlan]
Also, the call to __enter__() needs to be before the try/finally block (as it
is
in PEP 310). Otherwise we get the releasing a lock you failed to acquire
problem.
I did that on purpose. There's a separate object ('abc' in the
pseudo-code of the translation) whose __enter__ and __exit__ methods
are called, and in __enter__ it can keep track of the reversible
actions it has taken.
Consider an application where you have to acquire *two* locks regularly:
def lockBoth():
got1 = got2 = False
lock1.acquire(); got1 = True
lock2.acquire(); got2 = True
yield None
if got2: lock2.release()
if got1: lock1.release()
If this gets interrupted after locking lock1 but before locking lock2,
it still has some cleanup to do.
That code is incorrect, though. Say lockBoth() acquires lock1 but then
lock2.acquire() throws an exception. (Maybe the lock requires some I/O
operation, and the operation fails.) The interpreter will never reach
the yield statement and lock1 will never be released.
You really have to write it like this:
def lockBoth():
lock1.acquire()
try:
lock2.acquire()
except:
lock1.release()
raise
yield None
try:
lock2.release()
finally:
lock1.release()
I know that this complicates simpler use cases, and I'm not 100% sure
this is the right solution; but I don't know how else to handle this
use case.
If __enter__ raises an exception, it has to clean up after itself before
propagating the exception. __exit__ shouldn't be called if __enter__ fails.
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Brett C. wrote: Guido van Rossum wrote: PEP 340 is still my favorite, but it seems there's too much opposition to it, so I'm trying to explore alternatives; at the same time I *really* dislike the complexities of some of the non-looping counterproposals (e.g. Nick Coghlan's PEP 3XX or the proposals that make every keyword associated with 'try' a method). Nick's was obviously directly against looping, but, with no offense to Nick, how many other people were against it looping? It never felt like it was a screaming mass with pitchforks but more of a I don't love it, but I can deal crowd. PEP 340 is very nice, but it became less appealing to me when I saw what it would do to break and continue statements. text = 'diamond' for fn in filenames: opening(fn) as f: if text in f.read(): print 'I found the text in %s' % fn break I think it would be pretty surprising if the break didn't stop the loop. Here's a new suggestion for PEP 340: use one keyword to start a block you don't want to loop, and a different keyword to start a block that can loop. If you specify the non-looping keyword but the block template produces more than one result, a RuntimeError results. Here is example A, a non-looping block statement using try: text = 'diamond' for fn in filenames: try opening(fn) as f: if text in f.read(): print 'I found the text in %s' % fn break In example A, the break statement breaks the for loop. If the opening() iterator returns more than one result, a RuntimeError will be generated by the Python interpreter. Here is example B, a looping block statement using in, adapted from PEP 340: in auto_retry(3, IOError) as attempt: f = urllib.urlopen(http://python.org/peps/pep-0340.html;) print f.read() Note that I introduced no new keywords except as, and the syntax in both cases is currently illegal. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merging PEP 310 and PEP 340-redux?
Guido van Rossum wrote: Going for all-out simplicity, I would like to be able to write these examples: class locking: def __init__(self, lock): self.lock = lock def __enter__(self): self.lock.acquire() def __exit__(self, *args): self.lock.release() class opening: def __init__(self, filename): self.filename = filename def __enter__(self): self.f = open(self.filename); return self.f def __exit__(self, *args): self.f.close()\ And do EXPR as VAR: BLOCK would mentally be translated into itr = EXPR VAR = itr.__enter__() try: BLOCK finally: itr.__exit__(*sys.exc_info()) # Except sys.exc_info() isn't defined by finally If it's this simple, it should be possible to write something that combines the acquisition of multiple resources in a single statement. For example: with combining(opening(src_fn), opening(dst_fn, 'w')) as src, dst: copy(src, dst) I think the following class would do it. class combining: def __init__(self, *resources): self.resources = resources self.entered = 0 def __enter__(self): results = [] try: for r in self.resources: results.append(r.__enter__()) self.entered += 1 return results except: # exit resources before re-raising the exception self.__exit__() raise def __exit__(self, *args): last_exc = None # exit only the resources successfully entered to_exit = self.resources[:self.entered] while to_exit: r = to_exit.pop() try: r.__exit__(*args) except: # re-raise the exception after exiting the others last_exc = sys.exc_info() if last_exc is not None: raise last_exc[0], last_exc[1], last_exc[2] Would that work? Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merging PEP 310 and PEP 340-redux?
Phillip J. Eby wrote: FYI, there are still use cases for clearing the exception state in an __exit__ method, that might justify allowing a true return from __exit__ to suppress the error. e.g.: Maybe __exit__ could suppress exceptions using a new idiom: def __exit__(self,*exc): if exc and not last and issubclass(exc[0],self.type): # suppress the exception raise None This seems clearer than return True. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New Py_UNICODE doc
M.-A. Lemburg wrote: All this talk about UTF-16 vs. UCS-2 is not very useful and strikes me a purely academic. The reference to possibly breakage by slicing a Unicode and breaking a surrogate pair is valid, the idea of UCS-4 being less prone to breakage is a myth: Fair enough. The original point is that the documentation is unclear about what a Py_UNICODE[] contains. I deduced that it contains either UCS2 or UCS4 and implemented accordingly. Not only did I guess wrong, but others will probably guess wrong too. Something in the docs needs to spell this out. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New Py_UNICODE doc
Martin v. Löwis wrote: Define correctly. Python, in ucs2 mode, will allow to address individual surrogate codes, e.g. in indexing. So you get u\U00012345[0] When Python encodes characters internally in UCS-2, I would expect u\U00012345 to produce a UnicodeError(character can not be encoded in UCS-2). u'\ud808' This will never work correctly, and never should, because an efficient implementation isn't possible. If you want safe indexing and slicing, you need ucs4. I agree that UCS4 is needed. There is a balancing act here; UTF-16 is widely used and takes less space, while UCS4 is easier to treat as an array of characters. Maybe we can have both: unicode objects start with an internal representation in UTF-16, but get promoted automatically to UCS4 when you index or slice them. The difference will not be visible to Python code. A compile-time switch will not be necessary. What do you think? Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New Py_UNICODE doc
Martin v. Löwis wrote: Shane Hathaway wrote: I agree that UCS4 is needed. There is a balancing act here; UTF-16 is widely used and takes less space, while UCS4 is easier to treat as an array of characters. Maybe we can have both: unicode objects start with an internal representation in UTF-16, but get promoted automatically to UCS4 when you index or slice them. The difference will not be visible to Python code. A compile-time switch will not be necessary. What do you think? This breaks backwards compatibility with existing extension modules. Applications that do PyUnicode_AsUnicode get a Py_UNICODE*, and can use that to directly access the characters. Py_UNICODE would always be 32 bits wide. PyUnicode_AsUnicode would cause the unicode object to be promoted automatically. Extensions that break as a result are technically broken already, aren't they? They're not supposed to depend on the size of Py_UNICODE. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New Py_UNICODE doc
Nicholas Bastin wrote: On May 6, 2005, at 3:42 PM, James Y Knight wrote: It means all the string operations treat strings as if they were UCS-2, but that in actuality, they are UTF-16. Same as the case in the windows APIs and Java. That is, all string operations are essentially broken, because they're operating on encoded bytes, not characters, but claim to be operating on characters. Well, this is a completely separate issue/problem. The internal representation is UTF-16, and should be stated as such. If the built-in methods actually don't work with surrogate pairs, then that should be fixed. Wait... are you saying a Py_UNICODE array contains either UTF-16 or UTF-32 characters, but never UCS-2? That's a big surprise to me. I may need to change my PyXPCOM patch to fit this new understanding. I tried hard to not care how Python encodes unicode characters, but details like this are important when combining two frameworks with different unicode APIs. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New Py_UNICODE doc
Nicholas Bastin wrote: On May 6, 2005, at 5:21 PM, Shane Hathaway wrote: Wait... are you saying a Py_UNICODE array contains either UTF-16 or UTF-32 characters, but never UCS-2? That's a big surprise to me. I may need to change my PyXPCOM patch to fit this new understanding. I tried hard to not care how Python encodes unicode characters, but details like this are important when combining two frameworks with different unicode APIs. Yes. Well, in as much as a large part of UTF-16 directly overlaps UCS-2, then sometimes unicode strings contain UCS-2 characters. However, characters which would not be legal in UCS-2 are still encoded properly in python, in UTF-16. And yes, I feel your pain, that's how I *got* into this position. Mapping from external unicode types is an important aspect of writing extension modules, and the documentation does not help people trying to do this. The fact that python's internal encoding is variable is a huge problem in and of itself, even if that was documented properly. This is why tools like Xerces and ICU will be happy to give you whatever form of unicode strings you want, but internally they always use UTF-16 - to avoid having to write two internal implementations of the same functionality. If you look up and down Objects/unicodeobject.c you'll see a fair amount of code written a couple of different ways (using #ifdef's) because of the variability in the internal representation. Ok. Thanks for helping me understand where Python is WRT unicode. I can work around the issues (or maybe try to help solve them) now that I know the current state of affairs. If Python correctly handled UTF-16 strings internally, we wouldn't need the UCS-4 configuration switch, would we? Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [OT] Re: PEP 340 -- Clayton's keyword?
Just a little offtopic note to Jeff Bone: Jeff, every time I send a message to Python-Dev, your Mail.app 2.0 sends me a nasty auto-reply that I can't quote in public. Please stop. Since I can't seem to reach you by email, I'm trying to reach you through this mailing list. The note refers to something about Shantar; maybe that will help you figure out what's wrong. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New Py_UNICODE doc
Nicholas Bastin wrote: On May 4, 2005, at 6:20 PM, Shane Hathaway wrote: On a related note, it would be help if the documentation provided a little more background on unicode encoding. Specifically, that UCS-2 is not the same as UTF-16, even though they're both two bytes wide and most of the characters are the same. UTF-16 can encode 4 byte characters, while UCS-2 can't. A Py_UNICODE is either UCS-2 or UCS-4. It took me I'm not sure the Python documentation is the place to teach someone about unicode. The ISO 10646 pretty clearly defines UCS-2 as only containing characters in the BMP (plane zero). On the other hand, I don't know why python lets you choose UCS-2 anyhow, since it's almost always not what you want. Then something in the Python docs ought to say why UCS-2 is not what you want. I still don't know; I've heard differing opinions on the subject. Some say you'll never need more than what UCS-2 provides. Is that incorrect? More generally, how should a non-unicode-expert writing Python extension code find out the minimum they need to know about unicode to use the Python unicode API? The API reference [1] ought to at least have a list of background links. I had to hunt everywhere. .. [1] http://docs.python.org/api/unicodeObjects.html Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Breaking out.
Alex Martelli wrote: Looking for a file with a certain magicnumber in its 1st two bytes...? for name in filenames: opening(name) as f: if f.read(2) == 0xFEB0: break This does seem to make real-life sense to me... I'd like to suggest a small language enhancement that would fix this example. Allow the break and continue statements to use a keyword, either for or while, to state that the code should break out of both the block statement and the innermost for or while statement. The example above would change to: for name in filenames: opening(name) as f: if f.read(2) == 0xFEB0: break for This could be a separate PEP if necessary. When a break for is used in a block statement, it should raise a new kind of exception, BreakForLoop, and the block statement should propagate the exception. When used outside a block statement, break for can use existing Python byte code to jump directly to the next appropriate statement. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: propose to get rid of 'as' keyword
Gustavo J. A. M. Carneiro wrote: In PEP 340 I read: block EXPR1 as VAR1: BLOCK1 I think it would be much clearer this (plus you save one keyword): block VAR1 = EXPR1: BLOCK1 I think you misunderstood the statement. EXPR1 creates an iterator, then VAR1 iterates over the values returns by the iterator. VAR1 never sees the iterator. Using your syntax would reinforce the misinterpretation that VAR1 sees the iterator. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Breaking out.
Ka-Ping Yee wrote: On Wed, 4 May 2005, Shane Hathaway wrote: for name in filenames: opening(name) as f: if f.read(2) == 0xFEB0: break for This is very elegant. Thanks. It works beautifully with break, though at first that natural analogs continue for, continue while appear to conflict with Guido's proposed extension to continue. But if we choose the keyword with to introduce an anonymous block, it comes out rather nicely: continue with 2 That's easier to read than continue 2, in my opinion. (If it's not too cute for you.) Or perhaps: continue yield 2 This would create some symmetry, since generators will retrieve the value passed by a continue statement using a yield expression. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Reinhold Birkenfeld wrote:
Noam Raphael wrote:
On 5/4/05, Reinhold Birkenfeld [EMAIL PROTECTED] wrote:
resource = opening(file.txt)
resource:
(...)
The latter would have to be forbidden.
Can you explain why it would have to be forbidden please?
Well, with it you could create suites with _any_ introducing
identifier. Consider:
[...]
transaction:
(...)
Do you understand my concern? It would be very, very hard to discern
these user-defined statements from real language constructs.
For each block statement, it is necessary to create a *new* iterator,
since iterators that have stopped are required to stay stopped. So at a
minimum, used-defined statements will need to call something, and thus
will have parentheses. The parentheses might be enough to make block
statements not look like built-in keywords.
PEP 340 seems to punish people for avoiding the parentheses:
transaction = begin_transaction()
transaction:
db.execute('insert 3 into mytable')
transaction:
db.execute('insert 4 into mytable')
I expect that only '3' would be inserted in mytable. The second use of
the transaction iterator will immediately raise StopIteration.
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Reinhold Birkenfeld wrote:
Shane Hathaway wrote:
For each block statement, it is necessary to create a *new* iterator,
Right.
since iterators that have stopped are required to stay stopped. So at a
minimum, used-defined statements will need to call something, and thus
will have parentheses. The parentheses might be enough to make block
statements not look like built-in keywords.
PEP 340 seems to punish people for avoiding the parentheses:
transaction = begin_transaction()
transaction:
db.execute('insert 3 into mytable')
transaction:
db.execute('insert 4 into mytable')
I expect that only '3' would be inserted in mytable. The second use of
the transaction iterator will immediately raise StopIteration.
Yes, but wouldn't you think that people would misunderstand it in this way?
Yes, they might. Just to be clear, the risk is that people will try to
write statements without parentheses and get burned because their code
doesn't get executed, right?
A possible workaround is to identify iterators that have already
finished. StopIteration doesn't distinguish between an iterator that
never yields any values from an iterator that has yielded all of its
values. Maybe there should be a subclass of StopIteration like
AlreadyStoppedIteration. Then, if a block statement gets an
AlreadyStoppedIteration exception from its iterator, it should convert
that to an error like InvalidBlockError.
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Shane Holloway (IEEE) wrote: Another thing. In the specification of the Anonymous Block function, is there a reason that itr = EXPR1 instead of itr = iter(EXPR1)? It seems to be a dis-symmetry with the 'for' loop specification. Hmm... yeah. That's strange. In fact, if it gets changed to itr = iter(EXPR1), as it probably ought to, all of the existing examples will continue to work. It will also be safe to start block iterators with a single variable, nullifying my argument about parentheses. So Reinhold's examples stand, except for the try block, since it clashes with a keyword. They read well, but when something goes wrong in the code, how would a new programmer crack these nuts? with: (...) synchronized: (...) transaction: (...) Thanks, -Shane (Holloway) ;) Once in a while I read a post by Shane Hway and start wondering when I wrote it and how I could've forgotten about it. And then I realize I didn't. :-) Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Greg Ewing wrote: Shane Hathaway wrote: For each block statement, it is necessary to create a *new* iterator, since iterators that have stopped are required to stay stopped. So at a minimum, used-defined statements will need to call something, and thus will have parentheses. Not necessarily! class Frobbing: def __neg__(self): begin_frobbing() try: yield finally: end_frobbing() frobbing = Frobbing() ... -frobbing: do_something() Larry Wall would hire you in a heartbeat. ;-) Maybe there's really no way to prevent people from writing cute but obscure block statements. A keyword like block or suite would give the reader something firm to hold on to. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
Luis P Caamano wrote: I've been skipping most of the anonymous block discussion and thus, I only had a very vague idea of what it was about until I read this explanation. Yes, it is understandable -- assuming it's correct :-) To my surprise, the explanation is now in the PEP. (Thanks, Guido!) Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
Guido van Rossum wrote: I don't know. What exactly is the audience supposed to be of this high-level statement? It would be pretty darn impossible to explain even the for-statement to people who are new to programming, let alone generators. And yet explaining the block-statement *must* involve a reference to generators. I'm guessing most introductions to Python, even for experienced programmers, put generators off until the advanced section, because this is pretty wild if you're not used to a language that has something similar. (I wonder how you'd explain Python generators to an experienced Ruby programmer -- their mind has been manipulated to the point where they'd be unable to understand Python's yield no matter how hard they tried. :-) I think this concept can be explained clearly. I'd like to try explaining PEP 340 to someone new to Python but not new to programming. I'll use the term block iterator to refer to the new type of iterator. This is according to my limited understanding. Good programmers move commonly used code into reusable functions. Sometimes, however, patterns arise in the structure of the functions rather than the actual sequence of statements. For example, many functions acquire a lock, execute some code specific to that function, and unconditionally release the lock. Repeating the locking code in every function that uses it is error prone and makes refactoring difficult. Block statements provide a mechanism for encapsulating patterns of structure. Code inside the block statement runs under the control of an object called a block iterator. Simple block iterators execute code before and after the code inside the block statement. Block iterators also have the opportunity to execute the controlled code more than once (or not at all), catch exceptions, or receive data from the body of the block statement. A convenient way to write block iterators is to write a generator. A generator looks a lot like a Python function, but instead of returning a value immediately, generators pause their execution at yield statements. When a generator is used as a block iterator, the yield statement tells the Python interpreter to suspend the block iterator, execute the block statement body, and resume the block iterator when the body has executed. The Python interpreter behaves as follows when it encounters a block statement based on a generator. First, the interpreter instantiates the generator and begins executing it. The generator does setup work appropriate to the pattern it encapsulates, such as acquiring a lock, opening a file, starting a database transaction, or starting a loop. Then the generator yields execution to the body of the block statement using a yield statement. When the block statement body completes, raises an uncaught exception, or sends data back to the generator using a continue statement, the generator resumes. At this point, the generator can either clean up and stop or yield again, causing the block statement body to execute again. When the generator finishes, the interpreter leaves the block statement. Is it understandable so far? Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: switch statement
Michael Chermside wrote: if x == 1:|if condition_1: do_1() |y = 1 elif x == 2: |elif condition_2: do_2() |y = 2 elif x == 3: |elif condition_3: do_3() |y = 3 else: |else: default() |y = 4 This inspired a twisted thought: if you just redefine truth, you don't have to repeat the variable. 0.9 wink True = x if 1: do_1() elif 2: do_2() elif 3: do_3() else: default() Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: [Pythonmac-SIG] zipfile still has 2GB boundary bug
Bob Ippolito wrote: The zipfile module is good enough to create input files for zipimport.. which is well tested and generally works -- barring the fact that zipimport has quite a few rough edges of its own. I certainly wouldn't recommend it for any heavy duty tasks in its current state. That's interesting because Java seems to suffer from similar problems. In the early days of Java, although a jar file was a zip file, Java wouldn't read jar files created by the standard zip utilities I used. I think the distinction was that the jar utility stored the files uncompressed. Java is fixed now, but I think it illustrates that zip files are non-trivial. BTW, I don't think the jar utility can delete files from a zip file either. ;-) Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defmacro (was: Anonymous blocks)
Robert Brewer wrote: So currently, all subclasses just override __set__, which leads to a *lot* of duplication of code. If I could write the base class' __set__ to call macros like this: def __set__(self, unit, value): self.begin() if self.coerce: value = self.coerce(unit, value) oldvalue = unit._properties[self.key] if oldvalue != value: self.pre() unit._properties[self.key] = value self.post() self.end() defmacro begin: pass defmacro pre: pass defmacro post: pass defmacro end: pass Here is a way to write that using anonymous blocks: def __set__(self, unit, value): with self.setting(unit, value): if self.coerce: value = self.coerce(unit, value) oldvalue = unit._properties[self.key] if oldvalue != value: with self.changing(oldvalue, value): unit._properties[self.key] = value def setting(self, unit, value): # begin code goes here yield None # end code goes here def changing(self, oldvalue, newvalue): # pre code goes here yield None # post code goes here ...(which would require macro-blocks which were decidedly *not* anonymous) then I could more cleanly write a subclass with additional macro methods: defmacro pre: old_children = self.children() defmacro post: for child in self.children: if child not in old_children: notify_somebody(New child %s % child) def changing(self, oldvalue, newvalue): old_children = self.children() yield None for child in self.children: if child not in old_children: notify_somebody(New child %s % child) Which do you prefer? I like fewer methods. ;-) Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defmacro (was: Anonymous blocks)
Robert Brewer wrote: I still prefer more methods, because my actual use-cases are more complicated. Your solution would work for the specific case I gave, but try factoring in: * A subclass which needs to share locals between begin and post, instead of pre and post. or * A set of 10 subclasses which need the same begin() but different end() code. Yielding seems both too restrictive and too inside-out to be readable, IMO. Ok, that makes sense. However, one of your examples seemingly pulls a name, 'old_children', out of nowhere. That's hard to fix. One of the greatest features of Python is the simple name scoping; we can't lose that. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Nick Coghlan wrote: An alternative would be to replace the 'yield None' with a 'break' or 'continue', and create an object which supports the resource protocol and NOT the iterator protocol. Something like: def my_resource(): print Hi! # Do entrance code continue # Go on with the contents of the 'with' block print Bye! # Do exit code (This is currently a SyntaxError, so it isn't ambiguous in any way) That's a very interesting suggestion. I've been lurking, thinking about a way to use something like PEP 310 to help manage database transactions. Here is some typical code that changes something under transaction control: begin_transaction() try: changestuff() changemorestuff() except: abort_transaction() raise else: commit_transaction() There's a lot of boilerplate code there. Using your suggestion, I could write that something like this: def transaction(): begin_transaction() try: continue except: abort_transaction() raise else: commit_transaction() with transaction(): changestuff() changemorestuff() Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
Fredrik Lundh wrote: Brian Sabbey wrote: doFoo(**): def func1(a, b): return a + b def func2(c, d): return c + d That is, a suite can be used to define keyword arguments. umm. isn't that just an incredibly obscure way to write def func1(a, b): return a + b def func2(c, d): return c + d doFoo(func1, func2) but with more indentation? Brian's suggestion makes the code read more like an outline. In Brian's example, the high-level intent stands out from the details, while in your example, there is no visual cue that distinguishes the details from the intent. Of course, lambdas are even better, when it's possible to use them: doFoo((lambda a, b: a + b), (lambda c, d: c + d)) Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
