[Python-Dev] test_cmd_line failure on Kubuntu 5.10 with GCC 4.0
Anyone else seeing any problems with test_cmd_line? I've got a few failures in test_cmd_line on Kubuntu 5.10 with GCC 4.0 relating to a missing "\n" line ending. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PythonCore\CurrentVersion
What happened to the CurrentVersion registry entry documented at http://www.python.org/windows/python/registry.html AFAICT, even the python15.wse file did not fill a value in this entry (perhaps I'm misinterpreting the wse file, though). So was this ever used? Why is it documented, and who documented it (unfortunately, registry.html is not in cvs/subversion, either)? Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency
On Saturday 08 October 2005 04:05, Josiah Carlson wrote: [ simplistic, informal benchmark of a test optimised versioned of the system, based on bouncing scaing rotating sprites around the screen. ] > Single process? Multi-process single machine? Multiprocess multiple > machine? SIngle process, single CPU, not very recent machine. (600MHz crusoe based machine so) That machine wasn't hardware accelerated though, so was only able to handle several dozen sprites before slowing down. The slowdown was due to the hardware not being able to keep up with pygame's drawing requests though rather than the framework. > I'm just offering the above as example benchmarks (you certainly don't > need to do them to satisfy me, but I'll be doing those when my tuple > space implementation is closer to being done). I'll note them as things worth doing - they look reasonable and interesting benchmarks. (I can think of a few modifications I might make though. For example in 3 you say "fastest". I might have that as a 3b. 3a could be "simplest to use/read" or "most likely to pick". Obviously there's a good chance that's not the fastest. (Could be optimised to be under the hood I suppose, but that wouldn't be the point of the test) > > [ Network controlled Networked Audio Mixing Matrix ] > I imagine you are using a C library/extension of some sort to do the > mixing...perhaps numarray, Numeric, ... Nope, just plain old python (I'm now using a 1.6Ghz centrino machine though). My mixing function is particularly naive as well. To me that says more about python than my code. I did consider using pyrex to wrap (or write) an optimised version, but there didn't seem to be any need for last week (Though for a non-prototype something faster would be nice :). I'll save responding the linda things until I have a chance to read in detail what you've written. It sounds very promising though - having multiple approaches to different styles of concurrency that work nicely with each other safely is always a positive thing IMO. Thanks for the suggestions and best regards, Michael. -- "Though we are not now that which in days of old moved heaven and earth, that which we are, we are: one equal temper of heroic hearts made weak by time and fate but strong in will to strive, to seek, to find and not to yield" -- "Ulysses", Tennyson ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sandboxed Threads in Python
On 10/7/05, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 07:17 PM 10/7/2005 -0600, Adam Olsen wrote: > >On 10/7/05, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > > > Note that implementing a root-based GC for Python is non-trivial, since > > > extension modules can store pointers to PyObjects anywhere they > > > like. Further, many Python objects don't even support being tracked by > > > the > > > current cycle collector. > > > > > > So, changing this would probably require a lot of C extensions to be > > > rewritten to support the needed API changes for the new garbage collection > > > strategy. > > > >They only need to be rewritten if you want them to provide an > >immutable type that can be transferred between sandboxes. > > No. You're missing my point. If they are able to *reference* these > objects, then the garbage collector has to know about it, or else it can't > know when to reclaim them. Ergo, these objects will leak, or else > extensions will crash when they refer to the deallocated memory. > > In other words, you can't handwave the whole problem away by assuming "a > garbage collector". The garbage collector has to actually be able to work, > and you haven't specified *how* it can work without changing the C API. Unfortunately the rammifications of your original statement didn't set in until well after I sent my reply. You are right, it does make it impossible without changing the C API, so that much of the idea is dead. I wonder if it would be possible to use a wrapper around the immutable type instead.. something to ponder anyway. > >I was aware that weakrefs needed some special handling (I just forgot > >to mention it), but I didn't know it was used by subclassing. > >Unfortunately I don't know what purpose it serves so I can't > >contemplate how to deal with it. > > It allows changes to a supertype's C-level slots to propagate to subclasses. I see. Well, I would have required the supertype to be immutable, so there couldn't be any changes to the C-level slots. -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sandboxed Threads in Python
On 10/7/05, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > Adam Olsen <[EMAIL PROTECTED]> wrote: > > I need to stress that *only* the new, immutable and "thread-safe > > mark-and-sweep" types would be affected by these changes. Everything > > else would continue to exist as it did before, and the benchmark > > exists to show they can coexist without killing performance. > > All the benchmark showed was that checking for a constant in the > refcount during in/decrefing, and not garbage collecting those objects, > didn't adversely affect performance. > > As an aside, there's also the ugly bit about being able to guarantee > that an object is immutable. I personally mutate Python strings in my C > code all the time (long story, not to be discussed here), and if I can > do it now, then any malicious or "inventive" person can do the same in > this "sandboxed thread" Python "of the future". Malicious use is hardly a serious concern. Someone using C code could just as well crash the interpreter. Modifying a python string you just created before you expose it to python code should be fine. If that's not what you're doing.. I'm not sure I want to know *wink* > At least in the case of integers, one could work the tagged integer idea > to bypass the freelist issue the Phillip offered, but in general, I > don't believe there exists a truely immutable type as long as there is C > extensions and/or cTypes. Further, the work to actually implement a new > garbage collector for Python in order to handle these 'immutable' types > seems to me to be more trouble than it is worth. Maybe.. I'm not convinced. There's a lot of payback IMO. -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] test_cmd_line failure on Kubuntu 5.10 with GCC 4.0
On 10/8/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Anyone else seeing any problems with test_cmd_line? I've got a few failures in > test_cmd_line on Kubuntu 5.10 with GCC 4.0 relating to a missing "\n" line > ending. > Same problem here. (FreeBSD 6.0 with GCC 3.4.4) In my short inspection, popen2.popen4.read() returned just an empty string. I'll investigate more in this weekend. Hye-Shik ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] test_cmd_line failure on Kubuntu 5.10 with GCC 4.0
Hye-Shik Chang wrote: > On 10/8/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > >>Anyone else seeing any problems with test_cmd_line? I've got a few failures in >>test_cmd_line on Kubuntu 5.10 with GCC 4.0 relating to a missing "\n" line >>ending. >> > > > Same problem here. (FreeBSD 6.0 with GCC 3.4.4) > In my short inspection, popen2.popen4.read() returned just an empty string. Good to know it isn't just a system quirk, as that's the same behaviour I'm getting. I noticed that the ones which appear to be failing (-E, -O, -S, -Q) are the ones which expect an interactive session to open. The tests which pass (-V, -h, directory as argument or stdin) are the ones which don't actually start the interpreter. If I explicitly write Ctrl-D to the subprocess's stdin for the tests which open the interpreter, then the tests pass. So it looks like some sort of buffering problem with standard out not getting flushed before the test tries to read the data. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sandboxed Threads in Python
Adam Olsen <[EMAIL PROTECTED]> wrote: > On 10/7/05, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > Adam Olsen <[EMAIL PROTECTED]> wrote: > > > I need to stress that *only* the new, immutable and "thread-safe > > > mark-and-sweep" types would be affected by these changes. Everything > > > else would continue to exist as it did before, and the benchmark > > > exists to show they can coexist without killing performance. > > > > All the benchmark showed was that checking for a constant in the > > refcount during in/decrefing, and not garbage collecting those objects, > > didn't adversely affect performance. > > > > As an aside, there's also the ugly bit about being able to guarantee > > that an object is immutable. I personally mutate Python strings in my C > > code all the time (long story, not to be discussed here), and if I can > > do it now, then any malicious or "inventive" person can do the same in > > this "sandboxed thread" Python "of the future". > > Malicious use is hardly a serious concern. Someone using C code could > just as well crash the interpreter. Your malicious user is my inventive colleague. Here's one: performing zero-copy inter-thread IPC by modifying shared immutables. Attempting to enforce a policy of "don't do that, it's not supported" is not going to be effective, especially when doing unsupported things increase speed. People have known for decades that having anything run in kernel space beyond the kernel is dangerous, but they still do because it is faster. I can (but won't) point out examples for days of bad decisions made for the sake of speed, or policy that has been ignored for the sake of speed (some of these overlap and some don't). > Modifying a python string you just created before you expose it to > python code should be fine. If that's not what you're doing.. I'm not > sure I want to know *wink* You really don't want to know. > Maybe.. I'm not convinced. There's a lot of payback IMO. You've not convinced me either. Good luck in getting a group together to make it happen. - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency
Michael Sparks <[EMAIL PROTECTED]> wrote:
> On Saturday 08 October 2005 04:05, Josiah Carlson wrote:
> > I'm just offering the above as example benchmarks (you certainly don't
> > need to do them to satisfy me, but I'll be doing those when my tuple
> > space implementation is closer to being done).
>
> I'll note them as things worth doing - they look reasonable and interesting
> benchmarks. (I can think of a few modifications I might make though. For
> example in 3 you say "fastest". I might have that as a 3b. 3a could be
> "simplest to use/read" or "most likely to pick". Obviously there's a good
> chance that's not the fastest. (Could be optimised to be under the hood
> I suppose, but that wouldn't be the point of the test)
Good point.
3a. Use 1024 byte blocks...
3b. Use whatever makes your system perform best (if you have the time to
tune it)...
> > > [ Network controlled Networked Audio Mixing Matrix ]
> > I imagine you are using a C library/extension of some sort to do the
> > mixing...perhaps numarray, Numeric, ...
>
> Nope, just plain old python (I'm now using a 1.6Ghz centrino machine
> though). My mixing function is particularly naive as well. To me that says
> more about python than my code. I did consider using pyrex to wrap (or
> write) an optimised version, but there didn't seem to be any need for
> last week (Though for a non-prototype something faster would be
> nice :).
Indeed. A quick array.array('h',...) implementation is able to run 7-8x
real time on 3->1 stream mixing on my 1.3 ghz laptop. Maybe numeric or
numarray isn't necessary.
> I'll save responding the linda things until I have a chance to read in detail
> what you've written. It sounds very promising though - having multiple
> approaches to different styles of concurrency that work nicely with each
> other safely is always a positive thing IMO.
>
> Thanks for the suggestions and best regards,
Thank you for the interesting and informative discussion.
- Josiah
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] test_cmd_line failure on Kubuntu 5.10 with GCC 4.0
On 10/8/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Hye-Shik Chang wrote: > > On 10/8/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > > > >>Anyone else seeing any problems with test_cmd_line? I've got a few failures > >>in > >>test_cmd_line on Kubuntu 5.10 with GCC 4.0 relating to a missing "\n" line > >>ending. > > If I explicitly write Ctrl-D to the subprocess's stdin for the tests which > open the interpreter, then the tests pass. So it looks like some sort of > buffering problem with standard out not getting flushed before the test tries > to read the data. Sorry, that's a new test I added recently. It works for me on gentoo. The test is very simple and shouldn't be hard to fix. Can you fix it? I assume Guido (or someone) added you as a developer. If not, if you can give me enough info, I can try to fix it. n ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sandboxed Threads in Python
On 10/8/05, Josiah Carlson <[EMAIL PROTECTED]> wrote: > Your malicious user is my inventive colleague. Here's one: performing > zero-copy inter-thread IPC by modifying shared immutables. Attempting to > enforce a policy of "don't do that, it's not supported" is not going to > be effective, especially when doing unsupported things increase speed. I actually have no problem with that, so long as you use a custom type. It may not technically be immutable but it does provide its own clearly defined semantics for simultaneous modification, and that's enough. Anyway, the idea as I presented it is dead at this point, so I'll leave it at that. -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sandboxed Threads in Python
> I can (but won't) point out examples for days of bad decisions made for > the sake of speed, or policy that has been ignored for the sake of speed > (some of these overlap and some don't). As long as you've entered premature-optimization land, how about decisions made because it's *assumed* that (A) We must have speed here and (B) This will make it happen. My hope would be that we could find a solution that would by default keep you out of trouble when writing concurrent programs, but provide a back door if you wanted to do something special. If you choose to go in the back door, you have to do it consciously and take responsibility for the outcome. With Java, in contrast, as soon as you step into the world of concurrency (even if you step in by accident, which is not uncommon), lots of rules change. What was an ordinary method call before is now something risky that can cause great damage. Should I make this variable volatile? Is an operation atomic? You have to learn a lot of things all over again. I don't want that for Python. I'd like the move into concurrency to be a gentle slope, not a sudden reality-shift. If a novice decides they want to try game programming with concurrency, I want there to be training wheels on by default, so that their first experience will be a successful one, and they can then start learning more features and ideas incrementally, without trying a feature and suddenly having the whole thing get weird and crash down on their heads and cause them to run screaming away ... I know there have been some technologies that have already been mentioned on this list and I hope that we can continue to experiment with and discuss those and also new ideas until we shake out the fundamental issues and maybe even come up with a list of possible solutions. Bruce Eckelhttp://www.BruceEckel.com mailto:[EMAIL PROTECTED] Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e" Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel Subscribe to my newsletter: http://www.mindview.net/Newsletter My schedule can be found at: http://www.mindview.net/Calendar ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] test_cmd_line failure on Kubuntu 5.10 with GCC 4.0
On 10/8/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > On 10/8/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > > Hye-Shik Chang wrote: > > > On 10/8/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > > > > > >>Anyone else seeing any problems with test_cmd_line? I've got a few > > >>failures in > > >>test_cmd_line on Kubuntu 5.10 with GCC 4.0 relating to a missing "\n" > > >>line ending. > > > > If I explicitly write Ctrl-D to the subprocess's stdin for the tests which > > open the interpreter, then the tests pass. So it looks like some sort of > > buffering problem with standard out not getting flushed before the test > > tries > > to read the data. > > Sorry, that's a new test I added recently. It works for me on gentoo. > The test is very simple and shouldn't be hard to fix. Can you fix > it? I assume Guido (or someone) added you as a developer. If not, if > you can give me enough info, I can try to fix it. I guess Neil's test was expecting at least one line of output from python at all times, but on most systems it is completely silent when the input is empty. I fixed the test (also in 2.4) to allow empty input as well as input ending in \n. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sandboxed Threads in Python
Bruce Eckel <[EMAIL PROTECTED]> wrote: > > > I can (but won't) point out examples for days of bad decisions made for > > the sake of speed, or policy that has been ignored for the sake of speed > > (some of these overlap and some don't). > > As long as you've entered premature-optimization land, how about > decisions made because it's *assumed* that (A) We must have speed here > and (B) This will make it happen. A. From what I understand about sandboxing threads, the point was to remove the necessity for the GIL, so that every thread can go out on its own and run on its own processor. B. Shared memory vs. queues vs. pipes vs. ... Concurrency without communication is almost totally worthless. Historically, shared memory has tended to be one of the fastest (if not the fastest) communication methods available. Whether or not mutable shared memory would be faster or slower than queues is unknown, but I'm going to stick with my experience until I am proved wrong by this mythical free threaded system with immutables. > My hope would be that we could find a solution that would by default > keep you out of trouble when writing concurrent programs, but provide > a back door if you wanted to do something special. If you choose to go > in the back door, you have to do it consciously and take > responsibility for the outcome. > > With Java, in contrast, as soon as you step into the world of > concurrency (even if you step in by accident, which is not uncommon), > lots of rules change. What was an ordinary method call before is now > something risky that can cause great damage. Should I make this > variable volatile? Is an operation atomic? You have to learn a lot of > things all over again. > > I don't want that for Python. I'd like the move into concurrency to be > a gentle slope, not a sudden reality-shift. If a novice decides they > want to try game programming with concurrency, I want there to be > training wheels on by default, so that their first experience will be > a successful one, and they can then start learning more features and > ideas incrementally, without trying a feature and suddenly having the > whole thing get weird and crash down on their heads and cause them to > run screaming away ... I don't want to get into an argument here. While I agree that concurrent programming should be easier, my experience with MPI (and other similar systems) and writing parallel algorithms leads me to believe that even if you have a simple method for communication, even if you can guarantee that thread/process A won't clobber thread/process B, actually writing software which executes in some way which made the effort of making the software concurrent worthwhile, is less than easy. I'd love to be proved wrong (I'm hoping to do it myself with tuple spaces). I do, however, doubt that free threading approaches will be the future for concurrent programming in CPython. - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods
On 10/7/05, Piet Delport <[EMAIL PROTECTED]> wrote: > Earlier this week, i proposed legalizing "return Result" inside a generator, > and making it act like "raise StopIteration( Result )", for exactly this > reason. > > IMHO, this is an elegant and straightforward extension of the current > semantics of returns inside generators, and is the final step toward making > generator-based concurrent tasks[1] look just like the equivalent synchronous > code (with the only difference, more-or-less, being the need for appropriate > "yield" keywords, and a task runner/scheduler loop). > > This change would make a huge difference to the practical usability of these > generator-based tasks. I think they're much less likely to catch on if you > have to write "raise StopIteration( Result )" (or "_return( Result )") all the > time. > > [1] a.k.a. coroutines, which i don't think is an accurate name, anymore. Before we do this I'd like to see you show some programming examples that show how this would be used. I'm having a hard time understanding where you would need this but I realize I haven't used this paradigm enough to have a good feel for it, so I'm open for examples. At least this makes more sense than mapping "return X" into "yield X; return" as someone previously proposed. :) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods
Guido van Rossum wrote: >>This change would make a huge difference to the practical usability of these >>generator-based tasks. I think they're much less likely to catch on if you >>have to write "raise StopIteration( Result )" (or "_return( Result )") all the >>time. >> >>[1] a.k.a. coroutines, which i don't think is an accurate name, anymore. > > > Before we do this I'd like to see you show some programming examples > that show how this would be used. I'm having a hard time understanding > where you would need this but I realize I haven't used this paradigm > enough to have a good feel for it, so I'm open for examples. > > At least this makes more sense than mapping "return X" into "yield X; > return" as someone previously proposed. :) It would be handy when the generators are being used as true pseudothreads with a scheduler like the one I posted earlier in this discussion. It allows these pseudothreads to "call" each other by yielding the call as a lambda or partial function application that produces a zero-argument callable. The called pseudothread can then yield as many times as it wants (either making its own calls, or just being a well-behaved member of a cooperatively MT environment), and then finally returning the value that the original caller requested. Using 'return' for this is actually a nice idea, and if we ever do make it legal to use 'return' in generators, these are the semantics it should have. However, I'm not sure its something we should be adding *right now* as part of PEP 342 - writing "raise StopIteration" and "raise StopIteration(result)", and saying that a generator includes an implied "raise StopIteration" after its last line of code really isn't that difficult to understand, and is completely explicit about what is going on. My basic concern is that I think replacing "raise StopIteration" with "return" and "raise StopIteration(EXPR)" with "return EXPR" would actually make such code easier to write at the expense of making it harder to *read*, because the fact that an exception is being raised is obscured. Consider the following two code snippets: def function(): try: return except StopIteration: print "We never get here." def generator(): yield try: return except StopIteration: print "But we would get here!" So, instead of having "return" automatically map to "raise StopIteration" inside generators, I'd like to suggest we keep it illegal to use "return" inside a generator, and instead add a new attribute "result" to StopIteration instances such that the following three conditions hold: # Result is None if there is no argument to StopIteration try: raise StopIteration except StopIteration, ex: assert ex.result is None # Result is the argument if there is exactly one argument try: raise StopIteration(expr) except StopIteration, ex: assert ex.result == ex.args[0] # Result is the argument tuple if there are multiple arguments try: raise StopIteration(expr1, expr2) except StopIteration, ex: assert ex.result == ex.args This precisely parallels the behaviour of return statements: return # Call returns None return expr# Call returns expr return expr1, expr2# Call returns (expr1, expr2) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sandboxed Threads in Python
Josiah Carlson wrote: > I do, however, doubt that free threading approaches will be the future > for concurrent programming in CPython. Hear, hear! IMO, it's the combination of the GIL with a compiler which never decides to change the code execution order under the covers that makes threading *not* a pain in Python (so long as one remembers to release the GIL around blocking calls to external libraries, and to use threading.Queue to get info between threads wherever possible). The desire to change that seems to be a classic case of wanting to write C/C++/Java/whatever in Python, rather than writing Python in Python. And thanks to Bruce for starting the recent multi-processing discussion - hopefully one day we will have mechanisms in the standard library that scale relatively smoothly from PEP 342 based logical threads, through threading.Thread based physical threads, to based subprocesses. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods
> Guido van Rossum wrote: > > Before we do this I'd like to see you show some programming examples > > that show how this would be used. I'm having a hard time understanding > > where you would need this but I realize I haven't used this paradigm > > enough to have a good feel for it, so I'm open for examples. On 10/8/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > It would be handy when the generators are being used as true pseudothreads > with a scheduler like the one I posted earlier in this discussion. It allows > these pseudothreads to "call" each other by yielding the call as a lambda or > partial function application that produces a zero-argument callable. The > called pseudothread can then yield as many times as it wants (either making > its own calls, or just being a well-behaved member of a cooperatively MT > environment), and then finally returning the value that the original caller > requested. > > Using 'return' for this is actually a nice idea, and if we ever do make it > legal to use 'return' in generators, these are the semantics it should have. > > However, I'm not sure its something we should be adding *right now* as part of > PEP 342 - writing "raise StopIteration" and "raise StopIteration(result)", and > saying that a generator includes an implied "raise StopIteration" after its > last line of code really isn't that difficult to understand, and is completely > explicit about what is going on. > > My basic concern is that I think replacing "raise StopIteration" with "return" > and "raise StopIteration(EXPR)" with "return EXPR" would actually make such > code easier to write at the expense of making it harder to *read*, because the > fact that an exception is being raised is obscured. Consider the following two > code snippets: > >def function(): > try: > return > except StopIteration: > print "We never get here." > >def generator(): > yield > try: > return > except StopIteration: > print "But we would get here!" Right. Plus, Piet also remarked that the value is silently ignored when the generator is used in a for-loop. Since that's likely to be the majority of generators, I'd worry that accepting "return X" would increase the occurrence of bugs caused by someone habitually writing "return X" where they meant "yield X". (Assuming there's another yield in the generator, otherwise it wouldn't be a generator and the error would reveal itself very differently.) > So, instead of having "return" automatically map to "raise StopIteration" > inside generators, I'd like to suggest we keep it illegal to use "return" > inside a generator, and instead add a new attribute "result" to StopIteration > instances such that the following three conditions hold: > > # Result is None if there is no argument to StopIteration > try: > raise StopIteration > except StopIteration, ex: > assert ex.result is None > > # Result is the argument if there is exactly one argument > try: > raise StopIteration(expr) > except StopIteration, ex: > assert ex.result == ex.args[0] > > # Result is the argument tuple if there are multiple arguments > try: > raise StopIteration(expr1, expr2) > except StopIteration, ex: > assert ex.result == ex.args > > This precisely parallels the behaviour of return statements: >return # Call returns None >return expr# Call returns expr >return expr1, expr2# Call returns (expr1, expr2) This seems a bit overdesigned; I'd expect that the trampoline scheduler could easily enough pick the args tuple apart to get the same effect without adding another attribute unique to StopIteration. I'd like to keep StopIteration really lightweight so it doesn't slow down its use in other places. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods
Guido van Rossum wrote: > On 10/8/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: >>So, instead of having "return" automatically map to "raise StopIteration" >>inside generators, I'd like to suggest we keep it illegal to use "return" >>inside a generator, and instead add a new attribute "result" to StopIteration >>instances such that the following three conditions hold: >> >> # Result is None if there is no argument to StopIteration >> try: >> raise StopIteration >> except StopIteration, ex: >> assert ex.result is None >> >> # Result is the argument if there is exactly one argument >> try: >> raise StopIteration(expr) >> except StopIteration, ex: >> assert ex.result == ex.args[0] >> >> # Result is the argument tuple if there are multiple arguments >> try: >> raise StopIteration(expr1, expr2) >> except StopIteration, ex: >> assert ex.result == ex.args >> >>This precisely parallels the behaviour of return statements: >> return # Call returns None >> return expr# Call returns expr >> return expr1, expr2# Call returns (expr1, expr2) > > > This seems a bit overdesigned; I'd expect that the trampoline > scheduler could easily enough pick the args tuple apart to get the > same effect without adding another attribute unique to StopIteration. > I'd like to keep StopIteration really lightweight so it doesn't slow > down its use in other places. True. And it would be easy enough for a framework to have a utility function that looked like: def getresult(ex): args = ex.args if not args: return None elif len(args) == 1: return args[0] else: return args Although, if StopIteration.result was a read-only property with the above definition, wouldn't that give us the benefit of "one obvious way" to return a value from a coroutine without imposing any runtime cost on normal use of StopIteration to finish an iterator? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods
On Oct 8, 2005, at 9:10 PM, Nick Coghlan wrote: > So, instead of having "return" automatically map to "raise > StopIteration" > inside generators, I'd like to suggest we keep it illegal to use > "return" > inside a generator Only one issue with that: it's _not currently illegal_ to use return inside a generator. From the view of the outsider, it currently effectively does currently map to "raise StopIteration". But not on the inside, just like you'd expect to happen. The only proposed change to the semantics is to also allow a value to be provided with the return. >def generator(): > yield > try: > return > except StopIteration: > print "But we would get here!" >>> def generator(): ... yield 5 ... try: ... return ... except StopIteration: ... print "But we would get here!" ... >>> x=generator() >>> x.next() 5 >>> x.next() Traceback (most recent call last): File "", line 1, in ? StopIteration >>> James ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods
James Y Knight wrote: > > On Oct 8, 2005, at 9:10 PM, Nick Coghlan wrote: > >> So, instead of having "return" automatically map to "raise >> StopIteration" >> inside generators, I'd like to suggest we keep it illegal to use >> "return" >> inside a generator > > > Only one issue with that: it's _not currently illegal_ to use return > inside a generator. From the view of the outsider, it currently > effectively does currently map to "raise StopIteration". But not on the > inside, just like you'd expect to happen. The only proposed change to > the semantics is to also allow a value to be provided with the return. Huh. I'd have sworn I'd tried that and it didn't work. Maybe I was using a value with the return, and had forgotten the details of the error message. In that case, I have far less of an objection to the idea - particularly since it *does* forcibly terminate the generator's block without triggering any exception handlers. I was forgetting that the StopIteration exception is actually raised external to the generator code block - it's created by the surrounding generator object once the code block terminates. That means the actual change being proposed is smaller than I thought: 1. Change the compiler to allow an argument to return inside a generator 2. Change generator objects to use the value returned by their internal code block as the argument to the StopIteration exception they create if the block terminates Note that this would change the behaviour of normal generators - they will raise "StopIteration(None)", rather than the current "StopIteration()". I actually kind of like that - it means that generators become even more like functions, with their return value being held in ex.args[0]. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
