Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Josiah Carlson
Michael Sparks [EMAIL PROTECTED] wrote: On Thursday 06 October 2005 23:15, Josiah Carlson wrote: [... 6 specific use cases ...] If Kamaelia is able to handle all of the above mechanisms in both a blocking and non-blocking fashion, then I would guess it has the basic requirements for

Re: [Python-Dev] Python 2.5 and ast-branch

2005-10-07 Thread Michael Hudson
Guido van Rossum [EMAIL PROTECTED] writes: How does this sound to the non-AST-branch developers who have to suffer the inevitable post-merge instability? I think it's now or never -- waiting longer isn't going to make this thing easier (not with several more language changes approved:

[Python-Dev] PyObject_Init documentation

2005-10-07 Thread Martin v. Löwis
says If type indicates that the object participates in the cyclic garbage detector, it is added to the detector's set of observed objects. Is this really correct? I thought you need to invoke PyObject_GC_TRACK explicitly? Regards, Martin ___

[Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Nick Coghlan
Based on Jason's comments regarding decimal.Context, and to explicitly cover the terminology agreed on during the documentation discussion back in July, I'm proposing a number of changes to PEP 343. I'll be updating the checked in PEP assuming there aren't any objections in the next week or so

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Fredrik Lundh
Nick Coghlan wrote: 9. Here's a proposed native context manager for decimal.Context: # This would be a new decimal.Context method @contextmanager def __with__(self): wouldn't it be better if the ContextWrapper class (or some variation thereof) could be used as a

[Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods

2005-10-07 Thread Nick Coghlan
I'm lifting Jason's PEP 342 suggestions out of the recent PEP 343 thread, in case some of the folks interested in coroutines stopped following that discussion. Jason suggested two convenience methods, .start() and .finish(). start() simply asserted that the generator hadn't been started yet,

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Nick Coghlan
Fredrik Lundh wrote: Nick Coghlan wrote: 9. Here's a proposed native context manager for decimal.Context: # This would be a new decimal.Context method @contextmanager def __with__(self): wouldn't it be better if the ContextWrapper class (or some variation

Re: [Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods

2005-10-07 Thread Nick Coghlan
Nick Coghlan wrote: It ends up looking like this: def __call__(self, value=None): Call a generator as a coroutine Returns the first argument supplied to StopIteration or None if no argument was supplied. Raises

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Michael Hudson
Nick Coghlan [EMAIL PROTECTED] writes: What if we simply special-cased the __with__ slot in type(), such that if it is populated with a generator object, that object is automatically wrapped using the @contextmanager decorator? (Jason actually suggested this idea previously) nit You don't

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Anders J. Munch
Nick Coghlan did a +1 job to write: 1. Amend the statement specification such that: with EXPR as VAR: BLOCK is translated as: abc = (EXPR).__with__() exc = (None, None, None) VAR = abc.__enter__() try: try:

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Eric Nieuwland
Nick Coghlan wrote: 1. Amend the statement specification such that: with EXPR as VAR: BLOCK is translated as: abc = (EXPR).__with__() exc = (None, None, None) VAR = abc.__enter__() try: try: BLOCK

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Nick Coghlan
Eric Nieuwland wrote: What happens to with 40*13+2 as X: print X It would fail with a TypeError because the relevant slot in the type object was NULL - the TypeError checks aren't shown for simplicity's sake. This behaviour isn't really any different from the existing PEP 343 -

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Nick Coghlan
Anders J. Munch wrote: Note that __with__ and __enter__ could be combined into one with no loss of functionality: abc,VAR = (EXPR).__with__() They can't be combined, because they're invoked on different objects. It would be like trying to combine __iter__() and next() into the same

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Nick Coghlan
Michael Hudson wrote: nit You don't want to check if it's a generator, you want to check if it's a function whose func_code has the relavent bit set. /nit Fair point :) Seems a bit magical to me, but haven't thought about it hard. Same here - I'm just starting to think that the alternative

[Python-Dev] Sourceforge CVS access

2005-10-07 Thread Nick Coghlan
Could one of the Sourceforge powers-that-be grant me check in access so I can update PEP 343 directly? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia ---

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Eric Nieuwland
Nick Coghlan wrote: Eric Nieuwland wrote: What happens to with 40*13+2 as X: print X It would fail with a TypeError because the relevant slot in the type object was NULL - the TypeError checks aren't shown for simplicity's sake. This behaviour isn't really any different

Re: [Python-Dev] Sourceforge CVS access

2005-10-07 Thread Guido van Rossum
I will, if you tell me your sourceforge username. On 10/7/05, Nick Coghlan [EMAIL PROTECTED] wrote: Could one of the Sourceforge powers-that-be grant me check in access so I can update PEP 343 directly? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Fredrik Lundh
Nick Coghlan wrote: That's not what the decorator is for - it's there to turn the generator used to implement the __with__ method into a context manager, rather than saying anything about decimal.Context as a whole. possibly, but using a decorated __with__ method doesn't make much sense if

[Python-Dev] Pythonic concurrency

2005-10-07 Thread Bruce Eckel
Early in this thread there was a comment to the effect that if you don't know how to use threads, don't use them, which I pointedly avoided responding to because it seemed to me to simply be inflammatory. But Ian Bicking just posted a weblog entry:

[Python-Dev] Extending tuple unpacking

2005-10-07 Thread Gustavo Niemeyer
Not sure if this has been proposed before, but one thing I occasionally miss regarding tuple unpack is being able to do: first, second, *rest = something Also in for loops: for first, second, *rest in iterator: pass This seems to match the current meaning for starred variables in

Re: [Python-Dev] Python 2.5 and ast-branch

2005-10-07 Thread Jeremy Hylton
On 10/6/05, Phillip J. Eby [EMAIL PROTECTED] wrote: At 07:34 PM 10/6/2005 -0700, Guido van Rossum wrote: How does this sound to the non-AST-branch developers who have to suffer the inevitable post-merge instability? I think it's now or never -- waiting longer isn't going to make this thing

Re: [Python-Dev] Extending tuple unpacking

2005-10-07 Thread Guido van Rossum
On 10/7/05, Gustavo Niemeyer [EMAIL PROTECTED] wrote: Not sure if this has been proposed before, but one thing I occasionally miss regarding tuple unpack is being able to do: first, second, *rest = something Also in for loops: for first, second, *rest in iterator: pass This

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Aahz
On Fri, Oct 07, 2005, Bruce Eckel wrote: I always have a problem with this. After many years of studying concurrency on-and-off, I continue to believe that threading is very difficult (indeed, the more I study it, the more difficult I understand it to be). And I admit this. The comments I

Re: [Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods

2005-10-07 Thread Phillip J. Eby
At 09:50 PM 10/7/2005 +1000, Nick Coghlan wrote: Notice how a non-coroutine callable can be yielded, and it will still work happily with the scheduler, because the desire to continue execution is indicated by the ContinueIteration exception, rather than by the type of the returned value. Wh?

Re: [Python-Dev] Extending tuple unpacking

2005-10-07 Thread Gustavo Niemeyer
Someone should really write up a PEP -- this was just discussed a week or two ago. Heh.. I should follow the list more closely. I personally think this is adequately handled by writing: (first, second), rest = something[:2], something[2:] That's an alternative indeed. But the the

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Phillip J. Eby
At 10:47 AM 10/7/2005 -0600, Bruce Eckel wrote: Also, look at the work that Scott Meyers, Andrei Alexandrescu, et al did on the Double Checked Locking idiom, showing that it was broken under threading. That was by no means trivial and obvious during all the years that people thought that it

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Shane Hathaway
Bruce Eckel wrote: But. I do happen to have contact with a lot of people who are at the forefront of the threading world, and *none* of them (many of whom have written the concurrency libraries for Java 5, for example) ever imply that threading is easy. In fact, they generally go out of their

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Barry Warsaw
On Fri, 2005-10-07 at 14:42, Shane Hathaway wrote: What's insanely difficult is really locking, and locking is driven by concurrency in general, not just threads. It's hard to reason about locks. I think that's a very interesting observation! I have not built a tremendous number of

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Antoine Pitrou
Hi, (my 2 cents, probably not very constructive) Recently, I've been simulating high concurrency on a PostgreSQL database, and I've discovered that the way you reason about row and table locks is very similar to the way you reason about locking among threads. The big difference is the

Re: [Python-Dev] __doc__ behavior in class definitions

2005-10-07 Thread Fredrik Lundh
Martin Maly wrote: I came across a case which I am not sure if by design or a bug in Python (Python 2.4.1 (#65, Mar 30 2005, 09:13:57)). Consider following Python module: # module begin module doc class c: print __doc__ __doc__ = class doc (1) print __doc__ print

Re: [Python-Dev] __doc__ behavior in class definitions

2005-10-07 Thread Phillip J. Eby
At 12:15 PM 10/7/2005 -0700, Martin Maly wrote: Based on the binding rules described in the Python documentation, I would expect the code to throw because binding created on the line (1) is local to the class block and all the other __doc__ uses should reference that binding. Apparently, it is not

Re: [Python-Dev] __doc__ behavior in class definitions

2005-10-07 Thread Steve Holden
Martin Maly wrote: Hello Python-Dev, My name is Martin Maly and I am a developer at Microsoft, working on the IronPython project with Jim Hugunin. I am spending lot of time making IronPython compatible with Python to the extent possible. I came across a case which I am not sure if by

Re: [Python-Dev] __doc__ behavior in class definitions

2005-10-07 Thread Jack Diederich
On Fri, Oct 07, 2005 at 12:15:04PM -0700, Martin Maly wrote: Hello Python-Dev, My name is Martin Maly and I am a developer at Microsoft, working on the IronPython project with Jim Hugunin. I am spending lot of time making IronPython compatible with Python to the extent possible. I came

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Michael Sparks
[ Possibly overlengthy reply. However given a multiple sets of cans of worms... ] On Friday 07 October 2005 07:25, Josiah Carlson wrote: One thing I notice is absent from the Kamaelia page is benchmarks. That's largely for one simple reason: we haven't done any yet. At least not anything I'd

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Jim Fulton
Shane Hathaway wrote: Antoine Pitrou wrote: A relational database has a very strict and regular data model. Also, it has transactions. This makes it easy to precisely define concurrency at the engine level. To apply the same thing to Python you would at least need : 1. a way to define a

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Antoine Pitrou
Well, I think you just described ZODB. ;-) *gasp* I'd be happy to explain how ZODB solves those problems, if you're interested. Well, yes, I'm interested :) (I don't anything about Zope internals though, and I've never even used it) ___

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Shane Hathaway
Antoine Pitrou wrote: I'd be happy to explain how ZODB solves those problems, if you're interested. Well, yes, I'm interested :) (I don't anything about Zope internals though, and I've never even used it) Ok. Quoting your list: To apply the same thing to Python you would at least need

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Bruce Eckel
//Theoretically// I suspect that the system /could/ perform as well as traditional approaches to dealing with concurrent problems single threaded (and multi-thread/process). I also think it's important to factor in the possibility of multiprocessors. If Kamaelia (for example) has a very safe

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Michael Sparks
On Friday 07 October 2005 23:26, Bruce Eckel wrote: I think the ease of use issue opens up far greater possibilities if you include multiprocessing ... That's what I'm looking for. In which case that's an area we need to push our work into sooner rather than later. After all, the PS3 and

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Michael Sparks
On Thursday 06 October 2005 21:06, Bruce Eckel wrote: So yes indeed, this is quite high on my list to research. Looks like people there have been doing some interesting work. Right now I'm just trying to cast a net, so that people can put in ideas, for when the Java book is done and I can

Re: [Python-Dev] __doc__ behavior in class definitions

2005-10-07 Thread Jason Orendorff
Martin, These two cases generate different bytecode. def foo(): # foo.func_code.co_flags == 0x43 print x# LOAD_FAST 0 x = 3 class Foo: # code object.co_flags == 0x40 print x# LOAD_NAME 'x' x = 3 In functions, local variables are just

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Nick Coghlan
Bruce Eckel wrote: I always have a problem with this. After many years of studying concurrency on-and-off, I continue to believe that threading is very difficult (indeed, the more I study it, the more difficult I understand it to be). And I admit this. The comments I sometimes get back are to

Re: [Python-Dev] Proposed changes to PEP 343

2005-10-07 Thread Nick Coghlan
Fredrik Lundh wrote: Nick Coghlan wrote: However, requiring a decorator to get a slot to work right looks pretty ugly to me, too. the whole concept might be perfectly fine on the this construct corre- sponds to this code level, but if you immediately end up with things that are not what

[Python-Dev] Sandboxed Threads in Python

2005-10-07 Thread Adam Olsen
Okay, basic principal first. You start with a sandboxed thread that has access to nothing. No modules, no builtins, *nothing*. This means it can run without the GIL but it can't do any work. To make it do something useful we need to give it two things: first, immutable types that can be safely

Re: [Python-Dev] Sandboxed Threads in Python

2005-10-07 Thread Phillip J. Eby
At 06:12 PM 10/7/2005 -0600, Adam Olsen wrote: Okay, basic principal first. You start with a sandboxed thread that has access to nothing. No modules, no builtins, *nothing*. This means it can run without the GIL but it can't do any work. It sure can't. You need at least the threadstate and a

Re: [Python-Dev] Sandboxed Threads in Python

2005-10-07 Thread Nick Coghlan
Phillip J. Eby wrote: Oh, and don't forget - newstyle classes keep weak references to all their subclasses, which means for example that every time you subclass 'dict', you're modifying the immutable 'dict' class. So, unless you recreate all the classes in each sandbox, you're back to

Re: [Python-Dev] Sandboxed Threads in Python

2005-10-07 Thread Adam Olsen
On 10/7/05, Phillip J. Eby [EMAIL PROTECTED] wrote: At 06:12 PM 10/7/2005 -0600, Adam Olsen wrote: Okay, basic principal first. You start with a sandboxed thread that has access to nothing. No modules, no builtins, *nothing*. This means it can run without the GIL but it can't do any work.

Re: [Python-Dev] PEP 342 suggestion: start(), __call__() and unwind_call() methods

2005-10-07 Thread Nick Coghlan
Phillip J. Eby wrote: At 09:50 PM 10/7/2005 +1000, Nick Coghlan wrote: Notice how a non-coroutine callable can be yielded, and it will still work happily with the scheduler, because the desire to continue execution is indicated by the ContinueIteration exception, rather than by the type

Re: [Python-Dev] Sandboxed Threads in Python

2005-10-07 Thread Josiah Carlson
Adam Olsen [EMAIL PROTECTED] wrote: I need to stress that *only* the new, immutable and thread-safe mark-and-sweep types would be affected by these changes. Everything else would continue to exist as it did before, and the benchmark exists to show they can coexist without killing

Re: [Python-Dev] Proposal for 2.5: Returning values from PEP 342 enhanced generators

2005-10-07 Thread James Y Knight
On Oct 3, 2005, at 1:53 AM, Piet Delport wrote: For generators written in this style, yield means suspend execution of the current call until the requested result/resource can be provided, and return regains its full conventional meaning of terminate the current call with a given result.

Re: [Python-Dev] Pythonic concurrency

2005-10-07 Thread Josiah Carlson
Michael Sparks [EMAIL PROTECTED] wrote: [ Possibly overlengthy reply. However given a multiple sets of cans of worms... ] On Friday 07 October 2005 07:25, Josiah Carlson wrote: One thing I notice is absent from the Kamaelia page is benchmarks. That's largely for one simple reason: we

Re: [Python-Dev] Sandboxed Threads in Python

2005-10-07 Thread Phillip J. Eby
At 07:17 PM 10/7/2005 -0600, Adam Olsen wrote: On 10/7/05, Phillip J. Eby [EMAIL PROTECTED] wrote: At 06:12 PM 10/7/2005 -0600, Adam Olsen wrote: Turns out it's quite easy and it doesn't harm performance of existing code or require modification (but a recompile is necessary). The idea is

Re: [Python-Dev] Sourceforge CVS access

2005-10-07 Thread Nick Coghlan
Guido van Rossum wrote: I will, if you tell me your sourceforge username. Sorry, forgot about that little detail ;) Anyway, its ncoghlan, same as the gmail account. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia