Re: [Python-Dev] Proposed addition to threading module - released
Martin v. Löwis wrote: > Nick Coghlan wrote: >> Do we want to add a "released" context manager to the threading module for >> 2.5? > > I don't think that should be added. I would consider it a dangerous > programming style: if the lock merely doesn't "need" to be held (i.e. > if it isn't necessary, but won't hurt), one should just keep holding > the lock. If it is essential to release the lock, because the code > would otherwise deadlock, the code should be dramatically revised > to avoid that situation, e.g. by redefining the granularity of the > lock, and moving the with statements accordingly. That isn't always possible or practical, though - Python's own GIL is a case where releasing it around long-running operations (such as blocking I/O operations) that don't need it provides significant benefits for threaded code, but redesigning the lock to use finer granularity causes its own problems. I'm not going to argue particularly strongly (or at all, really) for this one, since I think threading.Thread + Queue.Queue is a much better way to write threaded Python programs. The blocking IO 'asynchronize' use case in PEP 319 was just something I happened to notice in looking back at the various PEP's that were rejected in favour of PEP 343. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
On Mon, Apr 24, 2006 at 12:44:09PM +1000, Nick Coghlan wrote: > Paul Moore wrote: > > - Surely the __context__ method should be called __contextmgr__ now > > that it's producing a context manager? (Same naming issue, just the > > other side of it...) > > The __iter__ method isn't called __iterator__, so why would the __context__ > method need to be called "__contextmgr__"? It should be __ctxmgr__ to be in par with __iter__, __len__, dict and so on ;) Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Martin v. Löwis wrote: > - Paul Moore has contributed a Python build procedure for the > free version of the 2003 compiler. This one is without IDE, > but still, it should allow people without a VS 2003 license > to work on Python itself; it should also be possible to develop > extensions with that compiler (although I haven't verified > that distutils would pick that up correctly). Apparently, the status of this changed right now: it seems that the 2003 compiler is not available anymore; the page now says that it was replaced with the 2005 compiler. Should we reconsider? Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Buildbot messages and the build svn revision number
Would it be possible to get the buildbot error message subject lines to include the svn revision number of the build that failed? I committed some non-portable tests earlier today, but they should all be fixed in rev 45687. However, the various buildbots are still working through the different checkins that: a) included tests that weren't reliably correct on all platforms b) reverted the unreliable tests c) checked in more reliable tests that should work everywhere Until I open the message, click through to the details page and scroll down to the record of the svn update step, I don't actually know whether the buildbot is reporting a real problem or not. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Martin v. Löwis: > Apparently, the status of this changed right now: it seems that > the 2003 compiler is not available anymore; the page now says > that it was replaced with the 2005 compiler. > > Should we reconsider? I expect Microsoft means that Visual Studio Express will be available free forever, not that you will always be able to download Visual Studio 2005 Express. They normally only provide a particular product version for a limited time after it has been superceded. Neil ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
On 4/24/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: [...] > The __iter__ method isn't called __iterator__, so why would the __context__ > method need to be called "__contextmgr__"? Because. Hmm, Oleg already responded to this, and to be honest, I think the whole issue is a nitpick. Apologies for bringing it up, it doesn't help. > However, I'm definitely starting to agree with Just & Terry and whoever else > said it that the basic object needs to be called something other than "context > object". "context specifier" or something like that. (in fact, I might make a > pass through the PEP & documentation to see how that looks. . .) [...] > > (Oh, and by the way, writing this sentence > > reinforced the reason I perfer the old terminology - "context manager" > > feels like the name of something more concrete than "context", and yet > > under your new terminology, the "context" is more concrete than the > > "context manager"!) > > This is why I'm starting to think Terry and Just are right - the name of that > type needs to be something other than "context object" in order to reduce the > current ambiguity. Right. I'll still do as I promised, and have a better look through the latest documentation, but my gut feel is that this whole thing is getting way out of proportion. Naming and terminology is important, but we've now on our 3rd version of the docuentation. My instinct is STILL that the original version of the docs were fine. I can't express this in analytical terms, for which I apologise, but I'm not sure it's an analytical issue. So if the following sounds wooly, it's the best I can do :-) The terms "context" (an abstract thingy relating to the environment in which a block of code executes) and "context manager" (a concrete object that manages contexts) seem natural to me. The fact that you pass "context managers" to the "with" statement emphasizes the idea that they are the concrete one of the pair. Your version swaps the terms around, and so messes up my intuition. I concede that your version is self-consistent, but it doesn't match my intuition. And in my view, intuition matters here, so I have to prefer the variant that matches my intuition. Your point about the objects in the stdlib doesn't persuade me, I'm afraid. All the threading examples have __context__ return self, so they are, in some sense, both contexts and context managers (contexts which are their own managers, if you like). Only decimal.Context is a genuine context manager which creates independent context objects - I concede that the name doesn't match my theory here, but I'm afraid *I don't care*. I emphasize this, because it's an intuition thing again - I know my terminology is (slightly) inconsistent with respect to the decimal.Context class, but it doesn't disrupt my understanding. Worse (in terms of explaining myself!) it still feels natural to stick with decimal.Context even though I *know* that in my terms, it is formally a context manager. (The best explanation I can give is that it sits happily alongside the other places where I don't mind ignoring the difference between "function returning X" and "X" when describing things). OK, I really can't explain that any better, but the library object naming issue doesn't sway me. Right, this is my final position: I'm happy with the alpha-1 terminology. I would strongly prefer that we go back to that. I'd like to see the @contextmanager decorator split (conceptually, if not actually) into 2 parts, one for use with generator functions which are to be context managers, and one for use with __context__ functions (generator functions which are to be functions returning contexts). I'm willing to write up a patch doing the code changes for this, but I don't hold out much hope of being able to write docs any better than you or anyone else can! I won't argue against any self-consistent terminology. But that's because I'm not convinced of the value of having another round of change/review on the docs - it doesn't mean I support it over the alpha-1 version (and I'll probably still think in alpha-1 terms, whatever ends up being agreed). I think I've now read enough on the subject that my value as an unbiased reader is being lost... Paul. PS Is anyone else arguing with Nick on this? Now that I'm reduced to intuitive "well, I prefer it" arguments, it would be nice to hear what people's gut feelings are - "contexts create context managers", "context managers create contexts", "so confused I don't know"... :-) ? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (wasr45544 -peps/trunk/pep-0343.txt)
On 4/24/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > I'm going to try a pass through the docs using "context specifier", which > gives three separate terms: [...] > This removes the ambiguity between "context object" and "runtime context". That might just work. At the very least, I'd much rather see this as the alpha-2 alternative than what's there at the moment. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
On 4/24/06, Neil Hodgson <[EMAIL PROTECTED]> wrote: > Martin v. Löwis: > > > Apparently, the status of this changed right now: it seems that > > the 2003 compiler is not available anymore; the page now says > > that it was replaced with the 2005 compiler. > > > > Should we reconsider? > >I expect Microsoft means that Visual Studio Express will be > available free forever, not that you will always be able to download > Visual Studio 2005 Express. They normally only provide a particular > product version for a limited time after it has been superceded. No. Martin means that http://msdn.microsoft.com/visualc/vctoolkit2003/ no longer points to a downloadable version of MSVC which includes the optimizer, and generates VC 7.1 compatible binaries. This means that unless you've already downloaded it, or it's acceptable for someone else to host it, there's once again no way to build Python with free tools :-( (Is it worth the PSF asking MS if it's acceptable for python.org to host a copy of the toolkit compiler? As MS donated copies of MSVC 7.1 to the Python project, they may be willing to consider this...) Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 8 pylintrc?
On Sunday 23 April à 14:59, [EMAIL PROTECTED] wrote: > I had the possibly stupid idea today of running the stdlib through pylint. > Has anybody written a pylintrc file that attempts to reflect the > recommendations of PEP 8 the extent possible? default pylint configuration should not be that far from PEP8. I would be actually interested about what you think is not conformant, or even which test are missing for a better "PEP8 compliance test". -- Sylvain Thénault LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
Paul Moore wrote:
> Right. I'll still do as I promised, and have a better look through the
> latest documentation, but my gut feel is that this whole thing is
> getting way out of proportion. Naming and terminology is important,
> but we've now on our 3rd version of the docuentation.
Only the 2nd, really - prior to the work Phillip put into it, we didn't have
any user-oriented documentation at all. (the changes I made really were just
moving a bit of terminology).
I'm not counting the 'context object' -> 'context specifier' change as a
different version, as it was the process of updating the docs to match the
implementation that made me realise the term 'context object' was too
imprecise (the various comments here naturally helped in coming to that
understanding).
> My instinct is STILL that the original version of the docs were fine.
The original version of the docs moved the problem around, but it didn't solve
it - the fundamental issue was that we had two names ("context manager" and
"context"), but three entities (the original object with the __context__
method, the object it returned with __enter__ and __exit__ methods, and the
'runtime context' that the __enter__ and __exit__ methods manipulated).
The latest version of the docs uses "context specifier", "context manager" and
"context" for the three different things, and that seems to work reasonalbly
well.
> The terms "context" (an abstract thingy relating to the environment in
> which a block of code executes) and "context manager" (a concrete
> object that manages contexts) seem natural to me. The fact that you
> pass "context managers" to the "with" statement emphasizes the idea
> that they are the concrete one of the pair.
Agreed. In the latest version of the docs, you pass in either a context
specifier or a context manager to the with statement. A specifier just
provides a __context__ method, a manager provides __enter__ and __exit__ as
well.
> Your version swaps the terms around, and so messes up my intuition. I
> concede that your version is self-consistent, but it doesn't match my
> intuition. And in my view, intuition matters here, so I have to prefer
> the variant that matches my intuition.
Also agreed - the term 'context' needs to be reserved for the 'runtime
context', so I don't think we can use it for *any* of the concrete objects
involved.
I guess the python-dev discussion way back in the dim dark mists of time that
rejected "context" as the name for the pre-__context__ method context managers
was onto something :)
> I know my terminology is (slightly) inconsistent with respect to the
> decimal.Context class, but it doesn't disrupt my understanding. Worse
> (in terms of explaining myself!) it still feels natural to stick with
> decimal.Context even though I *know* that in my terms, it is formally
> a context manager.
With "context" no longer referring to any particular kind of object, this
naming issue goes away. The decimal.Context object is just a context specifier
for the active decimal context.
> OK, I really can't explain that any better, but the library object
> naming issue doesn't sway me.
It swayed me, but into (sort of) agreeing with you :)
> I'd like to see the @contextmanager decorator split (conceptually, if
> not actually) into 2 parts, one for use with generator functions which
> are to be context managers, and one for use with __context__ functions
> (generator functions which are to be functions returning contexts).
I believe the current docs avoid the need for doing this, as:
- the with statement expects a context specifier
- a context specifier's __context__ method must return a context manager
- all context managers can be used as context specifiers
So creating a standalone context manager properly parallels the creation of a
standalone iterator and creating a special purpose context manager as a
context specifier's __context__ method properly parallels creation of a
special purpose iterator as an iterable's __iter__ method.
This problem with the @contextmanager decorator is what clued me in to the
fact that the alpha 1 documentation just moved the terminology problem around
rather than actually fixing it.
> I think I've now read enough on the subject that my value as an
> unbiased reader is being lost...
Your input really helped me figure out where the problem was, though. Trying
to describe 3 different things using only 2 distinct terms was a recipe for
confusion in anybody's book :)
> PS Is anyone else arguing with Nick on this? Now that I'm reduced to
> intuitive "well, I prefer it" arguments, it would be nice to hear what
> people's gut feelings are - "contexts create context managers",
> "context managers create contexts", "so confused I don't know"... :-)
> ?
With the current docs:
"context specifiers create context managers"
"context specifiers define a runtime context"
"context managers enter and exit a runtime context"
"context managers
Re: [Python-Dev] Tkinter lockups.
On 4/24/06, Jeff Epler <[EMAIL PROTECTED]> wrote:
I just read the manpage for Tk_Init(3) (fc4 package tk-8.4.9-3) and itdoes not say that Tk_Init() may only be called once. While this doesn'tmean Python shouldn't work around it, I think the behavior should be
considered a bug in Tk, not _tkinter.FWIW, the Tk_Init manpage says "the Tk interpreter should not already be loaded", and it then goes on to say that if Tk_Init fails to *initialize* the interpreter, an error is returned. So it's unclear to me whether Tk_Init really "loads the interpreter", and whether it unloads after an error occurs (apparently not, I'd say ;)
http://www.tcl.tk/man/tcl8.4/TkLib/Tk_Init.htm
However, on this system, I couldn't recreate the problem you reportedwith either the "using _tkinter directly" instructions, or using this"C" test program:#include #include <
tk.h>int main(void) {Tcl_Interp *trp;unsetenv("DISPLAY");trp = Tcl_CreateInterp();printf("%d\n", Tk_Init(trp));printf("%d\n", Tk_Init(trp));
return 0;}Yes, this C snippet locks up on my systems, just as the python snippet does.-- Thomas Wouters <[EMAIL PROTECTED]
>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
On 4/24/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > > I think I've now read enough on the subject that my value as an > > unbiased reader is being lost... > > Your input really helped me figure out where the problem was, though. Trying > to describe 3 different things using only 2 distinct terms was a recipe for > confusion in anybody's book :) Glad it was of some use :-) > > PS Is anyone else arguing with Nick on this? Now that I'm reduced to > > intuitive "well, I prefer it" arguments, it would be nice to hear what > > people's gut feelings are - "contexts create context managers", > > "context managers create contexts", "so confused I don't know"... :-) > > ? [...] > The changes I made this weekend are designed to give everything its own name, > leaving "context" free to refer to either the runtime context or a context > specifier or whatever else makes sense in context. OK. At this point, the discussion seems to have mutated from a "Phillip vs Nick" debate to a "Paul vs Nick" debate. I think I need to duck out now and we'll see if this version passes by general acclaim (or sheer exhaustion :-)) Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Google Summer of Code proposal: New class for work with binary trees AVL and RB as with the standard dictionary.
I would like to participate in Google Summer of Code. The idea consists in creation of a specialized class for work with binary trees AVL and RB. The part of ideas is taken from Parrot (Perl6) where for pair values the specialized type is stipulated. As advantages it is possible to note: * High speed. On simple types at quantity of elements up to 1 speed of work concedes to the standard dictionary of all in 2.7 times. * Safety of work in a multithread operating mode. * The method for frosts of object is stipulated. * The data storage is carried out in the sorted kind. * A high overall performance `item' and `iteritem' methods as it is not necessary to create new objects. * At lots of objects in a tree memory is used much less intensively. * Absence of collisions. As consequence, it is impossible to generate bad data for creation DoS of the attacks influencing the dictionary of transferred arguments. * For objects existence of a method __ hash __ is not necessary. * The same kind of the dictionary by means of the overloaded operations probably change of its behaviour so that it supported the multivariational representation based on stratification given and subsequent recoils. Additional requirements to key objects: * The opportunity of comparison of key objects function cmp is necessary. There are the reasons, braking development of this project: * Compulsion of heading of a module `gc' for each compound object (this structure is unessential to some objects, but updating `gc' the module is required). * Absence of standard object the pair, probably depends on the previous item. Lacks of a binary tree: * Average time of search for hash - O (1), and for trees - O (log2 N). * A lot of memory under a single element of a tree: (3*sizeof (void *) + sizeof (int))*2, one element is used rather - the pair, the second site of memory is allocated under node of a tree. In protection of object "pair": * The logic of methods of the given object noticeably is easier than tuple, that as a result can affect speed of work of the program in the best party. * Alignment on 8 bytes has no big sense at present architecture where in cache sample a minimum on 64 bytes is made. Use of type "pair" will give an appreciable prize at use of alignment in 4 bytes. Otherwise on 64-bit platforms it is much more favourable to use tuple object. The given project can demand processing of the module `gcmodule.c' and `tupleobject.c'. It is necessary to reduce the size of static objects, for this purpose the opportunity is necessary is transparent to pass objects not having direct support from the module `gcmodule.c'. Also it will be partially necessary to process the module `obmalloc.c' for more effective distribution of memory. I shall be glad to answer questions on this theme. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
On Mon, 24 Apr 2006, Paul Moore wrote: On 4/24/06, Neil Hodgson <[EMAIL PROTECTED]> wrote: Martin v. Löwis: Apparently, the status of this changed right now: it seems that the 2003 compiler is not available anymore; the page now says that it was replaced with the 2005 compiler. Should we reconsider? [...] No. Martin means that http://msdn.microsoft.com/visualc/vctoolkit2003/ no longer points to a downloadable version of MSVC which includes the optimizer, and generates VC 7.1 compatible binaries. This means that unless you've already downloaded it, or it's acceptable for someone else to host it, there's once again no way to build Python with free tools :-( [...] Actually, it's apparently still there, just at a different URL. Somebody posted the new URL on c.l.py a day or two back (Alex Martelli started the thread, IIRC). I'm off to the dentist, no time to Google for it! John___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Google Summer of Code proposal: New class for work with binary trees AVL and RB as with the standard dictionary.
I would like to participate in Google Summer of Code. The idea consists in creation of a specialized class for work with binary trees AVL and RB. The part of ideas is taken from Parrot (Perl6) where for pair values the specialized type is stipulated. As advantages it is possible to note: * High speed. On simple types at quantity of elements up to 1 speed of work concedes to the standard dictionary of all in 2.7 times. * Safety of work in a multithread operating mode. * The method for frosts of object is stipulated. * The data storage is carried out in the sorted kind. * A high overall performance `item' and `iteritem' methods as it is not necessary to create new objects. * At lots of objects in a tree memory is used much less intensively. * Absence of collisions. As consequence, it is impossible to generate bad data for creation DoS of the attacks influencing the dictionary of transferred arguments. * For objects existence of a method __ hash __ is not necessary. * The same kind of the dictionary by means of the overloaded operations probably change of its behaviour so that it supported the multivariational representation based on stratification given and subsequent recoils. Additional requirements to key objects: * The opportunity of comparison of key objects function cmp is necessary. There are the reasons, braking development of this project: * Compulsion of heading of a module `gc' for each compound object (this structure is unessential to some objects, but updating `gc' the module is required). * Absence of standard object the pair, probably depends on the previous item. Lacks of a binary tree: * Average time of search for hash - O (1), and for trees - O (log2 N). * A lot of memory under a single element of a tree: (3*sizeof (void *) + sizeof (int))*2, one element is used rather - the pair, the second site of memory is allocated under node of a tree. In protection of object "pair": * The logic of methods of the given object noticeably is easier than tuple, that as a result can affect speed of work of the program in the best party. * Alignment on 8 bytes has no big sense at present architecture where in cache sample a minimum on 64 bytes is made. Use of type "pair" will give an appreciable prize at use of alignment in 4 bytes. Otherwise on 64-bit platforms it is much more favourable to use tuple object. The given project can demand processing of the module `gcmodule.c' and `tupleobject.c'. It is necessary to reduce the size of static objects, for this purpose the opportunity is necessary is transparent to pass objects not having direct support from the module `gcmodule.c'. Also it will be partially necessary to process the module `obmalloc.c' for more effective distribution of memory. I shall be glad to answer questions on this theme. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] gettext.py bug #1448060
Hi there, while playing with gettext catalog handling of a complex application, I've hit bug #1448060. This bug seems serious to me since gettext.py is not able to handle some .po files generated by msgcat/msgmerge due to lines like "#-#-#-#-# fr.po (2.0) #-#-#-#-#\n" in the metadata. I've posted a patch (#1475523) for this and assigned it to Martin Von Loewis since he was the core developper who has made some followup on the original bug. Could someone (Martin or someone else) quick review this patch ? I really need a fix for this, so if anyone feels my patch is not correct, and explain why and what should be done, I can rework on it. regards, -- Sylvain Thénault LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python Grammar Ambiguity
Hello all, I'm working on a parser for part of the Python language (expressions but not statements basically). I'm using PLY to generate the parser and it's mostly done. I've hit on what looks like a fundamental ambiguity in the Python grammar which is difficult to get round with PLY; and I'm wondering *why* the grammar is defined in this way. It's possible there is a reason that I've missed, which means I need to rethink my workaround. List displays (list comprehensions) are defined as (from http://docs.python.org/ref/lists.html ) : test ::= and_test ( "or" and_test )* | lambda_form testlist ::= test ( "," test )* [ "," ] list_display ::= "[" [listmaker] "]" listmaker ::= expression ( list_for | ( "," expression )* [","] ) list_iter ::= list_for | list_if list_for ::= "for" expression_list "in" testlist [list_iter] list_if ::= "if" test [list_iter] The problem is that list_for is defined as : "for" expression_list "in" testlist This allows arbitrary expressions in the 'assignment' part of a list comprehension. As a result, the following is valid syntax according to the grammar : [x for x + 1 in y] Obviously it isn't valid ! This parses to an ast, but the syntax error is thrown when you compile the resulting ast. The problem is that for the basic case of a list comprehension ( ``[x for x in y]``), ``x in y`` is a valid expression. That makes it extremely hard to disambiguate the grammar so that the ``in`` is treated correctly, and not part of an expression. My question is, why are arbitrary expressions allowed here in the grammar ? As far as I can tell, only identifiers (nested in parentheses or brackets) are valid here. I've got round the problem by creating a new node 'identifier_list' and just having that return the expected syntax tree (actually an expression list). This gets round the ambiguity [#]_. It worries me that there might be a valid expression allowed here that I haven't thought of. My current rules allow anything that looks like ``(a, [b, c, (d, e)], f)`` - any nested identifier list. Would anything else be allowed ? If not, why not modify the grammar so that the compiler has less possible invalid syntax trees to work with ? (Also the grammar definition of string conversion is wrong as it states that a trailing comma is valid, which isn't the case. As far as I can tell that is necessary to allow nesting string conversions.) Fuzzyman http://www.voidspace.org.uk/python/index.shtml .. [#] If I could make precedence work in PLY I could also solve it I guess. However I can't. :-) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Grammar Ambiguity
Well, yes, the syntax is supposed to be something like "for varlist in testlist". Could you report this as a doc bug (if you found this information in the docs)? On 4/24/06, Michael Foord <[EMAIL PROTECTED]> wrote: > Hello all, > > I'm working on a parser for part of the Python language (expressions but > not statements basically). I'm using PLY to generate the parser and it's > mostly done. > > I've hit on what looks like a fundamental ambiguity in the Python > grammar which is difficult to get round with PLY; and I'm wondering > *why* the grammar is defined in this way. It's possible there is a > reason that I've missed, which means I need to rethink my workaround. > > List displays (list comprehensions) are defined as (from > http://docs.python.org/ref/lists.html ) : > > > test ::= and_test ( "or" and_test )* | lambda_form > testlist ::= test ( "," test )* [ "," ] > list_display ::= "[" [listmaker] "]" > listmaker ::= expression ( list_for | ( "," expression )* [","] ) > list_iter ::= list_for | list_if > list_for ::= "for" expression_list "in" testlist [list_iter] > list_if ::= "if" test [list_iter] > > The problem is that list_for is defined as : > > "for" expression_list "in" testlist > > This allows arbitrary expressions in the 'assignment' part of a list > comprehension. > > As a result, the following is valid syntax according to the grammar : > > [x for x + 1 in y] > > Obviously it isn't valid ! This parses to an ast, but the syntax error > is thrown when you compile the resulting ast. > > The problem is that for the basic case of a list comprehension ( ``[x > for x in y]``), ``x in y`` is a valid expression. That makes it > extremely hard to disambiguate the grammar so that the ``in`` is treated > correctly, and not part of an expression. > > My question is, why are arbitrary expressions allowed here in the > grammar ? As far as I can tell, only identifiers (nested in parentheses > or brackets) are valid here. I've got round the problem by creating a > new node 'identifier_list' and just having that return the expected > syntax tree (actually an expression list). This gets round the ambiguity > [#]_. > > It worries me that there might be a valid expression allowed here that I > haven't thought of. My current rules allow anything that looks like > ``(a, [b, c, (d, e)], f)`` - any nested identifier list. Would anything > else be allowed ? > > If not, why not modify the grammar so that the compiler has less > possible invalid syntax trees to work with ? > > (Also the grammar definition of string conversion is wrong as it states > that a trailing comma is valid, which isn't the case. As far as I can > tell that is necessary to allow nesting string conversions.) > > Fuzzyman > http://www.voidspace.org.uk/python/index.shtml > > > .. [#] If I could make precedence work in PLY I could also solve it I > guess. However I can't. :-) > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
At 10:26 AM 4/24/2006 +0100, Paul Moore wrote: >OK. At this point, the discussion seems to have mutated from a >"Phillip vs Nick" debate to a "Paul vs Nick" debate. I only stepped aside so that other people would chime in. I still don't think the new terminology makes anything clearer, and would rather see tweaks to address the one minor issue of @contextmanager producing an object that's also a context than a complete reworking of the documentation. That was the only thing that was unclear in the a1 terminology and docs, and it's an extremely minor point that could easily be addressed. Throwing away an intuitive terminology because of a minor implementation issue in favor of a non-intutitive terminology that happens to be super-precise seems penny-wise and pound-foolish to me. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
On Apr 24, 2006, at 5:13 AM, John J Lee wrote: > On Mon, 24 Apr 2006, Paul Moore wrote: >> On 4/24/06, Neil Hodgson <[EMAIL PROTECTED]> wrote: >>> Martin v. Löwis: >>> Apparently, the status of this changed right now: it seems that the 2003 compiler is not available anymore; the page now says that it was replaced with the 2005 compiler. Should we reconsider? > [...] >> No. Martin means that http://msdn.microsoft.com/visualc/ >> vctoolkit2003/ >> no longer points to a downloadable version of MSVC which includes the >> optimizer, and generates VC 7.1 compatible binaries. >> >> This means that unless you've already downloaded it, or it's >> acceptable for someone else to host it, there's once again no way to >> build Python with free tools :-( > [...] > > Actually, it's apparently still there, just at a different URL. > Somebody posted the new URL on c.l.py a day or two back (Alex > Martelli started the thread, IIRC). I'm off to the dentist, no > time to Google for it! Yep, I was the one looking for that URL, and then at somebody else's request reposted it and also tinyurled it (since it's a very long URL it gives somebody problems). For the Toolkit 2003: http://tinyurl.com/gv8wr Also, for the Net SDK 1.1 (the 2.0 one apparently now is lacking the msvcrt.lib for x86...): http://tinyurl.com/5flob (original Url for the latter kindly supplied by Martin, btw). Martin also suggested using mingw instead, on that same thread. Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
On Apr 24, 2006, at 12:19 AM, Martin v. Löwis wrote: > Martin v. Löwis wrote: >> - Paul Moore has contributed a Python build procedure for the >> free version of the 2003 compiler. This one is without IDE, >> but still, it should allow people without a VS 2003 license >> to work on Python itself; it should also be possible to develop >> extensions with that compiler (although I haven't verified >> that distutils would pick that up correctly). > > Apparently, the status of this changed right now: it seems that > the 2003 compiler is not available anymore; the page now says > that it was replaced with the 2005 compiler. > > Should we reconsider? Personally, being a cheapskate, and with Windows only my tertiary system, I'm in favor of anything that makes it simpler and/or cheaper for people to work on Python and extensions. However, by the same token I cannot really gauge how stable and solid VS 2005 is -- if as current Windows experts you think it's still inferior to VS 2003, then that's a very big point against it. Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
On Apr 24, 2006, at 12:48 AM, Neil Hodgson wrote: > Martin v. Löwis: > >> Apparently, the status of this changed right now: it seems that >> the 2003 compiler is not available anymore; the page now says >> that it was replaced with the 2005 compiler. >> >> Should we reconsider? > >I expect Microsoft means that Visual Studio Express will be > available free forever, not that you will always be able to download > Visual Studio 2005 Express. They normally only provide a particular > product version for a limited time after it has been superceded. Yeah, that does sound like a normal commercial policy. If we want some compiler to be available "forever", we have to choose one that we can get permission to redistribute, and host it somewhere ourselves. Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
On Apr 24, 2006, at 1:24 AM, Paul Moore wrote: > On 4/24/06, Neil Hodgson <[EMAIL PROTECTED]> wrote: >> Martin v. Löwis: >> >>> Apparently, the status of this changed right now: it seems that >>> the 2003 compiler is not available anymore; the page now says >>> that it was replaced with the 2005 compiler. >>> >>> Should we reconsider? >> >>I expect Microsoft means that Visual Studio Express will be >> available free forever, not that you will always be able to download >> Visual Studio 2005 Express. They normally only provide a particular >> product version for a limited time after it has been superceded. > > No. Martin means that http://msdn.microsoft.com/visualc/vctoolkit2003/ > no longer points to a downloadable version of MSVC which includes the > optimizer, and generates VC 7.1 compatible binaries. > > This means that unless you've already downloaded it, or it's > acceptable for someone else to host it, there's once again no way to > build Python with free tools :-( I've posted a couple of tinyurl's that may help (no time right now to try everything out again myself). > > (Is it worth the PSF asking MS if it's acceptable for python.org to > host a copy of the toolkit compiler? As MS donated copies of MSVC 7.1 > to the Python project, they may be willing to consider this...) Small as the chance may be, it's still most definitely worth asking, so that everything doesn't suddenly break the instant MS wants to withdraw the URLs that currently still work. Also, we'd need the ability to redistribute the 1.1 SDK, as the 2.0 one seems to lack the key msvcrt.lib for x86. Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] SoC proposal: "fix some old, old bugs in sourceforge"
Hi all, I would like to participate in the Summer of Code as a student. At the moment it looks like the Python tracker on SF has about 2100 open bugs and patches, going back to late 2000. I'm assuming that a fair number of these are no longer be applicable, have been fixed/implemented already, etc., and somebody just needs to slog through the list and figure out what to do with them. My unglamorous proposal is to review bugs & patches (starting with the oldest) and resolve at least 200 of them. Is that too much? Too few? I'll fix as many as possible during the SoC time frame, but I wanted to set a realistically achievable minimum for the proposal. If anybody can offer helpful feedback on a good minimum number I'd appreciate it. Not-guru-ish-enough-to-found-a-new-web-framework'ly yours, Alan McIntyre ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
On 4/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 10:26 AM 4/24/2006 +0100, Paul Moore wrote: > >OK. At this point, the discussion seems to have mutated from a > >"Phillip vs Nick" debate to a "Paul vs Nick" debate. > > I only stepped aside so that other people would chime in. I still don't > think the new terminology makes anything clearer, and would rather see > tweaks to address the one minor issue of @contextmanager producing an > object that's also a context than a complete reworking of the > documentation. That was the only thing that was unclear in the a1 > terminology and docs, and it's an extremely minor point that could easily > be addressed. > > Throwing away an intuitive terminology because of a minor implementation > issue in favor of a non-intutitive terminology that happens to be > super-precise seems penny-wise and pound-foolish to me. That's *exactly* my feeling (thanks for expressing it better than I did). So I guess the 2 questions remaining are: 1. Does anyone feel that Nick's (re-)wording is better than the a1 version (apart from the @contextmanager issue, which we all agree needs a fix)? 2. Nick, what can we do to persuade you to go back to the a1 version, and simply look at @contextmanager? I've proposed splitting it into two, but that seems not to suit you (you've never responded to it specifically, so I may be misreading your silence here). As an alternative, I could just think about how to reword the docs. But I'm not confident I'm the best person to do that. One thing I would say is that Nick has added a section on context types to the "Built-in types" section of the libref. At the moment, it reflects his terminology, but even if the terminology gets reverted to a1 style, I'd like to see that stay (suitably reworded, of course!) It's a useful addition. Oh, and if there's a huge group of people who prefer Nick's terminology, now is the time to shout! Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 8 pylintrc?
>> Has anybody written a pylintrc file that attempts to reflect the >> recommendations of PEP 8 the extent possible? Sylvain> I would be actually interested about what you think is not Sylvain> conformant, or even which test are missing for a better "PEP8 Sylvain> compliance test". One concrete (but trivial) difference is that PyLint wants line length to max out at 80 columns while PEP 8 says 79. Since PEP 8 doesn't specify everything, there's going to be some fudging required. For things which are unspecified by PEP 8, I think you have to consider the body of code (the standard library) and the probably audience (people on this list) to consider how to set things. PyLint complains about the use of apply(), but it also calls the use of *args and **kwds "magic". The first complaint is correct in my opinion since apply() is deprecated, but the use of *args and **kwds is definitely not magic for this group. Skip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Grammar Ambiguity
Michael> I've hit on what looks like a fundamental ambiguity in the Michael> Python grammar which is difficult to get round with PLY; and Michael> I'm wondering *why* the grammar is defined in this way. Michael, You refer to the ref manual documentation: Michael> List displays (list comprehensions) are defined as (from Michael> http://docs.python.org/ref/lists.html ) Note that the BNF there is mostly designed for human consumption. Have you verified that the ambiguity is also present in the Grammar file? Skip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Mono-dev] IronPython Performance
Obviously all three might add to that. However I think that number 1 COULD already be enough for the noticed slowdown of 3 times. It depends a lot on which Classes/ Methods IronPython is using (And I guess it is now optimized towards the MS CLR). If you just have pure integer or floating point math in simple loops the difference between the CLR and Mono istn't very big (some percent usually), but as soon as you use other classes (even from the very core of the BCL) you will often see huge differences. It seems that the current goal of mono is to get everything running (In a way as simple as possible) instead of trying to optimize performance (e.g. I posted a patch some time ago to improve performance of common string operations of up to 100% which was turned down because it would have added about 100 LOC). Or another example: if your benchmark writes numbers to files or the console the culpit might be NumberFormatter, which itself is usually more than 5 times slower than on the CLR. But there are several others which could be the cause of that. By the way: Which optimizations did you enable when running IronPython on Mono? These also could potentially make a big speed difference. Also I guess number 2 could also make some difference in speed. I am aware of the following issues that might affect the results: 1) Mono vs. Microsoft's CLR. 2) Python 2.1 vs. Python 2.4 3) Changes in IronPython over the last two years. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python Grammar Ambiguity
Hello all, I'm working on a parser for part of the Python language (expressions but not statements basically). I'm using PLY to generate the parser and it's mostly done. I've hit on what looks like a fundamental ambiguity in the Python grammar which is difficult to get round with PLY; and I'm wondering *why* the grammar is defined in this way. It's possible there is a reason that I've missed, which means I need to rethink my workaround. List displays (list comprehensions) are defined as (from http://docs.python.org/ref/lists.html ) : test ::= and_test ( "or" and_test )* | lambda_form testlist ::= test ( "," test )* [ "," ] list_display ::= "[" [listmaker] "]" listmaker ::= expression ( list_for | ( "," expression )* [","] ) list_iter ::= list_for | list_if list_for ::= "for" expression_list "in" testlist [list_iter] list_if ::= "if" test [list_iter] The problem is that list_for is defined as : "for" expression_list "in" testlist This allows arbitrary expressions in the 'assignment' part of a list comprehension. As a result, the following is valid syntax according to the grammar : [x for x + 1 in y] Obviously it isn't valid ! This parses to an ast, but the syntax error is thrown when you compile the resulting ast. The problem is that for the basic case of a list comprehension ( ``[x for x in y]``), ``x in y`` is a valid expression. That makes it extremely hard to disambiguate the grammar so that the ``in`` is treated correctly, and not part of an expression. My question is, why are arbitrary expressions allowed here in the grammar ? As far as I can tell, only identifiers (nested in parentheses or brackets) are valid here. I've got round the problem by creating a new node 'identifier_list' and just having that return the expected syntax tree (actually an expression list). This gets round the ambiguity [#]_. It worries me that there might be a valid expression allowed here that I haven't thought of. My current rules allow anything that looks like ``(a, [b, c, (d, e)], f)`` - any nested identifier list. Would anything else be allowed ? If not, why not modify the grammar so that the compiler has less possible invalid syntax trees to work with ? (Also the grammar definition of string conversion is wrong as it states that a trailing comma is valid, which isn't the case. As far as I can tell that is necessary to allow nesting string conversions.) Fuzzyman http://www.voidspace.org.uk/python/index.shtml .. [#] If I could make precedence work in PLY I could also solve it I guess. However I can't. :-) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [IronPython] [Mono-dev] IronPython Performance
On the recursion limits: Until beta 6 IronPython didn't have proper support for limiting recursion depth. There was some minor support there, but it wasn't right. In beta 6 we have full support for limiting recursion depth, but by default we allow infinite recursion. If the user explicitly sets the recursion limit then we'll go ahead and enforce it. But all the reasons you outline below are great explanations of the differences. Do you want to help develop Dynamic languages on CLR? (http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Paolo Molaro Sent: Monday, April 24, 2006 2:53 AM To: Brent Fulgham Cc: [EMAIL PROTECTED]; [email protected]; [email protected]; [email protected] Subject: Re: [IronPython] [Mono-dev] IronPython Performance [I continued your large cross-post, even if I'm not subscribed to python-dev: hopefully the moderators will approve the post if needed.] On 04/21/06 Brent Fulgham wrote: > A while ago (nearly two years) an interesting paper was published by Jim > Hugunin > (http://www.python.org/pycon/dc2004/papers/9/) crowing about the significant > speed advantage observed in this interpreter running on top of Microsoft's > .NET > VM stack. I remember being surprised by these results, as Python has always > seemed fairly fast for an interpreted language. > > I've been working on the Programming Language Shootout for a few years now, > and > after growing tired of the repeated requests to include scores for IronPython > I > finally added the implementation this week. > > Comparison of IronPython (1.0 Beta 5) to Python 2.4.3 [1] and IronPython (1.0 > Beta 1) > to Python 2.4.2 [2] do not match the results reported in the 2004 paper. In > fact, > IronPython is consistenly 3 to 4 times slower (in some cases worse than > that), and > seemed to suffer from recursion/stack limitations. You're comparing different benchmarks, so it's not a surprise that you get different numbers. I only tried two benchmarks from the paper, pystone and the globals one (attached) and the results are largely equivalent to the ones from the paper: time mono IronPython-1.0-Beta6/IronPythonConsole.exe -O python-globals-bench2.py real 0m0.861s user 0m0.815s sys 0m0.043s (Note the above includes the startup time that I guess was not considered in the paper and is 0.620 seconds running test(1): just increasing the number of iterations will make the difference converge at about 10x faster mono vs python2.4). time python2.4 -O python-globals-bench2.py real 0m2.239s user 0m2.202s sys 0m0.025s python2.4 -O /usr/lib/python2.4/test/pystone.py Pystone(1.1) time for 5 passes = 1.4 This machine benchmarks at 35714.3 pystones/second mono IronPython-1.0-Beta6/IronPythonConsole.exe -O /usr/lib/python2.4/test/pystone.py Pystone(1.1) time for 5 passes = 0.989288 This machine benchmarks at 50541.4 pystones/second So IronPython is 40+% faster than CPython, like in the paper. The mono version I used is roughtly 1.1.14. As for the recursion limitations: what are they? I took the recursive benchmark from the shootout page and both CPython and IronPython need the call to setrecursionlimit (actually cpython needed it earlier for me, anyway). This might be a difference between IronPython beta 5 and the beta6 I'm using, though. It is also interesting because on my system running the attached python-ack.py with 8 as argument, mono/IronPython is 10% faster than CPython, while the shootout page reports CPython 2.4 50% faster. Feel free to investigate if this is because of improvements in mono or IronPython:) > I am aware of the following issues that might affect the results: > 1) Mono vs. Microsoft's CLR. This can certainly be a factor: some things in the MS runtime run faster and some slower than in the Mono runtime, so it depends which features are exercised the most in a particular benchmark. It is very likely that in most benchmarks the MS runtime will score better. > 2) Python 2.1 vs. Python 2.4 The python folks will have more details, but python2.4 seems to be faster than 2.1 and 2.3 in several areas. > 3) Changes in IronPython over the last two years. For some time the IronPython folks have focused on features and correctness issues, so I guess they'll start caring about performance in the future. It is also to note that IronPython now uses generics which have yet to see optimizations work in the mono runtime and libraries (they are a 2.0 feature). > I thought these findings might be interesting to some of the Python, > IronPython, > and Mono developers. Let the flames begin! ;-) Why flames? Hard numbers are just that: useful information. It's just your conclusions that might be incorrect;-) I think the summary to take away is: *) Sometimes IronPython on mono is faster than CPython *) Often CPython is faster than IronPyt
Re: [Python-Dev] Python Grammar Ambiguity
(oops - should have gone to list) Guido van Rossum wrote: > Well, yes, the syntax is supposed to be something like "for varlist in > testlist". Could you report this as a doc bug (if you found this > information in the docs)? > I think the documentation (which does put expression_list there) reflects the current state of the parser. First of all the grammar in SVN also has expression_list there *and* the following does successfully parse to an ast (but fails when you compile the ast) : [x for x + 1 in y] All the best, Michael Foord > On 4/24/06, Michael Foord <[EMAIL PROTECTED]> wrote: > >> Hello all, >> >> I'm working on a parser for part of the Python language (expressions but >> not statements basically). I'm using PLY to generate the parser and it's >> mostly done. >> >> I've hit on what looks like a fundamental ambiguity in the Python >> grammar which is difficult to get round with PLY; and I'm wondering >> *why* the grammar is defined in this way. It's possible there is a >> reason that I've missed, which means I need to rethink my workaround. >> >> List displays (list comprehensions) are defined as (from >> http://docs.python.org/ref/lists.html ) : >> >> >> test ::= and_test ( "or" and_test )* | lambda_form >> testlist ::= test ( "," test )* [ "," ] >> list_display ::= "[" [listmaker] "]" >> listmaker ::= expression ( list_for | ( "," expression )* [","] ) >> list_iter ::= list_for | list_if >> list_for ::= "for" expression_list "in" testlist [list_iter] >> list_if ::= "if" test [list_iter] >> >> The problem is that list_for is defined as : >> >> "for" expression_list "in" testlist >> >> This allows arbitrary expressions in the 'assignment' part of a list >> comprehension. >> >> As a result, the following is valid syntax according to the grammar : >> >> [x for x + 1 in y] >> >> Obviously it isn't valid ! This parses to an ast, but the syntax error >> is thrown when you compile the resulting ast. >> >> The problem is that for the basic case of a list comprehension ( ``[x >> for x in y]``), ``x in y`` is a valid expression. That makes it >> extremely hard to disambiguate the grammar so that the ``in`` is treated >> correctly, and not part of an expression. >> >> My question is, why are arbitrary expressions allowed here in the >> grammar ? As far as I can tell, only identifiers (nested in parentheses >> or brackets) are valid here. I've got round the problem by creating a >> new node 'identifier_list' and just having that return the expected >> syntax tree (actually an expression list). This gets round the ambiguity >> [#]_. >> >> It worries me that there might be a valid expression allowed here that I >> haven't thought of. My current rules allow anything that looks like >> ``(a, [b, c, (d, e)], f)`` - any nested identifier list. Would anything >> else be allowed ? >> >> If not, why not modify the grammar so that the compiler has less >> possible invalid syntax trees to work with ? >> >> (Also the grammar definition of string conversion is wrong as it states >> that a trailing comma is valid, which isn't the case. As far as I can >> tell that is necessary to allow nesting string conversions.) >> >> Fuzzyman >> http://www.voidspace.org.uk/python/index.shtml >> >> >> .. [#] If I could make precedence work in PLY I could also solve it I >> guess. However I can't. :-) >> >> ___ >> Python-Dev mailing list >> [email protected] >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> >> > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Grammar Ambiguity
[EMAIL PROTECTED] wrote: > Michael> I've hit on what looks like a fundamental ambiguity in the > Michael> Python grammar which is difficult to get round with PLY; and > Michael> I'm wondering *why* the grammar is defined in this way. > > Michael, > > You refer to the ref manual documentation: > > Michael> List displays (list comprehensions) are defined as (from > Michael> http://docs.python.org/ref/lists.html ) > > Note that the BNF there is mostly designed for human consumption. Have you > verified that the ambiguity is also present in the Grammar file? > From : http://svn.python.org/view/python/tags/r243/Grammar/Grammar?rev=43414&view=auto list_for: 'for' exprlist 'in' testlist_safe [list_iter] So in the Python grammar list_for *is* defined as an expression list. That follows, because using the parser module I can create an ast for a list comprehension like the following : import parser expr = '[1 for 1 in n]\n' ast = parser.expr(expr) print parser.compileast(ast) Traceback (most recent call last): File "ast_example.py", line 6, in ? print parser.compileast(ast) SyntaxError: can't assign to literal The syntax error is thrown at the compile stage, not the parse stage. Having list_for being defined in terms of something like varlist makes more sense, but isn't how the grammar is done currently. Michael Foord > Skip > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Grammar Ambiguity
This is probably because we have a similar ambiguity in assignments:
the grammar says something like
exprlist ('=' exprlist)*
but what is actually desired is
(varlist '=')* exprlist
Unfortunately the latter is not LL1 so we lie to the parser and tell
it the first form, and then in the code generator checks that the
actual parse tree for all exprlists except for the last conforms to
the more limited syntax for varlist (which is only in our head). If
not, it issues a "phase 2" SyntaxError. (One that doesn't have the
exact column information.)
We didn't have to do this for the syntax of the for-statement,
list-comprehensions etc. because the ambiguity doesn't exist there
(the 'for' keyword disambiguates the situation); but the current
approach allows more code sharing in the code generator.
I suggest you go ahead and write the second form for your own parser.
Coming up with the correct rules for varlist is not hard.
--Guido
On 4/24/06, Michael Foord <[EMAIL PROTECTED]> wrote:
> (oops - should have gone to list)
>
> Guido van Rossum wrote:
> > Well, yes, the syntax is supposed to be something like "for varlist in
> > testlist". Could you report this as a doc bug (if you found this
> > information in the docs)?
> >
>
> I think the documentation (which does put expression_list there)
> reflects the current state of the parser.
>
> First of all the grammar in SVN also has expression_list there *and* the
> following does successfully parse to an ast (but fails when you compile
> the ast) :
>
> [x for x + 1 in y]
>
> All the best,
>
>
> Michael Foord
>
>
> > On 4/24/06, Michael Foord <[EMAIL PROTECTED]> wrote:
> >
> >> Hello all,
> >>
> >> I'm working on a parser for part of the Python language (expressions but
> >> not statements basically). I'm using PLY to generate the parser and it's
> >> mostly done.
> >>
> >> I've hit on what looks like a fundamental ambiguity in the Python
> >> grammar which is difficult to get round with PLY; and I'm wondering
> >> *why* the grammar is defined in this way. It's possible there is a
> >> reason that I've missed, which means I need to rethink my workaround.
> >>
> >> List displays (list comprehensions) are defined as (from
> >> http://docs.python.org/ref/lists.html ) :
> >>
> >>
> >> test ::= and_test ( "or" and_test )* | lambda_form
> >> testlist ::= test ( "," test )* [ "," ]
> >> list_display ::= "[" [listmaker] "]"
> >> listmaker ::= expression ( list_for | ( "," expression )* [","] )
> >> list_iter ::= list_for | list_if
> >> list_for ::= "for" expression_list "in" testlist [list_iter]
> >> list_if ::= "if" test [list_iter]
> >>
> >> The problem is that list_for is defined as :
> >>
> >> "for" expression_list "in" testlist
> >>
> >> This allows arbitrary expressions in the 'assignment' part of a list
> >> comprehension.
> >>
> >> As a result, the following is valid syntax according to the grammar :
> >>
> >> [x for x + 1 in y]
> >>
> >> Obviously it isn't valid ! This parses to an ast, but the syntax error
> >> is thrown when you compile the resulting ast.
> >>
> >> The problem is that for the basic case of a list comprehension ( ``[x
> >> for x in y]``), ``x in y`` is a valid expression. That makes it
> >> extremely hard to disambiguate the grammar so that the ``in`` is treated
> >> correctly, and not part of an expression.
> >>
> >> My question is, why are arbitrary expressions allowed here in the
> >> grammar ? As far as I can tell, only identifiers (nested in parentheses
> >> or brackets) are valid here. I've got round the problem by creating a
> >> new node 'identifier_list' and just having that return the expected
> >> syntax tree (actually an expression list). This gets round the ambiguity
> >> [#]_.
> >>
> >> It worries me that there might be a valid expression allowed here that I
> >> haven't thought of. My current rules allow anything that looks like
> >> ``(a, [b, c, (d, e)], f)`` - any nested identifier list. Would anything
> >> else be allowed ?
> >>
> >> If not, why not modify the grammar so that the compiler has less
> >> possible invalid syntax trees to work with ?
> >>
> >> (Also the grammar definition of string conversion is wrong as it states
> >> that a trailing comma is valid, which isn't the case. As far as I can
> >> tell that is necessary to allow nesting string conversions.)
> >>
> >> Fuzzyman
> >> http://www.voidspace.org.uk/python/index.shtml
> >>
> >>
> >> .. [#] If I could make precedence work in PLY I could also solve it I
> >> guess. However I can't. :-)
> >>
> >> ___
> >> Python-Dev mailing list
> >> [email protected]
> >> http://mail.python.org/mailman/listinfo/python-dev
> >> Unsubscribe:
> >> http://mail.python.org/mailman/options/python-dev/guido%40python.org
> >>
> >>
> >
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
> >
> >
>
>
> _
Re: [Python-Dev] SoC proposal: "fix some old, old bugs in sourceforge"
On Mon, Apr 24, 2006 at 12:30:12PM -0400, Alan McIntyre wrote: > My unglamorous proposal is to review bugs & patches (starting with the > oldest) and resolve at least 200 of them. Is that too much? Too few? > I'll fix as many as possible during the SoC time frame, but I wanted to > set a realistically achievable minimum for the proposal. If anybody can > offer helpful feedback on a good minimum number I'd appreciate it. I'd suggest 75 or maybe 100 bugs or patches, not 200. Let's assume you'll spend 60 days working on the project. While some items are trivial (documentation typos, small bugfixes), you'll probably exhaust those pretty quickly. If a bug has been sitting around in the bug tracker for 3 years, it's probably because the bug isn't trivial: it may break compatibility to fix it, or the fix requires significant redesign of a component. 200 bugs is a little over 3 per day. During bug days I've spent three or four hours working on a single bug, which implies two bugs/day. So if you're working 8-hour days for those two months, you might be able to process about 120 items. Therefore, 75 seems a reasonable number, a little more than a bug a day. --amk ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed addition to threading module - released
On 4/23/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Nick Coghlan wrote: > > Do we want to add a "released" context manager to the threading module for > > 2.5? > > I don't think that should be added. I would consider it a dangerous > programming style: if the lock merely doesn't "need" to be held (i.e. > if it isn't necessary, but won't hurt), one should just keep holding > the lock. If it is essential to release the lock, because the code > would otherwise deadlock, the code should be dramatically revised > to avoid that situation, e.g. by redefining the granularity of the > lock, and moving the with statements accordingly. Actually, what Nick describes is *exactly* how one should write code using a condition variable: LOCK while nothing to do: UNLOCK wait for the condition variable (or sleep, or whatever) LOCK # here we have something to do with the lock held remove the to-do item UNLOCK except that the outer LOCK/UNLOCK pair should be using a try/except and the inner UNLOCK/LOCK pair should too. I don't see how you can do this easily by rewriting the code; the rewrite would be considerably ugly (or requires a GOTO :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Google Summer of Code proposal: New class for work with binary trees AVL and RB as with the standard dictionary.
There exists various C and Python implementations of both AVL and Red-Black trees. For users of Python who want to use AVL and/or Red-Black trees, I would urge them to use the Python implementations. In the case of *needing* the speed of a C extension, there already exists a CPython extension module for AVL trees: http://www.python.org/pypi/pyavl/1.1 I would suggest you look through some suggested SoC projects in the wiki: http://wiki.python.org/moin/SummerOfCode - Josiah "Vladimir 'Yu' Stepanov" <[EMAIL PROTECTED]> wrote: > > I would like to participate in Google Summer of Code. The idea consists in > creation of a specialized class for work with binary trees AVL and RB. The > part of ideas is taken from Parrot (Perl6) where for pair values the > specialized type is stipulated. As advantages it is possible to note: > * High speed. On simple types at quantity of elements up to 1 speed of > work concedes to the standard dictionary of all in 2.7 times. > * Safety of work in a multithread operating mode. > * The method for frosts of object is stipulated. > * The data storage is carried out in the sorted kind. > * A high overall performance `item' and `iteritem' methods as it is not > necessary to create new objects. > * At lots of objects in a tree memory is used much less intensively. > * Absence of collisions. As consequence, it is impossible to generate bad > data for creation DoS of the attacks influencing the dictionary of > transferred arguments. > * For objects existence of a method __ hash __ is not necessary. > * The same kind of the dictionary by means of the overloaded operations > probably change of its behaviour so that it supported the > multivariational > representation based on stratification given and subsequent recoils. > > Additional requirements to key objects: > * The opportunity of comparison of key objects function cmp is necessary. > > There are the reasons, braking development of this project: > * Compulsion of heading of a module `gc' for each compound object (this > structure is unessential to some objects, but updating `gc' the module > is required). > * Absence of standard object the pair, probably depends on the > previous item. > > Lacks of a binary tree: > * Average time of search for hash - O (1), and for trees - O (log2 N). > * A lot of memory under a single element of a tree: > (3*sizeof (void *) + sizeof (int))*2, > one element is used rather - the pair, the second site of memory is > allocated under node of a tree. > > In protection of object "pair": > * The logic of methods of the given object noticeably is easier than > tuple, > that as a result can affect speed of work of the program in the best > party. > * Alignment on 8 bytes has no big sense at present architecture where in > cache sample a minimum on 64 bytes is made. Use of type "pair" will give > an appreciable prize at use of alignment in 4 bytes. Otherwise on 64-bit > platforms it is much more favourable to use tuple object. > > The given project can demand processing of the module `gcmodule.c' and > `tupleobject.c'. It is necessary to reduce the size of static objects, > for this > purpose the opportunity is necessary is transparent to pass objects not > having > direct support from the module `gcmodule.c'. > > Also it will be partially necessary to process the module `obmalloc.c' > for more > effective distribution of memory. > > I shall be glad to answer questions on this theme. > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jcarlson%40uci.edu ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
Paul Moore wrote: > 2. Nick, what can we do to persuade you to go back to the a1 version, > and simply look at @contextmanager? Using two names to describe three different things isn't intuitive for anybody. You might persuade me to change the names around, but you aren't going to persuade me to go back to using "context" for either of the concrete object types. If we use "context object" as the name for any kind of concrete object, decimal.Context has to be an example of it. It wasn't in the alpha 1 docs, which was one of the main points of confusion (@contextmanager was the other). The alpha 2 docs have ended up sidestepping the issue by not using that term at all (introducing "context specifier" instead). The extent to which I'm willing to consider reversion to alpha 1 terminology is to make it so that the context management protocol is the protocol with the single __context__ method. However, the objects with the __enter__/__exit__ methods need to be called something more specific than context objects (more on that below). > I've proposed splitting it into > two, but that seems not to suit you (you've never responded to it > specifically, so I may be misreading your silence here). Wanting to have two names for the same function tells me there's a problem with the terminology, not that we should actually have two names for the same function :) I don't want to be fielding the question "what's the difference between contextlib.context and contextlib.contextmanager?" "Well, there isn't actually any difference." "Why two functions then?" "Ummm. . ." So, if we really want to use the alpha 1 terminology where a context manager provides just a __context__ method, then I think the right answer is to introduce a separate term for the objects that are returned by that method. However, while coming up with "when requested by the with statement, a context specifier object provides a context manager object to set up and tear down the desired runtime context" was pretty easy, I really struggle with that sentence when the starting object is a context manager. Trying "a context manager object provides a context object to set up and tear down the desired runtime context" seems initially appealing, until you think about all the "context objects" that already exist in various domains, such as: decimal arithmetic contexts GUI toolkit drawing contexts parsing & compilation contexts The fact that those are actually all candidate context specifiers, along with the temptation to abbreviate the term to just 'context', results in confusion just as bad as with the original PEP terminology. However, if you can successfully fill in the blank in: "when requested by the with statement, a context manager object provides a context object to set up and tear down the desired runtime context" then we can simply change the name of the contextmanager decorator and the various ContextManager objects in the implementation to that new term, and make the appropriate changes to the documentation. From my POV, the fact that the specifier says what the context should be, and the manager makes that happen makes sense. I suppose you could call the second object an effector, since it effects the necessary changes: "when requested by the with statement, a context manager object provides a context effector object to set up and tear down the desired runtime context" Actually using the transitive verb form of effect is fairly unnatural English, though :) > One thing I would say is that Nick has added a section on context > types to the "Built-in types" section of the libref. At the moment, it > reflects his terminology, but even if the terminology gets reverted to > a1 style, I'd like to see that stay (suitably reworded, of course!) > It's a useful addition. And one with a fairly long history - I first drafted something along those lines nearly 10 months ago [1]. FWIW, that discussion last year is the biggest reason I've been trying so hard to preserve "context manager" as the name for the objects with the __enter__/__exit__ methods. We spent a lot of effort coming up with it, and the way I wrote those draft docs (and tried to convey in the PEP's standard terminology section), it makes far more sense to me for that term to continue to apply to the objects with __enter__ and __exit__ methods than it does to relocate it to the johnny-come-lately objects which only have a __context__ method. But if we can agree on a different name for context managers (context objects is *not* an option), then changing those docs wouldn't be difficult. > Oh, and if there's a huge group of people who prefer Nick's > terminology, now is the time to shout! So far I've only got Greg commenting that it would be a shame for decimal.Context to not be a 'context' anymore. And he loses that either way - the only difference in that respect between alpha 1 and alpha 2 is that the alpha 2 docs don't call *any*
Re: [Python-Dev] PEP 343 update (with statement context terminology)
At 04:48 AM 4/25/2006 +1000, Nick Coghlan wrote: >Using two names to describe three different things isn't intuitive for >anybody. Um, what three things? I only count two: 1. Objects with __context__ 2. Objects with __enter__ and __exit__ What's the third thing? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reducing memory overhead for dictionaries by removing me_hash
On 4/23/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Kirat Singh wrote: > > The reason I looked into this to begin with was that my code used up a > > bunch of memory which was traceable to lots of little objects with > > instance dicts, so it seemed that if instancedicts took less memory I > > wouldn't have to go and add __slots__ to a bunch of my classes, or > > rewrite things as tuples/lists, etc. > > Ah. In that case, I would be curious if tuning PyDict_MINSIZE could > help. If you have many objects of the same type, am I right assuming > they all have the same number of dictionary keys? If so, what is the > dictionary size? Do they use ma_smalltable, or do they have an extra > ma_table? But the space savings by using __slots__ is so much bigger! (And less work than hacking the C code too. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed addition to threading module - released
[Guido] > Actually, what Nick describes is *exactly* how one should write code > using a condition variable: > > LOCK > while nothing to do: > UNLOCK > wait for the condition variable (or sleep, or whatever) > LOCK > # here we have something to do with the lock held > remove the to-do item > UNLOCK > > except that the outer LOCK/UNLOCK pair should be using a try/except > and the inner UNLOCK/LOCK pair should too. I don't see how you can do > this easily by rewriting the code; the rewrite would be considerably > ugly (or requires a GOTO :-). That didn't make much sense to me. If you're using a condition variable `cv`, the way that should be written is: cv.acquire() try: while nothing to do: cv.wait() # which unlocks on entry, and locks before return do something finally: cv.release() IOW, there is no "inner UNLOCK/LOCK" for the user to worry about (although the implementation of wait() has to worry about it). Does with cv: work to replace the outer (== only) acquire/try/finally/release dance? If so, that seems like plenty to me. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
On Mon, Apr 24, 2006, Phillip J. Eby wrote: > At 04:48 AM 4/25/2006 +1000, Nick Coghlan wrote: >> >>Using two names to describe three different things isn't intuitive for >>anybody. > > Um, what three things? I only count two: > > 1. Objects with __context__ > 2. Objects with __enter__ and __exit__ > > What's the third thing? The actual context that's used during the execution of BLOCK. It does not exist as a concrete object, but we must be able to refer to it in documentation. I called it a "namespace" in my docs, which I now realize is incorrect, but it's not entirely clear to me how to describe it. Perhaps "wrapper" will do. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Grammar Ambiguity
On 4/24/06, Michael Foord <[EMAIL PROTECTED]> wrote: > Hello all, > > I'm working on a parser for part of the Python language (expressions but > not statements basically). I'm using PLY to generate the parser and it's > mostly done. > > I've hit on what looks like a fundamental ambiguity in the Python > grammar which is difficult to get round with PLY; and I'm wondering > *why* the grammar is defined in this way. It's possible there is a > reason that I've missed, which means I need to rethink my workaround. > > List displays (list comprehensions) are defined as (from > http://docs.python.org/ref/lists.html ) : > Have you checked Grammar/Grammar to make sure it is the same? The grammar in the language ref is not always the most up-to-date version of the grammar. For instance, the grammar rule for 'test' is actually ``test: or_test ['if' or_test 'else' test] | lambdef``. -Brett > > test ::= and_test ( "or" and_test )* | lambda_form > testlist ::= test ( "," test )* [ "," ] > list_display ::= "[" [listmaker] "]" > listmaker ::= expression ( list_for | ( "," expression )* [","] ) > list_iter ::= list_for | list_if > list_for ::= "for" expression_list "in" testlist [list_iter] > list_if ::= "if" test [list_iter] > > The problem is that list_for is defined as : > > "for" expression_list "in" testlist > > This allows arbitrary expressions in the 'assignment' part of a list > comprehension. > > As a result, the following is valid syntax according to the grammar : > > [x for x + 1 in y] > > Obviously it isn't valid ! This parses to an ast, but the syntax error > is thrown when you compile the resulting ast. > > The problem is that for the basic case of a list comprehension ( ``[x > for x in y]``), ``x in y`` is a valid expression. That makes it > extremely hard to disambiguate the grammar so that the ``in`` is treated > correctly, and not part of an expression. > > My question is, why are arbitrary expressions allowed here in the > grammar ? As far as I can tell, only identifiers (nested in parentheses > or brackets) are valid here. I've got round the problem by creating a > new node 'identifier_list' and just having that return the expected > syntax tree (actually an expression list). This gets round the ambiguity > [#]_. > > It worries me that there might be a valid expression allowed here that I > haven't thought of. My current rules allow anything that looks like > ``(a, [b, c, (d, e)], f)`` - any nested identifier list. Would anything > else be allowed ? > > If not, why not modify the grammar so that the compiler has less > possible invalid syntax trees to work with ? > > (Also the grammar definition of string conversion is wrong as it states > that a trailing comma is valid, which isn't the case. As far as I can > tell that is necessary to allow nesting string conversions.) > > Fuzzyman > http://www.voidspace.org.uk/python/index.shtml > > > .. [#] If I could make precedence work in PLY I could also solve it I > guess. However I can't. :-) > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
At 12:24 PM 4/24/2006 -0700, Aahz wrote: >On Mon, Apr 24, 2006, Phillip J. Eby wrote: > > At 04:48 AM 4/25/2006 +1000, Nick Coghlan wrote: > >> > >>Using two names to describe three different things isn't intuitive for > >>anybody. > > > > Um, what three things? I only count two: > > > > 1. Objects with __context__ > > 2. Objects with __enter__ and __exit__ > > > > What's the third thing? > >The actual context that's used during the execution of BLOCK. It does >not exist as a concrete object, Um, huh? It's a thing but it's not an object? I'm lost now. I don't see why we should introduce a concept that has no concrete existence into something that's hard enough to explain when you stick to the objects that actually exist. :) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
On Mon, Apr 24, 2006, Phillip J. Eby wrote: > At 12:24 PM 4/24/2006 -0700, Aahz wrote: >>On Mon, Apr 24, 2006, Phillip J. Eby wrote: >>> At 04:48 AM 4/25/2006 +1000, Nick Coghlan wrote: Using two names to describe three different things isn't intuitive for anybody. >>> >>> Um, what three things? I only count two: >>> >>> 1. Objects with __context__ >>> 2. Objects with __enter__ and __exit__ >>> >>> What's the third thing? >> >>The actual context that's used during the execution of BLOCK. It does >>not exist as a concrete object, > > Um, huh? It's a thing but it's not an object? I'm lost now. I don't see > why we should introduce a concept that has no concrete existence into > something that's hard enough to explain when you stick to the objects that > actually exist. :) Let's go back to a pseudo-coded with statement: with EXPRESSION [as NAME]: BLOCK What happens while BLOCK is being executed? Again, here's what I said originally: EXPRESSION returns a value that the with statement uses to create a context (a special kind of namespace). The context is used to execute the BLOCK. The block might end normally, get terminated by a break or return, or raise an exception. No matter which of those things happens, the context contains code to clean up after the block. Do you have an alternate proposal for describing this that works well for newbies? Forget about the internal mechanics of what really happens, we must have a simple way of describing the with block itself! -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
At 12:49 PM 4/24/2006 -0700, Aahz wrote: >On Mon, Apr 24, 2006, Phillip J. Eby wrote: > > At 12:24 PM 4/24/2006 -0700, Aahz wrote: > >>On Mon, Apr 24, 2006, Phillip J. Eby wrote: > >>> At 04:48 AM 4/25/2006 +1000, Nick Coghlan wrote: > > Using two names to describe three different things isn't intuitive for > anybody. > >>> > >>> Um, what three things? I only count two: > >>> > >>> 1. Objects with __context__ > >>> 2. Objects with __enter__ and __exit__ > >>> > >>> What's the third thing? > >> > >>The actual context that's used during the execution of BLOCK. It does > >>not exist as a concrete object, > > > > Um, huh? It's a thing but it's not an object? I'm lost now. I don't see > > why we should introduce a concept that has no concrete existence into > > something that's hard enough to explain when you stick to the objects that > > actually exist. :) > >Let's go back to a pseudo-coded with statement: > > with EXPRESSION [as NAME]: > BLOCK > >What happens while BLOCK is being executed? Again, here's what I said >originally: > > EXPRESSION returns a value that the with statement uses to create a > context (a special kind of namespace). The context is used to > execute the BLOCK. The block might end normally, get terminated by > a break or return, or raise an exception. No matter which of those > things happens, the context contains code to clean up after the > block. > >Do you have an alternate proposal for describing this that works well for >newbies? No, I like your phrasing -- but it's quite concrete. EXPRESSION returns a value (object w/__context__) used to create a context (object w/__enter__ and __exit__). That's only two things. There is no *third* thing here. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Grammar Ambiguity
Guido van Rossum wrote:
> This is probably because we have a similar ambiguity in assignments:
> the grammar says something like
>
> exprlist ('=' exprlist)*
>
> but what is actually desired is
>
> (varlist '=')* exprlist
>
> Unfortunately the latter is not LL1 so we lie to the parser and tell
> it the first form, and then in the code generator checks that the
> actual parse tree for all exprlists except for the last conforms to
> the more limited syntax for varlist (which is only in our head). If
> not, it issues a "phase 2" SyntaxError. (One that doesn't have the
> exact column information.)
>
> We didn't have to do this for the syntax of the for-statement,
> list-comprehensions etc. because the ambiguity doesn't exist there
> (the 'for' keyword disambiguates the situation); but the current
> approach allows more code sharing in the code generator.
>
> I suggest you go ahead and write the second form for your own parser.
> Coming up with the correct rules for varlist is not hard.
>
>
We've already done that, I was just checking there wasn't some case we'd
missed were an expression was possible.
Thanks
Michael
> --Guido
>
> On 4/24/06, Michael Foord <[EMAIL PROTECTED]> wrote:
>
>> (oops - should have gone to list)
>>
>> Guido van Rossum wrote:
>>
>>> Well, yes, the syntax is supposed to be something like "for varlist in
>>> testlist". Could you report this as a doc bug (if you found this
>>> information in the docs)?
>>>
>>>
>> I think the documentation (which does put expression_list there)
>> reflects the current state of the parser.
>>
>> First of all the grammar in SVN also has expression_list there *and* the
>> following does successfully parse to an ast (but fails when you compile
>> the ast) :
>>
>> [x for x + 1 in y]
>>
>> All the best,
>>
>>
>> Michael Foord
>>
>>
>>
>>> On 4/24/06, Michael Foord <[EMAIL PROTECTED]> wrote:
>>>
>>>
Hello all,
I'm working on a parser for part of the Python language (expressions but
not statements basically). I'm using PLY to generate the parser and it's
mostly done.
I've hit on what looks like a fundamental ambiguity in the Python
grammar which is difficult to get round with PLY; and I'm wondering
*why* the grammar is defined in this way. It's possible there is a
reason that I've missed, which means I need to rethink my workaround.
List displays (list comprehensions) are defined as (from
http://docs.python.org/ref/lists.html ) :
test ::= and_test ( "or" and_test )* | lambda_form
testlist ::= test ( "," test )* [ "," ]
list_display ::= "[" [listmaker] "]"
listmaker ::= expression ( list_for | ( "," expression )* [","] )
list_iter ::= list_for | list_if
list_for ::= "for" expression_list "in" testlist [list_iter]
list_if ::= "if" test [list_iter]
The problem is that list_for is defined as :
"for" expression_list "in" testlist
This allows arbitrary expressions in the 'assignment' part of a list
comprehension.
As a result, the following is valid syntax according to the grammar :
[x for x + 1 in y]
Obviously it isn't valid ! This parses to an ast, but the syntax error
is thrown when you compile the resulting ast.
The problem is that for the basic case of a list comprehension ( ``[x
for x in y]``), ``x in y`` is a valid expression. That makes it
extremely hard to disambiguate the grammar so that the ``in`` is treated
correctly, and not part of an expression.
My question is, why are arbitrary expressions allowed here in the
grammar ? As far as I can tell, only identifiers (nested in parentheses
or brackets) are valid here. I've got round the problem by creating a
new node 'identifier_list' and just having that return the expected
syntax tree (actually an expression list). This gets round the ambiguity
[#]_.
It worries me that there might be a valid expression allowed here that I
haven't thought of. My current rules allow anything that looks like
``(a, [b, c, (d, e)], f)`` - any nested identifier list. Would anything
else be allowed ?
If not, why not modify the grammar so that the compiler has less
possible invalid syntax trees to work with ?
(Also the grammar definition of string conversion is wrong as it states
that a trailing comma is valid, which isn't the case. As far as I can
tell that is necessary to allow nesting string conversions.)
Fuzzyman
http://www.voidspace.org.uk/python/index.shtml
.. [#] If I could make precedence work in PLY I could also solve it I
guess. However I can't. :-)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailm
Re: [Python-Dev] PEP 343 update (with statement context terminology)
On Mon, Apr 24, 2006, Phillip J. Eby wrote: > At 12:49 PM 4/24/2006 -0700, Aahz wrote: >>On Mon, Apr 24, 2006, Phillip J. Eby wrote: >>> At 12:24 PM 4/24/2006 -0700, Aahz wrote: On Mon, Apr 24, 2006, Phillip J. Eby wrote: > At 04:48 AM 4/25/2006 +1000, Nick Coghlan wrote: >> >>Using two names to describe three different things isn't intuitive for >>anybody. > > Um, what three things? I only count two: > > 1. Objects with __context__ > 2. Objects with __enter__ and __exit__ > > What's the third thing? The actual context that's used during the execution of BLOCK. It does not exist as a concrete object, >>> >>> Um, huh? It's a thing but it's not an object? I'm lost now. I don't see >>> why we should introduce a concept that has no concrete existence into >>> something that's hard enough to explain when you stick to the objects that >>> actually exist. :) >> >>Let's go back to a pseudo-coded with statement: >> >> with EXPRESSION [as NAME]: >> BLOCK >> >>What happens while BLOCK is being executed? Again, here's what I said >>originally: >> >> EXPRESSION returns a value that the with statement uses to create a >> context (a special kind of namespace). The context is used to >> execute the BLOCK. The block might end normally, get terminated by >> a break or return, or raise an exception. No matter which of those >> things happens, the context contains code to clean up after the >> block. >> >>Do you have an alternate proposal for describing this that works well for >>newbies? > > No, I like your phrasing -- but it's quite concrete. EXPRESSION returns a > value (object w/__context__) used to create a context (object w/__enter__ > and __exit__). > > That's only two things. There is no *third* thing here. What is EXPRESSION, then? Not the value it returns, but EXPRESSION itself -- does it have a name? What about the kinds of things we use for EXPRESSION? -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
On 4/24/06, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Paul Moore wrote:
> > I've proposed splitting it into
> > two, but that seems not to suit you (you've never responded to it
> > specifically, so I may be misreading your silence here).
>
> Wanting to have two names for the same function tells me there's a problem
> with the terminology, not that we should actually have two names for the same
> function :)
My apologies. On rereading the code, I realised that I'd got muddled
by the decorator implementation. [For anyone reading along, if you
think I'm confused over contexts, watch me try to explain
decorators] So my explanation was unclear. I'll try again.
In the documentation of contextmanager, consider the examples:
@contextmanager
def tag(name):
...
class Tag:
...
@contextmanager
def __context__(self):
...
Now, tag should be a function which returns a context manager (a1
definition - object with a __context__ method) ("with" statement, item
1 of the definition). On the other hand, Tag.__context__ is a context
manager's __context__ method, which according to item 2 of the
definition of the "with" statement, should be a function which returns
a context object (a1 definition - object with __enter__ and __exit__
methods).
*These are two different types of function*.
Just because contextmanager is implemented in such a way that
decorated functions return objects with all 3 methods, doesn't make
the functions conceptually the same.
Does this help you to understand my concern? If not, I give up. (With
one proviso - you insist that objects with __enter__ and __exit__ must
supply __context__. I don't see why that's necessary, but I'll put
that in another post).
Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Must objects with __enter__/__exit__ also supply __context__?
The current, alpha 2, documentation insists that objects with __enter__ and __exit__ methods must also define __context__ in such a way that it returns self. I don't understand why that is necessary. I can understand that it is convenient, in cases where __context__ doesn't need to create a new object each time, but is it *necessary*? Specifically, is there a use case where you need to say "with x" where x is the return value of a __context__ method, or where you call __context__ on something you got from __context__? I can't find one in the PEP or in the code for contextlib... By insisting that things with __enter__ and __exit__ methods must implement __context__, there's a subtype relationship which I *think* means that Nick's insistence that the concepts are distinct, becomes difficult to support. But the terms are so confused now, that I'm utterly unable to frame my objection clearly. Can someone clarify this? Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
At 01:19 PM 4/24/2006 -0700, Aahz wrote: >What is EXPRESSION, then? Not the value it returns, but EXPRESSION >itself -- does it have a name? What about the kinds of things we use >for EXPRESSION? I read "EXPRESSION returns a value" as simply meaning that "value = EXPRESSION", i.e. that the result of computing EXPRESSION *is* the value. That's what it usually means when we talk about expressions returning a value -- that computing the expression produces a value. I still don't see a third thing here. "EXPRESSION returns a value" (Thing 1). That value is "used to create a context" (by calling __context__). This context (Thing 2) "is used to execute a block" (by calling __enter__ and __exit__). I don't get how you can have a difference between "EXPRESSION" and "value it returns" unless you're bringing functions into play. In everything else in Python, an expression *is* the value it returns. How could it be otherwise? Maybe you meant to write an explanation that included three objects, but what you wrote is actually a precise and accurate description of how things works. The value produced by EXPRESSION is used to create a context, and the context is used to execute the block. I don't know how you could explain it any more simply than that -- certainly not by adding a mysterious third gunman on the grassy knoll. :) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
At 04:48 AM 4/25/2006 +1000, Nick Coghlan wrote: >Wanting to have two names for the same function tells me there's a problem >with the terminology, not that we should actually have two names for the same >function :) It is purely an implementation detail of @contextmanager that it can be used to define __context__ methods. It would be perfectly legal to implement it in such a way that there were two helper classes, one with a __context__ method, and the other with the __enter__/__exit__ methods. I'm fine, however, with: 1. Changing the decorator name to @contextfactory 2. Requiring objects with __enter/__exit__ to also have __context__ (i.e., keep "context" as a subtype of "contextmanager") The truth is that @contextmanager is a misnomer anyway, because it doesn't turn the function into a context manager, it turns the function into a context factory - i.e., when called, it returns a context (that's also a contextmanager). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Must objects with __enter__/__exit__ also supply __context__?
At 09:35 PM 4/24/2006 +0100, Paul Moore wrote: >The current, alpha 2, documentation insists that objects with >__enter__ and __exit__ methods must also define __context__ in such a >way that it returns self. > >I don't understand why that is necessary. > >I can understand that it is convenient, in cases where __context__ >doesn't need to create a new object each time, but is it *necessary*? > >Specifically, is there a use case where you need to say "with x" where >x is the return value of a __context__ method, or where you call >__context__ on something you got from __context__? I can't find one in >the PEP or in the code for contextlib... The only benefit to this is that it allows us to have only one decorator. If the decorator is defined as returning a thing with __enter__ and __exit__, and such things must also have a __context__, then there is no need for a separate decorator that's defined as returning things that have __context__, nor to tweak the docs to explain that the single decorator does both, nor to have two names for the same decorator. So, it's sort of a documentation hack. :) By the way, I'm now on board with the idea that @contextmanager should be renamed, preferably to @contextfactory. (The objection someone made about "factory" implying a factory function is off-base; @contextfactory indeed *returns* a factory function that returns a context, so actually the name is perfect, and works for both wrapping __context__ methods and standalone generators.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Google Summer of Code proposal: improvement of long int and adding new types/modules.
2006/4/22, Mateusz Rukowicz <[EMAIL PROTECTED]>: > I am now quite sure, what I would like to do, and is possible by you to > accept - code decimal in C, most important things about that would: I'd be glad to mentor this. Regards, .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Builtin exit, good in interpreter, bad in code.
On 4/23/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Sean Reifschneider wrote: > > Thoughts? > > In Python 2.5, exit(0) exits. +1 > > Regards, > Martin > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/crutcher%40gmail.com > -- Crutcher Dunnavant <[EMAIL PROTECTED]> littlelanguages.com monket.samedi-studios.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed addition to threading module - released
Guido van Rossum wrote: > Actually, what Nick describes is *exactly* how one should write code > using a condition variable: > > LOCK > while nothing to do: > UNLOCK > wait for the condition variable (or sleep, or whatever) > LOCK > # here we have something to do with the lock held > remove the to-do item > UNLOCK > > except that the outer LOCK/UNLOCK pair should be using a try/except > and the inner UNLOCK/LOCK pair should too. I don't see how you can do > this easily by rewriting the code; the rewrite would be considerably > ugly (or requires a GOTO :-). I thought the trick is that the condition variable *atomically* releases the lock, waits for the condition, and then reacquires the condition variable. I.e. c = threading.Condition() c.lock() while nothing to do: c.wait() # here we have something to do with the lock held c.unlock() So the refactoring is to move the unlock/wait/lock sequence into the condition object. Using with, you could write this as with threading.Condition() as c: while nothing to do: c.wait() # do work So no need for an additional context manager here. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Neil Hodgson wrote: >I expect Microsoft means that Visual Studio Express will be > available free forever, not that you will always be able to download > Visual Studio 2005 Express. They normally only provide a particular > product version for a limited time after it has been superceded. Sure: they will remove download access to VS 2005 when VS 2007 comes available. Still, VS 2005 is available for download right now, and VS 2003 isn't (anymore). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SoC proposal: "fix some old, old bugs in sourceforge"
"Alan McIntyre" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Hi all, > > I would like to participate in the Summer of Code as a student. At the > moment it looks like the Python tracker on SF has about 2100 open bugs > and patches, going back to late 2000. The latest weekly tracker summary says about 1300, + 200 RFEs. Still too many. > I'm assuming that a fair number > of these are no longer be applicable, have been fixed/implemented > already, etc., and somebody just needs to slog through the list and > figure out what to do with them. I suspect so too, and plan to at least recheck some that I have reviewed. > My unglamorous proposal is to review bugs & patches (starting with the > oldest) and resolve at least 200 of them. Funny, and nice!, that you should propose this. I thought of adding something like this to the Python wiki as something I might mentor, but hesitated because reviewing *is* not glamourous, because Google wants code-writing projects, and because I am not one to mentor C code writing. > Is that too much? To review and close things that don't need a fix or are obsolete, no. To write code and fix, go with the response from someone who has done such. The thing I worry about, besides you or whoever getting too bored after a week, is that a batch of 50-100 nice new patches could then sit unreviewed on the patch tracker along with those already there. Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Alex Martelli wrote: > For the Toolkit 2003: > http://tinyurl.com/gv8wr When I go to this URL, I get redirected to http://www.microsoft.com/downloads/details.aspx?familyid=272BE09D-40BB-4&displaylang=en This doesn't look right - it ought to be a UUID. Anyway, I get a page that reads "The download you requested is unavailable. If you continue to see this message when trying to access this download, go to the "Search for a Download" area on the Download Center home page." both with Firefox and MSIE. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
John J Lee wrote: > Actually, it's apparently still there, just at a different URL. > Somebody posted the new URL on c.l.py a day or two back (Alex Martelli > started the thread, IIRC). I'm off to the dentist, no time to Google > for it! Please do. If you find the URL, please post it here. All URLs I found don't work (anymore). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbot messages and the build svn revision number
Nick Coghlan wrote: > Would it be possible to get the buildbot error message subject lines to > include the svn revision number of the build that failed? Only if somebody contributes a patch. Feel free to submit a bug report, but I see little chance of implementing that feature within the next 6 months. Requesting it from the buildbot guys has a higher chance to get it implemented, though - Brian Warner promised to listen to our feature requests (IIRC). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gettext.py bug #1448060
Sylvain Thénault wrote: > I've posted a patch (#1475523) for this and assigned it to Martin Von > Loewis since he was the core developper who has made some followup on > the original bug. Could someone (Martin or someone else) quick review > this patch ? I really need a fix for this, so if anyone feels my > patch is not correct, and explain why and what should be done, I can > rework on it. If you need quick patch, you should just go ahead and use it (unreviewed). It will take several more months until either Python 2.4.4 or Python 2.5 is released. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
""Martin v. Löwis"" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Sure: they will remove download access to VS 2005 when VS 2007 > comes available. Still, VS 2005 is available for download right > now, and VS 2003 isn't (anymore). Just yesterday, I downloaded the 2003 Toolkit using the link posted to c.l.p. I don't need it now, but though I should grab it 'just in case'. Retrying the link, it seems the future arrived already. Since, as I remember, there was no license agreement for the download, just for the install (if/when I do it), I should think it legal to send the file to someone else. But I don't really know, of course. Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed addition to threading module - released
Tim Peters wrote: > Does > > with cv: > > work to replace the outer (== only) acquire/try/finally/release dance? Indeed it does - although implemented in a somewhat convoluted way: A lock *is* a context (or is that "context manager"), i.e. it implements def __context__(self): return self __enter__=acquire def __exit__(self,*args): return self.release() #roughly A _Condition *has* a lock, so it could become the context (manager?) through def __context__(self): return self.lock However, instead of doing that, it does def __context__(self): return self # roughly: __enter__ is actually set in __init__ to self.lock.acquire def __enter__(self): return self.acquire() def __exit__(self): return self.release Looks somewhat redundant to me, but correct. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Terry Reedy wrote: > Since, as I remember, there was no license agreement for the download, just > for the install (if/when I do it), I should think it legal to send the file > to someone else. But I don't really know, of course. Going by the download page for 2005, it does not have any restrictions on the page: http://msdn.microsoft.com/vstudio/express/visualc/download/ so it seems plausible that vc2003 did not have such restrictions, either. However, the bottom of the page has a link (Terms of Use) to http://www.microsoft.com/info/cpyright.mspx which says WITHOUT LIMITING THE FOREGOING, COPYING OR REPRODUCTION OF THE SOFTWARE TO ANY OTHER SERVER OR LOCATION FOR FURTHER REPRODUCTION OR REDISTRIBUTION IS EXPRESSLY PROHIBITED, UNLESS SUCH REPRODUCTION OR REDISTRIBUTION IS EXPRESSLY PERMITTED BY THE LICENSE AGREEMENT ACCOMPANYING SUCH SOFTWARE. So I guess they don't want you to provide copies of that file. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed addition to threading module - released
On 4/24/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Tim Peters wrote: > > Does > > > > with cv: > > > > work to replace the outer (== only) acquire/try/finally/release dance? > > Indeed it does - although implemented in a somewhat convoluted way: > A lock *is* a context (or is that "context manager"), i.e. it implements > > def __context__(self): return self > __enter__=acquire > def __exit__(self,*args): return self.release() #roughly > > A _Condition *has* a lock, so it could become the context (manager?) > through > > def __context__(self): return self.lock > > However, instead of doing that, it does > > def __context__(self): return self > # roughly: __enter__ is actually set in __init__ to self.lock.acquire > def __enter__(self): > return self.acquire() > def __exit__(self): > return self.release > > Looks somewhat redundant to me, but correct. Thanks -- I didn't see the shortcut when I coded this. I'll fix it. Tim is right, the UNLOCK/LOCK part is implied in the wait() call. However, the wait() implementation really *does* provide a use case for the primitive operation that Nick proposed, and it can't be refactored to remove the pattern Martin disapproves of (though of course the existing try/finally is fine). I'm not sure if the use case is strong enough to warrant adding it; I think it's fine not to support it directly. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (wasr45544 -peps/trunk/pep-0343.txt)
On Apr 24, 2006, at 03:49, Nick Coghlan wrote: > Just van Rossum wrote: >> Baptiste Carvello wrote: >> >>> Terry Reedy a écrit : So I propose that the context maker be called just that: 'context maker'. That should pretty clearly not be the context that manages the block execution. >>> +1 for context maker. In fact, after reading the begining of the >>> thread, I came up with the very same idea. >> >> Or maybe "context factory"? Yes its a factory. That is traditionally that you call a function that makes objects isn't it? > That would be fine if we used __call__ to retrieve the context > manager - but > "factory" is too tightly bound to "factory function" in my mind. __call__ may or may not implement a factory, that is up to my design to decide. __context__ is always a factory because that is the interface that "with" mandates. factory function is one way to implement a factory so yes its tightly bound, but that not a reason to reject factory. From what I've read on this list this is all there is to it: with EXPR: block with needs a context. If the EXPR object has a __context__ method it is a factory that will make a suitable context for the EXPR object otherwise EXPR is the context. that context must have __enter__ and __exit__ methods to operate the with protocol. I haven't learn about decorators so I've no comment on why you need them as well as the special method names. But the docs should tell me why. Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reducing memory overhead for dictionaries by removing me_hash
very true, but python makes it oh so easy to be lazy :-)On 4/24/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: On 4/23/06, "Martin v. Löwis" <[EMAIL PROTECTED] > wrote:> Kirat Singh wrote:> > The reason I looked into this to begin with was that my code used up a> > bunch of memory which was traceable to lots of little objects with> > instance dicts, so it seemed that if instancedicts took less memory I > > wouldn't have to go and add __slots__ to a bunch of my classes, or> > rewrite things as tuples/lists, etc.>> Ah. In that case, I would be curious if tuning PyDict_MINSIZE could> help. If you have many objects of the same type, am I right assuming > they all have the same number of dictionary keys? If so, what is the> dictionary size? Do they use ma_smalltable, or do they have an extra> ma_table?But the space savings by using __slots__ is so much bigger! (And less work than hacking the C code too. :-)Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Google Summer of Code proposal: New class for work with binary trees AVL and RB as with the standard dictionary.
On 4/25/06, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > There exists various C and Python implementations of both AVL and > Red-Black trees. For users of Python who want to use AVL and/or > Red-Black trees, I would urge them to use the Python implementations. > In the case of *needing* the speed of a C extension, there already > exists a CPython extension module for AVL trees: > http://www.python.org/pypi/pyavl/1.1 > And a C implementation for redblack tree is here: http://python.org/sf/1324770 :) Hye-Shik ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
Paul Moore wrote:
> In the documentation of contextmanager, consider the examples:
>
> @contextmanager
> def tag(name):
> ...
>
> class Tag:
> ...
> @contextmanager
> def __context__(self):
> ...
>
> Now, tag should be a function which returns a context manager (a1
> definition - object with a __context__ method) ("with" statement, item
> 1 of the definition). On the other hand, Tag.__context__ is a context
> manager's __context__ method, which according to item 2 of the
> definition of the "with" statement, should be a function which returns
> a context object (a1 definition - object with __enter__ and __exit__
> methods).
>
> *These are two different types of function*.
Paraphrasing:
def iterseq(seq):
for x in range(len(seq)):
yield x[i]
class Container:
...
def __iter__(self):
for x in self.contents:
yield x
Now, iterseq should be a function which returns an iterable (object with an
__iter__() method that is passed to for statements). On the other hand,
Tag.__iter__ is an iterable's __iter__ method, which should be a function
which returns an iterator (an object with __iter__() and next() methods).
*These are two different types of function*.
Why yes, yes they are. When iterators were introduced, a deliberate design
decision was taken to make the iterator protocol a superset of the iterable
protocol, precisely to allow for loops not to care whether they were being
given an iterable or an iterator.
Iterator's can be standalone, or they can be intended specifically for use as
a particular iterable's native iterator. The terminology is the same either way.
PEP 343 made a *deliberate, conscious design decision* to copy the semantics
of iterators by making the context management protocol a superset of the
context protocol (or rather, the context specification protocol in alpha 2).
What I don't understand is why you and Phillip are so keen on breaking this
deliberate symmetry with an existing part of the language design. Of course it
isn't necessary, but then neither was it necessary to make the iterator
protocol a superset of the iterable protocol.
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Must objects with __enter__/__exit__ also supply __context__?
Phillip J. Eby wrote: > At 09:35 PM 4/24/2006 +0100, Paul Moore wrote: >> The current, alpha 2, documentation insists that objects with >> __enter__ and __exit__ methods must also define __context__ in such a >> way that it returns self. >> >> I don't understand why that is necessary. It's not necessary at all. It's a deliberate design decision in the PEP, deliberately copying the semantics of the iterator protocol. >> I can understand that it is convenient, in cases where __context__ >> doesn't need to create a new object each time, but is it *necessary*? >> >> Specifically, is there a use case where you need to say "with x" where >> x is the return value of a __context__ method, or where you call >> __context__ on something you got from __context__? I can't find one in >> the PEP or in the code for contextlib... There aren't any current use cases that require it, no. But how much harder would it have been to develop itertools if the "all iterators are iterables" identity hadn't been part of the original design? Requiring that all context managers also be context specifiers is harmless. Not requiring it breaks the parallel with iterators and iterables, which means that parallel can't be leveraged to explain how things work anymore. > The only benefit to this is that it allows us to have only one > decorator. If the decorator is defined as returning a thing with > __enter__ and __exit__, and such things must also have a __context__, > then there is no need for a separate decorator that's defined as > returning things that have __context__, nor to tweak the docs to explain > that the single decorator does both, nor to have two names for the same > decorator. > > So, it's sort of a documentation hack. :) It's a deliberate design decision to parallel the existing model provided by iterators and iterables. Otherwise, you have to have three concepts: context specifiers (have __context__ method) context managers (have __enter__/__exit__ methods) combined specifier/managers (which have all three) There was no need for iterators to be iterables, either - that was a deliberate design decision taken at the time iterators were introduced. I merely copied that pre-existing approach for PEP 343. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
Phillip J. Eby wrote: > At 12:24 PM 4/24/2006 -0700, Aahz wrote: >> On Mon, Apr 24, 2006, Phillip J. Eby wrote: >>> At 04:48 AM 4/25/2006 +1000, Nick Coghlan wrote: Using two names to describe three different things isn't intuitive for anybody. >>> Um, what three things? I only count two: >>> >>> 1. Objects with __context__ >>> 2. Objects with __enter__ and __exit__ >>> >>> What's the third thing? >> The actual context that's used during the execution of BLOCK. It does >> not exist as a concrete object, > > Um, huh? It's a thing but it's not an object? I'm lost now. I don't see > why we should introduce a concept that has no concrete existence into > something that's hard enough to explain when you stick to the objects that > actually exist. :) "the block executes with the lock held" "the block executes with the file open, and closes it when finished "the block executes with stdout redirected to the file f" "the block executes with opening and close HTML tags written to stdout before and after the code is executed" "the block executes with the active decimal context set to a copy of the supplied context object" Believe me, it may not be representable as a Python object, but the "runtime context" created by a with statement is very real, and its what users actually give a damn about. The concrete objects (whatever we call them) are implementation details. And this is why I now believe using "context object" for *either* of the concrete objects would be a mistake. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 update (with statement context terminology)
Phillip J. Eby wrote: > At 04:48 AM 4/25/2006 +1000, Nick Coghlan wrote: >> Wanting to have two names for the same function tells me there's a >> problem >> with the terminology, not that we should actually have two names for >> the same >> function :) > > It is purely an implementation detail of @contextmanager that it can be > used to define __context__ methods. It would be perfectly legal to > implement it in such a way that there were two helper classes, one with > a __context__ method, and the other with the __enter__/__exit__ methods. > > I'm fine, however, with: > > 1. Changing the decorator name to @contextfactory Due to the ambiguity of the term, I'm still strongly resistant to the idea of calling *any* of the objects involved context objects - so I don't like contextfactory either (as it implies that this is something which creates context objects). > 2. Requiring objects with __enter/__exit__ to also have __context__ Yay, we finally agree on something :) > The truth is that @contextmanager is a misnomer anyway, because it > doesn't turn the function into a context manager, it turns the function > into a context factory - i.e., when called, it returns a context (that's > also a contextmanager). A decorated generator function is as much a context manager as a normal generator function is actually an iterator :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SoC proposal: "fix some old, old bugs in sourceforge"
On 4/24/06, A.M. Kuchling <[EMAIL PROTECTED]> wrote: > On Mon, Apr 24, 2006 at 12:30:12PM -0400, Alan McIntyre wrote: > > My unglamorous proposal is to review bugs & patches (starting with the > > oldest) and resolve at least 200 of them. Is that too much? Too few? > > I'll fix as many as possible during the SoC time frame, but I wanted to > > set a realistically achievable minimum for the proposal. If anybody can > > offer helpful feedback on a good minimum number I'd appreciate it. > > I'd suggest 75 or maybe 100 bugs or patches, not 200. I agee with Andrew. There's not that much low hanging fruit. People review the old stuff from time to time. There are a lot of really hard bugs to fix. I guess there are also a lot that we can't reproduce and the submitter is MIA. Those might be easier. Ping them if not reproducible, if no response in a month, we close. n ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed addition to threading module - released
Guido van Rossum wrote: > Tim is right, the UNLOCK/LOCK part is implied in the wait() call. > However, the wait() implementation really *does* provide a use case > for the primitive operation that Nick proposed, and it can't be > refactored to remove the pattern Martin disapproves of (though of > course the existing try/finally is fine). I'm not sure if the use case > is strong enough to warrant adding it; I think it's fine not to > support it directly. Ah, you mean that the wait implementation *itself* should use the unlocked() context (which calls release on enter, and acquire on exit). That wouldn't work, as _Condition.wait doesn't use release/enter, but _release_save/_acquire_restore. So the unlocked context couldn't be used there if it existed. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
Greg Ewing wrote: > Travis Oliphant wrote: > >> For what it's worth, NumPy also defines a data-type object which it >> uses to describe the fundamental data-type of an array. In the context >> of this thread it is also yet another way to describe a binary-packed >> structure in Python. > > Maybe there should be a separate module providing > a data-packing facility that ctypes, NumPy, etc. > can all use (perhaps with their own domain-specific > extensions to it). > > It does seem rather silly to have about 3 or 4 > different incompatible ways to do almost exactly > the same thing (struct, ctypes, NumPy and now > Construct). I agree. Especially with ctypes and struct now in the standard library. The problem, however, is that every module does something a little-bit different with the object. NumPy needs a built-in object with at least a few fields defined. The idea of "specifying the data-type" is different then it's representation to NumPy. After looking at it, I'm not particularly fond of Construct's way to specify data-types, but then again we've been developing the array interface for just this purpose and so have some biased opinions. Some kind of data-type specification would indeed be useful. NumPy needs a built-in (i.e. written in C) data-type object internally. If that builtin object were suitable generally then all the better. For details, look at http://numeric.scipy.org/array_interface (in particular the __array_descr__ field of the interface for what we came up with last year over several months of discussion. -Travis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] interested in Google Summer of Code: what should I do?
Hello everyone. I'm interested in the Python Software Foundation's Google Summer of Code proposals. What should I do? 1. Should I post my resume here? 2. List a few of the proposals that really interest me? 3. Explain why I feel qualified to contribute? 4. Who should I talk to? Thanks for everyone's time, and I eagerly await any responses. -- Karl Pietrzak [EMAIL PROTECTED] pgpoBgrnox2wM.pgp Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] interested in Google Summer of Code: what should I do?
Hi Karol. Please see the wiki: http://wiki.python.org/moin/SummerOfCode/ There are a bunch of ideas up there. If you want to hash out new ideas, I suppose this list is as good as any. But please do some searching to find if your idea has been tried before. It might also help you to guage interest in your project. You can also post messages to comp.lang.python. You should start working on your proposal now. The final submission is due in about 2 weeks. Good luck! n -- On 4/24/06, Karol Pietrzak <[EMAIL PROTECTED]> wrote: > Hello everyone. > > I'm interested in the Python Software Foundation's Google Summer of Code > proposals. > > What should I do? > 1. Should I post my resume here? > 2. List a few of the proposals that really interest me? > 3. Explain why I feel qualified to contribute? > 4. Who should I talk to? > > Thanks for everyone's time, and I eagerly await any responses. > > > -- > Karl Pietrzak > [EMAIL PROTECTED] > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > > > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
