Re: [Python-Dev] Iterator version of contextlib.nested
> Part of the justification for the new with-statement syntax was > that nested() doesn't have a way to finalize the constructors > if one of them fails. I think the problem was a little bit more subtle: nested() gets passed managers, so their __init__()s should all have run when the first context is entered. The only problem comes up when the __exit__() of an outer manager tries to suppress an exception raised by the __enter__() of an inner one. This is a limited defect in that it doesn't affect the common situation where no __exit__() tries to suppress any exceptions. (In a quick glance over the std library I couldn't find a single instance of an exception-suppressing __exit__().). > And now > that we have the new with-statement syntax, it mostly just > represents a second-way-to-do-it (a second way that has > has the stated pitfall). So the functionalities of nested() and multi-with overlap in the common use cases, and each has its own limitation in an uncommon one. I agree that this situation is unfortunate, but I think introducing support for one uncommon case and removing it for another is not the way to go in 3.1. That's why I think nested() should stay un-deprecated until there is a replacement which handles a superset of its use cases. > The new statement was not designed to support passing in > tuples of context-managers. This issue was raised while > the new with-statement was being designed and it was > intentionally left-out (in part, because the use cases were > questionable FWIW, my use case (which made me notice the DeprecationWarning in the first place) is in a command dispatch function, which looks at the command to be executed and pre-processes its arguments in a uniform way. Part of that pre-processing is entering contexts of context manager before passing them along (and exiting them when the command finishes or raises an exception). > and in-part because there were other ways > to do it such as adding __enter__ and __exit__ to tuple). Which has not been done for 3.1. Granted, you could subclass tuple and add them yourself, but then you would mostly be copying what's already implemented in nested(). > I suggest a PEP for 2.7 and 3.2 for building-out the > with-statement to support tuples of context managers That sounds like a good idea. > IMO, this represents doing-it-the-right-way instead of preserving a > construct that is known to be problematic. > Leaving it in will enshrine it. I don't see the problem with deprecating it only after a completely suitable replacement is found. Why would it be any harder to deprecate nested() in 3.2? - Hagen ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Iterator version of contextlib.nested
Hagen Fürstenau wrote: >> I actually almost asked for that to be changed to a >> PendingDeprecationWarning when it was first added - Benjamin, do you >> mind if I downgrade this warning to a pending one post rc2? > > I'm not sure what that would buy us. For the use case I mentioned it > would be just as annoying to get a PendingDeprecationWarning. But if the > warning was completely removed now, nested could still get deprecated in > 3.2 as soon as some better mechanism for a variable number of managers > has been provided. Unlike a full DeprecationWarning, a PendingDeprecationWarning is ignored by default. You have to switch them on explicitly via code or a command line switch in order to see them. Pending warnings give the hint that we intend to get rid of something, but either don't have clear replacements for some legitimate use cases (as in the case of contextlib.nested) or have some other reason for not generating a warning by default (e.g. we may not have cleared all uses out of the standard library yet). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Iterator version of contextlib.nested
> Unlike a full DeprecationWarning, a PendingDeprecationWarning is ignored > by default. You have to switch them on explicitly via code or a command > line switch in order to see them. Sorry, I should have made myself more familiar with the warnings mechanism before writing. In that case I'm fine with a PendingDeprecationWarning. :-) - Hagen ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Iterator version of contextlib.nested
Raymond Hettinger wrote: > I suggest a PEP for 2.7 and 3.2 for building-out the > with-statement to support tuples of context managers > (perhaps modeled after the syntax for except-statements > which allows either individual exceptions or tuples of > exceptions). The reason I suggest a PEP is that use > cases need to be fully thought out and the failing > constructor problem needs to be dealt with. > IMO, this represents doing-it-the-right-way instead of preserving a > construct that is known to be problematic. > Leaving it in will enshrine it. Better to just provide our new > syntax that correctly handles the common case and then > wait to carefully think through how to add support for passed-in > nested cm's if in-fact those turn-out to have reasonable > use cases. I agree that the current incarnation of contextlib.nested needs to go - it isn't really salvagable in its current form. However, I don't think we should generate a warning for it by default until we provide a new mechanism for handling a variable number of context managers - PendingDeprecationWarning seems a much better fit. A 2.7/3.2 PEP can then address the two main issues with the current approach: 1. The __init__ calls for the inner context managers occur outside the scope of the outer context managers. Some form of lazy evaluation would be needed to deal with that. 2. If an inner __enter__ call raises an exception that is suppressed by an outer __exit__ call then the body of with statement should be skipped rather than raising RuntimeError. This means either new syntax with semantics along the lines of the previously rejected PEP 377 or else a custom exception and a second context manager that suppresses it. Personally, I don't think new syntax for the PEP 377 semantics is warranted for the same reason that PEP 377 itself was rejected - it complicates the statement definition significantly for a really obscure corner case. Better to come up with a new and improved version of nested that eliminates (or better handles) the quirks and leave the statement definition alone. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] CPython in the web browser under Native Client
Mark Seaborn wrote: > [3] > http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html Very cool! Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Exception for setting attributes of built-in type
Guido van Rossum wrote: > In general, CPython isn't always consistent in raising AttributeError > and TypeError when it comes to such policy issues: there are various > places that raise TypeError in typeobject.c (and probably elsewhere) > that simply forbid setting a specific attribute (another example is > __name__). We're pretty inconsistent when it comes to looking up special methods as well - those that are looked up through dedicated slots in abstract.c usually raise TypeError, while those that are looked up via a PyType method usually raise AttributeError. I'm not sure that there is any practical way to handle that other than for applications that care about cross-implementation compatibility in this area to catch a (TypeError, AttributeError) tuple rather than one or the other. CHeers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python sendmsg()/recvmsg() implementation
Jean-Paul Calderone wrote: On Tue, 09 Jun 2009 16:46:54 +0200, Kálmán Gergely wrote: Hello, my name is Greg. I've just started using python after many years of C programming, and I'm also new to the list. I wanted to clarify this first, so that maybe I will get a little less beating for my stupidity :) Welcome! [snip] Browsing the net I've found a patch to the python core (http://bugs.python.org/issue1194378), dated 2005. First of all, I would like to ask you guys, whether you know of any way of doing this FD passing magic, or that you know of any 3rd party module / patch / anything that can do this for me. Aside from the patch in the tracker, there are several implementations of these APIs as third-party extension modules. Since I'm fairly familiar with C and (not that much, but I feel the power) python, I would take the challenge of writing it, given that the above code is still somewhat usable. If all else fails I would like to have your help to guide me through this process. What would be great is if you could take the patch in the tracker and get it into shape so that it is suitable for inclusion. This would involve three things, I think: 1. Write unit tests for the functionality (since the patch itself provides none) 2. Update the patch so that it again applies cleanly against trunk 3. Add documentation for the new APIs Once this is done, you can get a committer to look at it and either provide more specific feedback or apply it. Thanks, Jean-Paul ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/synapse%40jasmin.hu Hello again So, after a little cleanup I've managed to integrate the code into socketmodule.c/h. It works fine now, though I needed to add it to Lib/socket.py, otherwise it wouldn't show up in the socket module (I've searched for recvfrom and added it). I've also cleaned up the code a little, fixed some codingstyle issues (which might still exist). Since I am not a python core developer the patch might still be in a pretty outdated state. I'd like someone to look it over and direct me to some documentation (the ones I've found so far were pretty sketchy), for it to be acceptable for inclusion. The sanity of the code is what matters to me the most. I've looked it over though and found it in a sound state, but I guess you guys might have a different opinion about that ;) With writing the test cases, some documentation would be nice. I've attached the patch generated with svn diff for revision 73434, and the test code that I use to pass a file descriptor between processes. It works just fine :). Thanks Kalman Gergely Index: Lib/socket.py === --- Lib/socket.py (revision 73434) +++ Lib/socket.py (working copy) @@ -159,14 +159,14 @@ # All the method names that must be delegated to either the real socket # object or the _closedsocket object. _delegate_methods = ("recv", "recvfrom", "recv_into", "recvfrom_into", - "send", "sendto") + "recvmsg", "send", "sendto", "sendmsg") class _closedsocket(object): __slots__ = [] def _dummy(*args): raise error(EBADF, 'Bad file descriptor') # All _delegate_methods must also be initialized here. -send = recv = recv_into = sendto = recvfrom = recvfrom_into = _dummy +send = recv = recv_into = sendto = recvfrom = recvfrom_into = recvmsg = sendmsg = _dummy __getattr__ = _dummy # Wrapper around platform socket objects. This implements Index: Modules/socketmodule.c === --- Modules/socketmodule.c (revision 73434) +++ Modules/socketmodule.c (working copy) @@ -251,6 +251,9 @@ #ifdef HAVE_SYS_TYPES_H #include #endif +#include +#include +#include "structmember.h" /* Generic socket object definitions and includes */ #define PySocket_BUILDING_SOCKET @@ -1840,6 +1843,7 @@ int optname; int res; PyObject *buf; + char *data; socklen_t buflen = 0; #ifdef __BEOS__ @@ -1852,6 +1856,36 @@ &level, &optname, &buflen)) return NULL; +#ifdef SO_PEERCRED + if( level == SOL_SOCKET && optname == SO_PEERCRED ) { + /* Buffer length for struct ucred, ignore parameter. */ + buflen = sizeof(int)*3; + + /* Allocate ucred structure and data buffer. */ + if( !( buf = PyType_GenericAlloc((PyTypeObject*)&ucred_type,1) ) ) + return NULL; + if( !( data = PyMem_Malloc(buflen) ) ) { + Py_DECREF(buf); + return NULL; + } + + /* Use getsockopt to retrieve data. */ + if( ( res = getsockopt(s->sock_fd,level,optname,data,&buflen) ) < + 0 ) { + Py_DECREF(buf); + return s->errorhandler(); + } + + /* Write it out to object. */ + ((PySocketUcredObject*)buf)->uid = ((int*)data)[1]; + ((PySocketU
Re: [Python-Dev] python sendmsg()/recvmsg() implementation
Kálmán Gergely wrote: > Since I am not a python core developer the patch might still be in a > pretty outdated state. I'd like someone to look it over > and direct me to some documentation (the ones I've found so far were > pretty sketchy), for it to be acceptable for inclusion. > The sanity of the code is what matters to me the most. I've looked it > over though and found it in a sound state, but I guess > you guys might have a different opinion about that ;) > > With writing the test cases, some documentation would be nice. Most unit tests these days are written based on either doctest or unittest. When adding new features to an existing module, it is usually best to follow the testing style already used for the rest of that module (in this case, that should be test/test_socket.py). The relevant question in the dev FAQ gives good pointers: http://www.python.org/dev/faq/#how-to-test-a-patch > I've attached the patch generated with svn diff for revision 73434, and > the test code that I use to pass a file descriptor > between processes. It works just fine :). Uploading files to the tracker is generally the best option - patches tend to get lost if they're just sent to the mailing list. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Exception for setting attributes of built-in type
On Sun, Jun 14, 2009 at 4:19 PM, Guido van Rossum wrote: > In general, CPython isn't always consistent in raising AttributeError > and TypeError when it comes to such policy issues: there are various > places that raise TypeError in typeobject.c (and probably elsewhere) > that simply forbid setting a specific attribute (another example is > __name__). I should add that this policy is also forced somewhat by the existence of the "multiple interpreters in one address space" feature, which is used e.g. by mod_python. This feature attempts to provide isolation between interpreters to the point that each one can have a completely different set of modules loaded and can be working on a totally different application. The implementation of CPython shares built-in types between multiple interpreters (and it wouldn't be easy to change this); if you were able to modify a built-in type from one interpreter, all other interpreters would see that same modification. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [IronPython] Exception for setting attributes of built-in type
Guido wrote: > I should add that this policy is also forced somewhat by the existence > of the "multiple interpreters in one address space" feature, which is > used e.g. by mod_python. This feature attempts to provide isolation > between interpreters to the point that each one can have a completely > different set of modules loaded and can be working on a totally > different application. The implementation of CPython shares built-in > types between multiple interpreters (and it wouldn't be easy to change > this); if you were able to modify a built-in type from one > interpreter, all other interpreters would see that same modification. IronPython is in the exact same boat here - we share built-in types Across multiple Python engines as well. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [IronPython] Exception for setting attributes of built-in type
Dino Viehland wrote: Guido wrote: I should add that this policy is also forced somewhat by the existence of the "multiple interpreters in one address space" feature, which is used e.g. by mod_python. This feature attempts to provide isolation between interpreters to the point that each one can have a completely different set of modules loaded and can be working on a totally different application. The implementation of CPython shares built-in types between multiple interpreters (and it wouldn't be easy to change this); if you were able to modify a built-in type from one interpreter, all other interpreters would see that same modification. IronPython is in the exact same boat here - we share built-in types Across multiple Python engines as well. And indeed it is needed - if you are working with multiple interpreters (engines in IronPython) you don't want isinstance(something, dict) to fail because it is a dictionary from a different interpreter... Michael ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [IronPython] Exception for setting attributes of built-in type
On Mon, Jun 15, 2009 at 9:10 AM, Michael Foord wrote: > Dino Viehland wrote: >> Guido wrote: >>> I should add that this policy is also forced somewhat by the existence >>> of the "multiple interpreters in one address space" feature, which is >>> used e.g. by mod_python. This feature attempts to provide isolation >>> between interpreters to the point that each one can have a completely >>> different set of modules loaded and can be working on a totally >>> different application. The implementation of CPython shares built-in >>> types between multiple interpreters (and it wouldn't be easy to change >>> this); if you were able to modify a built-in type from one >>> interpreter, all other interpreters would see that same modification. >> IronPython is in the exact same boat here - we share built-in types >> Across multiple Python engines as well. > And indeed it is needed - if you are working with multiple interpreters > (engines in IronPython) you don't want isinstance(something, dict) to fail > because it is a dictionary from a different interpreter... Ah, but that suggests you have sharing between different interpreters. If you're doing that, perhaps you shouldn't be using multiple interpreters, but instead multiple threads? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [IronPython] Exception for setting attributes of built-in type
Guido van Rossum wrote: On Mon, Jun 15, 2009 at 9:10 AM, Michael Foord wrote: Dino Viehland wrote: Guido wrote: I should add that this policy is also forced somewhat by the existence of the "multiple interpreters in one address space" feature, which is used e.g. by mod_python. This feature attempts to provide isolation between interpreters to the point that each one can have a completely different set of modules loaded and can be working on a totally different application. The implementation of CPython shares built-in types between multiple interpreters (and it wouldn't be easy to change this); if you were able to modify a built-in type from one interpreter, all other interpreters would see that same modification. IronPython is in the exact same boat here - we share built-in types Across multiple Python engines as well. And indeed it is needed - if you are working with multiple interpreters (engines in IronPython) you don't want isinstance(something, dict) to fail because it is a dictionary from a different interpreter... Ah, but that suggests you have sharing between different interpreters. If you're doing that, perhaps you shouldn't be using multiple interpreters, but instead multiple threads? Well, in our use case we use multiple engines to provide an isolated execution context for every document (the Resolver One spreadsheet written in IronPython). Each of these has their own calculation thread as well - but the engine per document structure is nice and clean and means each document can have its own set of modules loaded without affecting the other documents (although they share a core set of modules). Once we move these engines into their own app domains we can completely isolate each document and apply separate security permissions to each one. That might mean each document effectively paying the not-insubstantial startup time hit and we haven't begun to look at how to mitigate that. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Exception for setting attributes of built-in type
2009/6/15 Nick Coghlan : > Guido van Rossum wrote: >> In general, CPython isn't always consistent in raising AttributeError >> and TypeError when it comes to such policy issues: there are various >> places that raise TypeError in typeobject.c (and probably elsewhere) >> that simply forbid setting a specific attribute (another example is >> __name__). > > We're pretty inconsistent when it comes to looking up special methods as > well - those that are looked up through dedicated slots in abstract.c > usually raise TypeError, while those that are looked up via a PyType > method usually raise AttributeError. What's a PyType method? -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Iterator version of contextlib.nested
I don't consider changing a DeprecationWarning to a PendingDeprecationWarning "resurrecting" its target. Seems like resurrection to me. Pending warnings are hidden by default, so someone has to go look for it (and no one does this). The problem with the nested() construct is not so much that it duplicates the new with-statement; the problem is that it is a bug factory when used as advertised. The sole justification for keeping it around is that it handles an obscure use case (one that isn't even shown in its documentation or examples). I'm not opposing the idea to change the DeprecationWarning to a PendingDeprecationWarning, but I don't think we're doing the users any favors by hiding the warning message. Raymond P.S. If you switch to PendingDeprecationWarning, the example in the docs should probably be switched to show the one valid use case (passing in a prepackaged nest of context managers). Right now, the current example just shows the hazardous pattern that is much better served by the new with-statement syntax. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Iterator version of contextlib.nested
2009/6/15 Raymond Hettinger : > P.S. If you switch to PendingDeprecationWarning, the example > in the docs should probably be switched to show the one valid > use case (passing in a prepackaged nest of context managers). > Right now, the current example just shows the hazardous pattern > that is much better served by the new with-statement syntax. +1 I think this should be done anyway. -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Iterator version of contextlib.nested
Raymond Hettinger wrote: P.S. If you switch to PendingDeprecationWarning, the example in the docs should probably be switched to show the one valid use case (passing in a prepackaged nest of context managers). It could even suggest that it only be used for this, since it may disappear, and that other uses should use the new syntax. That would give people the best chance of writing future-proof code when they can. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Iterator version of contextlib.nested
2009/6/15 Terry Reedy : > Raymond Hettinger wrote: > >> P.S. If you switch to PendingDeprecationWarning, the example >> in the docs should probably be switched to show the one valid >> use case (passing in a prepackaged nest of context managers). > > It could even suggest that it only be used for this, since it may disappear, > and that other uses should use the new syntax. That would give people the > best chance of writing future-proof code when they can. And if the warning is *not* changed to a PendingDeprecationWarning, the documentation could also note the necessary warnings call needed to let the example code run warning-free. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Is the py3k mercurial mirror broken?
I get the following: >hg version Mercurial Distributed SCM (version 1.2) Copyright (C) 2005-2008 Matt Mackall and others This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. >hg clone http://code.python.org/hg/branches/py3k python-3k requesting all changes adding changesets transaction abort! rollback completed ** unknown exception encountered, details follow ** report bug details to http://www.selenic.com/mercurial/bts ** or [email protected] ** Mercurial Distributed SCM (version 1.2) ** Extensions loaded: Traceback (most recent call last): File "hg", line 27, in File "mercurial\dispatch.pyc", line 16, in run File "mercurial\dispatch.pyc", line 25, in dispatch File "mercurial\dispatch.pyc", line 41, in _runcatch File "mercurial\dispatch.pyc", line 372, in _dispatch File "mercurial\dispatch.pyc", line 247, in runcommand File "mercurial\dispatch.pyc", line 417, in _runcommand File "mercurial\dispatch.pyc", line 377, in checkargs File "mercurial\dispatch.pyc", line 371, in File "mercurial\util.pyc", line 718, in check File "mercurial\commands.pyc", line 603, in clone File "mercurial\hg.pyc", line 215, in clone File "mercurial\localrepo.pyc", line 2149, in clone File "mercurial\localrepo.pyc", line 1492, in pull File "mercurial\localrepo.pyc", line 2016, in addchangegroup File "mercurial\revlog.pyc", line 1192, in addgroup File "mercurial\changegroup.pyc", line 31, in chunkiter File "mercurial\changegroup.pyc", line 15, in getchunk File "mercurial\util.pyc", line 1674, in read File "mercurial\httprepo.pyc", line 18, in zgenerator zlib.error: Error -3 while decompressing: incorrect header check Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is the py3k mercurial mirror broken?
There seem to be some issues with Subversion, so that's probably affecting hg. 2009/6/15 Paul Moore : > I get the following: > >>hg version > Mercurial Distributed SCM (version 1.2) > > Copyright (C) 2005-2008 Matt Mackall and others > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > >>hg clone http://code.python.org/hg/branches/py3k python-3k > requesting all changes > adding changesets > transaction abort! > rollback completed > ** unknown exception encountered, details follow > ** report bug details to http://www.selenic.com/mercurial/bts > ** or [email protected] > ** Mercurial Distributed SCM (version 1.2) > ** Extensions loaded: > Traceback (most recent call last): > File "hg", line 27, in > File "mercurial\dispatch.pyc", line 16, in run > File "mercurial\dispatch.pyc", line 25, in dispatch > File "mercurial\dispatch.pyc", line 41, in _runcatch > File "mercurial\dispatch.pyc", line 372, in _dispatch > File "mercurial\dispatch.pyc", line 247, in runcommand > File "mercurial\dispatch.pyc", line 417, in _runcommand > File "mercurial\dispatch.pyc", line 377, in checkargs > File "mercurial\dispatch.pyc", line 371, in > File "mercurial\util.pyc", line 718, in check > File "mercurial\commands.pyc", line 603, in clone > File "mercurial\hg.pyc", line 215, in clone > File "mercurial\localrepo.pyc", line 2149, in clone > File "mercurial\localrepo.pyc", line 1492, in pull > File "mercurial\localrepo.pyc", line 2016, in addchangegroup > File "mercurial\revlog.pyc", line 1192, in addgroup > File "mercurial\changegroup.pyc", line 31, in chunkiter > File "mercurial\changegroup.pyc", line 15, in getchunk > File "mercurial\util.pyc", line 1674, in read > File "mercurial\httprepo.pyc", line 18, in zgenerator > zlib.error: Error -3 while decompressing: incorrect header check > > Paul. > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/benjamin%40python.org > -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Exception for setting attributes of built-in type
Benjamin Peterson wrote: > 2009/6/15 Nick Coghlan : >> Guido van Rossum wrote: >>> In general, CPython isn't always consistent in raising AttributeError >>> and TypeError when it comes to such policy issues: there are various >>> places that raise TypeError in typeobject.c (and probably elsewhere) >>> that simply forbid setting a specific attribute (another example is >>> __name__). >> We're pretty inconsistent when it comes to looking up special methods as >> well - those that are looked up through dedicated slots in abstract.c >> usually raise TypeError, while those that are looked up via a PyType >> method usually raise AttributeError. > > What's a PyType method? I was misremembering - for some reason I thought: a) _PyObject_LookupSpecial was a PyType method b) That it raised AttributeError itself instead of letting the caller decide what error to raise It's still the case that (e.g.) special_lookup() in ceval.c raises an AttributeError, as do special method lookups from Python of the form "type(obj).__method__(obj)". Whether CPython raises TypeError or AttributeError when it comes to special methods is really pretty arbitrary rather than a matter of any grand design. Cheers, Nick. P.S. If anyone feels like digging into the archives, the last time I can recall this topic coming up is when I was concerned about the original with statement implementation raising AttributeError rather than TypeError when it couldn't find an __enter__ or __exit__ method. Guido chimed in then (as now) to say that either exception was fine. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Iterator version of contextlib.nested
Paul Moore wrote: > 2009/6/15 Terry Reedy : >> Raymond Hettinger wrote: >> >>> P.S. If you switch to PendingDeprecationWarning, the example >>> in the docs should probably be switched to show the one valid >>> use case (passing in a prepackaged nest of context managers). >> It could even suggest that it only be used for this, since it may disappear, >> and that other uses should use the new syntax. That would give people the >> best chance of writing future-proof code when they can. > > And if the warning is *not* changed to a PendingDeprecationWarning, > the documentation could also note the necessary warnings call needed > to let the example code run warning-free. I think I like that option even better than downgrading the warning. I created http://bugs.python.org/issue6288 to make it clear to Benjamin when I have finished updating the docs. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: It seems like whenever I want to do some kind of opportunistic but non-blocking stuff with a remote service Do you actually do this with buffered streams? I find it's better to steer well clear of buffered I/O objects when doing non-blocking stuff, because they don't play well with other things like select(). Anyhow, I wouldn't be opposed to having a way of looking into the buffer, but it should be a separate API -- preferably with a better name than peek0(), which gives no clue at all about what it does differently from peek(). -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On 16Jun2009 11:21, Greg Ewing wrote: > Cameron Simpson wrote: >> It seems like whenever I want to do some kind of opportunistic but >> non-blocking stuff with a remote service > > Do you actually do this with buffered streams? Sure, in C, python and perl quite happily. In some circumstances. Provided you're careful about when to fflush() it can all go quite smoothly. It's certainly not applicable to everything. > I find > it's better to steer well clear of buffered I/O objects > when doing non-blocking stuff, because they don't play > well with other things like select(). Ah, the non-blockingness. Well that's the rub. I normally avoid non-blocking requirements by using threads, so that the thread gathering from the stream can block. Then the consumer can poll a Queue from the worker thread, etc. I really don't like select(/poll/epoll etc) much; aside from select's scaling issues with lots of files (hence poll/epoll) there are high performance things where having an event loop using select is a good way to go, but I generally prefer using threads and/or generators to make the code clear (single flow of control, single task for the block of code, etc) if there's no reason not to. > Anyhow, I wouldn't be opposed to having a way of looking > into the buffer, but it should be a separate API -- > preferably with a better name than peek0(), which gives > no clue at all about what it does differently from peek(). Indeed, though arguably read1() is a lousy name too, on the same basis. My itch is that peek() _feels_ like it should be "look into the buffer" but actually can block and/or change the buffer. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ You can't wait for inspiration. You have to go after it with a club. - Jack London ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Cameron Simpson wrote: On 16Jun2009 11:21, Greg Ewing wrote: Cameron Simpson wrote: It seems like whenever I want to do some kind of opportunistic but non-blocking stuff with a remote service Do you actually do this with buffered streams? Sure, in C, python and perl quite happily. In some circumstances. Provided you're careful about when to fflush() it can all go quite smoothly. It's certainly not applicable to everything. I find it's better to steer well clear of buffered I/O objects when doing non-blocking stuff, because they don't play well with other things like select(). Ah, the non-blockingness. Well that's the rub. I normally avoid non-blocking requirements by using threads, so that the thread gathering from the stream can block. Then the consumer can poll a Queue from the worker thread, etc. I really don't like select(/poll/epoll etc) much; aside from select's scaling issues with lots of files (hence poll/epoll) there are high performance things where having an event loop using select is a good way to go, but I generally prefer using threads and/or generators to make the code clear (single flow of control, single task for the block of code, etc) if there's no reason not to. Anyhow, I wouldn't be opposed to having a way of looking into the buffer, but it should be a separate API -- preferably with a better name than peek0(), which gives no clue at all about what it does differently from peek(). Indeed, though arguably read1() is a lousy name too, on the same basis. My itch is that peek() _feels_ like it should be "look into the buffer" but actually can block and/or change the buffer. Can block, but not if you don't want it too. You might just want to see what, if anything, is currently available, up to n bytes. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
On 16Jun2009 02:18, MRAB wrote: >> My itch is that peek() _feels_ like it should be "look into the buffer" >> but actually can block and/or change the buffer. >> > Can block, but not if you don't want it too. You might just want to see > what, if anything, is currently available, up to n bytes. Am I missing something? In the face of an _empty_ buffer (which I can't tell from outside) how do I prevent peek() blocking? More generally, if I go peek(n) and if n > bytes_in_buffer_right_now and the raw stream would block if a raw read is done? My concerns would go away if I could probe the buffer content size; then I could ensure peek(n) chose n <= the content size. If that's not enough, my problem - I can choose to read-and-block or go away and come back later. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ If all around you is darkness and you feel you're contending in vain, then the light at the end of the tunnel is the front of an oncoming train. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen()
On 14Jun2009 16:42, Mark Seaborn wrote: | I use a convenience function like this, so that GC takes care of the FDs: | | def make_pipe(): | read_fd, write_fd = os.pipe() | return os.fdopen(read_fd, "r"), os.fdopen(write_fd, "w") Not guarrenteed to be timely. The try/except at least closes things as control passes out of the relevant scope. I don't think all pythons do immediate ref-counted GC. But it's very neat! -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Trust the computer industry to shorten Year 2000 to Y2K. It was this thinking that caused the problem in the first place. - Mark Ovens ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
