Re: [Python-Dev] Adding a conditional expression in Py3.0
Guido van Rossum wrote: [Guido] Let me give you what you expect. If all the X if C else Y syntax does is prevent that atrocity from ever being introduced, it would be worth it. :) [Steve] Well, fine. However, it does allow atrocities like func(f for f in lst if f -1 if f 0 else +1) No it doesn't! Inside an 'if' (of any flavor), further ifs have to be nested. So you'd have to write func(f for f in lst if f (-1 if f 0 else +1)) or perhaps func(f for f in lst if (f -1 if f 0 else +1)) But I doubt you meant to write +1 where True could have sufficed. :) :) All that said, inside an if is hardly the best place for a conditional of any kind. Clearly such usage can go down as abuse. An if-else expression has lower priority than anything else except lambda; the expression lambda x: x if x = 0 else -x is equivalent to lambda x: (x if x = 0 else -x) That's about the only way it would make sense, I suppose. I realise that any chosen syntax is subject to abuse, but a conditional expression in a (currently allowed) conditional context will be guaranteed obscure. Your original instinct to omit conditional expressions was right! Now you've pushed me over the edge. I've made up my mind now, X if C else Y it will be. I hope to find time to implement it in Python 2.5. Let it be as controversial as bool or @decorator, I don't care. Not before time, either. If this ends the discussion then I'll consider I've done everyone a favour. Sometimes no decision is worse than the wrong decision ;-). regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 350: Codetags
Josiah Carlson wrote: Further, even if it becomes a spec, it doesn't guarantee implementation in Python editors (for which you are shooting for). Take a wander through current implementations of code tags in various editors to get a feel for what they support. I've read various posts about what code tags could support, but not what editors which implement code tags and/or its variants actually currently support; which is a better indication of what people want than an informal survey via email of python-dev, python-list, and/or the PEP submission process. An approach to this area that would make sense to me is: 1. Defer PEP 350 2. Publish a simple Python module for finding and processing code tags in a configurable fashion 3. Include a default configuration in the module that provides the behaviour described in PEP 350 4. After this hypothetical code tag processing module has been out in the wild for a while, re-open PEP 350 with an eye to including the module in the standard library The idea is that it should be possible to tailor the processing module in order to textually scan a codebase (possibly C or C++ rather than Python) in accordance with a project-specific system of code tagging, rather than requiring that the project necessarily use the default style included in the processing module (Although using a system other than the default one may result in reduced functionality, naturally). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Conditional Expression Resolution
Guido van Rossum wrote: Congratulations gracefully accepted. Hurrah! Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] linecache problem
On Fri, Sep 30, 2005 at 09:17:39AM +0200, Thomas Heller wrote: On several occasions I have seen tracebacks in my code pointing to PIL's __init__.py file. That is strange, because I have installed PIL but it is used nowhere. [skip] The bug is present in 2.3, 2.4, and current CVS. I have seen such tracebacks in all versions of Python AFAIR. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
(C) That scheduler is non-preemptive. A single greedy generator can starve all the others. Instead of looking at this as a problem, you could look at it as a feature. Since generators can't be switched at arbitrary places, the programmer has to define his/her synchronization points explicitly. Let me contrast this: - in preemptive MT, threads can switch at every moment by default, and you have to explicitly wrap some pieces of code in critical sections (or other primitives) to protect the semantics of your program; every time you forget a critical section, you introduce a bug - in cooperative MT, threads can only switch where the programmer explicitly allows it; every time you forget a synchronization point, you give your program worse performance (latency), but you *don't* introduce a bug So you have a scheme where you seek optimal performance (latency) but you take the risk of a huge number of difficult bugs, and you have a scheme where good performance needs more careful coding *but* the paradigm avoids difficult bugs /by construction/. By the way, the cooperative MT is exactly the Twisted approach, except implemented in a different manner (event loop instead of explicit cooperative tasks). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] linecache problem
Oleg Broytmann [EMAIL PROTECTED] writes: On Fri, Sep 30, 2005 at 09:17:39AM +0200, Thomas Heller wrote: On several occasions I have seen tracebacks in my code pointing to PIL's __init__.py file. That is strange, because I have installed PIL but it is used nowhere. [skip] The bug is present in 2.3, 2.4, and current CVS. I have seen such tracebacks in all versions of Python AFAIR. I think it's a severe bug that needs to be fixed. Tracebacks showing the wrong sourcelines leave the impression 'the Python installation is totally broken'. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency
On 9/30/05, Jim Jewett [EMAIL PROTECTED] wrote: Bruce Eckel wrote: 3) Tasks are cheap enough that I can make thousands of them, ... 4) Tasks are self-guarding, so they prevent other tasks from interfering with them. The only way tasks can communicate with each other is through some kind of formal mechanism (something queue-ish, I'd imagine). I think these two are the hardest to reconcile. Agreed. I think the biggest problem is that (4) is too strong. At the OS level, certain issues (such as the current directory, or the stdio streams) are per-process, so that taking (4) at face value requires multiprocessing, which violates (3)... If you constrain (4), you can get something much more effective. For example, if you accept that certain things are volatile (start with OS-specified per-process stuff, plus the Python builtins, maybe) then it's much easier to produce solutions. I don't think that shared state is that big an issue per se - after all, we're all used to the idea that the contents of a file might change behind our backs. The real issue is not having a *clearly defined* shared state. The big problem with threads is that the shared state is *everything*. There's no partitioning at all. Every thread-based abstraction is based around threads voluntarily restricting themselves to a limited set of safe communication operations, and otherwise scrupulously avoiding each other's space (Don't sit there, that's auntie Mary's chair...) If we had an abstraction (multiple interpreters, anyone?) which still had a shared state, but a much smaller one, which was defined clearly, such that unsafe operations were easy to identify, then day-to-day concurrent programming would be a lot easier. Yes, it's still possible to break things, but people can do that already by hacking about with builtins, or abusing the introspection API, and it's not made Python unusable, because they generally don't... This seems to me to be a perfect case for a Practicality beats Purity approach. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Conditional Expression Resolution
On 9/30/05, Guido van Rossum [EMAIL PROTECTED] wrote: Flames, pleas to reconsider, etc., to /dev/null. No flames from here. Congratulations gracefully accepted. Consider them supplied. For both your patience, and for supplying the decision we all desperately needed. It's still my language! :-) It always was :-) And just to prove it, we ended up with your original instinct after all... Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Conditional Expression Resolution
On Thu, 2005-09-29 at 21:21, Guido van Rossum wrote: Flames, pleas to reconsider, etc., to /dev/null. Congratulations gracefully accepted. It's still my language! :-) Congratulations! May this be as successful a syntax addition as decorators and print (of which I'm a fan too! :). -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
(C) That scheduler is non-preemptive. A single greedy generator can starve all the others. Antoine Instead of looking at this as a problem, you could look at it Antoine as a feature. Apple looked at it as a feature for years. Not anymore. wink Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency
Support for multiple interpreters already existsfrom the C API (mod_python, Java Embedded Python a few other add-ons use them) But: - it's not possible to createnew interpreter instancesfrom within Python. - there's no mechanism for passing information between interpreters. -interaction withextension modules instancesmay be a problem. Apart fromthese points they actually seem to work pretty well and it might be, as you suggest, a Practical approach. Implementing a 'subinterp' module could be interesting... Max On 9/30/05, Paul Moore [EMAIL PROTECTED] wrote: On 9/30/05, Jim Jewett [EMAIL PROTECTED] wrote: Bruce Eckel wrote: 3) Tasks are cheap enough that I can make thousands of them, ... 4) Tasks are self-guarding, so they prevent other tasks from interfering with them. The only way tasks can communicate with each other is through some kind of formal mechanism (something queue-ish, I'd imagine). I think these two are the hardest to reconcile. Agreed. I think the biggest problem is that (4) is too strong. At theOS level, certain issues (such as the current directory, or the stdiostreams) are per-process, so that taking (4) at face value requires multiprocessing, which violates (3)...If you constrain (4), you can get something much more effective. Forexample, if you accept that certain things are volatile (start withOS-specified per-process stuff, plus the Python builtins, maybe) then it's much easier to produce solutions.I don't think that shared state is that big an issue per se - afterall, we're all used to the idea that the contents of a file mightchange behind our backs. The real issue is not having a *clearly defined* shared state.The big problem with threads is that the shared state is *everything*.There's no partitioning at all. Every thread-based abstraction isbased around threads voluntarily restricting themselves to a limited set of safe communication operations, and otherwise scrupulouslyavoiding each other's space (Don't sit there, that's auntie Mary'schair...)If we had an abstraction (multiple interpreters, anyone?) which still had a shared state, but a much smaller one, which was defined clearly,such that unsafe operations were easy to identify, then day-to-dayconcurrent programming would be a lot easier. Yes, it's still possible to break things, but people can do that already by hacking about withbuiltins, or abusing the introspection API, and it's not made Pythonunusable, because they generally don't...This seems to me to be a perfect case for a Practicality beats Purity approach.Paul.___Python-Dev mailing listPython-Dev@python.org http://mail.python.org/mailman/listinfo/python-devUnsubscribe: http://mail.python.org/mailman/options/python-dev/anothermax%40gmail.com -- flickr: http://www.flickr.com/photos/anothermax/sets ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
Le vendredi 30 septembre 2005 à 08:32 -0500, [EMAIL PROTECTED] a écrit : (C) That scheduler is non-preemptive. A single greedy generator can starve all the others. Antoine Instead of looking at this as a problem, you could look at it Antoine as a feature. Apple looked at it as a feature for years. Not anymore. wink You are missing the context: - at the OS level, it is not good to have cooperative MT because any badly-written app (there are lots of ;-)) can cause lack of responsiveness for the whole machine - at the application level, cooperative MT can be good to enforce robust semantics without disrupting other apps The latter is what we are discussing AFAIK: a different concurrency scheme inside a single application - not accross the whole OS or desktop. Actually, I think different concurrency schemes must be chosen and mixed depending on the semantics. For example, when you have a networked GUI app, it can be good to have the network in one preemptive thread (using e.g. Twisted) and the GUI in another preemptive thread (using GTK, wx...). This is because GUI and network have different latency characteristics and requirements. (of course, you can replace preemptive thread with process in the above description) So whatever innovatice concurrency scheme Python may come out, it should still be mixable with more traditional concurrency schemes, because required properties vary wildly even inside a single app. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 350: Codetags
On 9/30/05, Nick Coghlan [EMAIL PROTECTED] wrote: An approach to this area that would make sense to me is: 1. Defer PEP 350 2. Publish a simple Python module for finding and processing code tags in a configurable fashion 3. Include a default configuration in the module that provides the behaviour described in PEP 350 4. After this hypothetical code tag processing module has been out in the wild for a while, re-open PEP 350 with an eye to including the module in the standard library The idea is that it should be possible to tailor the processing module in order to textually scan a codebase (possibly C or C++ rather than Python) in accordance with a project-specific system of code tagging, rather than requiring that the project necessarily use the default style included in the processing module (Although using a system other than the default one may result in reduced functionality, naturally). Maybe I'm just an old fart, but this all seems way over-engineered. Even for projects the size of Python, a simple grep+find is sufficient. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Conditional Expression Resolution
Barry Warsaw wrote: On Thu, 2005-09-29 at 21:21, Guido van Rossum wrote: Flames, pleas to reconsider, etc., to /dev/null. Congratulations gracefully accepted. It's still my language! :-) Congratulations! May this be as successful a syntax addition as decorators and print (of which I'm a fan too! :). I have to say, now Guido is getting rid of print along with the rest of the print statement I have become nostalgically fond of it. Perhaps it's just that I can't help siding with the underdog, doomed for the knacker's yard in Python 3.0. Sniff. The decision on X if C else Y is a cause for much jubilation. Congratulations, Guido! nothing-to-see-here-folks-let's-move-along-ly y'rs - steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] linecache problem
On 9/30/05, Thomas Heller [EMAIL PROTECTED] wrote: On several occasions I have seen tracebacks in my code pointing to PIL's __init__.py file. That is strange, because I have installed PIL but it is used nowhere. Finally I traced it down to a problem in the linecache code, which tries to be smart in up updatecache function. When os.stat() on the filename fails with os.error, it walks along sys.path and returns the first file with a matching basename. This *may* make sense for toplevel modules, but never for modules in packages. So, if the traceback's stack contains an entry for a non-existing file (for example because the .py file for a .pyc file is no longer present), linecache returns absolute garbage. Example, on my system (the \a\b\c\__init__.py file doesn't exist): C:\python -c import linecache; print linecache.getlines(r'\a\b\c\__init__.py') ['#\n', '# The Python Imaging Library.\n', '# $Id: //modules/pil/PIL/__init__.py#2 $\n', '#\n', '# package placeholder\n', '#\n', '# Copyright (c) 1999 by Secret Labs AB.\n', '#\n', '# See the README file for information on usage and redistribution.\n', '#\n', '\n', '# ;-)\n'] C:\ The bug is present in 2.3, 2.4, and current CVS. Probably my fault (I wrote linecache, 13 years ago (before Python had packages!). But looking at the code I don't understand why this is; there's no code to descend into subdirectories of directories found on sys.path. I wonder if the problem isn't on PIL's end, which puts the PIL directory on sys.path? Anyway, don't hesitate to suggest a patch on sourceforge -- python-dev really isn't the forum for further discussion of this issue. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
On 9/30/05, Antoine Pitrou [EMAIL PROTECTED] wrote: (C) That scheduler is non-preemptive. A single greedy generator can starve all the others. Instead of looking at this as a problem, you could look at it as a feature. Since generators can't be switched at arbitrary places, the programmer has to define his/her synchronization points explicitly. I use this approach extensively, using tasks which are defined using generators. The scheduler I developed for this can be viewed here: http://metaplay.dyndns.org:82/svn/nanothreads/nanothreads.py Synchronization using yield statements is an important feature, as I use these cooperative threads for game development. These threads are _very_ useful for games, as they have very low overhead, and allow the programmer to exercise more control over their execution. Sw. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
On 9/30/05, Antoine Pitrou [EMAIL PROTECTED] wrote: Le vendredi 30 septembre 2005 à 08:32 -0500, [EMAIL PROTECTED] a écrit : (C) That scheduler is non-preemptive. A single greedy generator can starve all the others. Antoine Instead of looking at this as a problem, you could look at it Antoine as a feature. Apple looked at it as a feature for years. Not anymore. wink You are missing the context: - at the OS level, it is not good to have cooperative MT because any badly-written app (there are lots of ;-)) can cause lack of responsiveness for the whole machine - at the application level, cooperative MT can be good to enforce robust semantics without disrupting other apps The latter is what we are discussing AFAIK: a different concurrency scheme inside a single application - not accross the whole OS or desktop. I like this analysis. Actually, I think different concurrency schemes must be chosen and mixed depending on the semantics. For example, when you have a networked GUI app, it can be good to have the network in one preemptive thread (using e.g. Twisted) and the GUI in another preemptive thread (using GTK, wx...). This is because GUI and network have different latency characteristics and requirements. (of course, you can replace preemptive thread with process in the above description) I'm skeptical of this. I think a traditional GUI+network app can combine the network and UI tasks in one process -- usually they interact somewhat and that's where the bugs show up. Also note that this is not an application area where MP is very interesting -- most of the time you're blocked waiting for the user or for a socket (or likely both :) so a single CPU is all you need. I've never heard someone complain that the GIL is in the way for these types of apps. So whatever innovatice concurrency scheme Python may come out, it should still be mixable with more traditional concurrency schemes, because required properties vary wildly even inside a single app. I don't think you've proved that yet. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Conditional Expression Resolution
[Guido] After a long discussion I've decided to add a shortcut conditional expression to Python 2.5. The syntax will be A if C else B ... The priorities will be such that you can write ... x = A if C else B if D else E I assume this groups as A if C else (B if D else E) rather than as (A if C else B) if D else E ? So that C is evaluated first, and if C is true D isn't evaluated at all. ... Flames, pleas to reconsider, etc., to /dev/null. Compared to the postfix A then C if B it's a positive delight. Congratulations gracefully accepted. Congratulations gracefully tendered. It's still my language! :-) If that was in doubt, you just proved it wink. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] linecache problem
Guido van Rossum [EMAIL PROTECTED] writes: On 9/30/05, Thomas Heller [EMAIL PROTECTED] wrote: On several occasions I have seen tracebacks in my code pointing to PIL's __init__.py file. That is strange, because I have installed PIL but it is used nowhere. Finally I traced it down to a problem in the linecache code, which tries to be smart in up updatecache function. When os.stat() on the filename fails with os.error, it walks along sys.path and returns the first file with a matching basename. This *may* make sense for toplevel modules, but never for modules in packages. The bug is present in 2.3, 2.4, and current CVS. Probably my fault (I wrote linecache, 13 years ago (before Python had packages!). But looking at the code I don't understand why this is; there's no code to descend into subdirectories of directories found on sys.path. I wonder if the problem isn't on PIL's end, which puts the PIL directory on sys.path? It seems PIL cannot decide if it wants to be a package or not, but better would be to ask the author. /F, ? It installs a PIL.pth file in lib/site-packages, which contains 'PIL' only - that's where the sys.path entry comes from. OTOH, the lib/site-packages/PIL directory contains an __init__.py file. PIL.pth is the only .pth file that I have where the directory contains an __init__.py file. Anyway, don't hesitate to suggest a patch on sourceforge -- python-dev really isn't the forum for further discussion of this issue. https://sourceforge.net/tracker/index.php?func=detailaid=1309567group_id=5470atid=105470 Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding a conditional expression in Py3.0
I did this for my favorite proposal, and ended up with the list shown further down below. I think they all looks great! The fact that so few were found in whole of the standard library does put the use case into question, though, no? Though I am sure more could be found with a more thorough scan. There's lots of code in it that looks like this: def __init__(self, foo = None): if foo is not None: self.foo = foo else: self.foo = self.getFooDefault() With if/else it can be written: def __init__(self, foo = None): self.foo = foo if foo is not None else self.getFooDefault() However, I'm convinced that the latter version is less readable than the former. Often the line becomes more than 80 characters so there's no gain in translating it to an if-else expression because you would still have to split the line, other times the condition is so complex that it deserves its own line. Many opportunities for uses, but atleast as many for abuses. -- mvh Björn ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
Hi, I've never heard someone complain that the GIL is in the way for these types of apps. I've never said so either. I was just saying that it can be useful to mix cooperative threading and preemptive threading in the same app, i.e. have different domains of cooperative threading which are preemptively scheduled by the OS. That has nothing to do with the GIL, I think (but I don't know much in Python internals). For instance the mix between Twisted and GUI event loops is a very common topic on the twisted mailing-list, because people have trouble getting it right inside a single system thread. The Twisted people have started advocating something called the ThreadedSelectReactor, which I haven't looked at but whose name suggests that some operations are performed in a helper system thread. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] bool(container) [was bool(iter([])) changed between 2.3 and 2.4]
Python doesn't worry about a precise boolean object, it distinguishes between something and nothing. Is there anything left? is a pretty good analogy for iterators. It isn't always cheaply available, and having might encourage poor style -- so iterators are going back to the default for non-containers of always True. How general strong is this default-to-true rule? If I submit a documentation patch, should I say that numbers, lists, strings, dictionaries, and tuples are a special case, or should I just warn that some container-like objects (including iterators) are always True? More specific question: A Queue.Queue is always true. Should I submit a bug patch or add it as a beware of custom classes example? def __nonzero__(self): return not self.empty() ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency
On 9/30/05, Jeremy Maxfield [EMAIL PROTECTED] wrote: Support for multiple interpreters already exists from the C API (mod_python, Java Embedded Python a few other add-ons use them) I'm aware of that (didn't I mention it in my message - sorry). But: - it's not possible to create new interpreter instances from within Python. Yet... - there's no mechanism for passing information between interpreters. But one could possibly be added when exposing the functionality to Python. - interaction with extension modules instances may be a problem. This is the issue I have heard mentioned before as the problematic one. Unfortunately, I don't know enough about the problems to be able to comment. If it's as simple as saying that any global state maintained by an extension module is part of the shared state, then I'm OK with that (although it would be nice to encourage extension authors to document their global state a bit more clearly :-)) Actually, it might be necessary to use global extension state (and the fact that it's shared) to implement inter-interpreter communication... Apart from these points they actually seem to work pretty well and it might be, as you suggest, a Practical approach. Implementing a 'subinterp' module could be interesting... Indeed. Maybe I'll have a go (in my copious free time :-)) Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] C API doc fix
On 9/29/05, Robey Pointer robey at lag.net wrote: Yesterday I ran into a bug in the C API docs. The top of this page: http://docs.python.org/api/unicodeObjects.html says: Py_UNICODE This type represents a 16-bit unsigned storage type which is used by Python internally as basis for holding Unicode ordinals. On platforms where wchar_t is available and also has 16-bits, Py_UNICODE is a typedef alias for wchar_t to enhance native platform compatibility. On all other platforms, Py_UNICODE is a typedef alias for unsigned short. Steven Bethard wrote: I believe this is the same issue that was brought up in May[1]. My impression was that people could not agree on a documentation patch. [1] http://www.python.org/dev/summary/2005-05-01_2005-05-15.html I thought the problem was disagreement over how the system *should* pick an underlying type to alias. Given the current policy, are there objections to a patch that at least steers people away from assuming they can use the underlying type directly? Py_UNICODE Python uses this type to store Unicode ordinals. It is typically a typedef alias, but the underlying type -- and the size of that type -- varies across different systems. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding a conditional expression in Py3.0
Guido: Now you've pushed me over the edge. I've made up my mind now, X if C else Y it will be. I hope to find time to implement it in Python 2.5. Steve Holden: Not before time, either. If this ends the discussion then I'll consider I've done everyone a favour. Sometimes no decision is worse than the wrong decision ;-). Exactly how I felt about bringing the decorator decision to a close. ;) Congratulations to you both! Robert Brewer System Architect Amor Ministries [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bool(container) [was bool(iter([])) changed between 2.3 and 2.4]
On 9/30/05, Jim Jewett [EMAIL PROTECTED] wrote: If I submit a documentation patch, should I say that numbers, lists, strings, dictionaries, and tuples are a special case, or should I just warn that some container-like objects (including iterators) are always True? You seem to be going at this from the wrong direction. Boolean value are defined by calling __nonzero__ or __len__, whichever exists; if neither exists the answer is true (except for None which is special-cased only for historical reasons -- there's no reason why it couldn't have a __nonzero__ method. __nonzero__ is intended for object types that either want to have number-like behavior or have a special reason for wanting to act like a Boolean. __len__ is for sequences and mappings specifically -- everything that supports __getitem__ should have __len__ and everything that has __len__ should have __getitem__. (This is what broke for iterators in 2.4.) So if anything's an exception, it's numbers -- strings, lists, tuples are sequences and dicts are mappings, and that's where they get their definition of Booleanness from. Always remember, user-defined classes can define __nonzero__ any way they wish, and they get what they deserve. Library designers however should try to follow established patterns. Testing for Queue emptiness via __nonzero__ seems unwarranted since a Queue doesn't have any other sequence behavior. Containerish behavior isn't enough to warrant empty -- false; in some sense every object is a container (at least every object with a __dict__ attribute) and you sure don't want to map __len__ to self.__dict__.__len__... -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
On Fri, 30 Sep 2005 17:26:27 +0200, Antoine Pitrou [EMAIL PROTECTED] wrote: Hi, I've never heard someone complain that the GIL is in the way for these types of apps. I've never said so either. I was just saying that it can be useful to mix cooperative threading and preemptive threading in the same app, i.e. have different domains of cooperative threading which are preemptively scheduled by the OS. That has nothing to do with the GIL, I think (but I don't know much in Python internals). For instance the mix between Twisted and GUI event loops is a very common topic on the twisted mailing-list, because people have trouble getting it right inside a single system thread. The Twisted people have started advocating something called the ThreadedSelectReactor, which I haven't looked at but whose name suggests that some operations are performed in a helper system thread. Advocating might be putting it too strongly :) Experimenting with describes the current state of things most accurately. The problem it aims to solve is integration with cooperative threading systems which don't work very well. An example of such a loop is the wx event loop. Whenever a modal dialog is displayed or a menu is activated, wx's loop stops cooerating. The only way we've found thus far to avoid this is to run wx in a separate thread, so even when it stops cooperating, network code can still get scheduled. Jp ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bool(container) [was bool(iter([])) changedbetween 2.3 and 2.4]
Jim Jewett writes:
Python doesn't worry about a precise boolean object, it
distinguishes between something and nothing.
Is there anything left? is a pretty good analogy for iterators.
[...]
A Queue.Queue is always true. Should I submit a bug patch
I would have phrased this very differently. I would have said
it as follows:
All Python objects can be converted to a boolean value. Most
objects are True, except for False (of course), None, any number
equal to 0, an empty string (), and the empty containers
(), [], and {}. User defined objects can override __len__ or
__nonzero__ to return False; everything else is True.
I don't believe that there is an overarching rule that all containers
must be false when empty. I'm not sure there is even a clear
definition of container that would satisfy everyone. It makes
complete sense to me that empty iterators and Queue.Queues are True...
it follows the general rule that everything is true except the above
(short) list of objects and user-defined (or library) classes that
want to mimic the behavior of one of these or have some obvious
meaning for non-true.
-- Michael Chermside
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bool(container) [was bool(iter([])) changedbetween 2.3 and 2.4]
[Guido van Rossum] __len__ is for sequences and mappings specifically -- everything that supports __getitem__ should have __len__ and everything that has __len__ should have __getitem__. That's going a bit far. Unordered collections (like sets and bags) are a good counter-example. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bool(container) [was bool(iter([])) changedbetween 2.3 and 2.4]
Guido van Rossum wrote: Containerish behavior isn't enough to warrant empty -- false; in some sense every object is a container (at least every object with a __dict__ attribute) and you sure don't want to map __len__ to self.__dict__.__len__... the ElementTree experience shows that doing def __getitem__(self, index): return self.items[index] def __len__(self): return len(self.items) def __nonzero__(self): return True is sometimes a good idea, even for objects that are mostly sequences. (ET 1.2.X lacks the __nonzero__ method, and accidentally treating elements with no subelements as if the element itself doesn't exist is by far the most common gotcha in ET code). /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
Hi Jp, Le vendredi 30 septembre 2005 à 12:20 -0400, Jp Calderone a écrit : Advocating might be putting it too strongly :) Experimenting with describes the current state of things most accurately. Ok :) The problem it aims to solve is integration with cooperative threading systems which don't work very well. An example of such a loop is the wx event loop. Whenever a modal dialog is displayed or a menu is activated, wx's loop stops cooerating. This specific problem hides the more general problem, which is that GUI and network activities have different typical latencies. As I explained on the twisted ML, a GUI needs very good response times to feel friendly (typically below 20 or even 10 ms.), whereas some network protocols have non-trivial calculations which can go above 100 ms. Moreover, you don't want a sudden flood of network events to block events in the GUI. This is why even without considering wx's specificities, it is still useful to keep GUI and network activities in separate threads. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
I was just saying that it can be useful to mix cooperative threading and preemptive threading in the same app, i.e. have different domains of cooperative threading which are preemptively scheduled by the OS. That has nothing to do with the GIL, I think (but I don't know much in Python internals). This is one of the interesting questions I'd like to answer (but which probably requires some kind of mathematician to prove one way or another): Does a complete concurrency solution require both a preemptive task management system and a cooperative one? OS Processes are generally quite limited in number (although I understand that BeOS was designed to create tons of lightweight processes, which changed that particular picture). In Java, you can usually create several hundred threads before it craps out. Simulations and games could easily have thousands or even hundreds of thousands of independent simulation units. If there are limits to the number of concurrency drivers (threads, etc.), then that would suggest that at some point you have to move to a cooperative system. Which would suggest that a complete concurrency solution might require both. That wouldn't be my ideal. My ideal would be a single solution that would scale up to large numbers of concurrency units, and that wouldn't require the programmer to remember to explicitly yield control. Whether my ideal is possible is a question I'd like to answer. Bruce Eckelhttp://www.BruceEckel.com mailto:[EMAIL PROTECTED] Contains electronic books: Thinking in Java 3e Thinking in C++ 2e Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel Subscribe to my newsletter: http://www.mindview.net/Newsletter My schedule can be found at: http://www.mindview.net/Calendar ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bool(container) [was bool(iter([])) changedbetween 2.3 and 2.4]
On 9/30/05, Raymond Hettinger [EMAIL PROTECTED] wrote: [Guido van Rossum] __len__ is for sequences and mappings specifically -- everything that supports __getitem__ should have __len__ and everything that has __len__ should have __getitem__. That's going a bit far. Unordered collections (like sets and bags) are a good counter-example. I've implemented many extension types in C, always to interface external libraries (the main examples, MPI and PETSc). In those libraries, some object are invalid (MPI_COMM_NULL) or can get deallocated in the C side. I always implement a __nonzero__ method for testing those situations. It sounds a bit bizarre to me to have to implement a number method for a object that is far from being a number, and of course __len__ do not make any sense. Perhaps Py3k could have a more explicit mechanism for definig the truth value of an object? Something like __bool__ method to be filled in type object? Any opinions? Lisandro Dalcín ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Conditional Expression Resolution
On 9/29/05, Guido van Rossum [EMAIL PROTECTED] wrote: [SNIP] Flames, pleas to reconsider, etc., to /dev/null. Congratulations gracefully accepted. And gladly given! All proposals should be resolved this cleanly in the end. It's still my language! :-) Yes it is, thank goodness! -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
On Fri, 2005-09-30 at 18:33 +0200, Antoine Pitrou wrote: Hi Jp, Le vendredi 30 septembre 2005 à 12:20 -0400, Jp Calderone a écrit : Advocating might be putting it too strongly :) Experimenting with describes the current state of things most accurately. Ok :) The problem it aims to solve is integration with cooperative threading systems which don't work very well. An example of such a loop is the wx event loop. Whenever a modal dialog is displayed or a menu is activated, wx's loop stops cooerating. That is wx's problem. Try PyGTK some day; I hear it's really good! ;-) This specific problem hides the more general problem, which is that GUI and network activities have different typical latencies. As I explained on the twisted ML, a GUI needs very good response times to feel friendly (typically below 20 or even 10 ms.), whereas some network protocols have non-trivial calculations which can go above 100 ms. With PyGTK a typical solution for this is to use a generator function executing an idle function, which would make the non-trivial calculations, but yield control back to the main loop periodically, in order to process GUI events. For example, see the last code block in http://www.async.com.br/faq/pygtk/index.py?req=showfile=faq23.020.htp Moreover, you don't want a sudden flood of network events to block events in the GUI. Process one event at a time, after each event give back control to the main loop, and give low priority to socket IO events. Problem solved. This is why even without considering wx's specificities, it is still useful to keep GUI and network activities in separate threads. You are considering a scenario that seldom happens to design a solution that is far too complicated for most cases. Regards. -- Gustavo J. A. M. Carneiro [EMAIL PROTECTED] [EMAIL PROTECTED] The universe is always one step beyond logic ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Active Objects in Python
[ I don't post often, but hopefully the following is of interest in this discussion ] Bruce Eckel wrote: Yes, defining an class as active would: 1) Install a worker thread and concurrent queue in each object of that class. 2) Automatically turn method calls into tasks and enqueue them 3) Prevent any other interaction other than enqueued messages Here I think the point is an object that has some form of thread with enforced communications preventing (or limiting) shared data. This essentially describes the project we've been working on with Kamaelia. Kamaelia is divided into two halves: * A simple framework (Axon) for building components with the following characteristics: * Have a main() method which is a generator. * Have inbound and outbound named queues implemented using lists. (akin to stdin/out) * Access to an environmental system which is key/value lookup (similar purpose to the unix environment). (This could easily grow into a much more linda like system) * A collection of components which form a library. We also have a threaded component which uses a thread and queues rather than a generator lists. With regard to the criteria you listed in a later post: 1) Has been tested against (so far) 2 novice programmers, who just picked up the framework and ran with in. (learnt python one week, axon the next, then implemented interesting things rapidly) 2) We don't deal with distribution (yet), but this is on the cards. Probably before EuroOSCON. (I want to have a program with two top level pygame windows) 3) We use python generators and have experimented initially with several tens of thousands of them, which works suprisingly well, even on an old 500Mhz machine. 4) Self guarding is difficult. However generators are pretty opaque, and getting at values inside their stack frame is beyond my python skills. 5) Deadlock can be forced in all scenarios, but you would actually have to work at it. In practice if we yield at all points or use the threaded components for everything then we'd simply livelock. 6) Regarding an actor. Our pygame sprite component actually is very, very actor like. This is to the extent we've considered implementing a script reading and director component for orchestrating them. (Time has prevented that though) 7) Since our target has been novices, this has come by default. 8) We don't do mobility of components/tasks as yet, however a colleague at work interested in mobility agent software has proposed looking at that using Kamaelia for the coming year. We also ran the framework by people at the WoTUG (*) conference recently, and the general viewpoint came back that it's tractable from a formal analysis and mobility perspective(if we change x,y,z). (*) http://www.wotug.org/ I wrote a white paper based on my Python UK talk, which is here: * http://www.bbc.co.uk/rd/pubs/whp/whp11.shtml (BBC RD white papers are aimed at imparting info in a such a way that it's useful and readable, unlike some white papers :) So far we have components for building things from network servers through to audio/video decoders and players, and presentation tools. (Using pygame and tkinter) Also we have a graphical editting tool for putting together systems and introspecting systems. An aim in terms of API was to enable novices to implement a microcosm version of the system. Specifically we tested this idea on a pre-university trainee who built a fairly highly parallel system which took video from an mpg file (simulating a PVR), and sent periodic snapshots to a mobile phone based client. He had minimal programming experience at the start, and implemented this over a 3 months period. We've since repeated that experience with a student who has had 2 years at university, but no prior concurrency, python or networks experience. In his 6 weeks with us he was able to do some interesting things. (Such as implement a system for joining multicast islands and sending content over a simple reliable multicast protocol). The tutorial we use to get people up to speed rapidly is here: * http://kamaelia.sourceforge.net/MiniAxon/ However that doesn't really cover the ideas of how to write components, use pipelines, graphlines and services. At the moment we're adding in the concept of a chassis into which you plug in existing components. The latest concept we're bouncing round is this: # (This was fleshed out in respect to a question by a novice on c.l.p - # periodically send the the same message to all connected clients) # start from Kamaelia.Chassis.ConnectedServer import SimpleServer from Kamaelia.Chassis.Splitter import splitter, publish, Subscriber class message_source(component): [ ... snippety ... ] def main(self): last = self.scheduler.time while 1: yield 1 if
Re: [Python-Dev] PEP 350: Codetags
Guido van Rossum wrote: On 9/30/05, Nick Coghlan [EMAIL PROTECTED] wrote: An approach to this area that would make sense to me is: 1. Defer PEP 350 2. Publish a simple Python module for finding and processing code tags in a configurable fashion 3. Include a default configuration in the module that provides the behaviour described in PEP 350 4. After this hypothetical code tag processing module has been out in the wild for a while, re-open PEP 350 with an eye to including the module in the standard library The idea is that it should be possible to tailor the processing module in order to textually scan a codebase (possibly C or C++ rather than Python) in accordance with a project-specific system of code tagging, rather than requiring that the project necessarily use the default style included in the processing module (Although using a system other than the default one may result in reduced functionality, naturally). Maybe I'm just an old fart, but this all seems way over-engineered. Even for projects the size of Python, a simple grep+find is sufficient. I expect many people would agree with you, but Micah was interested enough in the area to write a PEP about it. The above was just a suggestion for a different way of looking at the problem, so that writing a PEP would actually make sense. At the moment, if the tags used are project-specific, and the method used to find them is a simple grep+find, then I don't see a reason for the idea to be a *Python* Enhancement Proposal. Further, I see some interesting possibilities for automation if such a library exists. For example, a cron job that scans the checked in sources, and automatically converts new TODO's to RFE's in the project tracker, and adds a tracker cross-link into the source code comment. The job could similarly create bug reports for FIXME's. If the project tracker was one that supported URL links, and the project had a URL view of the source tree, then the cross-links between the code tag and the tracker could be actual URL references to each other. However, the starting point for exploring any such ideas would be a library that made it easier to work with code tags. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 350: Codetags
On 9/30/05, Nick Coghlan [EMAIL PROTECTED] wrote: Further, I see some interesting possibilities for automation if such a library exists. For example, a cron job that scans the checked in sources, and automatically converts new TODO's to RFE's in the project tracker, and adds a tracker cross-link into the source code comment. The job could similarly create bug reports for FIXME's. If the project tracker was one that supported URL links, and the project had a URL view of the source tree, then the cross-links between the code tag and the tracker could be actual URL references to each other. With all respect for the OP, that's exactly the kind of enthusiastic over-engineering that I'm afraid the PEP will encourage. I seriously doubt that any of that work will contribute towards a project's success (compared to simply having a convention of putting XXX in the code). -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
