Re: [Python-Dev] Google Summer of Code proposal: New class for work with binary trees AVL and RB as with the standard dictionary.
Josiah Carlson wrote: There exists various C and Python implementations of both AVL and Red-Black trees. For users of Python who want to use AVL and/or Red-Black trees, I would urge them to use the Python implementations. In the case of *needing* the speed of a C extension, there already exists a CPython extension module for AVL trees: http://www.python.org/pypi/pyavl/1.1 I would suggest you look through some suggested SoC projects in the wiki: http://wiki.python.org/moin/SummerOfCode - Josiah Thanks for the answer! I already saw pyavl-1.1. But for this reason I would like to see the module in a standard package python. Functionality for pyavl and dict to compare difficultly. Functionality of my module will differ from functionality dict in the best party. I have lead some tests on for work with different types both for a package pyavl-1.1, and for the prototype of own module. The script of check is resulted in attached a file avl-timeit.py In files timeit-result-*-*.txt results of this test. The first in the name of a file means quantity of the added elements, the second - argument of a method timeit. There it is visible, that in spite of the fact that the module xtree is more combined in comparison with pyavl the module (for everyone again inserted pair [the key, value], is created two elements: python object - pair, and an internal element of a tree), even in this case it works a little bit more quickly. Besides the module pyavl is unstable for work in multithread appendices (look the attached file module-avl-is-thread-unsafe.py). I think, that creation of this type (together with type of pair), will make programming more convenient since sorting by function sort will be required less often. I can probably borrow in this module beyond the framework of the project google. The demand of such type of data is interesting. Because of necessity of processing `gcmodule.c' and `obmalloc.c' this module cannot be realized as the external module. #!/usr/local/bin/python # Initialize modules import avl import xtree import types import sys import timeit if len(sys.argv) != 3: iterations = 100 count = 10 else: iterations = int(sys.argv[1]) count = int(sys.argv[2]) print Iterations, iterations print Repeats, count print result_avl = timeit.Timer( import avl a = avl.new() for i in xrange(%d): a.insert(i) % iterations).timeit(count) print object avl.new(), result_avl result_xtree = timeit.Timer( import xtree a = xtree.xtree() for i in xrange(%d): a[i] = True % iterations).timeit(count) print object xtree.xtree(), result_xtree result_dict = timeit.Timer( import types a = dict() for i in xrange(%d): a[i] = True % iterations).timeit(count) print object dict(), result_dict print Dict Xtree AVL print Dict %.2f %.2f %.2f % (float(result_dict)/result_dict, float(result_dict)/result_xtree, float(result_dict)/result_avl) print Xtree %.2f %.2f %.2f % (float(result_xtree)/result_dict, float(result_xtree)/result_xtree, float(result_xtree)/result_avl) print AVL %.2f %.2f %.2f % (float(result_avl)/result_dict, float(result_avl)/result_xtree, float(result_avl)/result_avl) fox:root!~/Downloads/Python/PYTHON/pyavl-1.1 python setup.py install running install running build running build_ext building 'avl' extension creating build creating build/temp.freebsd-5.4-RELEASE-i386-2.4 cc -fno-strict-aliasing -DNDEBUG -O -pipe -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x2 -O2 -fPIC -DHAVE_AVL_VERIFY -DAVL_FOR_PYTHON -I/usr/local/include/python2.4 -c avl.c -o build/temp.freebsd-5.4 -RELEASE-i386-2.4/avl.o -O2 -Wno-parentheses -Wno-uninitialized cc -fno-strict-aliasing -DNDEBUG -O -pipe -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x2 -O2 -fPIC -DHAVE_AVL_VERIFY -DAVL_FOR_PYTHON -I/usr/local/include/python2.4 -c avlmodule.c -o build/temp.freeb sd-5.4-RELEASE-i386-2.4/avlmodule.o -O2 -Wno-parentheses -Wno-uninitialized creating build/lib.freebsd-5.4-RELEASE-i386-2.4 cc -shared -pthread -O2 build/temp.freebsd-5.4-RELEASE-i386-2.4/avl.o build/temp.freebsd-5.4-RELEASE -i386-2.4/avlmodule.o -o build/lib.freebsd-5.4-RELEASE-i386-2.4/avl.so -Wl,-x running install_lib copying build/lib.freebsd-5.4-RELEASE-i386-2.4/avl.so - /usr/local/lib/python2.4/site-packages ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Google Summer of Code proposal: New class for work with binary trees AVL and RB as with the standard dictionary.
Josiah Carlson wrote: There exists various C and Python implementations of both AVL and Red-Black trees. For users of Python who want to use AVL and/or Red-Black trees, I would urge them to use the Python implementations. In the case of *needing* the speed of a C extension, there already exists a CPython extension module for AVL trees: http://www.python.org/pypi/pyavl/1.1 I would suggest you look through some suggested SoC projects in the wiki: http://wiki.python.org/moin/SummerOfCode - Josiah Thanks for the answer! I already saw pyavl-1.1. But for this reason I would like to see the module in a standard package python. Functionality for pyavl and dict to compare difficultly. Functionality of my module will differ from functionality dict in the best party. I have lead some tests on for work with different types both for a package pyavl-1.1, and for the prototype of own module. The script of check is resulted in attached a file avl-timeit.py In files timeit-result-*-*.txt results of this test. The first in the name of a file means quantity of the added elements, the second - argument of a method timeit. There it is visible, that in spite of the fact that the module xtree is more combined in comparison with pyavl the module (for everyone again inserted pair [the key, value], is created two elements: python object - pair, and an internal element of a tree), even in this case it works a little bit more quickly. Besides the module pyavl is unstable for work in multithread appendices (look the attached file module-avl-is-thread-unsafe.py). I think, that creation of this type (together with type of pair), will make programming more convenient since sorting by function sort will be required less often. I can probably borrow in this module beyond the framework of the project google. The demand of such type of data is interesting. Because of necessity of processing `gcmodule.c' and `obmalloc.c' this module cannot be realized as the external module. #!/usr/local/bin/python # Initialize modules import avl import xtree import types import sys import timeit if len(sys.argv) != 3: iterations = 100 count = 10 else: iterations = int(sys.argv[1]) count = int(sys.argv[2]) print Iterations, iterations print Repeats, count print result_avl = timeit.Timer( import avl a = avl.new() for i in xrange(%d): a.insert(i) % iterations).timeit(count) print object avl.new(), result_avl result_xtree = timeit.Timer( import xtree a = xtree.xtree() for i in xrange(%d): a[i] = True % iterations).timeit(count) print object xtree.xtree(), result_xtree result_dict = timeit.Timer( import types a = dict() for i in xrange(%d): a[i] = True % iterations).timeit(count) print object dict(), result_dict print Dict Xtree AVL print Dict %.2f %.2f %.2f % (float(result_dict)/result_dict, float(result_dict)/result_xtree, float(result_dict)/result_avl) print Xtree %.2f %.2f %.2f % (float(result_xtree)/result_dict, float(result_xtree)/result_xtree, float(result_xtree)/result_avl) print AVL %.2f %.2f %.2f % (float(result_avl)/result_dict, float(result_avl)/result_xtree, float(result_avl)/result_avl) fox:root!~/Downloads/Python/PYTHON/pyavl-1.1 python setup.py install running install running build running build_ext building 'avl' extension creating build creating build/temp.freebsd-5.4-RELEASE-i386-2.4 cc -fno-strict-aliasing -DNDEBUG -O -pipe -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x2 -O2 -fPIC -DHAVE_AVL_VERIFY -DAVL_FOR_PYTHON -I/usr/local/include/python2.4 -c avl.c -o build/temp.freebsd-5.4 -RELEASE-i386-2.4/avl.o -O2 -Wno-parentheses -Wno-uninitialized cc -fno-strict-aliasing -DNDEBUG -O -pipe -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x2 -O2 -fPIC -DHAVE_AVL_VERIFY -DAVL_FOR_PYTHON -I/usr/local/include/python2.4 -c avlmodule.c -o build/temp.freeb sd-5.4-RELEASE-i386-2.4/avlmodule.o -O2 -Wno-parentheses -Wno-uninitialized creating build/lib.freebsd-5.4-RELEASE-i386-2.4 cc -shared -pthread -O2 build/temp.freebsd-5.4-RELEASE-i386-2.4/avl.o build/temp.freebsd-5.4-RELEASE -i386-2.4/avlmodule.o -o build/lib.freebsd-5.4-RELEASE-i386-2.4/avl.so -Wl,-x running install_lib copying build/lib.freebsd-5.4-RELEASE-i386-2.4/avl.so - /usr/local/lib/python2.4/site-packages #!/usr/local/bin/python import avl import time a = avl.new() crash = 61215 # may be any in range 0 .. 8 a.insert(crash) a.insert(crash + 1) b = iter(a) b.next() for i in xrange(10): a.insert(i) a.remove(crash) k=[] for i in xrange(100): # fill memory with padding for 32-bit machine k.append(pow(65536, 2)) # fill memory with padding for 64-bit machine k.append(pow(65536, 10)) for i in b: print i Iterations 30 Repeats 100 object avl.new() 44.3226678371 object xtree.xtree() 30.8406031132 object dict() 12.9507939816 Dict Xtree AVL Dict 1.00 0.42 0.29 Xtree 2.38 1.00 0.70 AVL 3.42 1.44
Re: [Python-Dev] draft of externally maintained packages PEP
Brett Cannon wrote: here is the rough draft of the PEP for packages maintained externally from Python itself. There is some missing, though, that I would like help filling in. I don't know who to list as the contact person (i.e., the Python developer in charge of the code) for Expat, Optik or pysqlite. [...] I also thought Gerhard Haering was in charge of pysqlite, but he has never responded to any emails about this topic. Sorry for not answering any sooner. Please list me as contact person for the SQLite module. Maybe the responsibility should go to Anthony since I know he worked to get the package in and probably cares about keeping it updated? As for Expat (the parser), is that Fred? [...] pysqlite - Web site http://www.sqlite.org/ - Standard library name sqlite3 - Contact person XXX - Synchronisation history * 2.1.3 (2.5) You can add * 2.2.0 * 2.2.2 here. -- Gerhard ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
Brett Cannon wrote: OK, I am going to write the PEP I proposed a week or so ago, listing all modules and packages within the stdlib that are maintained externally so we have a central place to go for contact info or where to report bugs on issues. This should only apply to modules that want bugs reported outside of the Python tracker and have a separate dev track. People who just use the Python repository as their mainline version can just be left out. [...] I prefer to have bugs on the sqlite module reported in the pysqlite tracker at http://pysqlite.org/ aka http://initd.org/tracker/pysqlite For bug fixes I have the same position as Fredrik: security fixes, bugs that block the build and warnings from automatic checkers should be done through the Python repository and I will port them to the external version. For any changes that reach deeper I'd like to have them handed over to me. As it's unrealistic that all bugs are reported through the pysqlite tracker, can it please be arranged that if somebody enters a SQLite related bug through the Sourceforge tracker, that I get notified? Perhaps by defining a category SQLite here and adding me as default responsible, if that's possible on SF. Currently I'm not subscribed to python-checkins and didn't see a need to. Is there a need to for Python core developers? I think there's no better way except subscribing and defining a filter for SQLite-related commits to be notified if other people commit changes to the SQLite module in Python? It's not that I'm too lazy, but I'd like to keep the number of things I need to monitor low. -- Gerhard ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviewed patches [was: SoC proposal: fix some old, old bugs in sourceforge]
On 4/25/06, Jim Jewett [EMAIL PROTECTED] wrote: Perhaps; part of the problem is with the SF workflow. Yes. Brett should probably add that to the list of what's wanted from a new tracker (good alerting of new items, and maybe some specific Request commit functionality, tied to a listing of committers) Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Updated context management documentation
Phillip J. Eby wrote: At 12:08 AM 4/26/2006 +1000, Nick Coghlan wrote: Secondly, the documentation now shows an example of a class with a close() method using contextlib.closing directly as its own __context__() method. Sadly, that would only work if closing() were a function. Classes don't get turned into methods, so you'll need to change that example to use: def __context__(self): return closing(self) instead. D'oh! I'll fix the example :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] big-memory tests
Neal and I wrote a few tests that exercise the Py_ssize_t code on 64bit hardware:http://python.org/sf/1471578Now that it's configurable and integrated with test_support and all, we think it's time to include it in the normal testsuite. I'd really like it to be in one of the next alphas, so people with an interest in insane-sized objects (and insane memory in their hardware) can run the tests, and (hopefully) add more. The tests that are there passed on AMD64 with 16Gb of RAM (although the tests that took 48Gb, and thus a ton of swap, took literally days to run,) but there are still many potential tests to add. I still want to add some 'find memory requirements for bigmem tests' and maybe 'run only bigmem tests' code to regrtest, and the bigmemtest decorator should be moved to test_support so we can intersperse bigmem tests with regular tests (say, in test_mmap.) But other than that, anyone object to committing this to the trunk (not sandbox)? The tests *are* run by default, but only with very small limits (so they'll use less memory than most of the tests), to make sure the tests do function. -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Must objects with __enter__/__exit__ also supply __context__?
Guido van Rossum wrote: I'm not convinced. Phillip managed to capture most of my reasons for documenting it that way. However, trying to find a better term than the with statement context phrase used in the latest docs is causing me to have doubts. The term I'm currently favouring is managed context because of the obvious relationship with context manager. But with the current documentation, it's potentially unclear which object is doing the managing, as all managed contexts are required to serve as context managers as well. So the term ends up potentially a little confusing (is the manager of a managed context the original context manager, or the managed context itself?). I'd have to actually try it out to see if the terminology could be kept clear. OTOH, if the two protocols are made orthogonal, then it's clear that the manager is always the original object with the __context__ method. Then the three cases are: - a pure context manager (only provides __context__) - a pure managed context (only provides __enter__/__exit__) - a managed context which can be its own context manager (provides all three) In the standard library, decimal.Context and threading.Condition would be examples of the first, decimal.ManagedContext would be an example of the second, and contextlib.GeneratorContext, contextlib.closing, files and the various threading module lock objects would be examples of the third. The final thing that occurred to me is that if we do find a use case for mixing and matching context managers and managed contexts, it would be easy enough to add a function to contextlib to assist in doing so: class SimpleContextManager(object): def __init__(ctx): self.context = ctx def __context__(self): return self.context def contextmanager(obj): if hasattr(obj, __context__): return obj if hasattr(obj, __enter__) and hasattr(obj, __exit__): return SimpleContextManager(obj) raise TypeError(Require a context manager or a managed context) The ability to add that function if we eventually decide we need it means that we don't have to worry about inadvertently ruling out use cases we simply haven't thought of yet (which was one of my concerns). Since I can supply reasonable answers to all of my earlier reservations, I now believe we could drop this requirement and still end up with comprehensible documentation. Whether it's a good idea to do so. . . I'm not sure. I still quite like the idea of having to describe only two kinds of object instead of three, but making the protocols orthogonal has its merits, too. Cheers, Nick. On 4/25/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 11:29 PM 4/25/2006 -0400, Phillip J. Eby wrote: See, if @contextfactory functions return a thing *with* a __context__ method, how is that usable with with? It isn't, unless the thing also happens to have __enter__/__exit__ methods. This was the hole in the documentation that caused Nick to seek to revisit the decorator name in the first place. Argh. I seem to be tongue-tied this evening. What I mean is, if @contextfactory functions' return value is usable as a with expression, that means it must have a __context__ method. But, if you are using @contextfactory to *define* a __context__ method, the return value should clearly have __enter__ and __exit__ methods. What this means is that if we describe the one method and the two methods as independent things, there is no *single* name we can use to describe the return value of a @contextfactory function. It's a wave and a particle, so we either have to start talking about wavicles or have some detailed explanation of why @contextfactory function return values are both waves and particles at the same time. However, if we say that particles are a kind of wave, and have all the same features as waves but just add a few others, then we can simply say @contextfactory functions return particles, and the waviness is implied, and all is right in the world. At least, until AMK comes along and asks why you can't separate the particleness from the waveness, which was what started this whole thing in the first place... :) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
Gerhard Häring [EMAIL PROTECTED] writes: Currently I'm not subscribed to python-checkins and didn't see a need to. Is there a need to for Python core developers? I would say it's encouraged. I think there's no better way except subscribing and defining a filter for SQLite-related commits to be notified if other people commit changes to the SQLite module in Python? That sounds like the least effort yes. mailman has 'topic filters' so one could define a sqlite topic for python-checkins and you could just subscribe to that, but that requires Barry doing something :-) (I think). Cheers, mwh -- Two things I learned for sure during a particularly intense acid trip in my own lost youth: (1) everything is a trivial special case of something else; and, (2) death is a bunch of blue spheres. -- Tim Peters, 1 May 1998 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] big-memory tests
Thomas Wouters [EMAIL PROTECTED] writes: Neal and I wrote a few tests that exercise the Py_ssize_t code on 64bit hardware: http://python.org/sf/1471578 Now that it's configurable and integrated with test_support and all, we think it's time to include it in the normal testsuite. I'd really like it to be in one of the next alphas, so people with an interest in insane-sized objects (and insane memory in their hardware) can run the tests, and (hopefully) add more. The tests that are there passed on AMD64 with 16Gb of RAM (although the tests that took 48Gb, and thus a ton of swap, took literally days to run,) but there are still many potential tests to add. I still want to add some 'find memory requirements for bigmem tests' and maybe 'run only bigmem tests' code to regrtest, and the bigmemtest decorator should be moved to test_support so we can intersperse bigmem tests with regular tests (say, in test_mmap.) But other than that, anyone object to committing this to the trunk (not sandbox)? Not at all, get it in pronto, I say! Cheers, mwh -- ... but I guess there are some things that are so gross you just have to forget, or it'll destroy something within you. perl is the first such thing I have known. -- Erik Naggum, comp.lang.lisp ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
Michael Hudson wrote: Gerhard Häring [EMAIL PROTECTED] writes: Currently I'm not subscribed to python-checkins and didn't see a need to. Is there a need to for Python core developers? I would say it's encouraged. I think there's no better way except subscribing and defining a filter for SQLite-related commits to be notified if other people commit changes to the SQLite module in Python? That sounds like the least effort yes. mailman has 'topic filters' so one could define a sqlite topic for python-checkins and you could just subscribe to that, but that requires Barry doing something :-) (I think). One could as well read python-checkins via gmane, and set up client-side filtering in the nntp reader. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] TRUNK FREEZE from 00:00 UTC, 27th April 2006 for 2.5a2
I'm going to be cutting 2.5a2 tomorrow. The trunk should be considered FROZEN from 00:00 UTC on the 27th of April (in about 12-13 hours time). Unless you're one of the release team, please don't make checkins to the trunk until the release is done. I'll send another message when it's done. Thanks, Anthony -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviewed patches [was: SoC proposal: fix some old, old bugs in sourceforge]
A.M. Kuchling wrote: On Tue, Apr 25, 2006 at 04:10:02PM -0400, Jim Jewett wrote: I don't see a good way to say It looks good to me. I don't see any way to say There were issues, but I think they're resolved now. So either way, I and the author are both sort of waiting for a committer to randomly happen back over old patches. If there are too many patches waiting for a committer to assess them, that probably points up the need for more committers. Simply adding more committers in general isn't necessarily the answer though, since it takes a while to feel entitled to check code in for different areas. I know I'm pretty reluctant to commit anything not directly related to the AST compiler, PEP 338 or PEP 343. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what do you like about other trackers and what do you hate about SF?
Brett Cannon wrote: I am starting to hash out what the Call for Trackers is going to say on the Infrastructure mailing list. Laura Creighton suggested we have a list of features that we would like to see and what we all hate about SF so as to provide some guidelines in terms of how to set up the test trackers that people try to sell us on. So, if you could, please reply to this message with ONE thing you have found in a tracker other than SF that you have liked (especially compared to SF) and ONE thing you dislike/hate about SF's tracker. I will use the replies as a quick-and-dirty features list of stuff that we would like to see demonstrated in the test trackers. A feature I can't recall seeing anywhere, but one I'd like: the ability to define a filter, and have the tracker email me when a new bug is submitted that matches the filter (being able to create an RSS feed based on a filter would be just as good) Something that I dislike about SF: the fact that I can't use email to participate in a bug discussion, but have to use SF's somewhat horrible web interface. OK, I guess I slipped in another feature request with that second one ;) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r45721 - python/trunk/Lib/test/test_with.py
tim.peters wrote: Author: tim.peters Date: Wed Apr 26 03:15:53 2006 New Revision: 45721 Modified: python/trunk/Lib/test/test_with.py Log: Rev 45706 renamed stuff in contextlib.py, but didn't rename uses of it in test_with.py. As a result, test_with has been skipped (due to failing imports) on all buildbot boxes since. Alas, that's not a test failure -- you have to pay attention to the 1 skip unexpected on PLATFORM: test_with kinds of output at the ends of test runs to notice that this got broken. That would be my fault - I've got about four unexpected skips I actually expect because I don't have the relevant external modules built. I must have missed this new one joining the list. It's likely that more renaming in test_with.py would be desirable. Yep. I can see why you deferred fixing it, though :) I'm also curious as to why that test uses its own version of Nested rather than the one in contextlib. (granted, you can't inherit from the contextlib version, but containment should work fine) Anyway, I've created SF bug #1476845 to keep track of the things that need to be done to finish the PEP 343 terminology cleanup. (What we have now should be fine for alpha 2) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Google Summer of Code proposal: New class for work with binary trees AVL and RB as with the standard dictionary.
Hye-Shik Chang wrote: On 4/25/06, Vladimir 'Yu' Stepanov [EMAIL PROTECTED] wrote: Thanks for the answer, but after application of a patch on python-2.4.3 I could not compile it. A conclusion of a stage of compilation in the attached file. Aah. The patch is for Python in Subversion trunk. I think you can add the following line somewhere in collectionsmodule.c to avoid to error. #define Py_ssize_t int Thank you! Hye-Shik Thanks! I have compared realizations. The object collections.rbtree works very well. Why it still extends in the form of a patch? Speed of work can be compared to speed of work of the standard dictionary. Here comparative results of speed of work. My module concedes in speed of work of that realization without the index on a parental element and allocation is used goes two objects on each element. There is an offer on improvement of its work. To add new exceptions for indication of the fact of blocking, for example: Exception |__ StandartError |__ LockError |__ FreezeError |__ RDLockError |__ WRLockError That speed of work was is high enough for check on each kind of blocking it is necessary to check only one identifier. For example these macro: - void _lock_except(int lockcnt) { if (lockcnt 0) { if ((lockcnt1) == 0) PyErr_SetNone(PyExc_RDLockError); else PyErr_SetNone(PyExc_FreezeLockError); } else PyErr_SetNone(PyExc_WRLockError); } #define wrlock_SET(x,r) if ((x)-ob_lockcnt != 0) { \ _lock_except((x)-ob_lockcnt); \ return r; \ } \ (x)-ob_lockcnt = -1 #define wrlock_UNSET(x) (x)-ob_lockcnt = 0 #define rdlock_SET(x,r) if ((x)-ob_lockcnt 0) { \ _lock_except((x)-ob_lockcnt); \ return r; \ } \ (x)-ob_lockcnt += 2 #define rdlock_UNSET(x) (x)-ob_lockcnt -= 2 #define freezelock_SET(x, r) if ((x)-ob_lockcnt 0) { \ _lock_except((x)-ob_lockcnt); \ return r; \ } \ (x)-ob_lockcnt |= 1; - In object the counter in the form of an integer with a sign should be certain - ob_lockcnt: - struct newobject { PyObject_HEAD // or PyObject_HEAD_VAR ... int ob_lockcnt; ... }; - And by any call depending on if the method only reads data of object, the sequence rdlock_* should be used. Example: - PyObject * method_for_read_something(PyObject *ob) { rdlock_SET(ob, NULL); ... do something ... if (failure) goto failed; ... do some else ... rdlock_UNSET(ob); Py_INCREF(Py_None); return Py_None; failed: rdlock_UNSET(ob); return NULL; } - If the given method can make any critical changes that the sequence wrlock_* should be used. Example: - PyObject * method_for_some_change(PyObject *ob) { wrlock_SET(ob, NULL); ... do something ... if (failure) goto failed; ... do some else ... wrlock_UNSET(ob); Py_INCREF(Py_None); return Py_None; failed: wrlock_UNSET(ob); return NULL; } - If the object needs to be frozen, it is equivalent to installation of constant blocking on reading. Thus it is possible to establish blocking on already readable object. For the open iterators it is necessary to carry out opening of blocking on reading. Closing can be carried out or on end of work of iterator, or on its closing: - PyObject * map_object_keys(PyObject *ob) { rdlock_SET(ob, NULL); ... allocate iterator and initialize ... } map_object_keys_iter_destroy(PyObject *ob) { ... destroy data ... rdlock_UNSET(ob); } - ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
Re: [Python-Dev] Reviewed patches
On Tue, Apr 25, 2006 at 06:04:12PM -0400, Jim Jewett wrote: So how are the committers supposed to even know that it is waiting for assessment? The solutions that I've seen work are Could we mark the bug/patch as status 'pending'? This status exists in the SF bug tracker but no bugs or patches are assigned this status, so we're probably not using it. --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] suggestion: except in list comprehension
a friend of mine suggested this, and i thought i should share it with the mailing list.many times when you would want to use list/generator comprehensions, you have tofall back to the old for/append way, because of exceptions. so it may be a good idea to allow an exception handling mechanism in these language constructs.since list comprehensions are expressions, an exceptions thrown inside one meansthe entire list is discarded. you may want to provide some, at least fundamental, error handling, like if this item raises an exception, just ignore it, or terminate theloop in that case and return whatever you got so far.the syntax is quite simple:[ expr for expr in expr [if cond] [except exception-class-or-tuple: action] ] where action is be one of continue or break:* continue would mean ignore this item* break would mean return what you got so farfor example: a = [i, fart, in, your, general, 5, direction, you, silly, english, kniggits]give me every word that starts with y, ignoring all errors b = [x for x in a if x.startswith(y) except: continue]# returns [your, you] or only AttributeError to be more speciifcb = [x for x in a if x.startswith(y) except AttributeError: continue]# returns [your, you]and with break b = [x for x in a if x.startswith(y) except AttributeError: continue] # returns only [your] -- we didn't get past the 5 in order to do something like this today, you have to resort to the old way,b = []for x in a: try: if x.startswith(y):b.append(x) except ...: pass which really breaks the idea behind list comprehension. so it's true that this example i provided can be done with a more complex if condition (first doing hasattr), but it's not always possible, and how would you do it if the error occurs at the first part of the _expression_? y = [4, 3, 2, 1, 0, -1, -2, -3] [1.0 / x for x in y except ZeroDivisionError: break][0.25, 0.333, 0.5, 1.0] [1.0 / x for x in y except ZeroDivisionError: continue] [0.25, 0.333, 0.5, 1.0, -1.0, -0.5, -0.333]again, in this example you can add if x != 0, but it's not always possible to tell whichelement will fail. for example:filelist = [a, b, c, \\/invalid file name:?*, e] openfiles = [open(filename) for filename in filelist except IOError: continue]the example is hypothetical, but in this case you can't tell *prior to the exception*that the operation is invalid. the same goes for generator expressions, of course. -tomer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
[EMAIL PROTECTED] wrote on 25/04/2006 13:22:27: Neil Hodgson [EMAIL PROTECTED] writes: Martin v. Löwis: Apparently, the status of this changed right now: it seems that the 2003 compiler is not available anymore; the page now says that it was replaced with the 2005 compiler. Should we reconsider? I expect Microsoft means that Visual Studio Express will be available free forever, not that you will always be able to download Visual Studio 2005 Express. I don't think that's what Herb Sutter said in his ACCU keynote, which is where I'm pretty sure Guido got his information at the start of this thread (he was there too and the email appeared soon after). If I remember right, he said that 2005 was free, forever, and they'd think about later versions. I may be misremembering, and I certainly haven't read any official stuff from Microsoft... Hi Michael, I was there and that's how I remember it too. Cheers, Ben Cheers, mwh -- I also feel it essential to note, [...], that Description Logics, non-Monotonic Logics, Default Logics and Circumscription Logics can all collectively go suck a cow. Thank you. -- http://advogato.org/person/Johnath/diary.html?start=4 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python- dev/python%40theyoungfamily.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Accessing DLL objects from other DLLs
Hi - I'm looking at implementing Python as a scripting language for an existing C++ code base. However, there are many interdependencies between the C++ modules that I already have, and I'm not sure how to deal with this in a Python context. I thought this would be the appropriate place to pose my question. As background, I'm familiar with the basics of creating Python DLLs from scratch and using SWIG (my preferred approach).For example: Let's say I have a Foo class, which has it's own independent set of functionality, and so is worthy of making into it's own Python module. Let us also say that I have a Bar class, which has some independent functionality and so could be made into it's own module, but has a method which can take an object of type Foo.If I want to keep these classes as distinct modules, but retain this kind of module interdependency, how can I architect this with Python? The current architecture has a sort of COM-like messaging middleman instantiated by "Main" that allows DLLs to call functionality in classes contained in other DLLs.Thanks,Mike___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Addressing Outstanding PEPs
The following PEPs are open according to PEP 0 and haven't seen much activity recently IIRC. I'd like everyone to take a cut and bring these up to date. For all the PEPs that aren't going anywhere, can we close them? Please update your PEP if appropriate. PEP 237 mentions changing an OverflowWarning in 2.5. Did we do that? Are any of the PEPs below in a similar boat? 302? If you want to discuss any of these, please start a new thread. n -- S 209 Adding Multidimensional Arrays Barrett, Oliphant S 228 Reworking Python's Numeric Model Zadka, GvR S 243 Module Repository Upload Mechanism Reifschneider S 256 Docstring Processing System FrameworkGoodger S 258 Docutils Design SpecificationGoodger S 266 Optimizing Global Variable/Attribute Access Montanaro S 267 Optimized Access to Module NamespacesHylton S 268 Extended HTTP functionality and WebDAV Stein S 275 Switching on Multiple Values Lemburg S 280 Optimizing access to globals GvR S 286 Enhanced Argument Tuples von Loewis I 287 reStructuredText Docstring FormatGoodger S 297 Support for System Upgrades Lemburg S 298 The Locked Buffer Interface Heller S 302 New Import Hooks JvR, Moore S 304 Controlling Generation of Bytecode Files Montanaro S 314 Metadata for Python Software Packages v1.1 Kuchling, Jones S 321 Date/Time Parsing and Formatting Kuchling S 323 Copyable Iterators Martelli S 331 Locale-Independent Float/String Conversions Reis S 334 Simple Coroutines via SuspendIteration Evans S 335 Overloadable Boolean Operators Ewing S 337 Logging Usage in the Standard LibraryDubner S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2Jones I 350 Codetags Elliott S 354 Enumerations in Python Finney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Addressing Outstanding PEPs
Neal S 266 Optimizing Global Variable/Attribute Access Montanaro Should be closed/rejected. I assume this concept will not be address in 2.x but resurface in a completely different form in 3.x. Neal S 304 Controlling Generation of Bytecode Files Montanaro Probably ditto. There were some problems reported with the concept on Windows (which unfortunately I lost). I have no particular interest in this for the environments in which I work. I've asked for a couple times on c.l.py for someone who is interested in this to take it over, but nobody's ever bitten. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviewed patches
A.M. Kuchling wrote: On Tue, Apr 25, 2006 at 06:04:12PM -0400, Jim Jewett wrote: So how are the committers supposed to even know that it is waiting for assessment? The solutions that I've seen work are Could we mark the bug/patch as status 'pending'? This status exists in the SF bug tracker but no bugs or patches are assigned this status, so we're probably not using it. If the SF tracker help is to be believed, pending status won't stick for very long (unless somebody has changed the default timeout): You can set the status to 'Pending' if you are waiting for a response from the tracker item author. When the author responds the status is automatically reset to that of 'Open'. Otherwise, if the author doesn't respond with an admin-defined amount of time (default is 14 days) then the item is given a status of 'Deleted'. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PY_FORMAT_SIZE_T warnings on OS X
Brett Cannon wrote: I created patch 1474907 with a fix for it. Checks if %zd works for size_t and if so sets PY_FORMAT_SIZE_T to z, otherwise just doesn't set the macro def. Assigned to Martin to make sure I didn't foul it up, but pretty much anyone could probably double-check it. Unfortunately, SF stopped sending emails when you get assigned an issue, so I didn't receive any message when you created that patch. I now reviewed it, and think the test might not fail properly on systems where %zd isn't supported. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inheriting basic types more efficiently
Have never seen such an answer before. Please excuse if I read it wrong. The method would just return the new values, for shure. That is what it shall do. The interpreter calls that method for receiving the new (to be used) values to pass them on to the real target (the called attribute). The method is a kind of filter to be used intermediately. Dennis On Wed, 26 Apr 2006 17:35:22 +0200 [EMAIL PROTECTED] wrote: The results of your email command are provided below. Attached is your original message. - Unprocessed: quite now: The new-style classes support the inheritance of basic types, like int, long, etc. Inheriting them is nice but not always efficient because one can not *control* the inherited methods but only overwrite them. For example, I want to implement a bitarray type. Because the long type is unlimited and implemented as an integer array (as far as I know), it seems to be a good choice for basing a bitarray type on it. Boolean arithmetics on long integers should work quite fast, I expect. However, the bitarray class should accept different values than the long type accepts, like 101010. This is not only true for initialization but also for comparisons, etc.: if x 101010: print True. Here now the trouble appears. There seems to be no way to catch an input value before it reaches the attribute, to parse and convert it into the appropriate integer, and to send this integer to the attribute instead. One actually has to overwrite all the inherited methods to call the validation method from inside each of them individually. This renders the inheritance of the long type useless. One could just write a new class instead. This is not optimal. Please think about implementing a method that catches all input values. - Ignored: This method would just return the new values. Thanks, Dennis - Done. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 304 (Was: Addressing Outstanding PEPs)
[EMAIL PROTECTED] wrote: [...] Should be closed/rejected. [...] Neal S 304 Controlling Generation of Bytecode Files Montanaro Probably ditto. Rejected would be a wrong description, IMO; withdrawn describes it better. It's not that the feature is undesirable or the specific approach at solving the problem - just nobody is interested to work on it. So future contributors shouldn't get the impression that this was discussed and rejected, but that it was discussed and abandoned. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Addressing Outstanding PEPs
At 11:55 PM 4/25/2006 -0700, Neal Norwitz wrote: S 243 Module Repository Upload Mechanism Reifschneider This one needs to be withdrawn or updated - it totally doesn't match the implementation in Python 2.5. S 302 New Import Hooks JvR, Moore Unless somebody loans me the time machine, I won't be able to finish all of PEP 302's remaining items before the next alpha. Some, like the idea of moving the built-in logic to sys.meta_path, can't be done until Py3K without having adverse impact on deployed software. S 334 Simple Coroutines via SuspendIteration Evans IIRC, this one's use cases can be met by using a coroutine trampoline as described in PEP 342. S 345 Metadata for Python Software Packages 1.2Jones This one should not be accepted in its current state; too much specification of syntax, too little of semantics. And the specifications that *are* there, are often wrong. For example, the strict version syntax wouldn't even support *Python's own* version numbering scheme, let alone the many package versioning schemes in actual use. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accessing DLL objects from other DLLs
Michael Hearne wrote: If I want to keep these classes as distinct modules, but retain this kind of module interdependency, how can I architect this with Python? Please understand that python-dev is for discussions about the development *of* Python, not for the development *with* Python. Use news:comp.lang.python (mailto:python-list@python.org) for the latter. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Addressing Outstanding PEPs
Neal Norwitz wrote: The following PEPs are open according to PEP 0 and haven't seen much activity recently IIRC. I'd like everyone to take a cut and bring these up to date. For all the PEPs that aren't going anywhere, can we close them? Please update your PEP if appropriate. S 298 The Locked Buffer Interface Heller I withdraw this. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Dropping __init__.py requirement for subpackages
(Context: There's a large crowd with pitchforks and other sharp pointy farm implements just outside the door of my office at Google. They are making an unbelievable racket. It appears they are Google engineers who have been bitten by a misfeature of Python, and they won't let me go home before I have posted this message.) The requirement that a directlry must contain an __init__.py file before it is considered a valid package has always been controversial. It's designed to prevent the existence of a directory with a common name like time or string from preventing fundamental imports to work. But the feature often trips newbie Python programmers (of which there are a lot at Google, at our current growth rate we're probably training more new Python programmers than any other organization worldwide :-). One particular egregious problem is that *subpackage* are subject to the same rule. It so happens that there is essentially only one top-level package in the Google code base, and it already has an __init__.py file. But developers create new subpackages at a frightening rate, and forgetting to do touch __init__.py has caused many hours of lost work, not to mention injuries due to heads banging against walls. So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inheriting basic types more efficiently
To bring the real problem more upfront. Up to now, inheriting classes is all about processing (the output channel) but not about retrieving (the input channel). However, though a new type can advance from an existing type if it just needs to provide some few more methods, it can not advance from an existing type if it needs to support some few other input formats--even if they all convert to the inherited type easily. The input channel is completely forgotten. Dennis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
* Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. Not that it would count in any way, but I'd prefer to keep it. How would I mark a subdirectory as not-a-package otherwise? echo raise ImportError __init__.py ? nd -- Das, was ich nicht kenne, spielt stückzahlmäßig *keine* Rolle. -- Helmut Schellong in dclc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. So this would mean that current non-package subdirectories in a package (that contain things like data files or configuration info) would become packages with no modules in them? sharpening-my-farm-implements-ly y'rs, -- Benji York ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, André Malo [EMAIL PROTECTED] wrote: * Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. Not that it would count in any way, but I'd prefer to keep it. How would I mark a subdirectory as not-a-package otherwise? What's the use case for that? Have you run into this requirement? And even if you did, was there a requirement that the subdirectory's name be the same as a standard library module? If the subdirectory's name is not constrained, the easiest way to mark it as a non-package is to put a hyphen or dot in its name; if you can't do that, at least name it something that you don't need to import. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Benji York [EMAIL PROTECTED] wrote: Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. So this would mean that current non-package subdirectories in a package (that contain things like data files or configuration info) would become packages with no modules in them? Yup. Of course unless you try to import from them that wouldn't particularly hurt, except if the subdir name happens to be the same as a module name. Note that absolute import (which will be turned on for all in 2.6) will solve the ambiguity; the only ambiguity left would be if you had a module foo.py and also a non-package subdirectory foo. But that's just asking for trouble. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inheriting basic types more efficiently
I doubt you'll get many answers. I have no idea what you're talking about. How about an example or two? On 4/26/06, Dennis Heuer [EMAIL PROTECTED] wrote: To bring the real problem more upfront. Up to now, inheriting classes is all about processing (the output channel) but not about retrieving (the input channel). However, though a new type can advance from an existing type if it just needs to provide some few more methods, it can not advance from an existing type if it needs to support some few other input formats--even if they all convert to the inherited type easily. The input channel is completely forgotten. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] suggestion: except in list comprehension
tomer filiba [EMAIL PROTECTED] wrote: [ expr for expr in expr [if cond] [except exception-class-or-tuple: action] ] Note that of the continue cases you offer, all of them are merely simple if condition (though the file example could use a better test than os.path.isfile). [x for x in a if x.startswith(y) except AttributeError: continue] [x for x in a if hasattr(x, 'startswith') and x.startswith(y)] [1.0 / x for x in y except ZeroDivisionError: continue] [1.0 / x for x in y if x != 0] [open(filename) for filename in filelist except IOError: continue] [open(filename) for filename in filelist if os.path.isfile(filename)] The break case can be implemented with particular kind of instance object, though doesn't have the short-circuiting behavior... class StopWhenFalse: def __init__(self): self.t = 1 def __call__(self, t): if not t: self.t = 0 return 0 return self.t z = StopWhenFalse() Assuming you create a new instance z of StopWhenFalse before doing the list comprehensions... [x for x in a if z(hasattr(x, 'startswith') and x.startswith(y))] [1.0 / x for x in y if z(x != 0)] [open(filename) for filename in filelist if z(os.path.isfile(filename))] If you couldn't guess; -1, you can get equivalent behavior without complicating the generator expression/list comprension syntax. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PY_FORMAT_SIZE_T warnings on OS X
[Brett Cannon] I created patch 1474907 with a fix for it. Checks if %zd works for size_t and if so sets PY_FORMAT_SIZE_T to z, otherwise just doesn't set the macro def. Assigned to Martin to make sure I didn't foul it up, but pretty much anyone could probably double-check it. [Martin v. Löwis] Unfortunately, SF stopped sending emails when you get assigned an issue, so I didn't receive any message when you created that patch. I now reviewed it, and think the test might not fail properly on systems where %zd isn't supported. I agree with Martin's patch comments. The C standards don't require that printf fail if an unrecognized format gimmick is used (behavior is explicitly undefined then). For example, #include stdio.h #include stdlib.h int main() { int i; i = printf(%zd\n, (size_t)0); printf(%d\n, i); return 0; } prints zd\n3\n under all of MSVC 6.0, MSVC 7.1, and gcc (Cygwin) on WinXP. Using sprintf instead and checking the string produced seems much more likely to work. After the trunk freeze melts, I suggest just _trying_ stuff. The buildbot system covers a bunch of platforms now, and when trying to out-hack ill-defined C stuff try it and see is easier than thinking 0.5 wink. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Barry Warsaw [EMAIL PROTECTED] wrote: On Wed, 2006-04-26 at 10:16 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. But if it's there, then nothing changes, right? IOW, if we want to expose names in the subpackage's namespace, we can still do it in the subpackage's __init__.py. It's just that otherwise empty subpackage __init__.py files won't be required. Correct. This is an important clarification. What would the following print? import toplevel.sub1.sub2 print toplevel.sub1.sub2.__file__ If it's path/sub1/sub2 then that kind of breaks a common idiom of using os.path.dirname() on a module's __file__ to find co-located resources. Or at least, you have to know whether to dirname its __file__ or not (which might not be too bad, since you'd probably know how that package dir is organized anyway). Oh, cool gray area. I propose that if there's no __init__.py it prints 'path/sub1/sun2/' i.e. with a trailing slash; that causes dirname to just strip the '/'. (It would be a backslash on Windows of course). I dunno. Occasionally it trips me up too, but it's such an obvious and easy fix that it's never bothered me enough to care. But you're not a newbie. for a newbie who's just learned about packages, is familiar with Java, and missed one line in the docs, it's an easy mistake to make and a tough one to debug. I can't think of an example, but I suppose it's still possible that lifting this requirement could cause some in-package located data directories to be mistakenly importable. I'd be somewhat more worried about frameworks that dynamically import things having to be more cautious about cleansing their __import__() arguments now. But (assuming 2.6 and absolute import) what would be the danger of importing such a package? Presumably it contains no *.py or *.so files so there's no code there; but even so you'd have to go out of your way to import it (since if the directory exists it can't also be a subpackage or submodule name that's in actual use). I'd be -1 but the remote possibility of you being burned at the stake by your fellow Googlers makes me -0 :). I'm not sure I understand what your worry is. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
* Guido van Rossum wrote: On 4/26/06, André Malo [EMAIL PROTECTED] wrote: * Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. Not that it would count in any way, but I'd prefer to keep it. How would I mark a subdirectory as not-a-package otherwise? What's the use case for that? Have you run into this requirement? And even if you did, was there a requirement that the subdirectory's name be the same as a standard library module? If the subdirectory's name is not constrained, the easiest way to mark it as a non-package is to put a hyphen or dot in its name; if you can't do that, at least name it something that you don't need to import. Actually I have no problems with the change from inside python, but from the POV of tools, which walk through the directories, collecting/separating python packages and/or supplemental data directories. It's an explicit vs. implicit issue, where implicit would mean kind of heuristics from now on. IMHO it's going to break existing stuff [1] and should at least not be done in such a rush. nd [1] Well, it does break some of mine ;-) -- Da fällt mir ein, wieso gibt es eigentlich in Unicode kein i mit einem Herzchen als Tüpfelchen? Das wär sooo süüss! -- Björn Höhrmann in darw ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 10:16 AM 4/26/2006 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. Note that many tools exist which have grown to rely on the presence of __init__ modules. Also, although your proposal would allow imports to work reasonably well, tools that are actively looking for packages would need to have some way to distinguish package directories from others. My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. This tweak would also make it usable for top-level directories, since the mere presence of a 'time' directory wouldn't get in the way of anything. The thing more likely to have potential for problems is that many Python projects have a test directory that isn't intended to be a package, and thus may interfere with imports from the stdlib 'test' package. Whether this is really a problem or not, I don't know. But, we could treat packages without __init__ as namespace packages. That is, set their __path__ to encompass similarly-named directories already on sys.path, so that the init-less package doesn't interfere with other packages that have the same name. This would require a bit of expansion to PEP 302, but probably not much. Most of the rest is existing technology, and we've already begun migrating stdlib modules away from doing their own hunting for __init__ and other files, towards using the pkgutil API. By the way, one small precedent for packages without __init__: setuptools generates such packages using .pth files when a package is split between different distributions but are being installed by a system packaging tool. In such cases, *both* parts of the package can't include an __init__, because the packaging tool (e.g. RPM) is going to complain that the shared file is a conflict. So setuptools generates a .pth file that creates a module object with the right name and initializes its __path__ to point to the __init__-less directory. This should be a small change. Famous last words. :) There's a bunch of tools that it's not going to work properly with, and not just in today's stdlib. (Think documentation tools, distutils extensions, IDEs...) Are you sure you wouldn't rather just write a GoogleImporter class to fix this problem? Append it to sys.path_hooks, clear sys.path_importer_cache, and you're all set. For that matter, if you have only one top-level package, put the class and the installation code in that top-level __init__, and you're set to go. And that approach will work with Python back to version 2.3; no waiting for an upgrade (unless Google is still using 2.2, of course). Let's see, the code would look something like: class GoogleImporter: def __init__(self, path): if not os.path.isdir(path): raise ImportError(Not for me) self.path = os.path.realpath(path) def find_module(self, fullname, path=None): # Note: we ignore 'path' argument since it is only used via meta_path subname = fullname.split(.)[-1] if os.path.isdir(os.path.join(self.path, subname)): return self path = [self.path] try: file, filename, etc = imp.find_module(subname, path) except ImportError: return None return ImpLoader(fullname, file, filename, etc) def load_module(self, fullname): import sys, new subname = fullname.split(.)[-1] path = os.path.join(self.path, subname) module = sys.modules.setdefault(fullname, new.module(fullname)) module.__dict__.setdefault('__path__',[]).append(path) return module class ImpLoader: def __init__(self, fullname, file, filename, etc): self.file = file self.filename = filename self.fullname = fullname self.etc = etc def load_module(self, fullname): try: mod = imp.load_module(fullname, self.file, self.filename, self.etc) finally: if self.file: self.file.close() return mod import sys sys.path_hooks.append(GoogleImporter) sys.path_importer_cache.clear() ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 02:07 PM 4/26/2006 -0400, Phillip J. Eby wrote: def find_module(self, fullname, path=None): # Note: we ignore 'path' argument since it is only used via meta_path subname = fullname.split(.)[-1] if os.path.isdir(os.path.join(self.path, subname)): return self path = [self.path] try: file, filename, etc = imp.find_module(subname, path) except ImportError: return None return ImpLoader(fullname, file, filename, etc) Feh. The above won't properly handle the case where there *is* an __init__ module. Trying again: def find_module(self, fullname, path=None): subname = fullname.split(.)[-1] path = [self.path] try: file, filename, etc = imp.find_module(subname, path) except ImportError: if os.path.isdir(os.path.join(self.path, subname)): return self else: return None return ImpLoader(fullname, file, filename, etc) There, that should only fall back to __init__-less handling if there's no foo.py or foo/__init__.py present. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Google Summer of Code proposal: New class for work with binary trees AVL and RB as with the standard dictionary.
Vladimir 'Yu' Stepanov [EMAIL PROTECTED] wrote: Josiah Carlson wrote: There exists various C and Python implementations of both AVL and Red-Black trees. For users of Python who want to use AVL and/or Red-Black trees, I would urge them to use the Python implementations. In the case of *needing* the speed of a C extension, there already exists a CPython extension module for AVL trees: http://www.python.org/pypi/pyavl/1.1 I would suggest you look through some suggested SoC projects in the wiki: http://wiki.python.org/moin/SummerOfCode - Josiah Thanks for the answer! I already saw pyavl-1.1. But for this reason I would like to see the module in a standard package python. Functionality for pyavl and dict to compare difficultly. Functionality of my module will differ from functionality dict in the best party. I have lead some tests on for work with different types both for a package pyavl-1.1, and for the prototype of own module. The script of check is resulted in attached a file avl-timeit.py In files timeit-result-*-*.txt results of this test. The first in the name of a file means quantity of the added elements, the second - argument of a method timeit. There it is visible, that in spite of the fact that the module xtree is more combined in comparison with pyavl the module (for everyone again inserted pair [the key, value], is created two elements: python object - pair, and an internal element of a tree), even in this case it works a little bit more quickly. Besides the module pyavl is unstable for work in multithread appendices (look the attached file module-avl-is-thread-unsafe.py). I'm not concerned about the speed of the external AVL module, and I'm not concerned about the speed of trees in Python; so far people have found that dictionaries are generally sufficient. More specifically, while new lambda syntaxes are presented almost weekly, I haven't heard anyone complain about Python's lack of a tree module in months. As a result, I don't find the possible addition of any tree structure to the collections module as being a generally compelling addition. Again, I believe that the existance of 3rd party extension modules which implement AVL and red-black trees, regardless of their lack of thread safety, slower than your module by a constant, or lack of particular functionality to be basically immaterial to this discussion. In my mind, there are only two relevant items to this discussion: 1. Is having a tree implementation (AVL or Red-Black) important, and if so, is it important enough to include in the collections module? 2. Is a tree implementation important enough for google to pay for its inclusion, given that there exists pyavl and a collectionsmodule.c patch for a red-black tree implementation? Then again, you already have your own implementation of a tree module, and it seems as though you would like to be paid for this already-done work. I don't know how google feels about such things, but you should remember to include this information in your application. I think, that creation of this type (together with type of pair), will make programming more convenient since sorting by function sort will be required less often. How often are you sorting data? I've found that very few of my operations involve sorting data of any kind, and even if I did have need of sorting, Python's sorting algorithm is quite fast, likely faster by nearly a factor of 2 (in the number of comparisons) than the on-line construction of an AVL or Red-Black tree. I can probably borrow in this module beyond the framework of the project google. The demand of such type of data is interesting. Because of necessity of processing `gcmodule.c' and `obmalloc.c' this module cannot be realized as the external module. It seems to me (after checking collectionsmodule.c and a few others) that proper C modules only seem to import Python.h and maybe structmember.h . If you are manually including gcmodule.c and obmalloc.c, you may be doing something wrong; I'll leave it to others to further comment on this aspect. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: On 4/26/06, Benji York [EMAIL PROTECTED] wrote: Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. I don't particularly like it. You still need __init__.py's for 'import *' to work (not that I like or use 'import *' :). The first question that pops into my mind when I think file-less modules is where does the package-module come from. That question is a lot easier to answer (not to mention explain) when all packages have an __init__.py. It also adds to Python's consistency (which means people learn something from it that they can apply to other things later; in that case, removing it just hampers their growth.) And besides, it's just not that big a deal. I don't feel strongly enough about it to object, though. However, I would suggest adding a warning for existing, __init__.py-less directories that would-have-been imported in 2.5. (There's an alpha3 scheduled, so it doesn't have to go in alpha2 tonight, and it could probably be last-minuted into beta1 too.) That should fix both Google's problems and that of everyone having existing non-package subdirs :-) Then, if it really matters, we can change the import in 2.6.Note that absolute import (which will be turned on for all in 2.6) 2.7, see the PEP. -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Sounds a bit like the tail wagging the dog. I thought the Google geeks were a smart bunch. ISTM that something like Phillip Eby's code would be the most expedient solution. I would add one extension: if a package directory without an __init__.py file *is* encountered, an empty __init__.py file should automatically be created (and perhaps even svn add or equivalent called), and the code should loudly complain Packages need __init__.py files, noob! Add sound and light effects for extra credit. -- David Goodger http://python.net/~goodger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, André Malo [EMAIL PROTECTED] wrote: * Guido van Rossum wrote: On 4/26/06, André Malo [EMAIL PROTECTED] wrote: * Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. Not that it would count in any way, but I'd prefer to keep it. How would I mark a subdirectory as not-a-package otherwise? What's the use case for that? Have you run into this requirement? And even if you did, was there a requirement that the subdirectory's name be the same as a standard library module? If the subdirectory's name is not constrained, the easiest way to mark it as a non-package is to put a hyphen or dot in its name; if you can't do that, at least name it something that you don't need to import. Actually I have no problems with the change from inside python, but from the POV of tools, which walk through the directories, collecting/separating python packages and/or supplemental data directories. It's an explicit vs. implicit issue, where implicit would mean kind of heuristics from now on. IMHO it's going to break existing stuff [1] and should at least not be done in such a rush. nd [1] Well, it does break some of mine ;-) Can you elaborate? You could always keep the __init__.py files, you know... -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 10:16 AM 4/26/2006 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. Note that many tools exist which have grown to rely on the presence of __init__ modules. Also, although your proposal would allow imports to work reasonably well, tools that are actively looking for packages would need to have some way to distinguish package directories from others. My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. I'm not sure what you mean by one directory read. You'd have to list the entire directory, which may require reading more than one block if the directory is large. But I'd be happy to define it like this from the POV of tools that want to know about sub-packages; my users complain because they have put .py files in a directory that they consider a sub-package so it would work fine for them. Python itself might attempt to consider the directory as a package and raise ImportError because the requested sub-module isn't found; the creation of a dummy entry in sys.modules in that case doesn't bother me. This tweak would also make it usable for top-level directories, since the mere presence of a 'time' directory wouldn't get in the way of anything. Actually, no; the case I remember was a directory full of Python code (all experiments by the user related to a particular topic -- I believe it was string). The thing more likely to have potential for problems is that many Python projects have a test directory that isn't intended to be a package, and thus may interfere with imports from the stdlib 'test' package. Whether this is really a problem or not, I don't know. test is a top-level package. I'm not proposing to change the rules for toplevel packages. Now you have the reason why. (And the new absolute import feature in 2.6 will prevent aliasing problems between subdirectories and top-level modules.) But, we could treat packages without __init__ as namespace packages. That is, set their __path__ to encompass similarly-named directories already on sys.path, so that the init-less package doesn't interfere with other packages that have the same name. Let's stick to the one feature I'm actually proposing please. This would require a bit of expansion to PEP 302, but probably not much. Most of the rest is existing technology, and we've already begun migrating stdlib modules away from doing their own hunting for __init__ and other files, towards using the pkgutil API. By the way, one small precedent for packages without __init__: setuptools generates such packages using .pth files when a package is split between different distributions but are being installed by a system packaging tool. In such cases, *both* parts of the package can't include an __init__, because the packaging tool (e.g. RPM) is going to complain that the shared file is a conflict. So setuptools generates a .pth file that creates a module object with the right name and initializes its __path__ to point to the __init__-less directory. This should be a small change. Famous last words. :) There's a bunch of tools that it's not going to work properly with, and not just in today's stdlib. (Think documentation tools, distutils extensions, IDEs...) Are you worried about the tools not finding directories that are now subpackages? Then fix the tools. Or are you worried about flagging subdirectories as (empty) packages since they exist, have a valid name (no hyphens, dots etc.) and contain no modules? I'm not sure I would call that failing. I can't see how a tool would crash or produce incorrect results with this change, *unless* you consider it incorrect to list a data directory as an empty package. To me, that's an advantage. Are you sure you wouldn't rather just write a GoogleImporter class to fix this problem? No, because that would require more setup code with a requirement to properly enable it, etc., etc., more failure modes, etc., etc. Append it to sys.path_hooks, clear sys.path_importer_cache, and you're all set. For that matter, if you have only one top-level package, put the class and the installation code in that top-level __init__, and you're set to go. I wish it were that easy. If there was such an easy solution, there wouldn't be pitchforks involved. I can't go into the details, but that just wouldn't work; and the problem happens most frequently to people who are already overloaded with learning new stuff. This is just one more bit of insanity they have to deal with. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: On 4/26/06, Barry Warsaw [EMAIL PROTECTED] wrote: On Wed, 2006-04-26 at 10:16 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. I'm not really sure what this would buy us. Newbies would still forget the __init__.py in top-level packages (not all newbies work for Google). Oldies would have trouble recognizing a directory as being a Python package, rather than just a collection of modules - you wouldn't go hunting up the path to find the top-level __init__.py file which identifies the directory as being a sub-package of some top-level package (not all oldies work for Google either where you only have a single top-level package). -1. It doesn't appear to make things easier and breaks symmetry. But if it's there, then nothing changes, right? IOW, if we want to expose names in the subpackage's namespace, we can still do it in the subpackage's __init__.py. It's just that otherwise empty subpackage __init__.py files won't be required. Correct. This is an important clarification. What would the following print? import toplevel.sub1.sub2 print toplevel.sub1.sub2.__file__ If it's path/sub1/sub2 then that kind of breaks a common idiom of using os.path.dirname() on a module's __file__ to find co-located resources. Or at least, you have to know whether to dirname its __file__ or not (which might not be too bad, since you'd probably know how that package dir is organized anyway). Oh, cool gray area. I propose that if there's no __init__.py it prints 'path/sub1/sun2/' i.e. with a trailing slash; that causes dirname to just strip the '/'. (It would be a backslash on Windows of course). I dunno. Occasionally it trips me up too, but it's such an obvious and easy fix that it's never bothered me enough to care. But you're not a newbie. for a newbie who's just learned about packages, is familiar with Java, and missed one line in the docs, it's an easy mistake to make and a tough one to debug. Why not make the ImportError's text a little smarter instead, e.g. let the import mechanism check for this particular gotcha ? This would solve the newbie problem without any changes to the import scheme. I can't think of an example, but I suppose it's still possible that lifting this requirement could cause some in-package located data directories to be mistakenly importable. I'd be somewhat more worried about frameworks that dynamically import things having to be more cautious about cleansing their __import__() arguments now. But (assuming 2.6 and absolute import) what would be the danger of importing such a package? Presumably it contains no *.py or *.so files so there's no code there; but even so you'd have to go out of your way to import it (since if the directory exists it can't also be a subpackage or submodule name that's in actual use). Wasn't there agreement on postponing the absolute imports to Py3k due to the common use-case of turning e.g. 3rd party top-level packages into sub-packages of an application ? Without absolute imports your proposed scheme is going to break, since can easily mask top-level packages or modules. I'd be -1 but the remote possibility of you being burned at the stake by your fellow Googlers makes me -0 :). I'm not sure I understand what your worry is. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 26 2006) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
* Guido van Rossum wrote: [me] Actually I have no problems with the change from inside python, but from the POV of tools, which walk through the directories, collecting/separating python packages and/or supplemental data directories. It's an explicit vs. implicit issue, where implicit would mean kind of heuristics from now on. IMHO it's going to break existing stuff [1] and should at least not be done in such a rush. nd [1] Well, it does break some of mine ;-) [Guido] Can you elaborate? You could always keep the __init__.py files, you know... Okay, here's an example. It's about a non-existant __init__.py, though ;-) I have a test system which collects the test suites from one or more packages automatically by walking through the tree. Now there are subdirectories which explicitly are not packages (no __init__.py), but do contain some python files (helper scripts, spawned for particular tests). The test collector doesn't consider now these subdirectories at all, but in future it would need to (because it should search like python itself). Another point is that one can even hide supplementary packages within such a subdirectory. It's only visible to scripts inside the dir (I admit, that the latter is not a real usecase, just a thought that came up while writing this up). Anyway: sure, one could tweak the naming - just not for existing a.k.a. already released stuff. It's not very nice to force that, too ;-) nd -- If God intended people to be naked, they would be born that way. -- Oscar Wilde ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: On 4/26/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 10:16 AM 4/26/2006 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. Note that many tools exist which have grown to rely on the presence of __init__ modules. Also, although your proposal would allow imports to work reasonably well, tools that are actively looking for packages would need to have some way to distinguish package directories from others. My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. I'm not sure what you mean by one directory read. You'd have to list the entire directory, which may require reading more than one block if the directory is large. Sounds like a lot of filesystem activity to me :-) Currently, __init__.py is used as landmark by the import code to easily determine whether a directory is a package using two simple I/O operations (fstat on __init__.py, __init__.py[co]). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 26 2006) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Wednesday 26 April 2006 15:05, André Malo wrote: Another point is that one can even hide supplementary packages within such a subdirectory. It's only visible to scripts inside the dir (I admit, that the latter is not a real usecase, just a thought that came up while writing this up). I have tests that do this. This is a very real use case. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] suggestion: except in list comprehension
tomer filiba [EMAIL PROTECTED] wrote: first, i posted it two days ago, so it's funny it got posted only now... the moderators are sleeping on the job :) I don't believe python-dev has moderators...or at least my posts have never been delayed like that. Note that of the continue cases you offer, all of them are merely simple if condition yes, i said that explicitly, did you *read* my mail? After reading your syntax I read over the samples you provided. Generally speaking, I get far too much email to read the ful content of every I think Python should have feature/syntax/whatever X, especially when those posts begin with my friend had this idea. Let your friend post if if (s)he thinks it important, or at least post the idea on python-list first. Python-dev shouldn't be the starting point of these syntax discussions, it should be near the ending point. but i also said it's not always possible. you *can't* always tell prior to doing something. Of course you can't. But you know what? In all the times I've used list comprehensions over the last few years, I've never wanted, nor needed, the functionality you describe. Would it be convenient? Sure. But there are many other additions that I would find more convenient, and would likely find more use, which have been rejected. anyway, i guess my friend may have better show-cases, as he's the one who found the need for it. i just thought i should bring this up here. if you think better show cases would convince you, i can ask him. I think that the syntax is unnecessary, and no amount of use-cases that your friend can supply will likely convince me otherwise (unless your friend happens to be a core Python developer - in which case he would have posted him/herself). If you couldn't guess; -1, you can get equivalent behavior without complicating the generator expression/list comprension syntax. so how come list comprehensions aren't just a complication to the syntax? you can always do them the old way, x = [] for : x.append(...) List comprehensions were designed with this one particular simple use-case in mind. Look through the standard library at the number of times that simple for loops with appends are used. Go ahead. Take your time. Now go ahead and count the cases where the only difference is a try/except clause. Compare that count. Note how the simple case is likely close to 10 times as common as the try/except case? THAT's why the simple version was included, and that's why I don't believe that except is necessary in list comprehensions. don't worry, i'm not going to argue it past this. Good, I think it's a nonstarter. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: On 4/26/06, Barry Warsaw [EMAIL PROTECTED] wrote: On Wed, 2006-04-26 at 10:16 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. [...] I'd be -1 but the remote possibility of you being burned at the stake by your fellow Googlers makes me -0 :). I'm not sure I understand what your worry is. I happen to be a Googler too, but I was a Pythonista first... I'm -1 for minor mainly subjective reasons; 1) explicit is better than implicit. I prefer to be explicit about what is and isn't a module. I have plenty of doc and test and other directories inside python module source tree's that I don't want to be python modules. 2) It feels more consistant to always require it. /foo/ is a python package because it contains an __init__.py... so package /foo/bar/ should have one one too. 3) It changes things for what feels like very little gain. I've never had problems with it, and don't find the import exception hard to diagnose. Note that I think the vast majority of newbie missing __init__.py problems within google occur because people are missing __init__.py at the root of package import tree. This change would not not solve that problem. It wouldn't surprise me if this change would introduce a slew of newbies complaining that I have /foo on my PYTHONPATH, why can't I import foo/bar/ because they're forgotten the (now) rarely required __init__.py -- Donovan Baarda ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 11:50 AM 4/26/2006 -0700, Guido van Rossum wrote: I'm not sure what you mean by one directory read. You'd have to list the entire directory, which may require reading more than one block if the directory is large. You have to do this to find an __init__.py too, don't you? Technically, there's going to be a search for a .pyc or .pyo first, anyway. I'm just saying you can stop as soon as you hit an extension that's in imp.get_suffixes(). Are you sure you wouldn't rather just write a GoogleImporter class to fix this problem? No, because that would require more setup code with a requirement to properly enable it, etc., etc., more failure modes, etc., etc. I don't understand. I thought you said you had only *one* top-level package. Fix *that* package, by putting the code in its __init__.py. Job done. Append it to sys.path_hooks, clear sys.path_importer_cache, and you're all set. For that matter, if you have only one top-level package, put the class and the installation code in that top-level __init__, and you're set to go. I wish it were that easy. Well, if there's more than one top-level package, put the code in a module called google_imports and import google_import in each top-level package's __init__.py. I'm not sure I understand why a solution that works with released versions of Python that allows you to do exactly what you want, is inferior to a hypothetical solution for an unreleased version of Python that forces everybody else to update their tools. Unless of course the problem you're trying to solve is a political one rather than a technical one, that is. Or perhaps it wasn't clear from my explanation that my proposal will work the way you need it to, or I misunderstand what you're trying to do. Anyway, I'm not opposed to the idea of supporting this in future Pythons, but I definitely think it falls under the but sometimes never is better than RIGHT now rule where 2.5 is concerned. :) In particular, I'm worried that you're shrugging off the extent of the collateral damage here, and I'd be happiest if we waited until 3.0 before changing this particular rule -- and if we changed it in favor of namespace packages, which will more closely match naive user expectations. However, the fix the tools argument is weak, IMO. Zipfile imports have been a fairly half-assed feature for 2.3 and 2.4 because nobody took the time to make the *rest* of the stdlib work with zip imports. It's not a good idea to change machinery like this without considering at least what's going to have to be fixed in the stdlib. At a minimum, pydoc and distutils have embedded assumptions regarding __init__ modules, and I wouldn't be surprised if ihooks, imputil, and others do as well. If we can't keep the stdlib up to date with changes in the language, how can we expect anybody else to keep their code up to date? Finally, as others have pointed out, requiring __init__ at the top level just means that this isn't going to help anybody but Google. ISTM that in most cases outside Google, Python newbies are more likely to be creating top-level packages *first*, so the implicit __init__ doesn't help them. So, to summarize: 1. It only really helps Google 2. It inconveniences others who have to update their tools in order to support people who end up using it (even if by accident) 3. It's not a small change, unless you leave the rest of the stdlib unreviewed for impacts 4. It could be fixed at Google by adding a very small amount of code to the top of your __init__.py files (although apparently this is prevented for mysterious reasons that can't be shared) What's not to like? ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby wrote: My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. I assume you want import x.y to fail if y is an empty directory (or non-empty, but without .py files). I don't see a value in implementing such a restriction. If there are no .py files in a tree, then there would be no point in importing it, so applications will typically not import an empty directory. Implementing an expensive test that will never give a positive result and causes no problems if skipped should be skipped. I can't see the problem this would cause to tools: they should assume any subdirectory can be a package, with all consequences this causes. If the consequences are undesirable, users should just stop putting non-package subdirectories into a package if they want to use the tool. However, I doubt there are undesirable consequences (although consequences might be surprising at first). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] suggestion: except in list comprehension
On Wed, Apr 26, 2006, Josiah Carlson wrote: tomer filiba [EMAIL PROTECTED] wrote: first, i posted it two days ago, so it's funny it got posted only now... the moderators are sleeping on the job :) I don't believe python-dev has moderators...or at least my posts have never been delayed like that. I'm not active, but I've seen Tomer's posts getting delayed due to non-subscriber moderation. Tomer, you should make sure that all addresses you use to post are subscribed; if you have multiple addresses, you can set all but one to no-mail. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ Argue for your limitations, and sure enough they're yours. --Richard Bach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. -1 from me. I had never a problem with __init__.py to mark a package or subpackage. Better add __namespace__.py to state a package dir as namespace package. And support multiple occurrences of on_python_path/namespace_name/package. -- bye by Wolfgang ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 09:56 PM 4/26/2006 +0200, Martin v. Löwis wrote: Phillip J. Eby wrote: My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. I assume you want import x.y to fail if y is an empty directory (or non-empty, but without .py files). I don't see a value in implementing such a restriction. No, I'm saying that tools which are looking for packages and asking, Is this directory a package? should decide no in the case where it contains no modules. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby wrote: At 11:50 AM 4/26/2006 -0700, Guido van Rossum wrote: I'm not sure what you mean by one directory read. You'd have to list the entire directory, which may require reading more than one block if the directory is large. You have to do this to find an __init__.py too, don't you? Technically, there's going to be a search for a .pyc or .pyo first, anyway. No. Python does stat(2) and open(2) to determine whether a file is present in a directory. Whether or not that causes a full directory scan depends on the operating system. On most operating systems, it is *not* a full directory scan: - on network file systems, the directory is read only on the server; a full directory read would also cause a network transmission of the entire directory contents - on an advanced filesystem (such as NTFS), a lookup operation is a search in a balanced tree, rather than a linear search, bypassing many directory blocks for a large directory - on an advanced operating system (such as Linux), a repeated directory lookup for the file will not go to the file system each time, but cache the result of the lookup in an efficient memory structure. In all cases, the directory contents (whether read from disk into memory or not) does not have to be copied into python's address space for stat(2), but does for readdir(3). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby wrote: I assume you want import x.y to fail if y is an empty directory (or non-empty, but without .py files). I don't see a value in implementing such a restriction. No, I'm saying that tools which are looking for packages and asking, Is this directory a package? should decide no in the case where it contains no modules. Ah. Tools are of course free to do that. It would slightly deviate from Python's implementation of import, but the difference wouldn't matter for all practical purposes. So from a language lawyers' point of view, I would specify: A sub-package is a sub-directory of a package that contains at least one module file. Python implementations MAY accept sub-directories as sub-packages even if they contain no module files as package, instead of raising ImportError on attempts to import that sub-package. (a module file, in that context, would be a file file which matches imp.get_suffixes(), in case that isn't clear) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] (Context: There's a large crowd with pitchforks and other sharp pointy farm implements just outside the door of my office at Google. They are making an unbelievable racket. It appears they are Google engineers who have been bitten by a misfeature of Python, and they won't let me go home before I have posted this message.) One particular egregious problem is that *subpackage* are subject to the same rule. It so happens that there is essentially only one top-level package in the Google code base, and it already has an __init__.py file. But developers create new subpackages at a frightening rate, and forgetting to do touch __init__.py has caused many hours of lost work, not to mention injuries due to heads banging against walls. It seems to me that the right way to fix this is to simply make a small change to the error message. On a failed import, have the code check if there is a directory that would have been the requested package if it had contained an __init__ module. If there is then append a message like You might be missing an __init__.py file. It might also be good to check that the directory actually contained python modules. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 04:33 PM 4/26/2006 -0400, Joe Smith wrote: It seems to me that the right way to fix this is to simply make a small change to the error message. On a failed import, have the code check if there is a directory that would have been the requested package if it had contained an __init__ module. If there is then append a message like You might be missing an __init__.py file. It might also be good to check that the directory actually contained python modules. This is a great idea, but might be hard to implement in practice with the current C implementation of import, at least for the general case. But if we're talking about subpackages only, the common case is a one-element __path__, and for that case there might be something we could do. (The number of path items is relevant because the existence of a correctly-named but init-less directory should not stop later path items from being searched, so the actual error occurs far from the point where the empty directory would've been detected.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 01:49 PM 4/26/2006 -0700, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. I know exactly how you feel. :) But there's always Python 3.0, and if we're refactoring the import machinery there, we can do this the right way, not just the right now way. ;) IMO, if Py3K does this, it can and should be inclusive of top-level packages and assemble __path__ using all the sys.path entries. If we're going to break it, let's break it all the way. :) I'm still really curious why the importer solution (especially if tucked away in a Google-defined sitecustomize) won't work, though. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Terry Jones [EMAIL PROTECTED] wrote: I would suggest adding a hook to their version control system to automatically create (and preferably also check out) an __init__.py file whenever a new (source code) directory was placed under version control (supposing you can distinguish source code directories from the check in dirname). This wouldn't work of course -- the newbie would try to test it before checking it in, so the hook would not be run. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inheriting basic types more efficiently
OK, let's get back to the bitarray type. To first be clear about the *type* of bitarray: The type I talk about is really a *bit*-array and not a *bytewise*-array, as most developers seem to think of it. The array doesn't provide the boolean manipulation of single bytes in the array, it provides the manipulation of single bits or slices of bits in a bit sequence of unlimited length. Think of it like of a data sequence for a touring mashine (just bits). The reason why I'd like to use the long type as the base of my bitarray type is that the long type is already implemented as an array and working efficiently with large amounts of bytes. Also, it already supports a lot of necessary (including boolean) operators. The slicing/ manipulation of ranges can be implemented in the bitarray class. That's working similar to the example below: class bitarray(long): ... def __setitem__(self, key, value): # in this example only decimal values are supported if type(value) not in (int, long): raise ValueError # slices are supported if type(key) not in (int, long, slice): raise KeyError # checking if the target is a single bit if type(key) in (int, long) and value not in (0, 1): raise ValueError # checking if a range of bits is targeted and if the value fits if type(key) == slice: # let's assume that the slice object # provides only positive values and # that step is set to default (1) if value 0 or value (key.stop - key.start): raise ValueError key = key.start # only the offset is needed from now on # pushing up value to the offset # and adding it to the bit sequence long.__add__(2**key * value) # Should actually overwrite the # range but I couldn't even get # this to work. long complains # about the value being of type # int. But how can I convert it # to long if the term long is # already reserved for the # superclass? At least I was # too stupid to find it out. This small hack shows well that the long type could be a very good base for the bitarray type when it comes to processing (what comes out). However, when it comes to retrieving (what comes in), the situation is quite different. The bitarray type is a type for fiddling around. It shall also provide configurable truthtables and carry rules. To be able to work with the bitarray type more *natively*, it shall support bitstring input like 101010. But if the bitarray type is based on the long type and one tries the following: x = bitarray() x += 101010 the above bitstring is interpreted as the decimal value 101010, which is wrong. There seems to be no other way to change this than overwriting all the inherited methods to let them interpret the bitstring correctly. This is counterproductive. If there was a way to advice the interpreter to *filter* the input values before they are passed to the called attribute, one could keep the original methods of the long type and still provide bitstring input support. A method like __validate__ or similar could provide this filter. The interpreter first calls __validate__ to receive the return values and pass them on to the called method/attribute. Do you now understand the case? Dennis On Wed, 26 Apr 2006 10:47:55 -0700 Guido van Rossum [EMAIL PROTECTED] wrote: I doubt you'll get many answers. I have no idea what you're talking about. How about an example or two? On 4/26/06, Dennis Heuer [EMAIL PROTECTED] wrote: To bring the real problem more upfront. Up to now, inheriting classes is all about processing (the output channel) but not about retrieving (the input channel). However, though a new type can advance from an existing type if it just needs to provide some few more methods, it can not advance from an existing type if it needs to support some few other input formats--even if they all convert to the inherited type easily. The input channel is completely forgotten. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 01:49 PM 4/26/2006 -0700, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. I know exactly how you feel. :) Hardly -- you're not the BDFL. :) But there's always Python 3.0, and if we're refactoring the import machinery there, we can do this the right way, not just the right now way. ;) IMO, if Py3K does this, it can and should be inclusive of top-level packages and assemble __path__ using all the sys.path entries. If we're going to break it, let's break it all the way. :) No -- I'm actually quite happy with most of the existing behavior (though not with the APIs). I'm still really curious why the importer solution (especially if tucked away in a Google-defined sitecustomize) won't work, though. Well, we have a sitecustomize, and it's been decided that it's such a pain to get it to do the right thing that we're trying to get rid of it. So proposing to add more magic there would not work. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Apr 26, 2006, at 4:49 PM, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. No, you can not make a change which has a tiny (and arguably negative) advantage but large compatibility risks. Like it or not, Python is a stable, widely deployed program. Making almost arbitrary sideways changes is not something you can do to such a system without a lot of pushback. Breaking things requires (and *should* require) well thought out reasoning as to why the new way is actually better enough to justify the breakage. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] It might also be good to check that the directory actually contained python modules. This is a great idea, but might be hard to implement in practice with the current C implementation of import, at least for the general case. Perhaps checking and autogeneration of __init__.py should better be part of a Python-aware editor/IDE. A File menu could have a New Package entry to create a directory + empty __init__.py. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inheriting basic types more efficiently
I still don't understand what you are asking for. However there's one problem with your code: On 4/26/06, Dennis Heuer [EMAIL PROTECTED] wrote: class bitarray(long): ... def __setitem__(self, key, value): [...] long.__add__(2**key * value) # Should actually overwrite the [...] What on earth are you trying to do here? There are at least two bugs in this line: (a) long.__add__ is an unbound method so requires two arguments; (b) __setitem__ suggests that you're creating a mutable object; but long is immutable and even if your subclass isn't immutable, it still can't modify the immutable underlying long. You probably want to write a class that *has* a long (I agree that it is a good type to implement a bit array) instead of one that *is* a long. Subclassing is overrated! I don't understand at all what you're saying about decimal. You should take this to comp.lang.python; python-dev is about developing Python, not about programming questions. Even if you believe that your problem can only be solved by changes to Python, I gurantee you that it's much better to discuss this on c.l.py; most likely there's a solution to your problem (once you understand it) that does not involve changing the language or its implementation. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Google Summer of Code proposal: New class for workwith binary trees AVL and RB as with the standard dictionary.
Josiah Carlson [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Then again, you already have your own implementation of a tree module, and it seems as though you would like to be paid for this already-done work. I don't know how google feels about such things, They are explicitly against that in the student FAQ. They are paying for new code writen after the project is approved and before the endpoint. but you should remember to include this information in your application. Existing code can legitimately be used to show feasibility of the project and competancy of the student Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Joe Smith wrote: It seems to me that the right way to fix this is to simply make a small change to the error message. On a failed import, have the code check if there is a directory that would have been the requested package if it had contained an __init__ module. If there is then append a message like You might be missing an __init__.py file. +1. It's not that putting an __init__.py file in is hard, it's that people have a hard time realizing when they've forgotten to do it. -- Ian Bicking / [EMAIL PROTECTED] / http://blog.ianbicking.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: OK, forget it. I'll face the pitchforks.Maybe this'll help:http://python.org/sf/1477281 (You can call it 'oldtimer-repellant' if you want to use it to convince people there isn't any *real* backward-compatibility issue.) I'm disappointed though -- it sounds like we can never change anythingabout Python any more because it will upset the oldtimers. That's a bit unfair, Guido. There are valid reasons not to change Python's behaviour in this respect, regardless of upset old-timers. Besides, you're the BDFL; if you think the old-timers are wrong, I implore you to put their worries aside (after dutiful contemplation.) I've long since decided that any change what so ever will have activist luddites opposing it. I think most of them would stop when you make a clear decision -- how much whining have you had about the if-else syntax since you made the choice? I've heard lots of people gripe about it in private (at PyCon, of course, I never see Pythonistas anywhere else :-P), but I haven't seen any python-dev rants about it. I certainly hate PEP-308's guts, but if if-else is your decision, if-else is what we'll do. And so it is, I believe, with this case. -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Thomas Wouters [EMAIL PROTECTED] wrote: On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: OK, forget it. I'll face the pitchforks. Maybe this'll help: http://python.org/sf/1477281 (You can call it 'oldtimer-repellant' if you want to use it to convince people there isn't any *real* backward-compatibility issue.) I'd worry that it'll cause complaints when the warning is incorrect and a certain directory is being skipped intentionally. E.g. the string directory that someone had. Getting a warning like this can be just as upsetting to newbies! I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. That's a bit unfair, Guido. There are valid reasons not to change Python's behaviour in this respect, regardless of upset old-timers. Where are the valid reasons? All I see is knee-jerk -1, -1, -1, and this might cause tools to do the wrong thing. Not a single person attempted to see it from the newbie POV; several people explicitly rejected the newbie POV as invalid. I still don't know the name of any tool that would break due to this *and where the breakage wouldn't be easy to fix by adjusting the tool's behavior*. Yes, fixing tools is a pain. But they have to be fixed for every new Python version anyway -- new keywords, new syntax, new bytecodes, etc. Besides, you're the BDFL; if you think the old-timers are wrong, I implore you to put their worries aside (after dutiful contemplation.) I can only do that so many times before I'm no longer the BDFL. It's one thing to break a tie when there is widespread disagreement amongst developers (like about the perfect decorator syntax). It's another to go against a see of -1's. I've long since decided that any change what so ever will have activist luddites opposing it. I think most of them would stop when you make a clear decision -- how much whining have you had about the if-else syntax since you made the choice? I've heard lots of people gripe about it in private (at PyCon, of course, I never see Pythonistas anywhere else :-P), but I haven't seen any python-dev rants about it. I certainly hate PEP-308's guts, but if if-else is your decision, if-else is what we'll do. And so it is, I believe, with this case. OK. Then I implore you, please check in that patch (after adding error checking for PyErr_Warn() -- and of course after a2 hs been shipped), and damn the torpedoes. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Guido van Rossum [EMAIL PROTECTED] wrote: On 4/26/06, Thomas Wouters [EMAIL PROTECTED] wrote: On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: OK, forget it. I'll face the pitchforks. Maybe this'll help: http://python.org/sf/1477281 (You can call it 'oldtimer-repellant' if you want to use it to convince people there isn't any *real* backward-compatibility issue.)I'd worry that it'll cause complaints when the warning is incorrectand a certain directory is being skipped intentionally. E.g. thestring directory that someone had. Getting a warning like this can be just as upsetting to newbies!I don't think getting a spurious warning is as upsetting as getting no warning but the damned thing just not working. At least you have something to google for. And the warning includes the original line of source that triggered it *and* the directory (or directories) it's complaining about, which is quite a lot of helpful hints. The clashes with directories that aren't intended to be packages *and* a module of the same name is imported, yes, that's a real problem. It's not any worse than if we change package imports the way you originally proposed, though, and I think the actual number of spurious errors is very small (which self-respecting module still does 'import string', eh? :-) I don't think the fix for such a warning is going to be non-trivial. Where are the valid reasons?Of course, I only consider *my* reasons to be valid, and mine weren't knee-jerk or tool-related. I don't think Python should be going Oh, what you wanted wasn't possible, but I think I know what you wanted, let me do it for you, first of all because it's not very Pythonic, and second of all because it doesn't lower the learning curve, it just delays some upward motion a little (meaning the curve may become steeper, later.) A clear warning, on the other hand, can be a helpful nudge towards the 'a-HA' moment. Then I implore you, please check in that patch (after adding errorchecking for PyErr_Warn() -- and of course after a2 hs been shipped), and damn the torpedoes.Alrighty then. The list has about 12 hours to convince me (and you) that it's a bad idea to generate that warning. I'll be asleep by the time the trunk un-freezes, and I have a string of early meetings tomorrow. I'll get to it somewhere in the afternoon :) -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: http://python.org/sf/1477281 (You can call it 'oldtimer-repellant' if you want to use it to convince people there isn't any *real* backward-compatibility issue.) I'd worry that it'll cause complaints when the warning is incorrect and a certain directory is being skipped intentionally. E.g. the string directory that someone had. Getting a warning like this can be just as upsetting to newbies! I really think it would be more useful having an ImportError containing a suggestion than having a warning. Anyone who knows it's bogus can just ignore it. I'd probably make it that all directories that could have been imports get listed. FWIW I was definitely a kneejerk -1. After reading all the messages in this thread, I think I'm now a non-kneejerk -1. It seems like gratuitous change introducing inconsistency for minimal benefit - esp. if there is a notification that a directory *could* have been a package on import failure. I think it's a useful feature of Python that it's simple to distinguish a package from a non-package. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Thomas Wouters [EMAIL PROTECTED] wrote: Of course, I only consider *my* reasons to be valid, and mine weren't knee-jerk or tool-related. I don't think Python should be going Oh, what you wanted wasn't possible, but I think I know what you wanted, let me do it for you, first of all because it's not very Pythonic, and second of all because it doesn't lower the learning curve, it just delays some upward motion a little (meaning the curve may become steeper, later.) A clear warning, on the other hand, can be a helpful nudge towards the 'a-HA' moment. That still sounds like old-timer reasoning. Long ago we were very close to defining a package as a directory -- with none of this must contain __init__.py or another *.py file nonsense. IIRC the decision to make __init__.py mandatory faced opposition too, since people were already doing packages with just directories (which is quite clean and elegant, and that's also how it was in Java), but I added it after seeing a few newbies tear out their hair. I believe that if at that time __init__.py had remained optional, and today I had proposed to require it, the change would have been derided as unpythonic as well. There's nothing particularly unpythonic about optional behavior; e.g. classes may or may not provide an __init__ method. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Delaney, Timothy (Tim) [EMAIL PROTECTED] wrote: I really think it would be more useful having an ImportError containing a suggestion than having a warning. Anyone who knows it's bogus can just ignore it. That's effectively what Thomas's patch does though -- if at the end the path the package isn't found, you'll still get an ImportError; but it will be preceded by ImportWarnings showing you the non-package directories found on the way. The difference is that if you find a valid module package later on the path, you'll still get warnings. But occasionally those will be useful too -- when you are trying to create a package that happens to have the same name as a standard library package or module, it's a lot easier to debug the missing __init__.py if you get a warning about it instead of having to figure out that the foo you imported is not the foo you intended. The symptom is usually a mysterious AttributeError that takes a bit of determination to debug. Of course print foo.__file__ usually sets you right, but that's not the first thing a newbie would try -- they aren't quite sure about all the rules of import, so they are likely to first try to fiddle with $PYTHONPATH or sys.path, then put print statements in their package, etc. I'd probably make it that all directories that could have been imports get listed. Thomas' patch does this automatically -- you get a warning for each directory that is weighed and found too light. FWIW I was definitely a kneejerk -1. After reading all the messages in this thread, I think I'm now a non-kneejerk -1. It seems like gratuitous change introducing inconsistency for minimal benefit - esp. if there is a notification that a directory *could* have been a package on import failure. I think it's a useful feature of Python that it's simple to distinguish a package from a non-package. The change is only gratuitous if you disagree with it. The inconsistency is real but we all know the line about a foolish consistency. The benefit is minimal unless you've wasted an hour debugging this situation. Is it also useful to distinguish a subpackage from a non-subpackage? Anyway, the warning is more compatible and just as helpful so we'll go with that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: The difference is that if you find a valid module package later on the path, you'll still get warnings. This is the bit I don't like about it. Currently the warnings are displayed as the packages are found. I'd be quite happy with the warnings if they were suppressed until there is an actual ImportError - at which time they'll be displayed. But occasionally those will be useful too -- when you are trying to create a package that happens to have the same name as a standard library package or module, it's a lot easier to debug the missing __init__.py if you get a warning about it instead of having to figure out that the foo you imported is not the foo you intended. I suspect that more often any warnings when there is a successful import will be spurious. But testing in the field will reveal that. So I say alpha 3 should have Thomas' patch as is, and it can be changed afterwards if necessary. Thomas' patch does this automatically -- you get a warning for each directory that is weighed and found too light. Not exactly - you'll get warnings for directories earlier in sys.path than a matching package. Potential packages later in the path won't be warned about. If you're trying to resolve import problems, it's just as likely that the package you really want is later in sys.path than earlier. Obviously in the case that you get an ImportError this goes away. Is it also useful to distinguish a subpackage from a non-subpackage? Possibly. Perhaps it would be useful to have `is_package(dirname)`, `is_rootpackage(dirname)` and `is_subpackage(dirname)` functions somewhere (pkgutils?). Then the tools objections go away, and whatever mechanism is necessary to determine this (e.g. is_rootpackage checks if the directory is importable via sys.path) can be implemented. Anyway, the warning is more compatible and just as helpful so we'll go with that. Fair enough. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 01:10 AM 4/27/2006 +0200, Thomas Wouters wrote: On 4/27/06, Guido van Rossum mailto:[EMAIL PROTECTED][EMAIL PROTECTED] wrote: I'd worry that it'll cause complaints when the warning is incorrect and a certain directory is being skipped intentionally. E.g. the string directory that someone had. Getting a warning like this can be just as upsetting to newbies! I don't think getting a spurious warning is as upsetting as getting no warning but the damned thing just not working. At least you have something to google for. And the warning includes the original line of source that triggered it *and* the directory (or directories) it's complaining about, which is quite a lot of helpful hints. +1. If the warning is off-base, you can rename the directory or suppress the warning. As for the newbie situation, ISTM that this warning will generally come close enough in time to an environment change (new path entry, newly created conflicting directory) to be seen as informative. The only time it might be confusing is if you had just added an import foo after having a foo directory sitting around for a while. But even then, the warning is saying, hey, it looked like you might have meant *this* foo directory... if so, you're missing an __init__. So, at that point I rename the directory... or maybe add the __init__ and break my code. So then I back it out and put up with the warning and complain to c.l.p, or maybe threaten Guido with a pitchfork if I work at Google. Or maybe just a regular-sized fork, since the warning is just annoying. :) Alrighty then. The list has about 12 hours to convince me (and you) that it's a bad idea to generate that warning. I'll be asleep by the time the trunk un-freezes, and I have a string of early meetings tomorrow. I'll get to it somewhere in the afternoon :) I like the patch in general, but may I suggest PackageWarning or maybe BrokenPackageWarning instead of ImportWarning as the class name? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Delaney, Timothy (Tim) [EMAIL PROTECTED] wrote: Potential packages later in the path won't be warned about. If you're trying to resolve import problems, it's just as likely that the package you really want is later in sys.path than earlier. But module hiding is a feature, and anyway, we're not going to continue to search sys.path after we've found a valid match (imagine doing this on *every* successful import!), so you're not going to get warnings about those either way. Obviously in the case that you get an ImportError this goes away. Right. Is it also useful to distinguish a subpackage from a non-subpackage? Possibly. Perhaps it would be useful to have `is_package(dirname)`, `is_rootpackage(dirname)` and `is_subpackage(dirname)` functions somewhere (pkgutils?). YAGNI. Also note that not all modules or packages are represented by pathnames -- they could live in zip files, or be accessed via whatever other magic an import handler users. (Thomas's warning won't happen in those cases BTW -- it only affects the default import handler.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 04:57 PM 4/26/2006 -0700, Guido van Rossum wrote: On 4/26/06, Delaney, Timothy (Tim) [EMAIL PROTECTED] wrote: Possibly. Perhaps it would be useful to have `is_package(dirname)`, `is_rootpackage(dirname)` and `is_subpackage(dirname)` functions somewhere (pkgutils?). YAGNI. Also note that not all modules or packages are represented by pathnames -- they could live in zip files, or be accessed via whatever other magic an import handler users. FYI, pkgutil in 2.5 has utilities to walk a package tree, starting from sys.path or a package __path__, and it's PEP 302 compliant. pydoc now uses this in place of directory inspection, so that documenting zipped packages works correctly. These functions aren't documented yet, though, and probably won't be until next week at the earliest. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] suggestion: except in list comprehension
first, i posted it two days ago, so it's funny it got posted only now... the moderators are sleeping on the job :)anyway. Note that of the continue cases you offer, all of them are merely simple if conditionyes, i said that explicitly, did you *read* my mail?but i also said it's not always possible. you *can't* always tell prior to doing somethingthat it's bound to fail. you may have os.path.isfile , but not an arbitrary is_this_going_to_fail(x)and doing [1.0 / x for x in y if z(x != 0)]is quite an awkward way to say break... if then_break(cond) instead of if cond then break- - - - -anyway, i guess my friend may have better show-cases, as he's the one who found theneed for it. i just thought i should bring this up here. if you think better show cases would convince you, i can ask him. If you couldn't guess; -1, you can get equivalent behavior without complicating the generator _expression_/list comprension syntax.so how come list comprehensions aren't just a complication to the syntax? you can always do them the old way,x = []for : x.append(...)but i since people find {shorter / more to-the-point / convenience} reason enough to have introduced the list-comprehension syntax in the first place, there's also a case for adding exception handling to it.if the above snippet deserves a unique syntax, why not extend it to cover this as well?x = [] for : try: x.append(...) except: ...and it's such a big syntactic change.don't worry, i'm not going to argue it past this.-tomer On 4/26/06, Josiah Carlson [EMAIL PROTECTED] wrote: tomer filiba [EMAIL PROTECTED] wrote: [ expr for expr in expr [if cond] [except exception-class-or-tuple: action] ] Note that of the continue cases you offer, all of them are merely simpleif condition (though the file example could use a better test thanos.path.isfile).[x for x in a if x.startswith(y) except AttributeError: continue] [x for x in a if hasattr(x, 'startswith') and x.startswith(y)][1.0 / x for x in y except ZeroDivisionError: continue][1.0 / x for x in y if x != 0][open(filename) for filename in filelist except IOError: continue] [open(filename) for filename in filelist if os.path.isfile(filename)]The break case can be implemented with particular kind of instanceobject, though doesn't have the short-circuiting behavior... class StopWhenFalse:def __init__(self):self.t = 1def __call__(self, t):if not t:self.t = 0return 0return self.tz = StopWhenFalse() Assuming you create a new instance z of StopWhenFalse before doing thelist comprehensions...[x for x in a if z(hasattr(x, 'startswith') and x.startswith(y))][1.0 / x for x in y if z(x != 0)] [open(filename) for filename in filelist if z(os.path.isfile(filename))]If you couldn't guess; -1, you can get equivalent behavior withoutcomplicating the generator _expression_/list comprension syntax. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Thursday 27 April 2006 05:50, Phillip J. Eby wrote: Anyway, I'm not opposed to the idea of supporting this in future Pythons, but I definitely think it falls under the but sometimes never is better than RIGHT now rule where 2.5 is concerned. :) I agree fully. I don't think we should try and shove this into Python 2.5 on short notice, but I could be convinced otherwise. Right now, though, I'm a strong -1 for now for this in 2.5. If it's to go forward, I think it _definitely_ needs a PEP outlining the potential breakages (and I'm not sure we're aware of them all yet). In particular, I'm worried that you're shrugging off the extent of the collateral damage here, and I'd be happiest if we waited until 3.0 before changing this particular rule -- and if we changed it in favor of namespace packages, which will more closely match naive user expectations. The breakage of tools and the like is my concern, too. Python's import machinery is already a delicate mess of subtle rules. -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Thursday 27 April 2006 06:49, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. I'm not averse to changing this - just not to changing it on short notice for 2.5 and causing me pain from having to cut new releases to fix some breakage in the stdlib caused by this wink -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Anthony Baxter [EMAIL PROTECTED] wrote: On Thursday 27 April 2006 06:49, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. I'm not averse to changing this - just not to changing it on short notice for 2.5 and causing me pain from having to cut new releases to fix some breakage in the stdlib caused by this wink I wasn't proposing it for 2.5 (for this very reason) -- AFAICT most responses were again st this for 2.6 or 2.7. Hopefully you're okay with Thomas's patch going into 2.5a3 -- it *warns* about directories without __init__.py that otherwise match the requested name. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Boy, threads here sure move fast when there's work to be done :). Although largely moot now, I'll follow up for posterity's sake. On Wed, 2006-04-26 at 10:59 -0700, Guido van Rossum wrote: Oh, cool gray area. I propose that if there's no __init__.py it prints 'path/sub1/sun2/' i.e. with a trailing slash; that causes dirname to just strip the '/'. (It would be a backslash on Windows of course). Yep, that's the right thing to do. Given all the other traffic in this thread, I don't have much more to add except: I probably don't remember things accurately, but ISTR that __init__.py was added as a way to expose names in the package's namespace first. We probably went from that to requiring __init__.py as a way to be explicit about what directories are packages and which aren't. So I suspect you're right when you say that if the rule had already been relaxed and you were now proposing to tighten the rules, we probably get just as many complaints. alternative-universe-ly y'rs, -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Wednesday 26 April 2006 22:42, Barry Warsaw wrote: So I suspect you're right when you say that if the rule had already been relaxed and you were now proposing to tighten the rules, we probably get just as many complaints. Indeed. I think the problem many of us have with the proposal isn't the new behavior, but the change in the behavior. That's certainly it for me. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Fred L. Drake, Jr. [EMAIL PROTECTED] wrote: Indeed. I think the problem many of us have with the proposal isn't the new behavior, but the change in the behavior. That's certainly it for me. Which is why I said earlier that I felt disappointed that we can't change anything any more. But I'm fine with the warning -- it should be enough to keep Google's newbies from wasting their time. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] draft of externally maintained packages PEP
On 4/25/06, Thomas Heller [EMAIL PROTECTED] wrote: Brett Cannon wrote: here is the rough draft of the PEP for packages maintained externally from Python itself. There is some missing, though, that I would like help filling in. I don't know who to list as the contact person (i.e., the Python developer in charge of the code) for Expat, Optik or pysqlite. I know Greg Ward wrote Optik, but I don't know if he is still active at all (I saw AMK's email on Greg being busy). I also thought Gerhard Haering was in charge of pysqlite, but he has never responded to any emails about this topic. Maybe the responsibility should go to Anthony since I know he worked to get the package in and probably cares about keeping it updated? As for Expat (the parser), is that Fred? I also don't know what version of ctypes is in 2.5 . Thomas, can you tell me? ctypes -- - Web page http://starship.python.net/crew/theller/ctypes/ - Standard library name ctypes - Contact person Thomas Heller - Synchronisation history * 0.9.9.4 (2.5a1) * 0.9.9.6 (2.5a2) I am not going to worry about alphas, so I just set it to 0.9.9.6 for now and can update once 2.5.0 is released. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] draft of externally maintained packages PEP
On 4/26/06, Gerhard Häring [EMAIL PROTECTED] wrote: Brett Cannon wrote: here is the rough draft of the PEP for packages maintained externally from Python itself. There is some missing, though, that I would like help filling in. I don't know who to list as the contact person (i.e., the Python developer in charge of the code) for Expat, Optik or pysqlite. [...] I also thought Gerhard Haering was in charge of pysqlite, but he has never responded to any emails about this topic. Sorry for not answering any sooner. Please list me as contact person for the SQLite module. OK, great. Maybe the responsibility should go to Anthony since I know he worked to get the package in and probably cares about keeping it updated? As for Expat (the parser), is that Fred? [...] pysqlite - Web site http://www.sqlite.org/ - Standard library name sqlite3 - Contact person XXX - Synchronisation history * 2.1.3 (2.5) You can add * 2.2.0 * 2.2.2 I put in 2.2.2 for my copy since I am not going to worry about alphas. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Wed, Apr 26, 2006, Guido van Rossum wrote: Which is why I said earlier that I felt disappointed that we can't change anything any more. I've been here since Python 1.5.1. I don't understand why this issue in particular makes you feel disappointed. I also think your statement is just plain untrue. Looking at the What's New for Python 2.5, we have changes to the import machinery, the with statement, new functionality for generators, conditional expressions, ssize_t, exceptions as new-style classes, deleted modules (regex, regsub, and whrandom), and on and on and on. Some of these changes are going to cause moderate breakage for some people; they cumulatively represent a *lot* of change. Python 2.5 is gonna be a *huge* release. Quite honestly, I think you've let yourself get emotionally attached to this issue for some reason. On the other hand, I think that a lot of the blowback you got comes from you bringing up this issue right before 2.5a2. My suggestion is that you let this sit and make yourself a calendar entry to think about this again in October. If it still seems like a good idea, go ahead and bring it up. (Or just punt to 3.0.) -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ Argue for your limitations, and sure enough they're yours. --Richard Bach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
On 4/26/06, Gerhard Häring [EMAIL PROTECTED] wrote: Brett Cannon wrote: OK, I am going to write the PEP I proposed a week or so ago, listing all modules and packages within the stdlib that are maintained externally so we have a central place to go for contact info or where to report bugs on issues. This should only apply to modules that want bugs reported outside of the Python tracker and have a separate dev track. People who just use the Python repository as their mainline version can just be left out. [...] I prefer to have bugs on the sqlite module reported in the pysqlite tracker at http://pysqlite.org/ aka http://initd.org/tracker/pysqlite For bug fixes I have the same position as Fredrik: security fixes, bugs that block the build and warnings from automatic checkers should be done through the Python repository and I will port them to the external version. For any changes that reach deeper I'd like to have them handed over to me. Done in my copy. That just leaves who is to be listed for in charge of the expat parser. As it's unrealistic that all bugs are reported through the pysqlite tracker, can it please be arranged that if somebody enters a SQLite related bug through the Sourceforge tracker, that I get notified? Perhaps by defining a category SQLite here and adding me as default responsible, if that's possible on SF. It is, although I can't do it. Currently I'm not subscribed to python-checkins and didn't see a need to. Is there a need to for Python core developers? I think there's no better way except subscribing and defining a filter for SQLite-related commits to be notified if other people commit changes to the SQLite module in Python? It's not that I'm too lazy, but I'd like to keep the number of things I need to monitor low. Understandable. I think setting up a filter is probably the best way. Can always get the digest to also cut down on the email as well. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PY_FORMAT_SIZE_T warnings on OS X
On 4/26/06, Tim Peters [EMAIL PROTECTED] wrote: [Brett Cannon] I created patch 1474907 with a fix for it. Checks if %zd works for size_t and if so sets PY_FORMAT_SIZE_T to z, otherwise just doesn't set the macro def. Assigned to Martin to make sure I didn't foul it up, but pretty much anyone could probably double-check it. [Martin v. Löwis] Unfortunately, SF stopped sending emails when you get assigned an issue, so I didn't receive any message when you created that patch. I now reviewed it, and think the test might not fail properly on systems where %zd isn't supported. I agree with Martin's patch comments. The C standards don't require that printf fail if an unrecognized format gimmick is used (behavior is explicitly undefined then). For example, #include stdio.h #include stdlib.h int main() { int i; i = printf(%zd\n, (size_t)0); printf(%d\n, i); return 0; } prints zd\n3\n under all of MSVC 6.0, MSVC 7.1, and gcc (Cygwin) on WinXP. Using sprintf instead and checking the string produced seems much more likely to work. After the trunk freeze melts, I suggest just _trying_ stuff. The buildbot system covers a bunch of platforms now, and when trying to out-hack ill-defined C stuff try it and see is easier than thinking 0.5 wink. I uploaded a new version that uses sprintf() and then does strncmp() on the result to see if it matches what is expected. That should be explicit enough to make sure that only a properly supported z modifier results in a successful test. If anyone has a chance to check it again before the trunk unfreezes, that would be great. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inheriting basic types more efficiently
Dennis Heuer wrote: The reason why I'd like to use the long type as the base of my bitarray type is that the long type is already implemented as an array and working efficiently with large amounts of bytes. This is wrong; you are mixing up the is-a and has-a relationships. While it might be useful to have long as the *representation* of a bitarray, it's not the case that a bitarray *is* a long. A bitarray is a sequence type and should implement all aspects of the sequence protocol; long is a numeric type and should implement numeric operations. While the operators for these somewhat overlap (for + and *), the semantics of the operators never overlaps. So long and bitarray are completely distinct types, not subtypes of each other. IOW, you should do class bitarray: def __init__(self): self.value = 0L ... Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com