Re: [Python-Dev] Support for the Haiku OS
On Sun, Jan 18, 2009 at 11:03 PM, scott mc scott...@gmail.com wrote: I built 2.7 on Haiku, but am getting failures in the regression tests. Many of them are in math related tests, failing in the 15th decimal place on test_decimal and a few others like that, I posted a ticket on Haiku's trac for that as it might be related to Haiku's built in math lib? (libm is built into Haiku's libroot.so) http://dev.haiku-os.org/ticket/3308 Most of these look like libm/libc precision problems to me, of varying severity. Some particular comments: - the test_float result is worrying: there are a good few places where Python depends on eval(repr(.)) round-tripping for floats, and it looks as though either the eval or the repr is losing significant accuracy. Actually, there's so much accuracy loss that I wonder whether something's being cast from double precision to single precision at some point. - test_decimal failing was a bit of a surprise until I saw which test was failing: the decimal module quite deliberately does all computation using integer arithmetic, and avoids floating-point like the plague, so it should be ultra-portable. Except, of course, the recently added from_float method, which converts from floats to decimals. So fix up the floating-point and test_decimal should pass again. - I don't understand where the test_marshall and test_random failures are coming from. These could be Python problems (though I think it's more likely that they're Haiku floating-point problems). I'd be interested to see short code-snippets that reproduce these issues. - I wouldn't worry so much about the test_math and test_cmath failures until you get the others sorted out; the tests are probably stricter than they need to be. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Child process freezes during fork pipe exec
Hi, Summary: * In my organization, we have a *multi threaded* (threading library) python (python 2.4.1) daemon on Linux, which starts up various processes using the fork pipe exec model. * We use this fork , wait on pipe , exec model as a form of handshake between the parent and child processes. We want the child to go ahead only after the parent has noted down the fact that the child has been forked and what it's pid is. * This usually works fine, but for about 1 in every 20,000 processes started, the child process just freezes somewhere after the fork, before the exec. It does not die. It is alive and stuck. * Why does this happen? * Is there a better way for us to write a fork-wait_for_start_signal-exec construct? Here is what we do: One of the threads of the multi threaded python daemon does the following 1) Fork out a child process 2) Child process waits for a pipe message from parent --- Parent sends pipe message to child after noting down child details : pid, start time etc. --- 3) Child process prints various debug messages, including looking at os.environ values 4) Child process execs the right script Here it is again, in pseudo code: def start_job(): read_pipefd, write_pipefd = os.pipe() # 1) Fork out a child process pid = os.fork() if pid == 0: # 2) wait for excepted message on pipe os.close(write_pipefd) read_set, write_set, exp_set = select.select([read_pipefd], [], [], 300) if os.read(read_pipefd, len(expected message) expected message: os._exit(1) os.close(read_pipefd) # 3) print various debug messages, including os.environ values print sys.stderr, we print various debug messages here, including os.evniron values # 4) go ahead with exec os.execve(path, args, env) else: # parent process sends pipe message to child at the right time The problem: * Things work fine most of the time, but rarely, the process gets stuck after fork, before exec (In steps 2 or 3 above). Process makes no progress and does not die either. * When I do a gdb (gdb 6.5) attach on the process, bt fails as follows: (gdb) bt #0 0x2ba9fd5c6a68 in __lll_mutex_lock_wait () from /lib64/libpthread.so.0 #1 0x2ba9fd5c2a78 in _L_mutex_lock_106 () from /lib64/libpthread.so.0 dwarf2-frame.c:521: internal-error: Unknown CFI encountered. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) I looked into this error and found that pre-6.6 gdb throws this error when looking at the stack trace of a deadlocked process. This is certainly not a dead lock in my code as there is no locking involved in this area of code. * This problem happens for about 1 process in every 20,000. This statistics is gathered across about 80 machines in our cluster, so its not the case of a single machine having some hardware issue. * Note that the child is forked out by a *multi threaded* python application. I noticed some forums discussing how multi threaded (pthreads library) processes doing things between a fork and an exec can rarely get into a deadlock. I know that python ( atleast 2.4.1 ) multi threading does not use pthreads, but probably the python interpreter itself does use pthreads? Questions: * Why does this happen? * Is there a better way for us to write a fork-wait_for_start_signal-exec construct in a multi threaded application? Thanks, Gangadharan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Child process freezes during fork pipe exec
Gangadharan S.A. wrote: Hi, Summary: * In my organization, we have a *multi threaded* (threading library) python (python 2.4.1) daemon on Linux, which starts up various processes using the fork pipe exec model. The fork+threading combination had some fairly major issues that weren't resolved until the multiprocessing module was added for 2.6/3.0 [1] If you're able to upgrade to 2.5.4 things should be much better since many of the fork+threading fixes were backported to the 2.5 maintenance branch. For 2.4 I think you're pretty much out of luck though - it was already into security fix only mode by the time the child process deadlock problems in the fork/threading interaction were worked out. I don't personally know if your implementation could be modified in any way to make the deadlock less likely with Python 2.4, but the only way to eliminate the problem entirely (modulo any platform specific fork+threading problems) is to upgrade to the latest version of Python 2.5. Regards, Nick. [1] The bug you're probably encountering: http://bugs.python.org/issue874900 -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 3142: Add a while clause to generator expressions
Please find below PEP 3142: Add a while clause to generator expressions. I'm looking for feedback and discussion. PEP: 3142 Title: Add a while clause to generator expressions Version: $Revision: 68715 $ Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $ Author: Gerald Britton gerald.brit...@gmail.com Status: Draft Type: Standards Track Content-Type: text/plain Created: 12-Jan-2009 Python-Version: 3.0 Post-History: Abstract This PEP proposes an enhancement to generator expressions, adding a while clause to complement the existing if clause. Rationale A generator expression (PEP 289 [1]) is a concise method to serve dynamically-generated objects to list comprehensions (PEP 202 [2]). Current generator expressions allow for an if clause to filter the objects that are returned to those meeting some set of criteria. However, since the if clause is evaluated for every object that may be returned, in some cases it is possible that all objects would be rejected after a certain point. For example: g = (n for n in range(100) if n*n 50) which is equivalent to the using a generator function (PEP 255 [3]): def __gen(exp): for n in exp: if n*n 50: yield n g = __gen(iter(range(10))) would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider the numbers from 8 to 99 and reject them all since n*n = 50 for numbers in that range. Allowing for a while clause would allow the redundant tests to be short-circuited: g = (n for n in range(100) while n*n 50) would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8 since the condition (n*n 50) is no longer true. This would be equivalent to the generator function: def __gen(exp): for n in exp: if n*n 50: yield n else: break g = __gen(iter(range(100))) Currently, in order to achieve the same result, one would need to either write a generator function such as the one above or use the takewhile function from itertools: from itertools import takewhile g = takewhile(lambda n: n*n 50, range(100)) The takewhile code achieves the same result as the proposed syntax, albeit in a longer (some would say less-elegant) fashion. Also, the takewhile version requires an extra function call (the lambda in the example above) with the associated performance penalty. A simple test shows that: for n in (n for n in range(100) if 1): pass performs about 10% better than: for n in takewhile(lambda n: 1, range(100)): pass though they achieve similar results. (The first example uses a generator; takewhile is an iterator). If similarly implemented, a while clause should perform about the same as the if clause does today. The reader may ask if the if and while clauses should be mutually exclusive. There are good examples that show that there are times when both may be used to good advantage. For example: p = (p for p in primes() if p 100 while p 1000) should return prime numbers found between 100 and 1000, assuming I have a primes() generator that yields prime numbers. Of course, this could also be achieved like this: p = (p for p in (p for p in primes() if p 100) while p 1000) which is syntactically simpler. Some may also ask if it is possible to cover dropwhile() functionality in a similar way. I initially thought of: p = (p for p in primes() not while p 100) but I am not sure that I like it since it uses not in a non-pythonic fashion, I think. Adding a while clause to generator expressions maintains the compact form while adding a useful facility for short-circuiting the expression. Implementation: I am willing to assist in the implementation of this feature, although I have not contributed to Python thus far and would definitely need mentoring. (At this point I am not quite sure where to begin.) Presently though, I would find it challenging to fit this work into my existing workload. Acknowledgements Raymond Hettinger first proposed the concept of generator expressions in January 2002. References [1] PEP 289: Generator Expressions http://www.python.org/dev/peps/pep-0289/ [2] PEP 202: List Comprehensions http://www.python.org/dev/peps/pep-0202/ [3] PEP 255: Simple Generators http://www.python.org/dev/peps/pep-0255/ Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: PEP: 3142 Title: Add a while clause to generator expressions Version: $Revision: 68715 $ Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $ Author: Gerald Britton gerald.brit...@gmail.com Status: Draft Type: Standards Track Content-Type: text/plain Created: 12-Jan-2009 Python-Version: 3.0
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
On Mon, Jan 19, 2009 at 9:10 AM, Gerald Britton gerald.brit...@gmail.com wrote: Please find below PEP 3142: Add a while clause to generator expressions. I'm looking for feedback and discussion. PEP: 3142 Title: Add a while clause to generator expressions Version: $Revision: 68715 $ Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $ Author: Gerald Britton gerald.brit...@gmail.com Status: Draft Type: Standards Track Content-Type: text/plain Created: 12-Jan-2009 Python-Version: 3.0 Since 3.0 has already been released, the only versions this feature can be added to are 2.7 and 3.1. Do you intend this to be a 3.x only feature? -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
On Mon, Jan 19, 2009 at 9:10 AM, Gerald Britton gerald.brit...@gmail.comwrote: g = (n for n in range(100) if n*n 50) would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider the numbers from 8 to 99 and reject them all since n*n = 50 for numbers in that range. Allowing for a while clause would allow the redundant tests to be short-circuited: Instead of using a while clause, the above example could simply be rewritten: g = (n for n in range(8)) I appreciate that this is a toy example to illustrate the syntax. Do you have some slightly more complex examples, that could not be rewritten by altering the in clause? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
I am really unconvinced of the utility of this proposal and quite convinced of the confusing factor it may well add to the current syntax. I would like to see more applicable examples. It would replace uses of takewhile, but that isn't a really often used function. So, is there any evidence to support that making this a new syntax would find so many more uses of the construct to be worth it? I believe not. On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton gerald.brit...@gmail.com wrote: Please find below PEP 3142: Add a while clause to generator expressions. I'm looking for feedback and discussion. PEP: 3142 Title: Add a while clause to generator expressions Version: $Revision: 68715 $ Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $ Author: Gerald Britton gerald.brit...@gmail.com Status: Draft Type: Standards Track Content-Type: text/plain Created: 12-Jan-2009 Python-Version: 3.0 Post-History: Abstract This PEP proposes an enhancement to generator expressions, adding a while clause to complement the existing if clause. Rationale A generator expression (PEP 289 [1]) is a concise method to serve dynamically-generated objects to list comprehensions (PEP 202 [2]). Current generator expressions allow for an if clause to filter the objects that are returned to those meeting some set of criteria. However, since the if clause is evaluated for every object that may be returned, in some cases it is possible that all objects would be rejected after a certain point. For example: g = (n for n in range(100) if n*n 50) which is equivalent to the using a generator function (PEP 255 [3]): def __gen(exp): for n in exp: if n*n 50: yield n g = __gen(iter(range(10))) would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider the numbers from 8 to 99 and reject them all since n*n = 50 for numbers in that range. Allowing for a while clause would allow the redundant tests to be short-circuited: g = (n for n in range(100) while n*n 50) would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8 since the condition (n*n 50) is no longer true. This would be equivalent to the generator function: def __gen(exp): for n in exp: if n*n 50: yield n else: break g = __gen(iter(range(100))) Currently, in order to achieve the same result, one would need to either write a generator function such as the one above or use the takewhile function from itertools: from itertools import takewhile g = takewhile(lambda n: n*n 50, range(100)) The takewhile code achieves the same result as the proposed syntax, albeit in a longer (some would say less-elegant) fashion. Also, the takewhile version requires an extra function call (the lambda in the example above) with the associated performance penalty. A simple test shows that: for n in (n for n in range(100) if 1): pass performs about 10% better than: for n in takewhile(lambda n: 1, range(100)): pass though they achieve similar results. (The first example uses a generator; takewhile is an iterator). If similarly implemented, a while clause should perform about the same as the if clause does today. The reader may ask if the if and while clauses should be mutually exclusive. There are good examples that show that there are times when both may be used to good advantage. For example: p = (p for p in primes() if p 100 while p 1000) should return prime numbers found between 100 and 1000, assuming I have a primes() generator that yields prime numbers. Of course, this could also be achieved like this: p = (p for p in (p for p in primes() if p 100) while p 1000) which is syntactically simpler. Some may also ask if it is possible to cover dropwhile() functionality in a similar way. I initially thought of: p = (p for p in primes() not while p 100) but I am not sure that I like it since it uses not in a non-pythonic fashion, I think. Adding a while clause to generator expressions maintains the compact form while adding a useful facility for short-circuiting the expression. Implementation: I am willing to assist in the implementation of this feature, although I have not contributed to Python thus far and would definitely need mentoring. (At this point I am not quite sure where to begin.) Presently though, I would find it challenging to fit this work into my existing workload. Acknowledgements Raymond Hettinger first proposed the concept of generator expressions in January 2002. References [1] PEP 289: Generator Expressions http://www.python.org/dev/peps/pep-0289/ [2] PEP 202: List Comprehensions http://www.python.org/dev/peps/pep-0202/ [3] PEP 255: Simple Generators
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
Sure: Say I implement the sieve of Eratosthenes as a prime number generator. I want some primes for my application but there are an infinite number of primes. So I would like to write: prime = (p for p in sieve() while p 1000) instead of: import itertools prime = takewhile(lamda p:p1000, sieve()) to get the primes under 1000. On Mon, Jan 19, 2009 at 11:15 AM, Daniel Stutzbach dan...@stutzbachenterprises.com wrote: On Mon, Jan 19, 2009 at 9:10 AM, Gerald Britton gerald.brit...@gmail.com wrote: g = (n for n in range(100) if n*n 50) would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider the numbers from 8 to 99 and reject them all since n*n = 50 for numbers in that range. Allowing for a while clause would allow the redundant tests to be short-circuited: Instead of using a while clause, the above example could simply be rewritten: g = (n for n in range(8)) I appreciate that this is a toy example to illustrate the syntax. Do you have some slightly more complex examples, that could not be rewritten by altering the in clause? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
Thanks Calvin, Could you please expand on your thoughts about possible confusion? That is, how do you see a programmer becoming confused if this option were added to the syntax. On Mon, Jan 19, 2009 at 11:29 AM, Calvin Spealman ironfro...@gmail.com wrote: I am really unconvinced of the utility of this proposal and quite convinced of the confusing factor it may well add to the current syntax. I would like to see more applicable examples. It would replace uses of takewhile, but that isn't a really often used function. So, is there any evidence to support that making this a new syntax would find so many more uses of the construct to be worth it? I believe not. On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton gerald.brit...@gmail.com wrote: Please find below PEP 3142: Add a while clause to generator expressions. I'm looking for feedback and discussion. PEP: 3142 Title: Add a while clause to generator expressions Version: $Revision: 68715 $ Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $ Author: Gerald Britton gerald.brit...@gmail.com Status: Draft Type: Standards Track Content-Type: text/plain Created: 12-Jan-2009 Python-Version: 3.0 Post-History: Abstract This PEP proposes an enhancement to generator expressions, adding a while clause to complement the existing if clause. Rationale A generator expression (PEP 289 [1]) is a concise method to serve dynamically-generated objects to list comprehensions (PEP 202 [2]). Current generator expressions allow for an if clause to filter the objects that are returned to those meeting some set of criteria. However, since the if clause is evaluated for every object that may be returned, in some cases it is possible that all objects would be rejected after a certain point. For example: g = (n for n in range(100) if n*n 50) which is equivalent to the using a generator function (PEP 255 [3]): def __gen(exp): for n in exp: if n*n 50: yield n g = __gen(iter(range(10))) would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider the numbers from 8 to 99 and reject them all since n*n = 50 for numbers in that range. Allowing for a while clause would allow the redundant tests to be short-circuited: g = (n for n in range(100) while n*n 50) would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8 since the condition (n*n 50) is no longer true. This would be equivalent to the generator function: def __gen(exp): for n in exp: if n*n 50: yield n else: break g = __gen(iter(range(100))) Currently, in order to achieve the same result, one would need to either write a generator function such as the one above or use the takewhile function from itertools: from itertools import takewhile g = takewhile(lambda n: n*n 50, range(100)) The takewhile code achieves the same result as the proposed syntax, albeit in a longer (some would say less-elegant) fashion. Also, the takewhile version requires an extra function call (the lambda in the example above) with the associated performance penalty. A simple test shows that: for n in (n for n in range(100) if 1): pass performs about 10% better than: for n in takewhile(lambda n: 1, range(100)): pass though they achieve similar results. (The first example uses a generator; takewhile is an iterator). If similarly implemented, a while clause should perform about the same as the if clause does today. The reader may ask if the if and while clauses should be mutually exclusive. There are good examples that show that there are times when both may be used to good advantage. For example: p = (p for p in primes() if p 100 while p 1000) should return prime numbers found between 100 and 1000, assuming I have a primes() generator that yields prime numbers. Of course, this could also be achieved like this: p = (p for p in (p for p in primes() if p 100) while p 1000) which is syntactically simpler. Some may also ask if it is possible to cover dropwhile() functionality in a similar way. I initially thought of: p = (p for p in primes() not while p 100) but I am not sure that I like it since it uses not in a non-pythonic fashion, I think. Adding a while clause to generator expressions maintains the compact form while adding a useful facility for short-circuiting the expression. Implementation: I am willing to assist in the implementation of this feature, although I have not contributed to Python thus far and would definitely need mentoring. (At this point I am not quite sure where to begin.) Presently though, I would find it challenging to fit this work into my existing workload. Acknowledgements Raymond Hettinger first proposed the concept of generator expressions
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
2009/1/19 Gerald Britton gerald.brit...@gmail.com: Could you please expand on your thoughts about possible confusion? That is, how do you see a programmer becoming confused if this option were added to the syntax. My main concern about confusion is that you're adding a while that actually will behave like a break in the for. I know that with while you read it better, but the meaning is different in the generator/list comprehension for loop. Regards, -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
On Mon, Jan 19, 2009 at 10:37 AM, Gerald Britton gerald.brit...@gmail.comwrote: prime = (p for p in sieve() while p 1000) prime = takewhile(lamda p:p1000, sieve()) I'm pretty sure the extra cost of evaluating the lambda at each step is tiny compared to the cost of the sieve, so I don't you can make a convincing argument on performance. Also, you know the latter is actually fewer characters, right? :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Hmmm... __PyObject_NextNotImplemented when tracing lines
I see output like this in several tests on my Mac: test_array skipped -- dlopen(/Users/skip/src/python/trunk/build/lib.macosx-10.3-i386-2.7/cPickle.so, 2): Symbol not found: __PyObject_NextNotImplemented Referenced from: /Users/skip/src/python/trunk/build/lib.macosx-10.3-i386-2.7/cPickle.so Expected in: dynamic lookup This is in an up-to-date trunk sandbox running ./python.exe Lib/test/regrtest.py -T -D cover Didn't see that in a non-coverage pass. The following tests give this message: test_exceptions test_array test_collections test_copy_reg test_cpickle test_datetime test_deque test_fractions test_logging test_multiprocessing test_pickle test_pickletools test_re test_slice test_xpickle I see other weirdness as well including a failure of test_sys and the apparent inability to actually write out any useful coverage info, but this is the standout. In general it appears the tracing support in regrtest.py causes a bunch of problems. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
The sieve is just one example. The basic idea is that for some infinite generator (even a very simple one) you want to cut it off after some point. As for the number of characters, I spelled lambda incorrectly (left out a b) and there should be a space after the colon to conform to design guides. So, actually the takewhile version is two characters longer, not counting import itertools of course! On Mon, Jan 19, 2009 at 11:44 AM, Daniel Stutzbach dan...@stutzbachenterprises.com wrote: On Mon, Jan 19, 2009 at 10:37 AM, Gerald Britton gerald.brit...@gmail.com wrote: prime = (p for p in sieve() while p 1000) prime = takewhile(lamda p:p1000, sieve()) I'm pretty sure the extra cost of evaluating the lambda at each step is tiny compared to the cost of the sieve, so I don't you can make a convincing argument on performance. Also, you know the latter is actually fewer characters, right? :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
On Mon, Jan 19, 2009 at 11:41 AM, Gerald Britton gerald.brit...@gmail.com wrote: Thanks Calvin, Could you please expand on your thoughts about possible confusion? That is, how do you see a programmer becoming confused if this option were added to the syntax. I think that the difference between these two lines is too easy to miss among other code: prime = (p for p in sieve() while p 1000) and: prime = (p for p in sieve() if p 1000) The very distinctly different behavior is two important to look so similar at a glance. And, again, I just don't see takewhile being used all that often. Not nearly often enough to warrant replacing it with a special syntax! Only 178 results from Google CodeSearch, while chain, groupby, and repeat get 4000, 3000, and 1000 respectively. Should those be given their own syntax? On Mon, Jan 19, 2009 at 11:29 AM, Calvin Spealman ironfro...@gmail.com wrote: I am really unconvinced of the utility of this proposal and quite convinced of the confusing factor it may well add to the current syntax. I would like to see more applicable examples. It would replace uses of takewhile, but that isn't a really often used function. So, is there any evidence to support that making this a new syntax would find so many more uses of the construct to be worth it? I believe not. On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton gerald.brit...@gmail.com wrote: Please find below PEP 3142: Add a while clause to generator expressions. I'm looking for feedback and discussion. PEP: 3142 Title: Add a while clause to generator expressions Version: $Revision: 68715 $ Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $ Author: Gerald Britton gerald.brit...@gmail.com Status: Draft Type: Standards Track Content-Type: text/plain Created: 12-Jan-2009 Python-Version: 3.0 Post-History: Abstract This PEP proposes an enhancement to generator expressions, adding a while clause to complement the existing if clause. Rationale A generator expression (PEP 289 [1]) is a concise method to serve dynamically-generated objects to list comprehensions (PEP 202 [2]). Current generator expressions allow for an if clause to filter the objects that are returned to those meeting some set of criteria. However, since the if clause is evaluated for every object that may be returned, in some cases it is possible that all objects would be rejected after a certain point. For example: g = (n for n in range(100) if n*n 50) which is equivalent to the using a generator function (PEP 255 [3]): def __gen(exp): for n in exp: if n*n 50: yield n g = __gen(iter(range(10))) would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider the numbers from 8 to 99 and reject them all since n*n = 50 for numbers in that range. Allowing for a while clause would allow the redundant tests to be short-circuited: g = (n for n in range(100) while n*n 50) would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8 since the condition (n*n 50) is no longer true. This would be equivalent to the generator function: def __gen(exp): for n in exp: if n*n 50: yield n else: break g = __gen(iter(range(100))) Currently, in order to achieve the same result, one would need to either write a generator function such as the one above or use the takewhile function from itertools: from itertools import takewhile g = takewhile(lambda n: n*n 50, range(100)) The takewhile code achieves the same result as the proposed syntax, albeit in a longer (some would say less-elegant) fashion. Also, the takewhile version requires an extra function call (the lambda in the example above) with the associated performance penalty. A simple test shows that: for n in (n for n in range(100) if 1): pass performs about 10% better than: for n in takewhile(lambda n: 1, range(100)): pass though they achieve similar results. (The first example uses a generator; takewhile is an iterator). If similarly implemented, a while clause should perform about the same as the if clause does today. The reader may ask if the if and while clauses should be mutually exclusive. There are good examples that show that there are times when both may be used to good advantage. For example: p = (p for p in primes() if p 100 while p 1000) should return prime numbers found between 100 and 1000, assuming I have a primes() generator that yields prime numbers. Of course, this could also be achieved like this: p = (p for p in (p for p in primes() if p 100) while p 1000) which is syntactically simpler. Some may also ask if it is possible to cover dropwhile() functionality in a similar way. I initially thought of: p = (p for p in primes() not while p 100)
Re: [Python-Dev] Hmmm... __PyObject_NextNotImplemented when tracing lines
On Mon, Jan 19, 2009 at 11:51 AM, s...@pobox.com wrote:
I see output like this in several tests on my Mac:
test_array skipped --
dlopen(/Users/skip/src/python/trunk/build/lib.macosx-10.3-i386-2.7/cPickle.so,
2): Symbol not found: __PyObject_NextNotImplemented
Referenced from:
/Users/skip/src/python/trunk/build/lib.macosx-10.3-i386-2.7/cPickle.so
Expected in: dynamic lookup
This is in an up-to-date trunk sandbox running
./python.exe Lib/test/regrtest.py -T -D cover
..
I cannot reproduce this on my Mac. It looks like you may have an out
of date python.exe in your sandbox. Please check that
$ nm python.exe | grep PyObject_NextNotImplemented
00052940 T __PyObject_NextNotImplemented
has a T in the second column.
Note that this symbol seem to have been added recently:
$ svn blame -v Objects/object.c
..
68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
_PyObject_NextNotImplemented(PyObject *self)
68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009) {
68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
PyErr_Format(PyExc_TypeError,
68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
'%.200s' object is not iterable,
68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
Py_TYPE(self)-tp_name);
68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
return NULL;
68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009) }
68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
..
So it is possible that you did not pick it up in your build yet.
I am puzzled, however, that you don't see problems in a non-coverage run.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
On Mon, Jan 19, 2009 at 7:10 AM, Gerald Britton gerald.brit...@gmail.com wrote: PEP: 3142 Title: Add a while clause to generator expressions [snip] numbers in that range. Allowing for a while clause would allow the redundant tests to be short-circuited: g = (n for n in range(100) while n*n 50) would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8 since the condition (n*n 50) is no longer true. This would be equivalent to the generator function: def __gen(exp): for n in exp: if n*n 50: yield n else: break g = __gen(iter(range(100))) -1. As I pointed out on python-ideas, this proposal makes while mean something different in a generator expression. Currently, you can read any generator expression as a regular generator by simply indenting each clause and adding a yield statement. For example: (n for n in range(100) if n*n 50) turns into: for n in range(100): if n*n 50: yield n Applying that nice correspondence to the proposed while generator expression doesn't work though. For example: (n for n in range(100) while n*n 50) is, under the proposal, *not* equivalent to: for n in range(100): while n*n 50: yield n I'm strongly against making while mean something different in a generator expression than it does in a while statement. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
Gerald Britton wrote: Please find below PEP 3142: Add a while clause to generator expressions. I'm looking for feedback and discussion. This was already discussed on python-ideas where it got negative feedback. One objection, mentioned by Mathias Panzerbock and Georg Brandl, is that it is redundant with takewhile(). You did mention that in the PEP. The other, posted by Steven Bethard, is that it fundamentally breaks the current semantics of abbreviating (except for iteration variable scoping) an 'equivalent' for loop. This should have been listed in the PEP under Objections (or whatever the section. I did not bother to second his objection there but will now. -1 Steven summary: I'm probably just repeating myself here, but the reason not to do it is that the current generator expressions translate almost directly into the corresponding generator statements. Using while in the way you've suggested breaks this symmetry, and would make Python harder to learn. Longer presentation: I think this could end up being confusing. Current generator expressions turn into an equivalent generator function by simply indenting the clauses and adding a yield, for example: (i for i in range(100) if i % 2 == 0) is equivalent to: def gen(): for i in range(100): if i % 2 == 0: yield i Now you're proposing syntax that would no longer work like this. Taking your example: (i for i in range(100) while i = 50) I would expect this to mean: [meaning, one would expect this to mean, using current rules = tjr] def gen(): for i in range(100): while i = 50: yield i In short, -1. You're proposing to use an existing keyword in a new way that doesn't match how generator expressions are evaluated. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
Duly noted and thanks for the feedback! (just what I was looking for actually). I do disagree with the idea that the proposal, if implemented, would make Python harder to learn. Not sure who would find it harder. Having to find and use takewhile was harder for me. I still find that one counter-intuitive. I would have expected the parameters in the reverse order (take something, while something else is true). Tripped me up a few times, which got me thinking about an alternative. On Mon, Jan 19, 2009 at 12:51 PM, Terry Reedy tjre...@udel.edu wrote: Gerald Britton wrote: Please find below PEP 3142: Add a while clause to generator expressions. I'm looking for feedback and discussion. This was already discussed on python-ideas where it got negative feedback. One objection, mentioned by Mathias Panzerbock and Georg Brandl, is that it is redundant with takewhile(). You did mention that in the PEP. The other, posted by Steven Bethard, is that it fundamentally breaks the current semantics of abbreviating (except for iteration variable scoping) an 'equivalent' for loop. This should have been listed in the PEP under Objections (or whatever the section. I did not bother to second his objection there but will now. -1 Steven summary: I'm probably just repeating myself here, but the reason not to do it is that the current generator expressions translate almost directly into the corresponding generator statements. Using while in the way you've suggested breaks this symmetry, and would make Python harder to learn. Longer presentation: I think this could end up being confusing. Current generator expressions turn into an equivalent generator function by simply indenting the clauses and adding a yield, for example: (i for i in range(100) if i % 2 == 0) is equivalent to: def gen(): for i in range(100): if i % 2 == 0: yield i Now you're proposing syntax that would no longer work like this. Taking your example: (i for i in range(100) while i = 50) I would expect this to mean: [meaning, one would expect this to mean, using current rules = tjr] def gen(): for i in range(100): while i = 50: yield i In short, -1. You're proposing to use an existing keyword in a new way that doesn't match how generator expressions are evaluated. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/gerald.britton%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
- Mensaje original De: Gerald Britton gerald.brit...@gmail.com Para: Terry Reedy tjre...@udel.edu CC: python-dev@python.org Enviado: lunes, 19 de enero, 2009 15:03:47 Asunto: Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions Duly noted and thanks for the feedback! (just what I was looking for actually). I do disagree with the idea that the proposal, if implemented, would make Python harder to learn. Not sure who would find it harder. Having to find and use takewhile was harder for me. I still find that one counter-intuitive. I would have expected the parameters in the reverse order (take something, while something else is true). Tripped me up a few times, which got me thinking about an alternative. Are you even sure the list comprehension doesn't already shortcut evaluation? This quick test in 2.6 hints otherwise: a = (i for i in range(10) if i**210) a.next() 0 a.next() 1 a.next() 2 a.next() 3 a.next() Traceback (most recent call last): File pyshell#32, line 1, in module a.next() StopIteration ¡Todo sobre la Liga Mexicana de fútbol! Estadisticas, resultados, calendario, fotos y más:lt; http://espanol.sports.yahoo.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
Vitor Bosshard wrote: Are you even sure the list comprehension doesn't already shortcut evaluation? It does not. The body of the comprehension is evaluated all the way to completion, despite the fact that a.next() does not return until there is a successful test of the if expression. def print_range(n): ... for i in range(n): ... print(i) ... yield i ... a = (i for i in print_range(10) if i**210) a.next() 0 0 a.next() 1 1 a.next() 2 2 a.next() 3 3 a.next() 4 5 6 7 8 9 Traceback (most recent call last): File stdin, line 1, in module StopIteration -- Scott Dial sc...@scottdial.com scod...@cs.indiana.edu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
2009/1/19 Vitor Bosshard algor...@yahoo.com - Mensaje original De: Gerald Britton gerald.brit...@gmail.com Para: Terry Reedy tjre...@udel.edu CC: python-dev@python.org Enviado: lunes, 19 de enero, 2009 15:03:47 Asunto: Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions Duly noted and thanks for the feedback! (just what I was looking for actually). I do disagree with the idea that the proposal, if implemented, would make Python harder to learn. Not sure who would find it harder. Having to find and use takewhile was harder for me. I still find that one counter-intuitive. I would have expected the parameters in the reverse order (take something, while something else is true). Tripped me up a few times, which got me thinking about an alternative. Are you even sure the list comprehension doesn't already shortcut evaluation? This quick test in 2.6 hints otherwise: a = (i for i in range(10) if i**210) a.next() 0 a.next() 1 a.next() 2 a.next() 3 a.next() Traceback (most recent call last): File pyshell#32, line 1, in module a.next() StopIteration Does not prove anything. test.py source = iter(xrange(10)) a = (i for i in source if i**210) print list(a) print list(source) output --- $ python /tmp/test.py [0, 1, 2, 3] [] While 'a' is being evaluated, the source iterator is first completely exhausted. If the source iterator is infinite, an infinite loop is created and the program doesn't terminate. ¡Todo sobre la Liga Mexicana de fútbol! Estadisticas, resultados, calendario, fotos y más: http://espanol.sports.yahoo.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit The universe is always one step beyond logic. -- Frank Herbert ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
On Mon, Jan 19, 2009 at 1:41 PM, Scott Dial scott+python-...@scottdial.com wrote: Vitor Bosshard wrote: Are you even sure the list comprehension doesn't already shortcut evaluation? It does not. The body of the comprehension is evaluated all the way to completion, .. In addition, the test is evaluated on all items as well: def test(i): ...print testing, i ...return i**2 10 ... a = (i for i in range(10) if test(i)) a.next() testing 0 0 a.next() testing 1 1 a.next() testing 2 2 a.next() testing 3 3 a.next() testing 4 testing 5 testing 6 testing 7 testing 8 testing 9 Traceback (most recent call last): File stdin, line 1, in module StopIteration ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
2009/1/19 Vitor Bosshard algor...@yahoo.com: Are you even sure the list comprehension doesn't already shortcut evaluation? This quick test in 2.6 hints otherwise: a = (i for i in range(10) if i**210) Yes, but your test, once it becomes true, remains so. Consider list(n for n in range(10) if n%2 == 0) [0, 2, 4, 6, 8] I assume that the intention of the while syntax is: list(n for n in range(10) while n%2 == 0) [0] because 1%2 != 0 so the loop stops. Having said that, I'm -1 on the proposal. The requirement is rare enough that the correct place for it *is* in a module - and that's what itertools provides (along with a number of other, equally valid, manipulations). I certainly don't see it as justifying new syntax. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
On Mon, Jan 19, 2009 at 10:03, Gerald Britton gerald.brit...@gmail.com wrote: Duly noted and thanks for the feedback! (just what I was looking for actually). I do disagree with the idea that the proposal, if implemented, would make Python harder to learn. Not sure who would find it harder. Having to find and use takewhile was harder for me. I still find that one counter-intuitive. I would have expected the parameters in the reverse order (take something, while something else is true). Tripped me up a few times, which got me thinking about an alternative. The reason Python would be harder to learn is there is something more to learn. Removing confusion for something from the standard library by adding syntax does not warrant making the language easier. For something to be considered making the language easier it needs to solve a very common idiom in a way that is an improvement over the alternatives. In this instance I don't think the idiom is common enough to warrant the change. -Brett On Mon, Jan 19, 2009 at 12:51 PM, Terry Reedy tjre...@udel.edu wrote: Gerald Britton wrote: Please find below PEP 3142: Add a while clause to generator expressions. I'm looking for feedback and discussion. This was already discussed on python-ideas where it got negative feedback. One objection, mentioned by Mathias Panzerbock and Georg Brandl, is that it is redundant with takewhile(). You did mention that in the PEP. The other, posted by Steven Bethard, is that it fundamentally breaks the current semantics of abbreviating (except for iteration variable scoping) an 'equivalent' for loop. This should have been listed in the PEP under Objections (or whatever the section. I did not bother to second his objection there but will now. -1 Steven summary: I'm probably just repeating myself here, but the reason not to do it is that the current generator expressions translate almost directly into the corresponding generator statements. Using while in the way you've suggested breaks this symmetry, and would make Python harder to learn. Longer presentation: I think this could end up being confusing. Current generator expressions turn into an equivalent generator function by simply indenting the clauses and adding a yield, for example: (i for i in range(100) if i % 2 == 0) is equivalent to: def gen(): for i in range(100): if i % 2 == 0: yield i Now you're proposing syntax that would no longer work like this. Taking your example: (i for i in range(100) while i = 50) I would expect this to mean: [meaning, one would expect this to mean, using current rules = tjr] def gen(): for i in range(100): while i = 50: yield i In short, -1. You're proposing to use an existing keyword in a new way that doesn't match how generator expressions are evaluated. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/gerald.britton%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] stuck with dlopen...
If you run your python.exe under gdb you should be able to set a future
breakpoint on your _PyEval_EvalMiniFrameEx function and debug from there.
On Wed, Jan 14, 2009 at 8:28 PM, s...@pobox.com wrote:
I've recently been working on generating C functions on-the-fly which
inline
the C code necessary to implement the bytecode in a given Python function.
For example, this bytecode:
dis.dis(f)
2 0 LOAD_FAST0 (a)
3 LOAD_CONST 1 (1)
6 BINARY_ADD
7 RETURN_VALUE
is transformed into this rather boring bit of C code:
#include Python.h
#include code.h
#include frameobject.h
#include eval.h
#include opcode.h
#include structmember.h
#include opcode_mini.h
PyObject *
_PyEval_EvalMiniFrameEx(PyFrameObject *f, int throwflag)
{
static int jitting = 1;
PyEval_EvalFrameEx_PROLOG1();
co = f-f_code;
PyEval_EvalFrameEx_PROLOG2();
oparg = 0;
LOAD_FAST_IMPL(oparg);
oparg = 1;
LOAD_CONST_IMPL(oparg);
BINARY_ADD_IMPL();
RETURN_VALUE_IMPL();
PyEval_EvalFrameEx_EPILOG();
}
The PROLOG1, PROLOG2 and EPILOG macros are just chunks of code from
PyEval_EvalFrameEx.
I have the code compiling and linking, and dlopen and dlsym seem to work,
returning apparently valid pointers, but when I try to call the function I
get
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x000c
0x0058066d in _PyEval_EvalMiniFrameEx (f=0x230d30, throwflag=0) at
MwDLSf.c:17
Line 17 is the PROLOG1 macro. I presume it's probably barfed on the very
first instruction. (This is all on an Intel Mac running Leopard BTW.)
Here are the commands generated to compile and link the C code:
gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall \
-Wstrict-prototypes -g -DPy_BUILD_CORE -DNDEBUG \
-I/Users/skip/src/python/py3k-t/Include \
-I/Users/skip/src/python/py3k-t -c dTd5cl.c \
-o /tmp/MwDLSf.o
gcc -L/opt/local/lib -bundle -undefined dynamic_lookup -g \
/tmp/dTd5cl.o -L/Users/skip/src/python/py3k-t -lpython3.1 \
-o /tmp/MwDLSf.so
(It just uses the distutils compiler module to build .so files.) The .so
file looks more-or-less ok:
% otool -L /tmp/MwDLSf.so
/tmp/MwDLSf.so:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 111.1.3)
though nm doesn't show that any undefined _Py* symbols so I suspect I'm not
linking it correctly. The Python executable was built without
--enable-shared. I've tried building with that config flag, but that just
gives me fits during debugging because it always wants to find libpython in
the installation directory even if I'm running python.exe from the build
directory. Installing is a little tedious because it relies on a properly
functioning interpreter.
dlopen is called very simply:
handle = dlopen(shared, RTLD_NOW);
I used RTLD_NOW because that's what sys.getdlopenflags() returns. I'm not
calling dlclose for the time being.
I'm not exactly sure where I should go from here. I'd be more than happy
to
open an item in the issue tracker. I was hoping to get something a bit
closer to working before doing that though. The failure to properly load
the compiled function makes it pretty much impossble to debug the generated
code beyond what the compiler can tell me.
Any suggestions?
Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
On 1/19/2009 6:51 PM, Terry Reedy wrote: The other, posted by Steven Bethard, is that it fundamentally breaks the current semantics of abbreviating (except for iteration variable scoping) an 'equivalent' for loop. The proposed syntax would suggest that this should be legal as well: for i in iterable while cond: blahblah or perhaps: while cond for i in iterable: blahblah A while-for or for-while loop would be a novel invention, not seen in any other language that I know of. I seriously doubt its usefulness though... Sturla Molden ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
Steven Bethard wrote: -1. As I pointed out on python-ideas, this proposal makes while mean something different in a generator expression. While I initially found the suggestion in the PEP rather cute, that isn't enough to make it a good idea as a language addition. So, -1 for a few reasons: - the 'idiom' being replaced isn't common enough to justify dedicated syntax - the proposed syntax change isn't significantly easier to understand or significantly faster than the existing alternatives - Steven's more substantial objection that it would break the parallel between generator expressions/list comprehensions and the corresponding statements. Cheers, Nick. P.S. Some examples of past syntax changes that *were* accepted (and why): http://mail.python.org/pipermail/python-dev/2008-October/082831.html -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] compiling python2.5 (msys+mingw+wine) using msvcr80 assemblies
folks, hi, after some quiet advice i've tracked down a method for compiling python2.5 using msvcr80 that _will_ actually work both under native win32 and also under wine, but it's a _bit_ dodgy, as i couldn't track down where you're supposed to put Microsoft.VC80.CRT, except in the path of the application where it's running from. so, instead, i put the _contents_ of Microsoft.VC80.CRT.manifest into the manifest for the file, and this _does_ actually seem to work. i'm thinking of adding the Microsoft.VC80.CRT.manifest to the rc file (for compilation as a resource) to see if _that_ works, and will report back, but first i wanted to describe what i've done and see what people think: 1) created python_2.5_8.0_mingw_exe.manifest contents as follows, this is _normally_ what is in Microsoft.VC80.CRT _not_ in the .exe.manifest ?xml version='1.0' encoding='UTF-8' standalone='yes'? assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0' file name=msvcr80.dll hash=10f4cb2831f1e9288a73387a8734a8b604e5beaa hashalg=SHA1asmv2:hash xmlns:asmv2=urn:schemas-microsoft-com:asm.v2 xmlns:dsig=http://www.w3.org/2000/09/xmldsig#;dsig:Transformsdsig:Transform Algorithm=urn:schemas-microsoft-com:HashTransforms.Identity/dsig:Transform/dsig:Transformsdsig:DigestMethod Algorithm=http://www.w3.org/2000/09/xmldsig#sha1;/dsig:DigestMethoddsig:DigestValuen9On8FItNsK/DmT8UQxu6jYDtWQ=/dsig:DigestValue/asmv2:hash/file file name=msvcp80.dll hash=b2082dfd3009365c5b287448dcb3b4e2158a6d26 hashalg=SHA1asmv2:hash xmlns:asmv2=urn:schemas-microsoft-com:asm.v2 xmlns:dsig=http://www.w3.org/2000/09/xmldsig#;dsig:Transformsdsig:Transform Algorithm=urn:schemas-microsoft-com:HashTransforms.Identity/dsig:Transform/dsig:Transformsdsig:DigestMethod Algorithm=http://www.w3.org/2000/09/xmldsig#sha1;/dsig:DigestMethoddsig:DigestValue0KJ/VTwP4OUHx98HlIW2AdW1kuY=/dsig:DigestValue/asmv2:hash/file file name=msvcm80.dll hash=542490d0fcf8615c46d0ca487033ccaeb3941f0b hashalg=SHA1asmv2:hash xmlns:asmv2=urn:schemas-microsoft-com:asm.v2 xmlns:dsig=http://www.w3.org/2000/09/xmldsig#;dsig:Transformsdsig:Transform Algorithm=urn:schemas-microsoft-com:HashTransforms.Identity/dsig:Transform/dsig:Transformsdsig:DigestMethod Algorithm=http://www.w3.org/2000/09/xmldsig#sha1;/dsig:DigestMethoddsig:DigestValueYJuB+9Os2oxW4mY+2oC/r8lICZE=/dsig:DigestValue/asmv2:hash/file /assembly 2) created python_2.5_8.0_mingw_exe.rc contents as follows: #include winuser.h 2 RT_MANIFEST PC/python_2.5_8.0_mingw_exe.manifest you could get away with 2 24 PC/. and could exclude the #include 3) added a rule to Makefile.pre.in to create the .res as a binary: # This rule builds the .res file for the Python EXE, required when # linking and using msvcrt80 or above. good luck to us all... $(PYTHONEXEMSVRES): $(srcdir)/PC/python_$(VERSION)_$(MSRTVER)_exe.manifest \ $(srcdir)/PC/python_$(VERSION)_$(MSRTVER)_mingw_exe.rc windres --input $(srcdir)/PC/python_$(VERSION)_$(MSRTVER)_mingw_exe.rc \ --output $(PYTHONEXEMSVRES) \ --output-format=coff 4) added $(PYTHONEXEMSVRES) to the objects to be linked. stunningly, this actually works (of course, you need an msvcr80.dll for it to work duh). i tried finding a location to place the Microsoft.VC80.CRT.Manifest, prior to this hack - a wine dump showed this: 0009:trace:actctx:lookup_assembly looking for name=LMicrosoft.VC80.CRT version=8.0.50727.762 arch=Lx86 0009:trace:heap:RtlAllocateHeap (0x11,0002,0038): returning 0x115148 0009:trace:file:RtlDosPathNameToNtPathName_U (LC:\\windows\\winsxs\\manifests,0xff8c7d08,(nil),(nil)) 0009:trace:file:RtlGetFullPathName_U (LC:\\windows\\winsxs\\manifests 520 0xff8c79f4 (nil)) attempts to copy the manifest into that directory resulted in no joy. so, i'm a bit stuck, and would appreciate some advice on whether the above is acceptable (yes i know it makes sure that python.exe can only use one _very_ specific version of msvcr80.dll - and there are currently two: 8.0.50727.762 and 8.0.50608.0. also i'd appreciate some advice on what the _really_ best way to do this is. and on where the hell i'm supposed to put the VC80.CRT manifest so it will actually... _do_ something! l. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
Are you all certain that this mapping from a generator expression to a foor loop isn't just a happy coincidence? After all, the generator statement is just a generalization of the list comprehension and that doesn't map quite so directly. I have always taken both expressions at face value, and not tried to map them into something else. Why should you, since they are designed to match the grammar of the english language and make perfect sense if you read them as you would a herring recipe. The suggested while keyword expands on this easy to understand theme, and the fact that it doesn't fit a mapping that was probably never intentional shouldn't detract from that. K -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Terry Reedy Sent: 19. janúar 2009 17:52 To: python-dev@python.org Subject: Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions The other, posted by Steven Bethard, is that it fundamentally breaks the current semantics of abbreviating (except for iteration variable scoping) an 'equivalent' for loop. This should have been listed in the PEP under Objections (or whatever the section. I did not bother to second his objection there but will now. -1 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hmmm... __PyObject_NextNotImplemented when tracing lines
Alexander I cannot reproduce this on my Mac. It looks like you may Alexander have an out of date python.exe in your sandbox. Please check Alexander that Alexander $ nm python.exe | grep PyObject_NextNotImplemented Alexander 00052940 T __PyObject_NextNotImplemented Alexander has a T in the second column. Alexander Note that this symbol seem to have been added recently: ... I see what the problem is now. I configured with --enable-shared. Even though my sandbox was up-to-date, my python.exe was current and my libpython2.7.dylib held the symbol in question the *installed* version of libpython2.7.dylib didn't have the symbol: % nm python.exe | grep PyObject_NextNotImplemented % nm libpython2.7.dylib | egrep PyObject_NextNotImplemented 00054200 T __PyObject_NextNotImplemented % nm /Users/skip/local/lib/libpython2.7.dylib | egrep PyObject_NextNotImplemented % I've been bitten by this before. When building with --enable-shared the executable appears to have the installed shared library location bound into it: % otool -L python.exepython.exe: /Users/skip/local/lib/libpython2.7.dylib (compatibility version 2.7.0, current version 2.7.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.3) Is this something that we can fix/work around? It seems to me like a wart, probably platform-dependent. Alexander I am puzzled, however, that you don't see problems in a Alexander non-coverage run. Yeah, that mystifies me as well, especially given the above output from nm and otool. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a while clause to generator expressions
The proposal is similar to the c do statement do statement while (expression); which for whatever reason (infrequency?) like the switch statement have rightly not been adopted into python. winmail.dat___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
Kristján Valur Jónsson wrote: Are you all certain that this mapping from a generator expression to a foor loop isn't just a happy coincidence? Yes, The manual *defines* the meaning of a comprehension in terms of the corresponding nested statements. The comprehension consists of a single expression followed by at least one for clause and zero or more for or if clauses. In this case, the elements of the new container are those that would be produced by considering each of the for or if clauses a block, nesting from left to right, and evaluating the expression to produce an element each time the innermost block is reached. (3.0 doc) It was originally defined as exactly equivalent (with empty list initialization added). The only intended change of the slightly softer newer version is that the target name bindings do not escape the scope of the comprehension. The proposed change would BREAK the definition and intent of what a comprehension means. After all, the generator statement is just a generalization of the list comprehension I would call it a variation. Syntactically, a generator statement is a comprehension with parentheses instead of square brackets or curly braces. tjr ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Copyright notices in modules
Why does numbers.py say: # Copyright 2007 Google, Inc. All Rights Reserved. # Licensed to PSF under a Contributor Agreement. Weren't there multiple contributors including non-google people? Does Google want to be associated with code that was submitted with no tests? Do we want this sort of stuff in the code? If someone signs a contributor agreement, can we forgo the external copyright comments? Do we want to make a practice of every contributor commenting in the name of the company they were working for at the time (if so, I would have to add the comment to a lot of modules)? Does the copyright concept even apply to an abstract base class (I thought APIs were not subject to copyright, just like database layouts and language definitions)? Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
OK, I still don't like the general idea, but I have a suggestion for what I think is a more favorable syntax, anyway. Basically, adding the allowance of an 'else break' when there is an if clause in a generator expression or list comprehension. I still dont think it should be done, but it is a more consistent syntax. It has the obvious problem of looking like it might allow an alternative expression, like the if-expression. prime = (p for p in sieve() if p 1000 else break) On Mon, Jan 19, 2009 at 11:47 AM, Calvin Spealman ironfro...@gmail.com wrote: On Mon, Jan 19, 2009 at 11:41 AM, Gerald Britton gerald.brit...@gmail.com wrote: Thanks Calvin, Could you please expand on your thoughts about possible confusion? That is, how do you see a programmer becoming confused if this option were added to the syntax. I think that the difference between these two lines is too easy to miss among other code: prime = (p for p in sieve() while p 1000) and: prime = (p for p in sieve() if p 1000) The very distinctly different behavior is two important to look so similar at a glance. And, again, I just don't see takewhile being used all that often. Not nearly often enough to warrant replacing it with a special syntax! Only 178 results from Google CodeSearch, while chain, groupby, and repeat get 4000, 3000, and 1000 respectively. Should those be given their own syntax? On Mon, Jan 19, 2009 at 11:29 AM, Calvin Spealman ironfro...@gmail.com wrote: I am really unconvinced of the utility of this proposal and quite convinced of the confusing factor it may well add to the current syntax. I would like to see more applicable examples. It would replace uses of takewhile, but that isn't a really often used function. So, is there any evidence to support that making this a new syntax would find so many more uses of the construct to be worth it? I believe not. On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton gerald.brit...@gmail.com wrote: Please find below PEP 3142: Add a while clause to generator expressions. I'm looking for feedback and discussion. PEP: 3142 Title: Add a while clause to generator expressions Version: $Revision: 68715 $ Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $ Author: Gerald Britton gerald.brit...@gmail.com Status: Draft Type: Standards Track Content-Type: text/plain Created: 12-Jan-2009 Python-Version: 3.0 Post-History: Abstract This PEP proposes an enhancement to generator expressions, adding a while clause to complement the existing if clause. Rationale A generator expression (PEP 289 [1]) is a concise method to serve dynamically-generated objects to list comprehensions (PEP 202 [2]). Current generator expressions allow for an if clause to filter the objects that are returned to those meeting some set of criteria. However, since the if clause is evaluated for every object that may be returned, in some cases it is possible that all objects would be rejected after a certain point. For example: g = (n for n in range(100) if n*n 50) which is equivalent to the using a generator function (PEP 255 [3]): def __gen(exp): for n in exp: if n*n 50: yield n g = __gen(iter(range(10))) would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider the numbers from 8 to 99 and reject them all since n*n = 50 for numbers in that range. Allowing for a while clause would allow the redundant tests to be short-circuited: g = (n for n in range(100) while n*n 50) would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8 since the condition (n*n 50) is no longer true. This would be equivalent to the generator function: def __gen(exp): for n in exp: if n*n 50: yield n else: break g = __gen(iter(range(100))) Currently, in order to achieve the same result, one would need to either write a generator function such as the one above or use the takewhile function from itertools: from itertools import takewhile g = takewhile(lambda n: n*n 50, range(100)) The takewhile code achieves the same result as the proposed syntax, albeit in a longer (some would say less-elegant) fashion. Also, the takewhile version requires an extra function call (the lambda in the example above) with the associated performance penalty. A simple test shows that: for n in (n for n in range(100) if 1): pass performs about 10% better than: for n in takewhile(lambda n: 1, range(100)): pass though they achieve similar results. (The first example uses a generator; takewhile is an iterator). If similarly implemented, a while clause should perform about the same as the if clause does today. The reader may ask if the if and while clauses should be mutually exclusive. There are good examples that show that there are
[Python-Dev] Questions/comments on documentation formatting
I have been writing up the initial docs for importlib and four things struck me: 1. Why is three space indents the preferred indentation level? 2. Should we start using function annotations? 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, c=None]])``) really necessary when default argument values are present? And do we really need to nest the brackets when it is obvious that having on optional argument means the rest are optional as well? 4. The var directive is not working even though the docs list it as a valid directive; so is it still valid and something is broken, or the docs need to be updated? -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Copyright notices in modules
Raymond Hettinger writes: Does the copyright concept even apply to an abstract base class (I thought APIs were not subject to copyright, just like database layouts and language definitions)? Yes, it does, although a public API per se is not subject to copyright, because there's only one way to do it. Any comments, internal implementation (eg, names of persistent state variables, members, and constants, and the very existence of those identifiers), and tests are subject to copyright because they are expressive. I believe that a private API also can be subject to copyright, though I'm not as sure of that. The point being that there are good APIs and bad APIs that expose the same functionality, so that API design is expressive. However, if you expose the API and license people to use it, that license makes it impossible to restrict them from using it thereafter. Caveat: IANAL, and this is under U.S. law. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3142: Add a while clause to generator expressions
On 19Jan2009 19:42, Calvin Spealman ironfro...@gmail.com wrote: | OK, I still don't like the general idea, but I have a suggestion for | what I think is a more favorable syntax, anyway. Basically, adding the | allowance of an 'else break' when there is an if clause in a generator | expression or list comprehension. I still dont think it should be | done, but it is a more consistent syntax. It has the obvious problem | of looking like it might allow an alternative expression, like the | if-expression. | | prime = (p for p in sieve() if p 1000 else break) If I'm reading your suggestion correctly it changes feel of the if part quite fandamentally. A bare if acts as a filter, yielding only the items matching the condition. With an else break it yields only the items matching the condition up to the first which fails. For a range like p 1000 this isn't so different, but if the condition doesn't apply to just the leading items then this form becomes almost useless. Consider p % 3 == 0. Normally that would yield every third item. With else break it would yield just one item. Any non-continuous filter has the same issue. In my opinion this makes the construct far less applicable than it would appear at first glance. So -1 from me. I saw someone else mention takewhile(), and that's my preferred way of doing the original suggestion ([ ... while some-constraint ]), and thus I'm -1 on the original suggestion too (also for the if/while confusion mentioned in the thread). Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ Drive Agressively Rash Magnificently - Nankai Leathers ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon br...@python.org wrote: I have been writing up the initial docs for importlib and four things struck me: 1. Why is three space indents the preferred indentation level? Because it matches nicely up with the length of directives: .. somedirective:: blah ^^^ 2. Should we start using function annotations? No, I think that information is better stored in the function description. 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, c=None]])``) really necessary when default argument values are present? And do we really need to nest the brackets when it is obvious that having on optional argument means the rest are optional as well? Actually, the defaults are usually documented in the description not the signature. 4. The var directive is not working even though the docs list it as a valid directive; so is it still valid and something is broken, or the docs need to be updated? The docs should be updated. data is the one to use now. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
Brett Cannon wrote: 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, c=None]])``) really necessary when default argument values are present? And do we really need to nest the brackets when it is obvious that having on optional argument means the rest are optional as well? I can't think of an example off the top of my head, but I'm certain the point of nesting the brackets is to delimit the optional arguments into groups. Documenting your fxn() examples as fxn(a [, b=None, c=None]) would imply that if you provide 'b' then you must provide 'c', or if we abandon nested brackets, it's ambiguous as to the requirements. Imagine seeing foo(a [, b=None, c=None [, d=None]]) and I think the rationale for such notation becomes clear. -- Scott Dial sc...@scottdial.com scod...@cs.indiana.edu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson benja...@python.org wrote: On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon br...@python.org wrote: I have been writing up the initial docs for importlib and four things struck me: 1. Why is three space indents the preferred indentation level? Because it matches nicely up with the length of directives: .. somedirective:: blah ^^^ 2. Should we start using function annotations? No, I think that information is better stored in the function description. Why? Putting it in the signature makes it very succinct and a simple glance at the doc to see what type/ABC is expected. 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, c=None]])``) really necessary when default argument values are present? And do we really need to nest the brackets when it is obvious that having on optional argument means the rest are optional as well? Actually, the defaults are usually documented in the description not the signature. OK, but that doesn't make it optimal. And that still doesn't answer my question of whether all of those nested brackets are truly necessary. 4. The var directive is not working even though the docs list it as a valid directive; so is it still valid and something is broken, or the docs need to be updated? The docs should be updated. data is the one to use now. So the 'data' directive turns into any variable, not just a module variables? -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
On Mon, Jan 19, 2009 at 9:11 PM, Brett Cannon br...@python.org wrote: On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson benja...@python.org wrote: On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon br...@python.org wrote: 2. Should we start using function annotations? No, I think that information is better stored in the function description. Why? Putting it in the signature makes it very succinct and a simple glance at the doc to see what type/ABC is expected. Well, I guess it's just not been explored. Feel free to try it out if you wish, though. 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, c=None]])``) really necessary when default argument values are present? And do we really need to nest the brackets when it is obvious that having on optional argument means the rest are optional as well? Actually, the defaults are usually documented in the description not the signature. OK, but that doesn't make it optimal. And that still doesn't answer my question of whether all of those nested brackets are truly necessary. All I can say is that it is the style/convention. 4. The var directive is not working even though the docs list it as a valid directive; so is it still valid and something is broken, or the docs need to be updated? The docs should be updated. data is the one to use now. So the 'data' directive turns into any variable, not just a module variables? data is for module level objects. If you're documenting properties or attributes in classes, use attribute. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
From: Brett Cannon br...@python.org 1. Why is three space indents the preferred indentation level? I've also wondered about this. It is somewhat incovenient when bringing in code samples from files with four space indents. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
On Mon, Jan 19, 2009 at 9:23 PM, Raymond Hettinger pyt...@rcn.com wrote: From: Brett Cannon br...@python.org 1. Why is three space indents the preferred indentation level? I've also wondered about this. It is somewhat incovenient when bringing in code samples from files with four space indents. It's just reST indentation that is 3 spaces. Code examples in the reST can be 4 spaces. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
Benjamin Peterson benja...@python.org writes: On Mon, Jan 19, 2009 at 9:23 PM, Raymond Hettinger pyt...@rcn.com wrote: From: Brett Cannon br...@python.org 1. Why is three space indents the preferred indentation level? I've also wondered about this. It is somewhat incovenient when bringing in code samples from files with four space indents. It's just reST indentation that is 3 spaces. It doesn't have to be. When writing reST, I always make directives so they will line up nicely at 4-space indents: = Normal paragraph .. Comment .. foo:: bar = -- \ “I was the kid next door's imaginary friend.” —Emo Philips | `\ | _o__) | Ben Finney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
I have another question about doc formatting. What controls whether section headers get urls with a custom named jump target instead of a default name like id1? In particular, look at the urls for: http://docs.python.org/dev/library/collections.html#id1 versus http://docs.python.org/dev/library/collections.html#abcs-abstract-base-classes I would like all of the targets to have meaningful names. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
Raymond Hettinger pyt...@rcn.com writes: What controls whether section headers get urls with a custom named jump target instead of a default name like id1? In particular, look at the urls for: http://docs.python.org/dev/library/collections.html#id1 versus Hmm. Immediately preceding the h2 element, there is also an empty span element with a meaningful ID, counter-objects, that can be used to get to the same position in the document: http://docs.python.org/dev/library/collections.html#counter-objects However, the problem is that URL isn't used for the “Permalink to this headline” link. Perhaps a bug report to the Docutils folks is in order. -- \ “Nature is trying very hard to make us succeed, but nature does | `\ not depend on us. We are not the only experiment.” —Richard | _o__) Buckminster Fuller, 1978-04-30 | Ben Finney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
On Mon, Jan 19, 2009 at 19:19, Benjamin Peterson benja...@python.org wrote: On Mon, Jan 19, 2009 at 9:11 PM, Brett Cannon br...@python.org wrote: On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson benja...@python.org wrote: On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon br...@python.org wrote: 2. Should we start using function annotations? No, I think that information is better stored in the function description. Why? Putting it in the signature makes it very succinct and a simple glance at the doc to see what type/ABC is expected. Well, I guess it's just not been explored. Feel free to try it out if you wish, though. I just might. 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, c=None]])``) really necessary when default argument values are present? And do we really need to nest the brackets when it is obvious that having on optional argument means the rest are optional as well? Actually, the defaults are usually documented in the description not the signature. OK, but that doesn't make it optimal. And that still doesn't answer my question of whether all of those nested brackets are truly necessary. All I can say is that it is the style/convention. Right, which is why I am questioning it. =) 4. The var directive is not working even though the docs list it as a valid directive; so is it still valid and something is broken, or the docs need to be updated? The docs should be updated. data is the one to use now. So the 'data' directive turns into any variable, not just a module variables? data is for module level objects. If you're documenting properties or attributes in classes, use attribute. Then what are we supposed to use for arguments? Just ``literal``? -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
On Mon, Jan 19, 2009 at 19:50, Raymond Hettinger pyt...@rcn.com wrote: I have another question about doc formatting. What controls whether section headers get urls with a custom named jump target instead of a default name like id1? In particular, look at the urls for: http://docs.python.org/dev/library/collections.html#id1 versus http://docs.python.org/dev/library/collections.html#abcs-abstract-base-classes I would like all of the targets to have meaningful names. Not sure from a sphinx perspective, but Docutils does this automatically. You can also always specify the anchor point manually:: .. _abcs-abstract-base-classes Abstract Base Classes -- to get what you want. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
On Mon, Jan 19, 2009 at 19:02, Scott Dial scott+python-...@scottdial.com wrote: Brett Cannon wrote: 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, c=None]])``) really necessary when default argument values are present? And do we really need to nest the brackets when it is obvious that having on optional argument means the rest are optional as well? I can't think of an example off the top of my head, but I'm certain the point of nesting the brackets is to delimit the optional arguments into groups. Documenting your fxn() examples as fxn(a [, b=None, c=None]) would imply that if you provide 'b' then you must provide 'c', or if we abandon nested brackets, it's ambiguous as to the requirements. Imagine seeing foo(a [, b=None, c=None [, d=None]]) and I think the rationale for such notation becomes clear. Well, that is such a rare case that I don't know if it warrants the line noise in the argument declaration. That argument also doesn't make sense in the face of ``fxn(a [, a=None [, b=None]])`` where 'b' almost always has no connection to 'a', but is still supposed to be listed that way because of positional arguments being optional. I understand using them for C functions where there is no such thing as a default argument, but it just doesn't make a ton of sense for Python code. I don't know of anyone who was confused by what help() spit out and not having fancy bracketing. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions/comments on documentation formatting
In particular, look at the urls for: http://docs.python.org/dev/library/collections.html#id1 versus http://docs.python.org/dev/library/collections.html#abcs-abstract-base-classes I would like all of the targets to have meaningful names. [Brett] Not sure from a sphinx perspective, but Docutils does this automatically. You can also always specify the anchor point manually:: .. _abcs-abstract-base-classes Abstract Base Classes -- to get what you want. Thanks for the note. It pointed me to the real problem which was that manual anchor points can interfere with the automatically generated names if their names are the same. The solution was to *remove* the manually generated anchor points. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
