Re: [Python-Dev] Support for the Haiku OS

[Mark Dickinson]

On Sun, Jan 18, 2009 at 11:03 PM, scott mc  wrote:
> I built 2.7 on Haiku, but am getting failures in the regression tests.
>  Many of them are in math related tests, failing in the 15th decimal
> place on test_decimal and a few others like that, I posted a ticket on
> Haiku's trac for that as it might be related to Haiku's built in math
> lib? (libm is built into Haiku's libroot.so)
> http://dev.haiku-os.org/ticket/3308

Most of these look like libm/libc precision problems to me,
of varying severity.  Some particular comments:

 - the test_float result is worrying: there are a good few places
   where Python depends on eval(repr(.)) round-tripping for
   floats, and it looks as though either the eval or the repr
   is losing significant accuracy.  Actually, there's so much
   accuracy loss that I wonder whether something's being
   cast from double precision to single precision at some
   point.

- test_decimal failing was a bit of a surprise until I saw
  which test was failing:  the decimal module quite
  deliberately does all computation using integer
  arithmetic, and avoids floating-point like the plague,
  so it should be ultra-portable.  Except, of course, the
  recently added from_float method, which converts
  from floats to decimals.  So fix up the floating-point
  and test_decimal should pass again.

- I don't understand where the test_marshall and
  test_random failures are coming from.  These
  could be Python problems (though I think it's
  more likely that they're Haiku floating-point
  problems).  I'd be interested to see short
  code-snippets that reproduce these issues.

- I wouldn't worry so much about the test_math
  and test_cmath failures until you get the others
  sorted out;  the tests are probably stricter than
  they need to be.

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Child process freezes during fork pipe exec

[Gangadharan S.A.]

 Hi,

Summary:
* In my organization, we have a *multi threaded* (threading library)
python (python 2.4.1) daemon on Linux, which starts up various processes
using the fork pipe exec model.
* We use this fork , wait on pipe , exec model as  a form of handshake
between the parent and child processes. We want the child to go ahead only
after the parent has noted down the fact that the child has been forked and
what it's pid is.
* This usually works fine, but for about 1 in every 20,000 processes
started, the child process just freezes somewhere after the fork, before the
exec. It does not die. It is alive and stuck.
* Why does this happen?
* Is there a better way for us to write a
fork-wait_for_start_signal-exec construct?

Here is what we do:
One of the threads of the multi threaded python daemon does the
following
1) Fork out a child process
2) Child process waits for a pipe message from parent
--- Parent sends pipe message to child after noting down child
details : pid, start time etc. ---
3) Child process prints various debug messages, including looking at
os.environ values
4) Child process execs the right script

Here it is again, in pseudo code:
def start_job():
read_pipefd, write_pipefd = os.pipe()

# 1) Fork out a child process
pid = os.fork()

if pid == 0:
# 2) wait for excepted message on pipe
os.close(write_pipefd)
read_set, write_set, exp_set = select.select([read_pipefd], [],
[], 300)
if os.read(read_pipefd, len("expected message") <> "expected
message":
os._exit(1)
os.close(read_pipefd)

# 3) print various debug messages, including os.environ values
print >> sys.stderr, "we print various debug messages here,
including os.evniron values"

# 4) go ahead with exec
os.execve(path, args, env)
else:
# parent process sends pipe message to child at the right time


The problem:
* Things work fine most of the time, but rarely, the process gets
"stuck" after fork, before exec (In steps 2 or 3 above). Process makes no
progress and does not die either.
* When I do a gdb (gdb 6.5) attach on the process, bt fails as follows:

(gdb) bt
#0  0x2ba9fd5c6a68 in __lll_mutex_lock_wait () from
/lib64/libpthread.so.0
#1  0x2ba9fd5c2a78 in _L_mutex_lock_106 () from
/lib64/libpthread.so.0
dwarf2-frame.c:521: internal-error: Unknown CFI encountered.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n)

  I looked into this error and found that pre-6.6 gdb throws this error
when looking at the stack trace of a deadlocked process. This is certainly
not a dead lock in my code as there is no locking involved in this area of
code.
* This problem happens for about 1 process in every 20,000. This
statistics is gathered across about 80 machines in our cluster, so its not
the case of a single machine having some hardware issue.
* Note that the child is forked out by a *multi threaded* python
application. I noticed some forums discussing how multi threaded (pthreads
library) processes doing things between a fork and an exec can rarely get
into  a deadlock. I know that python ( atleast 2.4.1 ) multi threading does
not use pthreads, but probably the python interpreter itself does use
pthreads?

Questions:
* Why does this happen?
* Is there a better way for us to write a
fork-wait_for_start_signal-exec construct in a multi threaded application?

Thanks,
Gangadharan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Child process freezes during fork pipe exec

[Nick Coghlan]

Gangadharan S.A. wrote:
> Hi,
> 
> Summary:
> * In my organization, we have a *multi threaded* (threading library)
> python (python 2.4.1) daemon on Linux, which starts up various processes
> using the fork pipe exec model.

The fork+threading combination had some fairly major issues that weren't
resolved until the multiprocessing module was added for 2.6/3.0 [1]

If you're able to upgrade to 2.5.4 things should be much better since
many of the fork+threading fixes were backported to the 2.5 maintenance
branch.

For 2.4 I think you're pretty much out of luck though - it was already
into security fix only mode by the time the child process deadlock
problems in the fork/threading interaction were worked out.

I don't personally know if your implementation could be modified in any
way to make the deadlock less likely with Python 2.4, but the only way
to eliminate the problem entirely (modulo any platform specific
fork+threading problems) is to upgrade to the latest version of Python 2.5.

Regards,
Nick.

[1] The bug you're probably encountering:
http://bugs.python.org/issue874900

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Gerald Britton]

Please find below PEP 3142: Add a "while" clause to generator
expressions.  I'm looking for feedback and discussion.


PEP: 3142
Title: Add a "while" clause to generator expressions
Version: $Revision: 68715 $
Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $
Author: Gerald Britton 
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 12-Jan-2009
Python-Version: 3.0
Post-History:


Abstract

  This PEP proposes an enhancement to generator expressions, adding a
  "while" clause to complement the existing "if" clause.


Rationale

  A generator expression (PEP 289 [1]) is a concise method to serve
  dynamically-generated objects to list comprehensions (PEP 202 [2]).
  Current generator expressions allow for an "if" clause to filter
  the objects that are returned to those meeting some set of
  criteria.  However, since the "if" clause is evaluated for every
  object that may be returned, in some cases it is possible that all
  objects would be rejected after a certain point.  For example:

  g = (n for n in range(100) if n*n < 50)

  which is equivalent to the using a generator function
  (PEP 255 [3]):

  def __gen(exp):
  for n in exp:
  if n*n < 50:
  yield n
  g = __gen(iter(range(10)))

  would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider
  the numbers from 8 to 99 and reject them all since n*n >= 50 for
  numbers in that range.  Allowing for a "while" clause would allow
  the redundant tests to be short-circuited:

  g = (n for n in range(100) while n*n < 50)

  would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8
  since the condition (n*n < 50) is no longer true.  This would be
  equivalent to the generator function:

  def __gen(exp):
  for n in exp:
  if n*n < 50:
  yield n
  else:
  break
  g = __gen(iter(range(100)))

  Currently, in order to achieve the same result, one would need to
  either write a generator function such as the one above or use the
  takewhile function from itertools:

  from itertools import takewhile
  g = takewhile(lambda n: n*n < 50, range(100))

  The takewhile code achieves the same result as the proposed syntax,
  albeit in a longer (some would say "less-elegant") fashion.  Also,
  the takewhile version requires an extra function call (the lambda
  in the example above) with the associated performance penalty.
  A simple test shows that:

  for n in (n for n in range(100) if 1): pass

  performs about 10% better than:

  for n in takewhile(lambda n: 1, range(100)): pass

  though they achieve similar results.  (The first example uses a
  generator; takewhile is an iterator).  If similarly implemented,
  a "while" clause should perform about the same as the "if" clause
  does today.

  The reader may ask if the "if" and "while" clauses should be
  mutually exclusive.  There are good examples that show that there
  are times when both may be used to good advantage. For example:

  p = (p for p in primes() if p > 100 while p < 1000)

  should return prime numbers found between 100 and 1000, assuming
  I have a primes() generator that yields prime numbers.  Of course, this
  could also be achieved like this:

  p = (p for p in (p for p in primes() if p > 100) while p < 1000)

  which is syntactically simpler.  Some may also ask if it is possible
  to cover dropwhile() functionality in a similar way.  I initially thought
  of:

  p = (p for p in primes() not while p < 100)

  but I am not sure that I like it since it uses "not" in a non-pythonic
  fashion, I think.

  Adding a "while" clause to generator expressions maintains the
  compact form while adding a useful facility for short-circuiting
  the expression.

Implementation:

  I am willing to assist in the implementation of this feature, although I have
  not contributed to Python thus far and would definitely need mentoring. (At
  this point I am not quite sure where to begin.)  Presently though, I would
  find it challenging to fit this work into my existing workload.


Acknowledgements

  Raymond Hettinger first proposed the concept of generator
  expressions in January 2002.


References

  [1] PEP 289: Generator Expressions
  http://www.python.org/dev/peps/pep-0289/

  [2] PEP 202: List Comprehensions
  http://www.python.org/dev/peps/pep-0202/

  [3] PEP 255: Simple Generators
  http://www.python.org/dev/peps/pep-0255/


Copyright

  This document has been placed in the public domain.


Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
PEP: 3142
Title: Add a "while" clause to generator expressions
Version: $Revision: 68715 $
Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $
Author: Gerald Britton 
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 12-Jan-2009
Python-Version: 3.0
Post-H

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Benjamin Peterson]

On Mon, Jan 19, 2009 at 9:10 AM, Gerald Britton
 wrote:
> Please find below PEP 3142: Add a "while" clause to generator
> expressions.  I'm looking for feedback and discussion.
>
>
> PEP: 3142
> Title: Add a "while" clause to generator expressions
> Version: $Revision: 68715 $
> Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $
> Author: Gerald Britton 
> Status: Draft
> Type: Standards Track
> Content-Type: text/plain
> Created: 12-Jan-2009
> Python-Version: 3.0

Since 3.0 has already been released, the only versions this feature
can be added to are 2.7 and 3.1. Do you intend this to be a 3.x only
feature?




-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Daniel Stutzbach]

On Mon, Jan 19, 2009 at 9:10 AM, Gerald Britton wrote:

>  g = (n for n in range(100) if n*n < 50)
>
>  would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider
>  the numbers from 8 to 99 and reject them all since n*n >= 50 for
>  numbers in that range.  Allowing for a "while" clause would allow
>  the redundant tests to be short-circuited:
>

Instead of using a "while" clause, the above example could simply be
rewritten:

g = (n for n in range(8))

I appreciate that this is a toy example to illustrate the syntax.  Do you
have some slightly more complex examples, that could not be rewritten by
altering the "in" clause?

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Calvin Spealman]

I am really unconvinced of the utility of this proposal and quite
convinced of the confusing factor it may well add to the current
syntax. I would like to see more applicable examples. It would replace
uses of takewhile, but that isn't a really often used function. So, is
there any evidence to support that making this a new syntax would find
so many more uses of the construct to be worth it? I believe not.

On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton
 wrote:
> Please find below PEP 3142: Add a "while" clause to generator
> expressions.  I'm looking for feedback and discussion.
>
>
> PEP: 3142
> Title: Add a "while" clause to generator expressions
> Version: $Revision: 68715 $
> Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $
> Author: Gerald Britton 
> Status: Draft
> Type: Standards Track
> Content-Type: text/plain
> Created: 12-Jan-2009
> Python-Version: 3.0
> Post-History:
>
>
> Abstract
>
>  This PEP proposes an enhancement to generator expressions, adding a
>  "while" clause to complement the existing "if" clause.
>
>
> Rationale
>
>  A generator expression (PEP 289 [1]) is a concise method to serve
>  dynamically-generated objects to list comprehensions (PEP 202 [2]).
>  Current generator expressions allow for an "if" clause to filter
>  the objects that are returned to those meeting some set of
>  criteria.  However, since the "if" clause is evaluated for every
>  object that may be returned, in some cases it is possible that all
>  objects would be rejected after a certain point.  For example:
>
>  g = (n for n in range(100) if n*n < 50)
>
>  which is equivalent to the using a generator function
>  (PEP 255 [3]):
>
>  def __gen(exp):
>  for n in exp:
>  if n*n < 50:
>  yield n
>  g = __gen(iter(range(10)))
>
>  would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider
>  the numbers from 8 to 99 and reject them all since n*n >= 50 for
>  numbers in that range.  Allowing for a "while" clause would allow
>  the redundant tests to be short-circuited:
>
>  g = (n for n in range(100) while n*n < 50)
>
>  would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8
>  since the condition (n*n < 50) is no longer true.  This would be
>  equivalent to the generator function:
>
>  def __gen(exp):
>  for n in exp:
>  if n*n < 50:
>  yield n
>  else:
>  break
>  g = __gen(iter(range(100)))
>
>  Currently, in order to achieve the same result, one would need to
>  either write a generator function such as the one above or use the
>  takewhile function from itertools:
>
>  from itertools import takewhile
>  g = takewhile(lambda n: n*n < 50, range(100))
>
>  The takewhile code achieves the same result as the proposed syntax,
>  albeit in a longer (some would say "less-elegant") fashion.  Also,
>  the takewhile version requires an extra function call (the lambda
>  in the example above) with the associated performance penalty.
>  A simple test shows that:
>
>  for n in (n for n in range(100) if 1): pass
>
>  performs about 10% better than:
>
>  for n in takewhile(lambda n: 1, range(100)): pass
>
>  though they achieve similar results.  (The first example uses a
>  generator; takewhile is an iterator).  If similarly implemented,
>  a "while" clause should perform about the same as the "if" clause
>  does today.
>
>  The reader may ask if the "if" and "while" clauses should be
>  mutually exclusive.  There are good examples that show that there
>  are times when both may be used to good advantage. For example:
>
>  p = (p for p in primes() if p > 100 while p < 1000)
>
>  should return prime numbers found between 100 and 1000, assuming
>  I have a primes() generator that yields prime numbers.  Of course, this
>  could also be achieved like this:
>
>  p = (p for p in (p for p in primes() if p > 100) while p < 1000)
>
>  which is syntactically simpler.  Some may also ask if it is possible
>  to cover dropwhile() functionality in a similar way.  I initially thought
>  of:
>
>  p = (p for p in primes() not while p < 100)
>
>  but I am not sure that I like it since it uses "not" in a non-pythonic
>  fashion, I think.
>
>  Adding a "while" clause to generator expressions maintains the
>  compact form while adding a useful facility for short-circuiting
>  the expression.
>
> Implementation:
>
>  I am willing to assist in the implementation of this feature, although I have
>  not contributed to Python thus far and would definitely need mentoring. (At
>  this point I am not quite sure where to begin.)  Presently though, I would
>  find it challenging to fit this work into my existing workload.
>
>
> Acknowledgements
>
>  Raymond Hettinger first proposed the concept of generator
>  expressions in January 2002.
>
>
> References
>
>  [1] PEP 289: Generator Expressions
>  http://www.python.org/dev/peps/pep-0289/
>
>  [2] PEP 202: List Comprehen

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Gerald Britton]

Sure:  Say I implement the sieve of Eratosthenes as a prime number
generator.  I want some primes for my application but there are an
infinite number of primes.  So I would like to write:

prime = (p for p in sieve() while p < 1000)

instead of:

import itertools
prime = takewhile(lamda p:p<1000, sieve())

to get the primes under 1000.


On Mon, Jan 19, 2009 at 11:15 AM, Daniel Stutzbach
 wrote:
> On Mon, Jan 19, 2009 at 9:10 AM, Gerald Britton 
> wrote:
>>
>>  g = (n for n in range(100) if n*n < 50)
>>
>>  would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider
>>  the numbers from 8 to 99 and reject them all since n*n >= 50 for
>>  numbers in that range.  Allowing for a "while" clause would allow
>>  the redundant tests to be short-circuited:
>
> Instead of using a "while" clause, the above example could simply be
> rewritten:
>
> g = (n for n in range(8))
>
> I appreciate that this is a toy example to illustrate the syntax.  Do you
> have some slightly more complex examples, that could not be rewritten by
> altering the "in" clause?
>
> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Gerald Britton]

Thanks Calvin,

Could you please expand on your thoughts about possible confusion?
That is, how do you see a programmer becoming confused if this option
were added to the syntax.

On Mon, Jan 19, 2009 at 11:29 AM, Calvin Spealman  wrote:
> I am really unconvinced of the utility of this proposal and quite
> convinced of the confusing factor it may well add to the current
> syntax. I would like to see more applicable examples. It would replace
> uses of takewhile, but that isn't a really often used function. So, is
> there any evidence to support that making this a new syntax would find
> so many more uses of the construct to be worth it? I believe not.
>
> On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton
>  wrote:
>> Please find below PEP 3142: Add a "while" clause to generator
>> expressions.  I'm looking for feedback and discussion.
>>
>>
>> PEP: 3142
>> Title: Add a "while" clause to generator expressions
>> Version: $Revision: 68715 $
>> Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $
>> Author: Gerald Britton 
>> Status: Draft
>> Type: Standards Track
>> Content-Type: text/plain
>> Created: 12-Jan-2009
>> Python-Version: 3.0
>> Post-History:
>>
>>
>> Abstract
>>
>>  This PEP proposes an enhancement to generator expressions, adding a
>>  "while" clause to complement the existing "if" clause.
>>
>>
>> Rationale
>>
>>  A generator expression (PEP 289 [1]) is a concise method to serve
>>  dynamically-generated objects to list comprehensions (PEP 202 [2]).
>>  Current generator expressions allow for an "if" clause to filter
>>  the objects that are returned to those meeting some set of
>>  criteria.  However, since the "if" clause is evaluated for every
>>  object that may be returned, in some cases it is possible that all
>>  objects would be rejected after a certain point.  For example:
>>
>>  g = (n for n in range(100) if n*n < 50)
>>
>>  which is equivalent to the using a generator function
>>  (PEP 255 [3]):
>>
>>  def __gen(exp):
>>  for n in exp:
>>  if n*n < 50:
>>  yield n
>>  g = __gen(iter(range(10)))
>>
>>  would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider
>>  the numbers from 8 to 99 and reject them all since n*n >= 50 for
>>  numbers in that range.  Allowing for a "while" clause would allow
>>  the redundant tests to be short-circuited:
>>
>>  g = (n for n in range(100) while n*n < 50)
>>
>>  would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8
>>  since the condition (n*n < 50) is no longer true.  This would be
>>  equivalent to the generator function:
>>
>>  def __gen(exp):
>>  for n in exp:
>>  if n*n < 50:
>>  yield n
>>  else:
>>  break
>>  g = __gen(iter(range(100)))
>>
>>  Currently, in order to achieve the same result, one would need to
>>  either write a generator function such as the one above or use the
>>  takewhile function from itertools:
>>
>>  from itertools import takewhile
>>  g = takewhile(lambda n: n*n < 50, range(100))
>>
>>  The takewhile code achieves the same result as the proposed syntax,
>>  albeit in a longer (some would say "less-elegant") fashion.  Also,
>>  the takewhile version requires an extra function call (the lambda
>>  in the example above) with the associated performance penalty.
>>  A simple test shows that:
>>
>>  for n in (n for n in range(100) if 1): pass
>>
>>  performs about 10% better than:
>>
>>  for n in takewhile(lambda n: 1, range(100)): pass
>>
>>  though they achieve similar results.  (The first example uses a
>>  generator; takewhile is an iterator).  If similarly implemented,
>>  a "while" clause should perform about the same as the "if" clause
>>  does today.
>>
>>  The reader may ask if the "if" and "while" clauses should be
>>  mutually exclusive.  There are good examples that show that there
>>  are times when both may be used to good advantage. For example:
>>
>>  p = (p for p in primes() if p > 100 while p < 1000)
>>
>>  should return prime numbers found between 100 and 1000, assuming
>>  I have a primes() generator that yields prime numbers.  Of course, this
>>  could also be achieved like this:
>>
>>  p = (p for p in (p for p in primes() if p > 100) while p < 1000)
>>
>>  which is syntactically simpler.  Some may also ask if it is possible
>>  to cover dropwhile() functionality in a similar way.  I initially thought
>>  of:
>>
>>  p = (p for p in primes() not while p < 100)
>>
>>  but I am not sure that I like it since it uses "not" in a non-pythonic
>>  fashion, I think.
>>
>>  Adding a "while" clause to generator expressions maintains the
>>  compact form while adding a useful facility for short-circuiting
>>  the expression.
>>
>> Implementation:
>>
>>  I am willing to assist in the implementation of this feature, although I 
>> have
>>  not contributed to Python thus far and would definitely need mentoring. (At
>>  this point I am not q

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Facundo Batista]

2009/1/19 Gerald Britton :

> Could you please expand on your thoughts about possible confusion?
> That is, how do you see a programmer becoming confused if this option
> were added to the syntax.

My main concern about confusion is that you're adding a "while" that
actually will behave like a "break" in the "for".

I know that with "while" you read it better, but the meaning is
different in the generator/list comprehension "for" loop.

Regards,

-- 
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Daniel Stutzbach]

On Mon, Jan 19, 2009 at 10:37 AM, Gerald Britton
wrote:

>prime = (p for p in sieve() while p < 1000)
>prime = takewhile(lamda p:p<1000, sieve())
>

I'm pretty sure the extra cost of evaluating the lambda at each step is tiny
compared to the cost of the sieve, so I don't you can make a convincing
argument on performance.

Also, you know the latter is actually fewer characters, right? :-)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Hmmm... __PyObject_NextNotImplemented when tracing lines

[skip]

I see output like this in several tests on my Mac:

test_array skipped -- 
dlopen(/Users/skip/src/python/trunk/build/lib.macosx-10.3-i386-2.7/cPickle.so, 
2): Symbol not found: __PyObject_NextNotImplemented
  Referenced from: 
/Users/skip/src/python/trunk/build/lib.macosx-10.3-i386-2.7/cPickle.so
  Expected in: dynamic lookup

This is in an up-to-date trunk sandbox running

./python.exe Lib/test/regrtest.py -T -D cover

Didn't see that in a non-coverage pass.  The following tests give this
message:

test_exceptions
test_array
test_collections
test_copy_reg
test_cpickle
test_datetime
test_deque
test_fractions
test_logging
test_multiprocessing
test_pickle
test_pickletools
test_re
test_slice
test_xpickle

I see other weirdness as well including a failure of test_sys and the
apparent inability to actually write out any useful coverage info, but this
is the standout.  In general it appears the tracing support in regrtest.py
causes a bunch of problems.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Gerald Britton]

The sieve is just one example.  The basic idea is that for some
infinite generator (even a very simple one) you want to cut it off
after some point.  As for the number of characters, I spelled lambda
incorrectly (left out a b) and there should be a space after the colon
to conform to design guides.  So, actually the takewhile version is
two characters longer, not counting "import itertools" of course!

On Mon, Jan 19, 2009 at 11:44 AM, Daniel Stutzbach
 wrote:
> On Mon, Jan 19, 2009 at 10:37 AM, Gerald Britton 
> wrote:
>>
>>prime = (p for p in sieve() while p < 1000)
>>prime = takewhile(lamda p:p<1000, sieve())
>
> I'm pretty sure the extra cost of evaluating the lambda at each step is tiny
> compared to the cost of the sieve, so I don't you can make a convincing
> argument on performance.
>
> Also, you know the latter is actually fewer characters, right? :-)
>
> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Calvin Spealman]

On Mon, Jan 19, 2009 at 11:41 AM, Gerald Britton
 wrote:
> Thanks Calvin,
>
> Could you please expand on your thoughts about possible confusion?
> That is, how do you see a programmer becoming confused if this option
> were added to the syntax.

I think that the difference between these two lines is too easy to
miss among other code:

prime = (p for p in sieve() while p < 1000)

and:

prime = (p for p in sieve() if p < 1000)

The very distinctly different behavior is two important to look so
similar at a glance. And, again, I just don't see takewhile being used
all that often. Not nearly often enough to warrant replacing it with a
special syntax! Only 178 results from Google CodeSearch, while chain,
groupby, and repeat get 4000, 3000, and 1000 respectively. Should
those be given their own syntax?

> On Mon, Jan 19, 2009 at 11:29 AM, Calvin Spealman  
> wrote:
>> I am really unconvinced of the utility of this proposal and quite
>> convinced of the confusing factor it may well add to the current
>> syntax. I would like to see more applicable examples. It would replace
>> uses of takewhile, but that isn't a really often used function. So, is
>> there any evidence to support that making this a new syntax would find
>> so many more uses of the construct to be worth it? I believe not.
>>
>> On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton
>>  wrote:
>>> Please find below PEP 3142: Add a "while" clause to generator
>>> expressions.  I'm looking for feedback and discussion.
>>>
>>>
>>> PEP: 3142
>>> Title: Add a "while" clause to generator expressions
>>> Version: $Revision: 68715 $
>>> Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $
>>> Author: Gerald Britton 
>>> Status: Draft
>>> Type: Standards Track
>>> Content-Type: text/plain
>>> Created: 12-Jan-2009
>>> Python-Version: 3.0
>>> Post-History:
>>>
>>>
>>> Abstract
>>>
>>>  This PEP proposes an enhancement to generator expressions, adding a
>>>  "while" clause to complement the existing "if" clause.
>>>
>>>
>>> Rationale
>>>
>>>  A generator expression (PEP 289 [1]) is a concise method to serve
>>>  dynamically-generated objects to list comprehensions (PEP 202 [2]).
>>>  Current generator expressions allow for an "if" clause to filter
>>>  the objects that are returned to those meeting some set of
>>>  criteria.  However, since the "if" clause is evaluated for every
>>>  object that may be returned, in some cases it is possible that all
>>>  objects would be rejected after a certain point.  For example:
>>>
>>>  g = (n for n in range(100) if n*n < 50)
>>>
>>>  which is equivalent to the using a generator function
>>>  (PEP 255 [3]):
>>>
>>>  def __gen(exp):
>>>  for n in exp:
>>>  if n*n < 50:
>>>  yield n
>>>  g = __gen(iter(range(10)))
>>>
>>>  would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider
>>>  the numbers from 8 to 99 and reject them all since n*n >= 50 for
>>>  numbers in that range.  Allowing for a "while" clause would allow
>>>  the redundant tests to be short-circuited:
>>>
>>>  g = (n for n in range(100) while n*n < 50)
>>>
>>>  would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8
>>>  since the condition (n*n < 50) is no longer true.  This would be
>>>  equivalent to the generator function:
>>>
>>>  def __gen(exp):
>>>  for n in exp:
>>>  if n*n < 50:
>>>  yield n
>>>  else:
>>>  break
>>>  g = __gen(iter(range(100)))
>>>
>>>  Currently, in order to achieve the same result, one would need to
>>>  either write a generator function such as the one above or use the
>>>  takewhile function from itertools:
>>>
>>>  from itertools import takewhile
>>>  g = takewhile(lambda n: n*n < 50, range(100))
>>>
>>>  The takewhile code achieves the same result as the proposed syntax,
>>>  albeit in a longer (some would say "less-elegant") fashion.  Also,
>>>  the takewhile version requires an extra function call (the lambda
>>>  in the example above) with the associated performance penalty.
>>>  A simple test shows that:
>>>
>>>  for n in (n for n in range(100) if 1): pass
>>>
>>>  performs about 10% better than:
>>>
>>>  for n in takewhile(lambda n: 1, range(100)): pass
>>>
>>>  though they achieve similar results.  (The first example uses a
>>>  generator; takewhile is an iterator).  If similarly implemented,
>>>  a "while" clause should perform about the same as the "if" clause
>>>  does today.
>>>
>>>  The reader may ask if the "if" and "while" clauses should be
>>>  mutually exclusive.  There are good examples that show that there
>>>  are times when both may be used to good advantage. For example:
>>>
>>>  p = (p for p in primes() if p > 100 while p < 1000)
>>>
>>>  should return prime numbers found between 100 and 1000, assuming
>>>  I have a primes() generator that yields prime numbers.  Of course, this
>>>  could also be achieved like this:
>>>
>>>  p = (p f

Re: [Python-Dev] Hmmm... __PyObject_NextNotImplemented when tracing lines

[Alexander Belopolsky]

On Mon, Jan 19, 2009 at 11:51 AM,   wrote:
> I see output like this in several tests on my Mac:
>
>test_array skipped -- 
> dlopen(/Users/skip/src/python/trunk/build/lib.macosx-10.3-i386-2.7/cPickle.so,
>  2): Symbol not found: __PyObject_NextNotImplemented
>  Referenced from: 
> /Users/skip/src/python/trunk/build/lib.macosx-10.3-i386-2.7/cPickle.so
>  Expected in: dynamic lookup
>
> This is in an up-to-date trunk sandbox running
>
>./python.exe Lib/test/regrtest.py -T -D cover
..

I cannot reproduce this on my Mac.   It looks like you may have an out
of date python.exe in your sandbox.  Please check that

$ nm python.exe | grep PyObject_NextNotImplemented
00052940 T __PyObject_NextNotImplemented

has a "T" in the second column.

Note that this symbol seem to have been added recently:

$ svn blame -v Objects/object.c
..
 68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
_PyObject_NextNotImplemented(PyObject *self)
 68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009) {
 68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
 PyErr_Format(PyExc_TypeError,
 68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
  "'%.200s' object is not iterable",
 68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
  Py_TYPE(self)->tp_name);
 68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
 return NULL;
 68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009) }
 68560 amaury.forgeotdarc 2009-01-12 18:36:55 -0500 (Mon, 12 Jan 2009)
..

So it is possible that you did not pick it up in your build yet.

I am puzzled, however, that you don't see problems in a non-coverage run.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Steven Bethard]

On Mon, Jan 19, 2009 at 7:10 AM, Gerald Britton
 wrote:
> PEP: 3142
> Title: Add a "while" clause to generator expressions
[snip]
>  numbers in that range.  Allowing for a "while" clause would allow
>  the redundant tests to be short-circuited:
>
>  g = (n for n in range(100) while n*n < 50)
>
>  would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8
>  since the condition (n*n < 50) is no longer true.  This would be
>  equivalent to the generator function:
>
>  def __gen(exp):
>  for n in exp:
>  if n*n < 50:
>  yield n
>  else:
>  break
>  g = __gen(iter(range(100)))

-1. As I pointed out on python-ideas, this proposal makes "while" mean
something different in a generator expression. Currently, you can read
any generator expression as a regular generator by simply indenting
each clause and adding a yield statement. For example:

(n for n in range(100) if n*n < 50)

turns into:

for n in range(100):
if n*n < 50:
yield n

Applying that nice correspondence to the proposed "while" generator
expression doesn't work though. For example:

(n for n in range(100) while n*n < 50)

is, under the proposal, *not* equivalent to:

for n in range(100):
while n*n < 50:
yield n

I'm strongly against making "while" mean something different in a
generator expression than it does in a "while" statement.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
--- Bucky Katt, Get Fuzzy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Terry Reedy]

Gerald Britton wrote:

Please find below PEP 3142: Add a "while" clause to generator
expressions.  I'm looking for feedback and discussion.


This was already discussed on python-ideas where it got negative feedback.

One objection, mentioned by Mathias Panzerbock and Georg Brandl, is that 
it is redundant with takewhile().  You did mention that in the PEP.


The other, posted by Steven Bethard, is that it fundamentally breaks the 
current semantics of abbreviating (except for iteration variable 
scoping) an 'equivalent' for loop.  This should have been listed in the 
PEP under Objections (or whatever the section.  I did not bother to 
second his objection there but will now.


-1

Steven summary:
"I'm probably just repeating myself here, but the reason not to do it
is that the current generator expressions translate almost directly
into the corresponding generator statements. Using "while" in the way
you've suggested breaks this symmetry, and would make Python harder to
learn."

Longer presentation:
"I think this could end up being confusing. Current generator
expressions turn into an equivalent generator function by simply
indenting the clauses and adding a yield, for example:

(i for i in range(100) if i % 2 == 0)

is equivalent to:

def gen():
for i in range(100):
if i % 2 == 0:
yield i

Now you're proposing syntax that would no longer work like this.
Taking your example:

(i for i in range(100) while i <= 50)

I would expect this to mean:
[meaning, one would expect this to mean, using current rules = tjr]

def gen():
for i in range(100):
while i <= 50:
yield i

In short, -1. You're proposing to use an existing keyword in a new way
that doesn't match how generator expressions are evaluated."

Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Gerald Britton]

Duly noted and thanks for the feedback!  (just what I was looking for
actually).  I do disagree with the idea that the proposal, if
implemented, would make Python harder to learn.  Not sure who would
find it harder.  Having to find and use takewhile was harder for me.
I still find that one counter-intuitive.  I would have expected the
parameters in the reverse order (take something, while something else
is true).  Tripped me up a few times, which got me thinking about an
alternative.

On Mon, Jan 19, 2009 at 12:51 PM, Terry Reedy  wrote:
> Gerald Britton wrote:
>>
>> Please find below PEP 3142: Add a "while" clause to generator
>> expressions.  I'm looking for feedback and discussion.
>
> This was already discussed on python-ideas where it got negative feedback.
>
> One objection, mentioned by Mathias Panzerbock and Georg Brandl, is that it
> is redundant with takewhile().  You did mention that in the PEP.
>
> The other, posted by Steven Bethard, is that it fundamentally breaks the
> current semantics of abbreviating (except for iteration variable scoping) an
> 'equivalent' for loop.  This should have been listed in the PEP under
> Objections (or whatever the section.  I did not bother to second his
> objection there but will now.
>
> -1
>
> Steven summary:
> "I'm probably just repeating myself here, but the reason not to do it
> is that the current generator expressions translate almost directly
> into the corresponding generator statements. Using "while" in the way
> you've suggested breaks this symmetry, and would make Python harder to
> learn."
>
> Longer presentation:
> "I think this could end up being confusing. Current generator
> expressions turn into an equivalent generator function by simply
> indenting the clauses and adding a yield, for example:
>
>(i for i in range(100) if i % 2 == 0)
>
> is equivalent to:
>
>def gen():
>for i in range(100):
>if i % 2 == 0:
>yield i
>
> Now you're proposing syntax that would no longer work like this.
> Taking your example:
>
>(i for i in range(100) while i <= 50)
>
> I would expect this to mean:
> [meaning, one would expect this to mean, using current rules = tjr]
>
>def gen():
>for i in range(100):
>while i <= 50:
>yield i
>
> In short, -1. You're proposing to use an existing keyword in a new way
> that doesn't match how generator expressions are evaluated."
>
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/gerald.britton%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Vitor Bosshard]

- Mensaje original 
> De: Gerald Britton 
> Para: Terry Reedy 
> CC: python-dev@python.org
> Enviado: lunes, 19 de enero, 2009 15:03:47
> Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator 
> expressions
> 
> Duly noted and thanks for the feedback!  (just what I was looking for
> actually).  I do disagree with the idea that the proposal, if
> implemented, would make Python harder to learn.  Not sure who would
> find it harder.  Having to find and use takewhile was harder for me.
> I still find that one counter-intuitive.  I would have expected the
> parameters in the reverse order (take something, while something else
> is true).  Tripped me up a few times, which got me thinking about an
> alternative.


Are you even sure the list comprehension doesn't already shortcut evaluation?

This quick test in 2.6 hints otherwise:


>>> a = (i for i in range(10) if i**2<10)
>>> a.next()
0
>>> a.next()
1
>>> a.next()
2
>>> a.next()
3
>>> a.next()
Traceback (most recent call last):
  File "", line 1, in 
    a.next()
StopIteration


  ¡Todo sobre la Liga Mexicana de fútbol! Estadisticas, resultados, 
calendario, fotos y más:<
http://espanol.sports.yahoo.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Scott Dial]

Vitor Bosshard wrote:
> Are you even sure the list comprehension doesn't already shortcut evaluation?

It does not. The body of the comprehension is evaluated all the way to
completion, despite the fact that a.next() does not return until there
is a successful test of the if expression.

>>> def print_range(n):
... for i in range(n):
... print(i)
... yield i
...
>>> a = (i for i in print_range(10) if i**2<10)
>>> a.next()
0
0
>>> a.next()
1
1
>>> a.next()
2
2
>>> a.next()
3
3
>>> a.next()
4
5
6
7
8
9
Traceback (most recent call last):
  File "", line 1, in 
StopIteration

-- 
Scott Dial
sc...@scottdial.com
scod...@cs.indiana.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Gustavo Carneiro]

2009/1/19 Vitor Bosshard 

>
>
>
>
> - Mensaje original 
> > De: Gerald Britton 
> > Para: Terry Reedy 
> > CC: python-dev@python.org
> > Enviado: lunes, 19 de enero, 2009 15:03:47
> > Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator
> expressions
> >
> > Duly noted and thanks for the feedback!  (just what I was looking for
> > actually).  I do disagree with the idea that the proposal, if
> > implemented, would make Python harder to learn.  Not sure who would
> > find it harder.  Having to find and use takewhile was harder for me.
> > I still find that one counter-intuitive.  I would have expected the
> > parameters in the reverse order (take something, while something else
> > is true).  Tripped me up a few times, which got me thinking about an
> > alternative.
>
>
> Are you even sure the list comprehension doesn't already shortcut
> evaluation?
>
> This quick test in 2.6 hints otherwise:
>
>
> >>> a = (i for i in range(10) if i**2<10)
> >>> a.next()
> 0
> >>> a.next()
> 1
> >>> a.next()
> 2
> >>> a.next()
> 3
> >>> a.next()
> Traceback (most recent call last):
>   File "", line 1, in 
> a.next()
> StopIteration


Does not prove anything.

 test.py 
source = iter(xrange(10))
a = (i for i in source if i**2<10)
print list(a)
print list(source)

 output ---
$ python /tmp/test.py
[0, 1, 2, 3]
[]

While 'a' is being evaluated, the source iterator is first completely
exhausted.  If the source iterator is infinite, an infinite loop is created
and the program doesn't terminate.




>
>
>  ¡Todo sobre la Liga Mexicana de fútbol! Estadisticas, resultados,
> calendario, fotos y más:<
> http://espanol.sports.yahoo.com/
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com
>



-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Alexander Belopolsky]

On Mon, Jan 19, 2009 at 1:41 PM, Scott Dial
 wrote:
> Vitor Bosshard wrote:
>> Are you even sure the list comprehension doesn't already shortcut evaluation?
>
> It does not. The body of the comprehension is evaluated all the way to
> completion, ..

In addition, the test is evaluated on all items as well:

>>> def test(i):
...print "testing", i
...return i**2 < 10
...
>>> a = (i for i in range(10) if test(i))
>>> a.next()
testing 0
0
>>> a.next()
testing 1
1
>>> a.next()
testing 2
2
>>> a.next()
testing 3
3
>>> a.next()
testing 4
testing 5
testing 6
testing 7
testing 8
testing 9
Traceback (most recent call last):
  File "", line 1, in 
StopIteration
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Paul Moore]

2009/1/19 Vitor Bosshard :
> Are you even sure the list comprehension doesn't already shortcut evaluation?
>
> This quick test in 2.6 hints otherwise:
>
>
 a = (i for i in range(10) if i**2<10)

Yes, but your test, once it becomes true, remains so. Consider

>>> list(n for n in range(10) if n%2 == 0)
[0, 2, 4, 6, 8]

I assume that the intention of the while syntax is:

>>> list(n for n in range(10) while n%2 == 0)
[0]

because 1%2 != 0 so the loop stops.

Having said that, I'm -1 on the proposal. The requirement is rare
enough that the correct place for it *is* in a module - and that's
what itertools provides (along with a number of other, equally valid,
manipulations). I certainly don't see it as justifying new syntax.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Brett Cannon]

On Mon, Jan 19, 2009 at 10:03, Gerald Britton  wrote:
> Duly noted and thanks for the feedback!  (just what I was looking for
> actually).  I do disagree with the idea that the proposal, if
> implemented, would make Python harder to learn.  Not sure who would
> find it harder.  Having to find and use takewhile was harder for me.
> I still find that one counter-intuitive.  I would have expected the
> parameters in the reverse order (take something, while something else
> is true).  Tripped me up a few times, which got me thinking about an
> alternative.
>

The reason Python would be harder to learn is there is something more
to learn. Removing confusion for something from the standard library
by adding syntax does not warrant making the language easier.

For something to be considered making the language easier it needs to
solve a very common idiom in a way that is an improvement over the
alternatives. In this instance I don't think the idiom is common
enough to warrant the change.

-Brett


> On Mon, Jan 19, 2009 at 12:51 PM, Terry Reedy  wrote:
>> Gerald Britton wrote:
>>>
>>> Please find below PEP 3142: Add a "while" clause to generator
>>> expressions.  I'm looking for feedback and discussion.
>>
>> This was already discussed on python-ideas where it got negative feedback.
>>
>> One objection, mentioned by Mathias Panzerbock and Georg Brandl, is that it
>> is redundant with takewhile().  You did mention that in the PEP.
>>
>> The other, posted by Steven Bethard, is that it fundamentally breaks the
>> current semantics of abbreviating (except for iteration variable scoping) an
>> 'equivalent' for loop.  This should have been listed in the PEP under
>> Objections (or whatever the section.  I did not bother to second his
>> objection there but will now.
>>
>> -1
>>
>> Steven summary:
>> "I'm probably just repeating myself here, but the reason not to do it
>> is that the current generator expressions translate almost directly
>> into the corresponding generator statements. Using "while" in the way
>> you've suggested breaks this symmetry, and would make Python harder to
>> learn."
>>
>> Longer presentation:
>> "I think this could end up being confusing. Current generator
>> expressions turn into an equivalent generator function by simply
>> indenting the clauses and adding a yield, for example:
>>
>>(i for i in range(100) if i % 2 == 0)
>>
>> is equivalent to:
>>
>>def gen():
>>for i in range(100):
>>if i % 2 == 0:
>>yield i
>>
>> Now you're proposing syntax that would no longer work like this.
>> Taking your example:
>>
>>(i for i in range(100) while i <= 50)
>>
>> I would expect this to mean:
>> [meaning, one would expect this to mean, using current rules = tjr]
>>
>>def gen():
>>for i in range(100):
>>while i <= 50:
>>yield i
>>
>> In short, -1. You're proposing to use an existing keyword in a new way
>> that doesn't match how generator expressions are evaluated."
>>
>> Terry Jan Reedy
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/gerald.britton%40gmail.com
>>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] stuck with dlopen...

[Gregory P. Smith]

If you run your python.exe under gdb you should be able to set a future
breakpoint on your _PyEval_EvalMiniFrameEx function and debug from there.

On Wed, Jan 14, 2009 at 8:28 PM,  wrote:

>
> I've recently been working on generating C functions on-the-fly which
> inline
> the C code necessary to implement the bytecode in a given Python function.
> For example, this bytecode:
>
>>>> dis.dis(f)
>  2   0 LOAD_FAST0 (a)
>  3 LOAD_CONST   1 (1)
>  6 BINARY_ADD
>  7 RETURN_VALUE
>
> is transformed into this rather boring bit of C code:
>
>#include "Python.h"
>
>#include "code.h"
>#include "frameobject.h"
>#include "eval.h"
>#include "opcode.h"
>#include "structmember.h"
>
>#include "opcode_mini.h"
>
>PyObject *
>_PyEval_EvalMiniFrameEx(PyFrameObject *f, int throwflag)
>{
>
>static int jitting = 1;
>
>PyEval_EvalFrameEx_PROLOG1();
>co = f->f_code;
>PyEval_EvalFrameEx_PROLOG2();
>
>oparg = 0;
>LOAD_FAST_IMPL(oparg);
>oparg = 1;
>LOAD_CONST_IMPL(oparg);
>BINARY_ADD_IMPL();
>RETURN_VALUE_IMPL();
>
>PyEval_EvalFrameEx_EPILOG();
>}
>
> The PROLOG1, PROLOG2 and EPILOG macros are just chunks of code from
> PyEval_EvalFrameEx.
>
> I have the code compiling and linking, and dlopen and dlsym seem to work,
> returning apparently valid pointers, but when I try to call the function I
> get
>
>Program received signal EXC_BAD_ACCESS, Could not access memory.
>Reason: KERN_PROTECTION_FAILURE at address: 0x000c
>0x0058066d in _PyEval_EvalMiniFrameEx (f=0x230d30, throwflag=0) at
> MwDLSf.c:17
>
> Line 17 is the PROLOG1 macro.  I presume it's probably barfed on the very
> first instruction.  (This is all on an Intel Mac running Leopard BTW.)
>
> Here are the commands generated to compile and link the C code:
>
>gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall \
>-Wstrict-prototypes -g -DPy_BUILD_CORE -DNDEBUG \
>-I/Users/skip/src/python/py3k-t/Include \
>-I/Users/skip/src/python/py3k-t -c dTd5cl.c \
>-o /tmp/MwDLSf.o
>gcc -L/opt/local/lib -bundle -undefined dynamic_lookup -g \
>/tmp/dTd5cl.o -L/Users/skip/src/python/py3k-t -lpython3.1 \
>-o /tmp/MwDLSf.so
>
> (It just uses the distutils compiler module to build .so files.)  The .so
> file looks more-or-less ok:
>
>% otool -L /tmp/MwDLSf.so
>/tmp/MwDLSf.so:
>/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 111.1.3)
>
> though nm doesn't show that any undefined _Py* symbols so I suspect I'm not
> linking it correctly.  The Python executable was built without
> --enable-shared.  I've tried building with that config flag, but that just
> gives me fits during debugging because it always wants to find libpython in
> the installation directory even if I'm running python.exe from the build
> directory.  Installing is a little tedious because it relies on a properly
> functioning interpreter.
>
> dlopen is called very simply:
>
>handle = dlopen(shared, RTLD_NOW);
>
> I used RTLD_NOW because that's what sys.getdlopenflags() returns.  I'm not
> calling dlclose for the time being.
>
> I'm not exactly sure where I should go from here.  I'd be more than happy
> to
> open an item in the issue tracker.  I was hoping to get something a bit
> closer to working before doing that though.  The failure to properly load
> the compiled function makes it pretty much impossble to debug the generated
> code beyond what the compiler can tell me.
>
> Any suggestions?
>
> Skip
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Sturla Molden]

On 1/19/2009 6:51 PM, Terry Reedy wrote:

The other, posted by Steven Bethard, is that it fundamentally breaks the 
current semantics of abbreviating (except for iteration variable 
scoping) an 'equivalent' for loop. 


The proposed syntax would suggest that this should be legal as well:

for i in iterable while cond:
   blahblah

or perhaps:

while cond for i in iterable:
   blahblah


A while-for or for-while loop would be a novel invention, not seen in 
any other language that I know of. I seriously doubt its usefulness 
though...




Sturla Molden
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Nick Coghlan]

Steven Bethard wrote:
> -1. As I pointed out on python-ideas, this proposal makes "while" mean
> something different in a generator expression.

While I initially found the suggestion in the PEP rather cute, that
isn't enough to make it a good idea as a language addition.

So, -1 for a few reasons:
- the 'idiom' being replaced isn't common enough to justify dedicated syntax
- the proposed syntax change isn't significantly easier to understand or
significantly faster than the existing alternatives
- Steven's more substantial objection that it would break the parallel
between generator expressions/list comprehensions and the corresponding
statements.

Cheers,
Nick.

P.S. Some examples of past syntax changes that *were* accepted (and why):
http://mail.python.org/pipermail/python-dev/2008-October/082831.html

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] compiling python2.5 (msys+mingw+wine) using msvcr80 assemblies

[Luke Kenneth Casson Leighton]

folks, hi,
after some quiet advice i've tracked down a method for compiling
python2.5 using msvcr80 that _will_ actually work both under native
win32 and also under wine, but it's a _bit_ dodgy, as i couldn't track
down where you're supposed to put Microsoft.VC80.CRT, except in the
path of the application where it's running from.  so, instead, i put
the _contents_ of Microsoft.VC80.CRT.manifest into the manifest for
the file, and this _does_ actually seem to work.
i'm thinking of adding the Microsoft.VC80.CRT.manifest to the rc file
(for compilation as a resource) to see if _that_ works, and will
report back, but first i wanted to describe what i've done and see
what people think:

1) created python_2.5_8.0_mingw_exe.manifest contents as follows, this
is _normally_ what is in Microsoft.VC80.CRT _not_ in the .exe.manifest



http://www.w3.org/2000/09/xmldsig#";>http://www.w3.org/2000/09/xmldsig#sha1";>n9On8FItNsK/DmT8UQxu6jYDtWQ=
http://www.w3.org/2000/09/xmldsig#";>http://www.w3.org/2000/09/xmldsig#sha1";>0KJ/VTwP4OUHx98HlIW2AdW1kuY=
http://www.w3.org/2000/09/xmldsig#";>http://www.w3.org/2000/09/xmldsig#sha1";>YJuB+9Os2oxW4mY+2oC/r8lICZE=


2) created python_2.5_8.0_mingw_exe.rc contents as follows:

#include "winuser.h"
2 RT_MANIFEST PC/python_2.5_8.0_mingw_exe.manifest

you could get away with 2 24 PC/. and could exclude the #include

3) added a rule to Makefile.pre.in to create the .res as a binary:

# This rule builds the .res file for the Python EXE, required when
# linking and using msvcrt80 or above.  good luck to us all...
$(PYTHONEXEMSVRES): $(srcdir)/PC/python_$(VERSION)_$(MSRTVER)_exe.manifest \
$(srcdir)/PC/python_$(VERSION)_$(MSRTVER)_mingw_exe.rc
windres --input $(srcdir)/PC/python_$(VERSION)_$(MSRTVER)_mingw_exe.rc \
--output $(PYTHONEXEMSVRES) \
--output-format=coff

4) added $(PYTHONEXEMSVRES) to the objects to be linked.


stunningly, this actually works (of course, you need an msvcr80.dll
for it to work duh).

i tried finding a location to place the Microsoft.VC80.CRT.Manifest,
prior to this hack - a wine dump showed this:

0009:trace:actctx:lookup_assembly looking for
name=L"Microsoft.VC80.CRT" version=8.0.50727.762 arch=L"x86"
0009:trace:heap:RtlAllocateHeap (0x11,0002,0038): returning 0x115148
0009:trace:file:RtlDosPathNameToNtPathName_U
(L"C:\\windows\\winsxs\\manifests",0xff8c7d08,(nil),(nil))
0009:trace:file:RtlGetFullPathName_U
(L"C:\\windows\\winsxs\\manifests" 520 0xff8c79f4 (nil))

attempts to copy the manifest into that directory resulted in "no joy".

so, i'm a bit stuck, and would appreciate some advice on whether the
above is acceptable (yes i know it makes sure that python.exe can only
use one _very_ specific version of msvcr80.dll - and there are
currently two: 8.0.50727.762 and 8.0.50608.0.

also i'd appreciate some advice on what the _really_ best way to do this is.

and on where the hell i'm supposed to put the VC80.CRT manifest so it
will actually... _do_ something!

l.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Kristján Valur Jónsson]

Are you all certain that this mapping from a generator expression to a foor 
loop isn't just a happy coincidence?
After all, the generator statement is just a generalization of the list 
comprehension and that doesn't map quite so directly.

I have always taken both expressions at face value, and not tried to map them 
into something else.  Why should you, since they are designed to match the 
grammar of the english language and make perfect sense if you read them as you 
would a herring recipe.
The suggested "while" keyword expands on this easy to understand theme, and the 
fact that it doesn't fit a mapping that was probably never intentional 
shouldn't detract from that.

K


-Original Message-
From: python-dev-bounces+kristjan=ccpgames@python.org 
[mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Terry 
Reedy
Sent: 19. janúar 2009 17:52
To: python-dev@python.org
Subject: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator 
expressions


The other, posted by Steven Bethard, is that it fundamentally breaks the 
current semantics of abbreviating (except for iteration variable 
scoping) an 'equivalent' for loop.  This should have been listed in the 
PEP under Objections (or whatever the section.  I did not bother to 
second his objection there but will now.

-1

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hmmm... __PyObject_NextNotImplemented when tracing lines

[skip]

Alexander> I cannot reproduce this on my Mac.  It looks like you may
Alexander> have an out of date python.exe in your sandbox.  Please check
Alexander> that

Alexander> $ nm python.exe | grep PyObject_NextNotImplemented
Alexander> 00052940 T __PyObject_NextNotImplemented

Alexander> has a "T" in the second column.

Alexander> Note that this symbol seem to have been added recently:
...

I see what the problem is now.  I configured with --enable-shared.  Even
though my sandbox was up-to-date, my python.exe was current and my
libpython2.7.dylib held the symbol in question the *installed* version of
libpython2.7.dylib didn't have the symbol:

% nm python.exe | grep PyObject_NextNotImplemented
% nm libpython2.7.dylib | egrep PyObject_NextNotImplemented
00054200 T __PyObject_NextNotImplemented
% nm /Users/skip/local/lib/libpython2.7.dylib  | egrep 
PyObject_NextNotImplemented
%

I've been bitten by this before.  When building with --enable-shared the
executable appears to have the installed shared library location bound into
it:

% otool -L python.exepython.exe:
/Users/skip/local/lib/libpython2.7.dylib (compatibility version 
2.7.0, current version 2.7.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 111.1.3)

Is this something that we can fix/work around?  It seems to me like a wart,
probably platform-dependent.

Alexander> I am puzzled, however, that you don't see problems in a
Alexander> non-coverage run.

Yeah, that mystifies me as well, especially given the above output from nm
and otool.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a "while" clause to generator expressions

The proposal is similar to the c do statement

do statement while (expression);

which for whatever reason (infrequency?) like the switch statement have rightly 
not been adopted into python.
<>___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Terry Reedy]

Kristján Valur Jónsson wrote:

Are you all certain that this mapping from a generator expression to
a foor loop isn't just a happy coincidence?


Yes, The manual *defines* the meaning of a comprehension in terms of the 
corresponding nested statements.


"The comprehension consists of a single expression followed by at least 
one for clause and zero or more for or if clauses. In this case, the 
elements of the new container are those that would be produced by 
considering each of the for or if clauses a block, nesting from left to 
right, and evaluating the expression to produce an element each time the 
innermost block is reached." (3.0 doc)


It was originally defined as exactly equivalent (with empty list 
initialization added).  The only intended change of the slightly softer 
newer version is that the target name bindings do not escape the scope 
of the comprehension.


The proposed change would BREAK the definition and intent of what a 
comprehension means.


> After all, the generator statement is just a generalization
> of the list comprehension

I would call it a variation.  Syntactically, a generator statement is a 
comprehension with parentheses instead of square brackets or curly braces.


tjr

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Copyright notices in modules

[Raymond Hettinger]

Why does numbers.py say:

   # Copyright 2007 Google, Inc. All Rights Reserved.
   # Licensed to PSF under a Contributor Agreement.

Weren't there multiple contributors including non-google people?

Does Google want to be associated with code that
was submitted with no tests?

Do we want this sort of stuff in the code?

If someone signs a contributor agreement, can we
forgo the external copyright comments?

Do we want to make a practice of every contributor
commenting in the name of the company they were
working for at the time (if so, I would have to add
the comment to a lot of modules)?

Does the copyright concept even apply to an
abstract base class (I thought APIs were not
subject to copyright, just like database layouts
and language definitions)?


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Calvin Spealman]

OK, I still don't like the general idea, but I have a suggestion for
what I think is a more favorable syntax, anyway. Basically, adding the
allowance of an 'else break' when there is an if clause in a generator
expression or list comprehension. I still dont think it should be
done, but it is a more consistent syntax. It has the obvious problem
of looking like it might allow an alternative expression, like the
if-expression.

prime = (p for p in sieve() if p < 1000 else break)

On Mon, Jan 19, 2009 at 11:47 AM, Calvin Spealman  wrote:
> On Mon, Jan 19, 2009 at 11:41 AM, Gerald Britton
>  wrote:
>> Thanks Calvin,
>>
>> Could you please expand on your thoughts about possible confusion?
>> That is, how do you see a programmer becoming confused if this option
>> were added to the syntax.
>
> I think that the difference between these two lines is too easy to
> miss among other code:
>
>prime = (p for p in sieve() while p < 1000)
>
> and:
>
>prime = (p for p in sieve() if p < 1000)
>
> The very distinctly different behavior is two important to look so
> similar at a glance. And, again, I just don't see takewhile being used
> all that often. Not nearly often enough to warrant replacing it with a
> special syntax! Only 178 results from Google CodeSearch, while chain,
> groupby, and repeat get 4000, 3000, and 1000 respectively. Should
> those be given their own syntax?
>
>> On Mon, Jan 19, 2009 at 11:29 AM, Calvin Spealman  
>> wrote:
>>> I am really unconvinced of the utility of this proposal and quite
>>> convinced of the confusing factor it may well add to the current
>>> syntax. I would like to see more applicable examples. It would replace
>>> uses of takewhile, but that isn't a really often used function. So, is
>>> there any evidence to support that making this a new syntax would find
>>> so many more uses of the construct to be worth it? I believe not.
>>>
>>> On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton
>>>  wrote:
 Please find below PEP 3142: Add a "while" clause to generator
 expressions.  I'm looking for feedback and discussion.


 PEP: 3142
 Title: Add a "while" clause to generator expressions
 Version: $Revision: 68715 $
 Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $
 Author: Gerald Britton 
 Status: Draft
 Type: Standards Track
 Content-Type: text/plain
 Created: 12-Jan-2009
 Python-Version: 3.0
 Post-History:


 Abstract

  This PEP proposes an enhancement to generator expressions, adding a
  "while" clause to complement the existing "if" clause.


 Rationale

  A generator expression (PEP 289 [1]) is a concise method to serve
  dynamically-generated objects to list comprehensions (PEP 202 [2]).
  Current generator expressions allow for an "if" clause to filter
  the objects that are returned to those meeting some set of
  criteria.  However, since the "if" clause is evaluated for every
  object that may be returned, in some cases it is possible that all
  objects would be rejected after a certain point.  For example:

  g = (n for n in range(100) if n*n < 50)

  which is equivalent to the using a generator function
  (PEP 255 [3]):

  def __gen(exp):
  for n in exp:
  if n*n < 50:
  yield n
  g = __gen(iter(range(10)))

  would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider
  the numbers from 8 to 99 and reject them all since n*n >= 50 for
  numbers in that range.  Allowing for a "while" clause would allow
  the redundant tests to be short-circuited:

  g = (n for n in range(100) while n*n < 50)

  would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8
  since the condition (n*n < 50) is no longer true.  This would be
  equivalent to the generator function:

  def __gen(exp):
  for n in exp:
  if n*n < 50:
  yield n
  else:
  break
  g = __gen(iter(range(100)))

  Currently, in order to achieve the same result, one would need to
  either write a generator function such as the one above or use the
  takewhile function from itertools:

  from itertools import takewhile
  g = takewhile(lambda n: n*n < 50, range(100))

  The takewhile code achieves the same result as the proposed syntax,
  albeit in a longer (some would say "less-elegant") fashion.  Also,
  the takewhile version requires an extra function call (the lambda
  in the example above) with the associated performance penalty.
  A simple test shows that:

  for n in (n for n in range(100) if 1): pass

  performs about 10% better than:

  for n in takewhile(lambda n: 1, range(100)): pass

  though they achieve similar results.  (

[Python-Dev] Questions/comments on documentation formatting

[Brett Cannon]

I have been writing up the initial docs for importlib and four things struck me:

1. Why is three space indents the preferred indentation level?

2. Should we start using function annotations?

3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [,
c=None]])``) really necessary when default argument values are
present? And do we really need to nest the brackets when it is obvious
that having on optional argument means the rest are optional as well?

4. The var directive is not working even though the docs list it as a
valid directive; so is it still valid and something is broken, or the
docs need to be updated?

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Copyright notices in modules

[Stephen J. Turnbull]

Raymond Hettinger writes:

 > Does the copyright concept even apply to an abstract base class (I
 > thought APIs were not subject to copyright, just like database
 > layouts and language definitions)?

Yes, it does, although a public API per se is not subject to
copyright, because there's only one way to do it.  Any comments,
internal implementation (eg, names of persistent state variables,
members, and constants, and the very existence of those identifiers),
and tests are subject to copyright because they are expressive.

I believe that a private API also can be subject to copyright, though
I'm not as sure of that.  The point being that there are good APIs and
bad APIs that expose the same functionality, so that API design is
expressive.  However, if you expose the API and license people to use
it, that license makes it impossible to restrict them from using it
thereafter.

Caveat: IANAL, and this is under U.S. law.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions

[Cameron Simpson]

On 19Jan2009 19:42, Calvin Spealman  wrote:
| OK, I still don't like the general idea, but I have a suggestion for
| what I think is a more favorable syntax, anyway. Basically, adding the
| allowance of an 'else break' when there is an if clause in a generator
| expression or list comprehension. I still dont think it should be
| done, but it is a more consistent syntax. It has the obvious problem
| of looking like it might allow an alternative expression, like the
| if-expression.
| 
| prime = (p for p in sieve() if p < 1000 else break)

If I'm reading your suggestion correctly it changes feel of the "if"
part quite fandamentally. A bare "if" acts as a filter, yielding only
the items matching the condition. With an "else break" it yields only
the items matching the condition up to the first which fails.

For a range like "p < 1000" this isn't so different, but if the condition
doesn't apply to just the leading items then this form becomes almost
useless. Consider "p % 3 == 0". Normally that would yield every third
item. With "else break" it would yield just one item. Any non-continuous
filter has the same issue.

In my opinion this makes the construct far less applicable than it would
appear at first glance.

So -1 from me.

I saw someone else mention takewhile(), and that's my preferred way
of doing the original suggestion ("[ ... while some-constraint ]"),
and thus I'm -1 on the original suggestion too (also for the if/while
confusion mentioned in the thread).

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

Drive Agressively  Rash Magnificently   - Nankai Leathers
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Benjamin Peterson]

On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon  wrote:
> I have been writing up the initial docs for importlib and four things struck 
> me:
>
> 1. Why is three space indents the preferred indentation level?

Because it matches nicely up with the length of directives:

.. somedirective:: blah
^^^

>
> 2. Should we start using function annotations?

No, I think that information is better stored in the function description.

>
> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [,
> c=None]])``) really necessary when default argument values are
> present? And do we really need to nest the brackets when it is obvious
> that having on optional argument means the rest are optional as well?

Actually, the defaults are usually documented in the description not
the signature.

>
> 4. The var directive is not working even though the docs list it as a
> valid directive; so is it still valid and something is broken, or the
> docs need to be updated?

The docs should be updated. "data" is the one to use now.




-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Scott Dial]

Brett Cannon wrote:
> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [,
> c=None]])``) really necessary when default argument values are
> present? And do we really need to nest the brackets when it is obvious
> that having on optional argument means the rest are optional as well?

I can't think of an example off the top of my head, but I'm certain the
point of nesting the brackets is to delimit the optional arguments into
groups. Documenting your fxn() examples as "fxn(a [, b=None, c=None])"
would imply that if you provide 'b' then you must provide 'c', or if we
abandon nested brackets, it's ambiguous as to the requirements. Imagine
seeing "foo(a [, b=None, c=None [, d=None]])" and I think the rationale
for such notation becomes clear.

-- 
Scott Dial
sc...@scottdial.com
scod...@cs.indiana.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Brett Cannon]

On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson  wrote:
> On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon  wrote:
>> I have been writing up the initial docs for importlib and four things struck 
>> me:
>>
>> 1. Why is three space indents the preferred indentation level?
>
> Because it matches nicely up with the length of directives:
>
> .. somedirective:: blah
> ^^^
>
>>
>> 2. Should we start using function annotations?
>
> No, I think that information is better stored in the function description.
>

Why? Putting it in the signature makes it very succinct and a simple
glance at the doc to see what type/ABC is expected.

>>
>> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [,
>> c=None]])``) really necessary when default argument values are
>> present? And do we really need to nest the brackets when it is obvious
>> that having on optional argument means the rest are optional as well?
>
> Actually, the defaults are usually documented in the description not
> the signature.
>

OK, but that doesn't make it optimal. And that still doesn't answer my
question of whether all of those nested brackets are truly necessary.

>>
>> 4. The var directive is not working even though the docs list it as a
>> valid directive; so is it still valid and something is broken, or the
>> docs need to be updated?
>
> The docs should be updated. "data" is the one to use now.

So the 'data' directive turns into any variable, not just a module variables?

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Benjamin Peterson]

On Mon, Jan 19, 2009 at 9:11 PM, Brett Cannon  wrote:
> On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson  wrote:
>> On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon  wrote:
>>>
>>> 2. Should we start using function annotations?
>>
>> No, I think that information is better stored in the function description.
>>
>
> Why? Putting it in the signature makes it very succinct and a simple
> glance at the doc to see what type/ABC is expected.

Well, I guess it's just not been explored. Feel free to try it out if
you wish, though.

>
>>>
>>> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [,
>>> c=None]])``) really necessary when default argument values are
>>> present? And do we really need to nest the brackets when it is obvious
>>> that having on optional argument means the rest are optional as well?
>>
>> Actually, the defaults are usually documented in the description not
>> the signature.
>>
>
> OK, but that doesn't make it optimal. And that still doesn't answer my
> question of whether all of those nested brackets are truly necessary.

All I can say is that it is the style/convention.

>
>>>
>>> 4. The var directive is not working even though the docs list it as a
>>> valid directive; so is it still valid and something is broken, or the
>>> docs need to be updated?
>>
>> The docs should be updated. "data" is the one to use now.
>
> So the 'data' directive turns into any variable, not just a module variables?

"data" is for module level objects. If you're documenting properties
or attributes in classes, use "attribute".



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Raymond Hettinger]

From: "Brett Cannon" 

1. Why is three space indents the preferred indentation level?


I've also wondered about this.  It is somewhat incovenient
when bringing in code samples from files with four space indents.

Raymond


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Benjamin Peterson]

On Mon, Jan 19, 2009 at 9:23 PM, Raymond Hettinger  wrote:
> From: "Brett Cannon" 
>>
>> 1. Why is three space indents the preferred indentation level?
>
> I've also wondered about this.  It is somewhat incovenient
> when bringing in code samples from files with four space indents.

It's just reST indentation that is 3 spaces. Code examples in the reST
can be 4 spaces.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Ben Finney]

"Benjamin Peterson"  writes:

> On Mon, Jan 19, 2009 at 9:23 PM, Raymond Hettinger  wrote:
> > From: "Brett Cannon" 
> >>
> >> 1. Why is three space indents the preferred indentation level?
> >
> > I've also wondered about this.  It is somewhat incovenient
> > when bringing in code samples from files with four space indents.
> 
> It's just reST indentation that is 3 spaces.

It doesn't have to be. When writing reST, I always make directives so
they will line up nicely at 4-space indents:

=
Normal paragraph

..  Comment

..  foo::

bar
=

-- 
 \  “I was the kid next door's imaginary friend.” —Emo Philips |
  `\   |
_o__)  |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Raymond Hettinger]

I have another question about doc formatting.

What controls whether section headers get urls with a custom named jump target instead of 
a default name like "id1"?

In particular, look at the urls for:
   http://docs.python.org/dev/library/collections.html#id1 
versus
   http://docs.python.org/dev/library/collections.html#abcs-abstract-base-classes 


I would like all of the targets to have meaningful names.


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Ben Finney]

"Raymond Hettinger"  writes:

> What controls whether section headers get urls with a custom named
> jump target instead of a default name like "id1"?
> 
> In particular, look at the urls for:
>http://docs.python.org/dev/library/collections.html#id1 versus

Hmm. Immediately preceding the  element, there is also an empty
 element with a meaningful ID, "counter-objects", that can be
used to get to the same position in the document:

http://docs.python.org/dev/library/collections.html#counter-objects

However, the problem is that URL isn't used for the “Permalink to this
headline” link.

Perhaps a bug report to the Docutils folks is in order.

-- 
 \ “Nature is trying very hard to make us succeed, but nature does |
  `\   not depend on us. We are not the only experiment.” —Richard |
_o__)   Buckminster Fuller, 1978-04-30 |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Brett Cannon]

On Mon, Jan 19, 2009 at 19:19, Benjamin Peterson  wrote:
> On Mon, Jan 19, 2009 at 9:11 PM, Brett Cannon  wrote:
>> On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson  wrote:
>>> On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon  wrote:

 2. Should we start using function annotations?
>>>
>>> No, I think that information is better stored in the function description.
>>>
>>
>> Why? Putting it in the signature makes it very succinct and a simple
>> glance at the doc to see what type/ABC is expected.
>
> Well, I guess it's just not been explored. Feel free to try it out if
> you wish, though.
>

I just might.

>>

 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [,
 c=None]])``) really necessary when default argument values are
 present? And do we really need to nest the brackets when it is obvious
 that having on optional argument means the rest are optional as well?
>>>
>>> Actually, the defaults are usually documented in the description not
>>> the signature.
>>>
>>
>> OK, but that doesn't make it optimal. And that still doesn't answer my
>> question of whether all of those nested brackets are truly necessary.
>
> All I can say is that it is the style/convention.
>

Right, which is why I am questioning it. =)

>>

 4. The var directive is not working even though the docs list it as a
 valid directive; so is it still valid and something is broken, or the
 docs need to be updated?
>>>
>>> The docs should be updated. "data" is the one to use now.
>>
>> So the 'data' directive turns into any variable, not just a module variables?
>
> "data" is for module level objects. If you're documenting properties
> or attributes in classes, use "attribute".

Then what are we supposed to use for arguments? Just ``literal``?

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Brett Cannon]

On Mon, Jan 19, 2009 at 19:50, Raymond Hettinger  wrote:
> I have another question about doc formatting.
>
> What controls whether section headers get urls with a custom named jump
> target instead of a default name like "id1"?
>
> In particular, look at the urls for:
>   http://docs.python.org/dev/library/collections.html#id1 versus
>
> http://docs.python.org/dev/library/collections.html#abcs-abstract-base-classes
> I would like all of the targets to have meaningful names.

Not sure from a sphinx perspective, but Docutils does this
automatically. You can also always specify the anchor point manually::

  .. _abcs-abstract-base-classes

  Abstract Base Classes
  --

to get what you want.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Brett Cannon]

On Mon, Jan 19, 2009 at 19:02, Scott Dial
 wrote:
> Brett Cannon wrote:
>> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [,
>> c=None]])``) really necessary when default argument values are
>> present? And do we really need to nest the brackets when it is obvious
>> that having on optional argument means the rest are optional as well?
>
> I can't think of an example off the top of my head, but I'm certain the
> point of nesting the brackets is to delimit the optional arguments into
> groups. Documenting your fxn() examples as "fxn(a [, b=None, c=None])"
> would imply that if you provide 'b' then you must provide 'c', or if we
> abandon nested brackets, it's ambiguous as to the requirements. Imagine
> seeing "foo(a [, b=None, c=None [, d=None]])" and I think the rationale
> for such notation becomes clear.

Well, that is such a rare case that I don't know if it warrants the
line noise in the argument declaration. That argument also doesn't
make sense in the face of ``fxn(a [, a=None [, b=None]])`` where 'b'
almost always has no connection to 'a', but is still supposed to be
listed that way because of positional arguments being optional. I
understand using them for C functions where there is no such thing as
a default argument, but it just doesn't make a ton of sense for Python
code. I don't know of anyone who was confused by what help() spit out
and not having fancy bracketing.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Questions/comments on documentation formatting

[Raymond Hettinger]

In particular, look at the urls for:
  http://docs.python.org/dev/library/collections.html#id1 versus

http://docs.python.org/dev/library/collections.html#abcs-abstract-base-classes
I would like all of the targets to have meaningful names.


[Brett]

Not sure from a sphinx perspective, but Docutils does this
automatically. You can also always specify the anchor point manually::

 .. _abcs-abstract-base-classes

 Abstract Base Classes
 --

to get what you want.



Thanks for the note.  It pointed me to the real problem which was
that manual anchor points can interfere with the automatically
generated names if their names are the same.  The solution was
to *remove* the manually generated anchor points.


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com