Re: [Python-Dev] [Python-checkins] (2.7): Use simpler assert in basic example.

[Ezio Melotti]

On 11/03/2011 4.45, Nick Coghlan wrote:

On Thu, Mar 10, 2011 at 4:36 PM, ezio.melotti
  wrote:

http://hg.python.org/cpython/rev/9adc4792db9a
changeset:   68356:9adc4792db9a
branch:  2.7
user:Ezio Melotti
date:Thu Mar 10 23:35:39 2011 +0200
summary:
  Use simpler assert in basic example.

Please don't. Now the new methods are available, using assertTrue() on
supported expressions is actively throwing away data, and the docs
shouldn't recommend it *anywhere*.


I agree with you, but Raymond wants to keep the example as simple as 
possible and asked me to change it back.



If using assertIn() at this point in the docs is a genuine problem,
find an alternative test example that isn't better written using one
of the more specific methods.


That would be best.  Can you come up with a different example that only 
uses assertEqual/assertTrue/assertRaises?


Best Regards,
Ezio Melotti


Cheers,
Nick.


files:
  Doc/library/unittest.rst

diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -130,13 +130,13 @@

def test_choice(self):
element = random.choice(self.seq)
-   self.assertIn(element, self.seq)
+   self.assertTrue(element in self.seq)

def test_sample(self):
with self.assertRaises(ValueError):
random.sample(self.seq, 20)
for element in random.sample(self.seq, 5):
-   self.assertIn(element, self.seq)
+   self.assertTrue(element in self.seq)

if __name__ == '__main__':
unittest.main()

--
Repository URL: http://hg.python.org/cpython

___
Python-checkins mailing list
python-check...@python.org
http://mail.python.org/mailman/listinfo/python-checkins







___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] (2.6): Issue #11424: Fix bug in determining child loggers.

[Thomas Wouters]

On Thu, Mar 10, 2011 at 18:47, Nick Coghlan  wrote:

> On Thu, Mar 10, 2011 at 5:41 PM, Vinay Sajip 
> wrote:
> > - Original Message 
> >
> >> From: Éric Araujo 
> >> > From what I understand, we're supposed to forward-port in  Mercurial,
> >> Correct, but only in maintained branches, not security-mode  branches.
> >>
> >
> > Well, I saw this recent mail from Antoine:
> >
> > http://mail.python.org/pipermail/python-dev/2011-March/108766.html
> >
> > where he mentioned
> >
> >
> > - on one hand: 2.5 -> 2.6 -> 2.7 (if you still want to maintain 2.5)
> > - on the other hand: 3.1 -> 3.2 -> default
> >
> > and perhaps misunderstood. I was assuming that there wouldn't be any 2.6
> > releases from Mercurial, perhaps that assumption was unwarranted.
> >
> > Also, the section on forward-porting in the Dev Guide says "it should be
> applied
> > to the oldest branch applicable" and this could be interpreted to mean
> > applicable based on the content of the branch, rather than according to
> release
> > policy.
>
> It's the intersection: if the change applies to the content of that
> branch and that branch is still maintained, then start there.
>
> So security fixes start further back than ordinary bug fixes.
>
> This should be reverted in 2.6, with the dummy merge to prevent
> inadvertent reversion in 2.7.
>

Just to be clear on this, so people don't think this is more complicated
than it needs to be: you can still *develop* a patch based on 2.5, even
commit it to your local repository, and then only push it to 2.6 or 2.7 and
later. You don't have to consider where the patch should go beforehand.

-- 
Thomas Wouters 

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2011-03-04 - 2011-03-11)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open2709 (+18)
  closed 20540 (+53)
  total  23249 (+71)

Open issues with patches: 1151 


Issues opened (46)
==

#1271: Raw string parsing fails with backslash as last character
http://bugs.python.org/issue1271  reopened by ezio.melotti

#11397: os.path.realpath() may produce incorrect results
http://bugs.python.org/issue11397  opened by mkovtun

#11401: email.header error during .flatten()
http://bugs.python.org/issue11401  opened by sdaoden

#11402: _PyUnicode_Init leaks a little memory once
http://bugs.python.org/issue11402  opened by stutzbach

#11403: Add some generated files in PC/ to .hgignore.
http://bugs.python.org/issue11403  opened by santa4nt

#11406: There is no os.listdir() equivalent returning generator instea
http://bugs.python.org/issue11406  opened by mmarkk

#11407: unittest.TestCase.run should return result
http://bugs.python.org/issue11407  opened by michael.foord

#11409: pysetup --search should return non-zero when a dist is not ins
http://bugs.python.org/issue11409  opened by kelseyhightower

#11410: Use GCC visibility attrs in PyAPI_*
http://bugs.python.org/issue11410  opened by twouters

#11412: Section numbers in the Library Reference have a trailing perio
http://bugs.python.org/issue11412  opened by rhettinger

#11414: Add import fix for email.Message
http://bugs.python.org/issue11414  opened by kitterma

#11415: ZipFile don't overwrite compresed files at create
http://bugs.python.org/issue11415  opened by piranna

#11416: netrc module does not handle multiple entries for a single hos
http://bugs.python.org/issue11416  opened by saffroy

#11418: Method's global scope is module containing function definition
http://bugs.python.org/issue11418  opened by naoki

#11419: Python-ast.[ch] out-of-date w.r.t. sources in fresh 2.5 checko
http://bugs.python.org/issue11419  opened by skip.montanaro

#11420: Make testsuite pass with -B/DONTWRITEBYTECODE set.
http://bugs.python.org/issue11420  opened by twouters

#11421: Subversion keywords missing on 2.5 checkout
http://bugs.python.org/issue11421  opened by skip.montanaro

#11426: CSV examples can't close their files
http://bugs.python.org/issue11426  opened by gwideman

#11427: ctypes from_buffer no longer accepts bytes
http://bugs.python.org/issue11427  opened by benrg

#11429: ctypes is highly eclectic in its raw-memory support
http://bugs.python.org/issue11429  opened by benrg

#11430: can't change the sizeof a Structure that doesn't own its buffe
http://bugs.python.org/issue11430  opened by benrg

#11432: webbrowser.open on unix fails.
http://bugs.python.org/issue11432  opened by ideasman42

#11436: Clarify struct doc for format 's'.
http://bugs.python.org/issue11436  opened by terry.reedy

#11437: IDLE crash on startup with typo in config-keys.cfg
http://bugs.python.org/issue11437  opened by George.Dhoore

#11439: subversion keyword breakage
http://bugs.python.org/issue11439  opened by skrah

#11440: fix_callable should be dropped from lib2to3 / changed
http://bugs.python.org/issue11440  opened by SilentGhost

#11441: compile() raises SystemError if called from except clause
http://bugs.python.org/issue11441  opened by july

#11442: list_directory() in SimpleHTTPServer.py should add charset=...
http://bugs.python.org/issue11442  opened by gvanrossum

#11445: python.exe on OS X shared-llbrary build erroneously linked to 
http://bugs.python.org/issue11445  opened by skip.montanaro

#11446: Incorrect PEP link in Language Reference section 7.7
http://bugs.python.org/issue11446  opened by terrence

#11447: test_pydoc refleak
http://bugs.python.org/issue11447  opened by pitrou

#11448: docs for HTTPConnection.set_tunnel are ambiguous
http://bugs.python.org/issue11448  opened by rfk

#11449: tarfile tries to file_.tell() even when creating a new archive
http://bugs.python.org/issue11449  opened by strombrg

#11453: asyncore.file_wrapper should implement __del__ and call close 
http://bugs.python.org/issue11453  opened by amajorek

#11454: email.message import time
http://bugs.python.org/issue11454  opened by rosslagerwall

#11455: issue a warning when populating a CPython type dict with non-s
http://bugs.python.org/issue11455  opened by michael.foord

#11456: Documentation csv RFC4180
http://bugs.python.org/issue11456  opened by tonywallace

#11457: Expose nanosecond precision from system calls
http://bugs.python.org/issue11457  opened by khenriksson

#11459: Python select.select does not correctly report read readyness
http://bugs.python.org/issue11459  opened by amicitas

#11461: Reading UTF-16 with codecs.readline() breaks on surrogate pair
http://bugs.python.org/issue11461  opened by ply

#11462: Peephole creates duplicate and unused constants
http://bugs.python.org/issue11462  opened by eltoder

#1

[Python-Dev] Callable, non-descriptor class attributes.

[Thomas Wouters]

One of the things brought up at the language summit (and I believe at the VM
summit, although I wasn't there) was the unpredictable behaviour of
callables turning into methods when they're class attributes. Specifically,
things that are CFunctions in CPython (builtin functions, which are not
descriptors and do not turn into methods) but could be plain Python
functions in other Python implementations. (Or even turn into a plain
function in CPython itself, if the other-implementation argument isn't
persuasive enough for you.) To see the kind of thing that would be
(potentially) problematic, see for example Lib/encodings/ascii.py:

import codecs

### Codec APIs

class Codec(codecs.Codec):

# Note: Binding these as C functions will result in the class not
# converting them to methods. This is intended.
encode = codecs.ascii_encode
decode = codecs.ascii_decode

(I hope the irony of a two-line comment explaining the situation, as opposed
to the self-documenting nature of staticmethod, isn't lost on anyone :)
Obviously, if the codecs.ascii_encode and codecs.ascii_decode functions are
replaced *for whatever reason* by Python functions, this will break. Because
these kinds of class attributes are often used for dependency injection, the
potentially problematic cases are even more common in tests -- the io
testsuite, for example, specialcases pyio.open (in two different ways, no
less) but omits the same for io.open for no apparent reason (other than that
right now, in CPython, it isn't needed.)

At the summit we discussed some potential improvements here:

 1. Make staticmethod a callable object directly (it isn't, currently) and
apply it to any Python function that replaces a (in CPython) CFunction. The
change to staticmethod may be a good idea regardless, but the policy of
making other implementations comply to this quirk in CPython seems (to me)
like unnecessary descrimination[*].
 2. Make CFunctions turn into methods in CPython (after a period of warning
about the impending change, obviously.) The actual *usecase* for this is
hard to envision (I think Guido thought someone might do 'class str(str):
len = len', which doesn't seem like that big an improvement :) so it would
be a feature for symmetry's sake only.
 3. Produce a CompatibilityWarning making sure that the programmer knows
that this particular thing might not do the same thing in other Python
implementations *or versions*.
 4. Make it an error to have a callable class attribute that isn't a
descriptor (although maybe we only discussed this one in my head.)

To gauge the size of the problem I've implemented solution #3: I've uploaded
a patch that adds the warning (currently as a DeprecationWarning) and fixes
a bunch of the issues (not yet all of them):
http://bugs.python.org/issue11470. The patch currently warns for callable,
non-descriptor, *non-class* class attributes. The class check is necessary
because injected classes and -- more popularly, in some ORMs -- nested
classes would otherwise produce the warning. (Considering replacing a class
with a function has lots of other effects, like isinstance checks failing, I
don't think warning for that case is necessary anyway.) An alternate
strategy would be to warn only about uses of CFunctions in these cases, but
this misses out on a whole slew of potential issues (e.g. a module global
that is currently a bound instance method.)

If you look at the patch you'll notice it also fixes a couple of "dubious"
cases -- usually in a module's own testsuite -- where one could certainly
argue that the test is allowed to make the assumptions it's making. I don't
think writing it explicitly future-proof detracts from the readability,
stability and maintainability in most of these cases, although I should
point out that multiprocessing's testsuite produces a bunch more warnings
and isn't as easy to fix because of its use of locals().update() in a class
statement. (Yes, really.)

With regards to changing staticmethod, in an earlier iteration of the patch
I modified staticmethod to be directly callable *and* to return self in its
tp_descr_get, and this had a side-effect I hadn't considered: code that used
a staticmethod-wrapped object in any way other than calling it, broke. Right
now staticmethod is implemented as a generic container to inhibit descriptor
behaviour, in spite of its name, so it can be used to protect any kind of
descriptor in a class attribute. I'm not sure how common that use is, and
whether we should try to provide a different wrapper for those cases.
Unfortunately, without changing staticmethod to return itself, rather than
the callable it's wrapping, in tp_descr_get, code like this:

  class C:
open = builtin.open

  class D:
open = C.open

... would still produce different behaviour when builtin.open changes from a
CPython CFunction to a Python function wrapped in a staticmethod: C.open
would produce the Python function, and D.open would then become a method
after all. (This i

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

[Antoine Pitrou]

On Wed, 09 Mar 2011 07:15:07 +0100
Stefan Behnel  wrote:
> 
> Actually, why not put up a web page of "upcoming changes" somewhere, that 
> lists major decisions with user impact that were taken on python-dev? 
> Including a link to the relevant discussion and decision. Often enough, 
> decisions are taken inside of huge mailing list threads that get off-topic 
> before someone has "the right idea" and everyone who's still there to 
> listen agrees. Even for people lurking around on python-dev, it's easy 
> enough to miss these moments.

A better alternative IMHO would be to revive the community buildbots
(pybots.org), under the same form or another.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 03/11/2011 03:24 PM, Antoine Pitrou wrote:
> On Wed, 09 Mar 2011 07:15:07 +0100
> Stefan Behnel  wrote:
>>
>> Actually, why not put up a web page of "upcoming changes" somewhere, that 
>> lists major decisions with user impact that were taken on python-dev? 
>> Including a link to the relevant discussion and decision. Often enough, 
>> decisions are taken inside of huge mailing list threads that get off-topic 
>> before someone has "the right idea" and everyone who's still there to 
>> listen agrees. Even for people lurking around on python-dev, it's easy 
>> enough to miss these moments.
> 
> A better alternative IMHO would be to revive the community buildbots
> (pybots.org), under the same form or another.

How would more buildbots help the visibility of changes being landed to
the language or standard library?


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk16i68ACgkQ+gerLs4ltQ6RaACeMOXvAqUYByjx5lFUjMcVH0Z9
ZtIAn0MtoBD3LWc6GZsUCrQmL3DJqHVC
=rZ+o
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

[Guido van Rossum]

On Wed, Mar 9, 2011 at 3:40 PM, Doug Hellmann  wrote:
>
> On Mar 9, 2011, at 9:50 AM, Tim Lesher wrote:
>
>> On Wed, Mar 9, 2011 at 01:15, Stefan Behnel  wrote:
>>> Actually, why not put up a web page of "upcoming changes" somewhere, that
>>> lists major decisions with user impact that were taken on python-dev?
>>> Including a link to the relevant discussion and decision. Often enough,
>>> decisions are taken inside of huge mailing list threads that get off-topic
>>> before someone has "the right idea" and everyone who's still there to listen
>>> agrees. Even for people lurking around on python-dev, it's easy enough to
>>> miss these moments.
>>
>> We used to do biweekly-ish Python-Dev summaries for this reason.
>>
>> The original links at python.org appear to be down, but I found an
>> example mirrored at
>> ftp://ftp.ntua.gr/mirror/python/dev/summary/2005-02-01_2005-02-14.html
>>
>> Would resuming these and putting them back on python.org address the issue?
>>
>> It's been on my back burner for about two years now, but I want to
>> make sure I can keep up before diving in again.
>
> As Jesse mentioned, this topic came up on the board mailing list recently for 
> a reason completely unrelated to this thread. As a result of that discussion, 
> the board has asked me in my capacity as PSF Communications Director to help 
> the python-dev crew set up a blog (or other forum) through which you can 
> communicate news about major projects undertaken during development. This 
> would be in addition to, rather than a replacement for, individual developer 
> blogs, and would provide an official channel for the team to talk about 
> projects publicly after they are complete.
>
> Topics proposed as part of the discussion on the board list included the hg 
> migration, the new developer's guide, changes to the Mac installer, and the 
> updated release process for 3.2. Those are just examples, though. This 
> deprecation would make another good topic, and I'm sure everyone can think of 
> others. Consider the blog as an analog to the PEP process. Where PEPs come at 
> the beginning of a project, a blog post would come at a major milestones or 
> the completion of a project.
>
> The original request from the board was for the communications team to write 
> the messages, but I think it is more appropriate for the people doing the 
> work to talk about it. I will provide editorial guidance to anyone that wants 
> me to read their posts before they are published, and I will administer the 
> tool if needed (granting access and moderating comments that look like spam).
>
> I asked Michael to add this topic to the agenda for the language summit 
> tomorrow to get early feedback about whether this group thinks it is a good 
> idea. I was going to hold discussion for the mailing list until after that 
> meeting, but since the topic came up on its own please go ahead and respond 
> here with questions or comments, especially if you won't be in Atlanta 
> tomorrow. Let's table discussion of tools for now, though, because I want to 
> make sure there is enough support for the project before we spend too much 
> energy on implementation details.

I propose we try to find an "embedded blogger" who participates in
python-dev but is focused on making regular blog posts about the
interesting tidbits. There's no requirement to be complete (which I
think always weighed the python-dev-summaries task down).

Ditto for python-ideas -- it would be nice if it was the same person
but doesn't have to be.

This might be an opportunity for increasing diversity.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

[Antoine Pitrou]

On Fri, 11 Mar 2011 15:53:03 -0500
Tres Seaver  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> On 03/11/2011 03:24 PM, Antoine Pitrou wrote:
> > On Wed, 09 Mar 2011 07:15:07 +0100
> > Stefan Behnel  wrote:
> >>
> >> Actually, why not put up a web page of "upcoming changes" somewhere, that 
> >> lists major decisions with user impact that were taken on python-dev? 
> >> Including a link to the relevant discussion and decision. Often enough, 
> >> decisions are taken inside of huge mailing list threads that get off-topic 
> >> before someone has "the right idea" and everyone who's still there to 
> >> listen agrees. Even for people lurking around on python-dev, it's easy 
> >> enough to miss these moments.
> > 
> > A better alternative IMHO would be to revive the community buildbots
> > (pybots.org), under the same form or another.
> 
> How would more buildbots help the visibility of changes being landed to
> the language or standard library?

They would help check that changes don't cause regressions in major
third-party packages such as CPython.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

[Antoine Pitrou]

On Wed, 9 Mar 2011 15:40:56 -0500
Doug Hellmann  wrote:
> 
> The original request from the board was for the communications team to write 
> the messages, but I think it is more appropriate for the people doing the 
> work to talk about it. [...]
> 
> I asked Michael to add this topic to the agenda for the language summit 
> tomorrow to get early feedback about whether this group thinks it is a good 
> idea.

If you find someone who likes to do it then it's fine.
Otherwise, I'm not sure it's a good use of our (the developers') time.
We already explain changes in the documentation, in Misc/NEWS and in the
"what's new" documents...

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

[Brian Curtin]

On Mar 11, 2011 4:52 PM, "Guido van Rossum"  wrote:
>
> On Wed, Mar 9, 2011 at 3:40 PM, Doug Hellmann 
wrote:
> >
> > On Mar 9, 2011, at 9:50 AM, Tim Lesher wrote:
> >
> >> On Wed, Mar 9, 2011 at 01:15, Stefan Behnel 
wrote:
> >>> Actually, why not put up a web page of "upcoming changes" somewhere,
that
> >>> lists major decisions with user impact that were taken on python-dev?
> >>> Including a link to the relevant discussion and decision. Often
enough,
> >>> decisions are taken inside of huge mailing list threads that get
off-topic
> >>> before someone has "the right idea" and everyone who's still there to
listen
> >>> agrees. Even for people lurking around on python-dev, it's easy enough
to
> >>> miss these moments.
> >>
> >> We used to do biweekly-ish Python-Dev summaries for this reason.
> >>
> >> The original links at python.org appear to be down, but I found an
> >> example mirrored at
> >> ftp://ftp.ntua.gr/mirror/python/dev/summary/2005-02-01_2005-02-14.html
> >>
> >> Would resuming these and putting them back on python.org address the
issue?
> >>
> >> It's been on my back burner for about two years now, but I want to
> >> make sure I can keep up before diving in again.
> >
> > As Jesse mentioned, this topic came up on the board mailing list
recently for a reason completely unrelated to this thread. As a result of
that discussion, the board has asked me in my capacity as PSF Communications
Director to help the python-dev crew set up a blog (or other forum) through
which you can communicate news about major projects undertaken during
development. This would be in addition to, rather than a replacement for,
individual developer blogs, and would provide an official channel for the
team to talk about projects publicly after they are complete.
> >
> > Topics proposed as part of the discussion on the board list included the
hg migration, the new developer's guide, changes to the Mac installer, and
the updated release process for 3.2. Those are just examples, though. This
deprecation would make another good topic, and I'm sure everyone can think
of others. Consider the blog as an analog to the PEP process. Where PEPs
come at the beginning of a project, a blog post would come at a major
milestones or the completion of a project.
> >
> > The original request from the board was for the communications team to
write the messages, but I think it is more appropriate for the people doing
the work to talk about it. I will provide editorial guidance to anyone that
wants me to read their posts before they are published, and I will
administer the tool if needed (granting access and moderating comments that
look like spam).
> >
> > I asked Michael to add this topic to the agenda for the language summit
tomorrow to get early feedback about whether this group thinks it is a good
idea. I was going to hold discussion for the mailing list until after that
meeting, but since the topic came up on its own please go ahead and respond
here with questions or comments, especially if you won't be in Atlanta
tomorrow. Let's table discussion of tools for now, though, because I want to
make sure there is enough support for the project before we spend too much
energy on implementation details.
>
> I propose we try to find an "embedded blogger" who participates in
> python-dev but is focused on making regular blog posts about the
> interesting tidbits. There's no requirement to be complete (which I
> think always weighed the python-dev-summaries task down).
>
> Ditto for python-ideas -- it would be nice if it was the same person
> but doesn't have to be.
>
> This might be an opportunity for increasing diversity.
>
> --
> --Guido van Rossum (python.org/~guido)

I'm interested in the task and I guess I'll follow-up with Doug Hellman. I
don't follow -ideas close enough to summarize it, but I'd contribute to a
-dev blog.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

[Antoine Pitrou]

On Fri, 11 Mar 2011 22:56:49 +0100
Antoine Pitrou  wrote:

> On Fri, 11 Mar 2011 15:53:03 -0500
> Tres Seaver  wrote:
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA1
> > 
> > On 03/11/2011 03:24 PM, Antoine Pitrou wrote:
> > > On Wed, 09 Mar 2011 07:15:07 +0100
> > > Stefan Behnel  wrote:
> > >>
> > >> Actually, why not put up a web page of "upcoming changes" somewhere, 
> > >> that 
> > >> lists major decisions with user impact that were taken on python-dev? 
> > >> Including a link to the relevant discussion and decision. Often enough, 
> > >> decisions are taken inside of huge mailing list threads that get 
> > >> off-topic 
> > >> before someone has "the right idea" and everyone who's still there to 
> > >> listen agrees. Even for people lurking around on python-dev, it's easy 
> > >> enough to miss these moments.
> > > 
> > > A better alternative IMHO would be to revive the community buildbots
> > > (pybots.org), under the same form or another.
> > 
> > How would more buildbots help the visibility of changes being landed to
> > the language or standard library?
> 
> They would help check that changes don't cause regressions in major
> third-party packages such as CPython.

Sorry, I meant Cython, not CPython...

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] devguide: Suggest using mpatch in the FAQ

[Eric Smith]

On 03/11/2011 06:03 PM, antoine.pitrou wrote:

  If you want to try out or review a patch generated using Mercurial, do::

-   hg import --no-commit somework.patch
+   patch -p1<  somework.patch

  This will apply the changes in your working copy without committing them.
  If the patch was not created by Mercurial (for example, a patch created by
  Subversion and thus lacking any ``a``/``b`` directory prefixes in the patch),
-add ``-p0`` to the above command.
+replace ``-p0`` with ``-p1``.


Do you have this backwards? There is no -p0, it's already -p1.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

[Guido van Rossum]

On Fri, Mar 11, 2011 at 5:05 PM, Brian Curtin  wrote:
>
> On Mar 11, 2011 4:52 PM, "Guido van Rossum"  wrote:
>>
>> On Wed, Mar 9, 2011 at 3:40 PM, Doug Hellmann 
>> wrote:
>> >
>> > On Mar 9, 2011, at 9:50 AM, Tim Lesher wrote:
>> >
>> >> On Wed, Mar 9, 2011 at 01:15, Stefan Behnel 
>> >> wrote:
>> >>> Actually, why not put up a web page of "upcoming changes" somewhere,
>> >>> that
>> >>> lists major decisions with user impact that were taken on python-dev?
>> >>> Including a link to the relevant discussion and decision. Often
>> >>> enough,
>> >>> decisions are taken inside of huge mailing list threads that get
>> >>> off-topic
>> >>> before someone has "the right idea" and everyone who's still there to
>> >>> listen
>> >>> agrees. Even for people lurking around on python-dev, it's easy enough
>> >>> to
>> >>> miss these moments.
>> >>
>> >> We used to do biweekly-ish Python-Dev summaries for this reason.
>> >>
>> >> The original links at python.org appear to be down, but I found an
>> >> example mirrored at
>> >> ftp://ftp.ntua.gr/mirror/python/dev/summary/2005-02-01_2005-02-14.html
>> >>
>> >> Would resuming these and putting them back on python.org address the
>> >> issue?
>> >>
>> >> It's been on my back burner for about two years now, but I want to
>> >> make sure I can keep up before diving in again.
>> >
>> > As Jesse mentioned, this topic came up on the board mailing list
>> > recently for a reason completely unrelated to this thread. As a result of
>> > that discussion, the board has asked me in my capacity as PSF 
>> > Communications
>> > Director to help the python-dev crew set up a blog (or other forum) through
>> > which you can communicate news about major projects undertaken during
>> > development. This would be in addition to, rather than a replacement for,
>> > individual developer blogs, and would provide an official channel for the
>> > team to talk about projects publicly after they are complete.
>> >
>> > Topics proposed as part of the discussion on the board list included the
>> > hg migration, the new developer's guide, changes to the Mac installer, and
>> > the updated release process for 3.2. Those are just examples, though. This
>> > deprecation would make another good topic, and I'm sure everyone can think
>> > of others. Consider the blog as an analog to the PEP process. Where PEPs
>> > come at the beginning of a project, a blog post would come at a major
>> > milestones or the completion of a project.
>> >
>> > The original request from the board was for the communications team to
>> > write the messages, but I think it is more appropriate for the people doing
>> > the work to talk about it. I will provide editorial guidance to anyone that
>> > wants me to read their posts before they are published, and I will
>> > administer the tool if needed (granting access and moderating comments that
>> > look like spam).
>> >
>> > I asked Michael to add this topic to the agenda for the language summit
>> > tomorrow to get early feedback about whether this group thinks it is a good
>> > idea. I was going to hold discussion for the mailing list until after that
>> > meeting, but since the topic came up on its own please go ahead and respond
>> > here with questions or comments, especially if you won't be in Atlanta
>> > tomorrow. Let's table discussion of tools for now, though, because I want 
>> > to
>> > make sure there is enough support for the project before we spend too much
>> > energy on implementation details.
>>
>> I propose we try to find an "embedded blogger" who participates in
>> python-dev but is focused on making regular blog posts about the
>> interesting tidbits. There's no requirement to be complete (which I
>> think always weighed the python-dev-summaries task down).
>>
>> Ditto for python-ideas -- it would be nice if it was the same person
>> but doesn't have to be.
>>
>> This might be an opportunity for increasing diversity.
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>
> I'm interested in the task and I guess I'll follow-up with Doug Hellman. I
> don't follow -ideas close enough to summarize it, but I'd contribute to a
> -dev blog.

Awesome! (And we don't need to stop at one blogger. Many hands make light work.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Callable, non-descriptor class attributes.

[Greg Ewing]

Thomas Wouters wrote:

 2. Make CFunctions turn into methods in CPython (after a period of 
warning about the impending change, obviously.) The actual *usecase* for 
this is hard to envision


While not necessary for the case being discussed here, this would
be a big help for Pyrex and Cython, where currently some awkward
things have to be done to get a Python class with methods implemented
in C.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Callable, non-descriptor class attributes.

[Guido van Rossum]

On Fri, Mar 11, 2011 at 6:49 PM, Greg Ewing  wrote:
> Thomas Wouters wrote:
>
>>  2. Make CFunctions turn into methods in CPython (after a period of
>> warning about the impending change, obviously.) The actual *usecase* for
>> this is hard to envision
>
> While not necessary for the case being discussed here, this would
> be a big help for Pyrex and Cython, where currently some awkward
> things have to be done to get a Python class with methods implemented
> in C.

I appreciate the use case, but I believe the right solution there is
to create a custom decorator implementing the desired descriptor
behavior. There simply are too many places where existing code depends
on a built-in function placed in a class *not* turning into a method.

Also, there is a slippery-slope argument here: if we were to add such
a descriptor to CFunctions, why not to other callables? I think we
need to stick to the rule where only Python functions have a
method-making descriptor.

Off-line, I discussed the patch with Thomas and we ended up agreeing
that the warning should only be issued for CFunctions used as class
variables.

In summary, I don't think the problem is with the difference between
CFunctions and pure-Python functions. That difference is well-defined
and shouldn't go away. The problem is that it is fairly common that
something that is a CFunction in CPython is replaced by a pure-Python
function, either in another Python VM, or in a different environment
(think heapq.py vs. _heapq.c), or when some unittest uses dependency
injection (i.e. temporarily replaces a built-in with a wrapper written
in Python). Such substitutions are generally transparent, but not when
the function is used as a class attribute. We should do all these
things to address this:

- on the "consuming" side, wrap such class variables in staticmethod() calls

- on the "producing" side (e.g. in PyPy) wrap pure-Python functions
that are substitutes for CPython builtins in a custom descriptor that
prevents method-making

- in CPython, warn whenever a built-in function is used as a class variable

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

[Paul Moore]

On 11 March 2011 23:24, Guido van Rossum  wrote:
>> I'm interested in the task and I guess I'll follow-up with Doug Hellman. I
>> don't follow -ideas close enough to summarize it, but I'd contribute to a
>> -dev blog.
>
> Awesome! (And we don't need to stop at one blogger. Many hands make light 
> work.)

I'm interested but probably haven't got time to take something like
this on "properly". Helping out occasionally would probably be
something I could do, though. (Was that tentative enough? :-)) I'm on
-ideas, too, so I could do something there as well.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] The purpose of SETUP_LOOP, BREAK_LOOP, CONTINUE_LOOP

[Eugene Toder]

Hello,

What is the purpose of SETUP_LOOP instruction? From a quick look it
seems like it just pushes the size of the loop into blocks stack; that
size is only used by BREAK_LOOP instruction.
BREAK_LOOP could just contain the target address directly, like
CONTINUE_LOOP does. This would avoid SETUP_LOOP/POP_BLOCK overhead for
all loops.
Am I missing something? Does SETUP_LOOP serve any other purpose?

Similarly, it looks like BREAK_LOOP and CONTINUE_LOOP are just jumps
that respect try/finally blocks (i.e. jumping out of try executes
finally). Is there more semantics to them than this? If not, this can
be simplified to:
1) If not in try/finally, simply generate a direct jump outside of the
loop (break) or to the start of the loop (continue).
2) If in try/finally, generate a new instruction JUMP_FROM_TRY which
replaces both BREAK_LOOP and CONTINUE_LOOP. It behaves the same way as
CONTINUE_LOOP but without restriction to only jump backwards (could
reuse CONTINUE_LOOP, but the name would be misleading).
continue statement is already handled this way, but break always uses
BREAK_LOOP.

Any comments are appreciated.

Regards,
Eugene
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Raymond Hettinger]

Today, there was a significant check-in to the peephole optimizer that I think 
should be reverted:
   http://hg.python.org/cpython/rev/14205d0fee45/

The peephole optimizer pre-dated the introduction of the abstract syntax tree.  
Now that we have an AST, the preferred way to implement additional 
optimizations is with AST manipulation, upstream from code generation.  This 
approach is faster and much more reliable than the more brittle approach of 
disassembling, analyzing, and rewriting the bytecode created by the compiler.

There should no longer be any significant changes to the existing optimizer.  
It took a good deal of care to get it right in the first place.  The code is 
stable and has been proven successful by many years of deployment.  The nature 
of the peephole optimizer  is that changes to it are high risk, that the 
procedure is brittle, and that it is easily mucked-up in ways that are hard to 
detect. Accordingly, we've kept to simple conservative transformations and have 
assiduously avoided more complex (and interesting) optimizations.

FWIW, I've been the maintainer (as well as the original designer and 
implementer) of the peephole optimizer from the beginning; however, today's 
committer did not accept my advice to be very conservative with changes to it 
and to strongly prefer AST transformations instead.  Please consider reverting 
this change.


Raymond___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Alexander Belopolsky]

On Fri, Mar 11, 2011 at 9:28 PM, Raymond Hettinger
 wrote:
> Today, there was a significant check-in to the peephole optimizer that I
> think should be reverted:
>                http://hg.python.org/cpython/rev/14205d0fee45/

+1

I was going to comment on the corresponding issue #11244 more or less
supporting Raymond's arguments.  There is no end of optimization ideas
that can be implemented in peephole optimizer.  I know this first hand
having implemented several of those that have been ultimately
rejected.  At the end of the day, peephole optimizer is a hack that
predates proper AST design.  There is no need to continue squeezing
out last drops of juice from 10-year old technology when much better
approach is available with the modern design.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Benjamin Peterson]

2011/3/11 Raymond Hettinger :
> Today, there was a significant check-in to the peephole optimizer that I
> think should be reverted:
>                http://hg.python.org/cpython/rev/14205d0fee45/
> The peephole optimizer pre-dated the introduction of the abstract syntax
> tree.  Now that we have an AST, the preferred way to implement additional
> optimizations is with AST manipulation, upstream from code generation.  This
> approach is faster and much more reliable than the more brittle approach
> of disassembling, analyzing, and rewriting the bytecode created by the
> compiler.

The problem is no such AST optimizer exists. It's one thing avoid
changing old code because an improved version is in the works or
available (say argparse in lieu of getopt) and quite another when no
replacement code exists. At the moment, there is little reason not to
accept progressive improvements (with sights on overall design as
usual) to the code.

IMO, Ast or not ast statically optimizing python in any meaningful way
is a impossible task anyway. So, a better solution would be to just
rip the thing out.

> There should no longer be any significant changes to the existing
> optimizer.  It took a good deal of care to get it right in the first place.
>  The code is stable and has been proven successful by many years of
> deployment.  The nature of the peephole optimizer  is that changes to it are
> high risk, that the procedure is brittle, and that it is easily mucked-up in
> ways that are hard to detect. Accordingly, we've kept to simple conservative
> transformations and have assiduously avoided more complex (and interesting)
> optimizations.

Can you point to examples where changes introduced long-term
instability? ISTM, that the peepholer is one of the better tested
parts of the interpreter if not directly from its own tests, from
running the entire stdlib tests.

> FWIW, I've been the maintainer (as well as the original designer and
> implementer) of the peephole optimizer from the beginning; however, today's
> committer did not accept my advice to be very conservative with changes to
> it and to strongly prefer AST transformations instead.  Please consider
> reverting this change.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Callable, non-descriptor class attributes.

[Steven D'Aprano]

Thomas Wouters wrote:

One of the things brought up at the language summit (and I believe at the VM
summit, although I wasn't there) was the unpredictable behaviour of
callables turning into methods when they're class attributes.

[...]

 1. Make staticmethod a callable object directly (it isn't, currently) and
apply it to any Python function that replaces a (in CPython) CFunction. The
change to staticmethod may be a good idea regardless, but the policy of
making other implementations comply to this quirk in CPython seems (to me)
like unnecessary descrimination[*].


I've been bitten by the (to me) surprising fact that staticmethod 
objects are not directly callable. It's hard to write a method/function 
inside a class which is callable both when the class is being created, 
and afterwards, even though naively staticmethod seems like it should do 
the job. A toy example:



class Toy(object):
@staticmethod
def spam(n):
return ' '.join(['spam']*n)
lunch = spam(5) + "with a fried egg on top"


+1 on making staticmethods callable. I would have found that useful in 
the past.




 4. Make it an error to have a callable class attribute that isn't a
descriptor (although maybe we only discussed this one in my head.)


Do you mean to make one, or both, of these an error?

class C:
spam = len

C.__dict__['ham'] = lambda: None




--
Steven

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Guido van Rossum]

I recall several occasions where the peephole optimizer was subtly
buggy -- on one occasion the bug remained undetected for at least a
whole release cycle. (Sorry, I can't recall the details.) In fact, the
bug reported in http://bugs.python.org/issue11244 is another example
of how subtle the peephole optimizer is. Adding more complexity to it
just increases the chances that it will be broken by seemingly
unrelated changes.

As far as Antoine's commit goes, it looks like it went in hastily,
without review, and before any discussion. Clearly it *is*
controversial, and controversial fixes always need to be held back
until at least rough consensus is reached or until a BDFL decision.
Also it is not a fix to the bug reported in the issue, so the
Misc/NEWS text is misleading. A much simpler fix was proposed in the
bug and even approved by Antoine. (One note on the patch: it allocates
an extra stack which is dynamically grown; but there is no unittest to
exercise the stack-growing code. This note does constitute a full
review.)

I have always felt uncomfortable with *any* kind of optimization --
whether AST-based or bytecode-based. I feel the cost in code
complexity is pretty high and in most cases the optimization is not
worth the effort. Also I don't see the point in optimizing expressions
like "3 * 4 * 5" in Python. In C, this type of thing occurs frequently
due to macro expansion and the like, but in Python your code usually
looks pretty silly if you write that. So this is basically a solution
to a non-problem. The only two places where I personally see a need
for compile-time optimization are unary minus (since there is no way
to write negative constants otherwise) and string literal
concatenation (which is a common way to break long string literals).

Now, should Antoine's checkin be rolled back? I think the case is in
Antoine's court to convince us that his patch is correct *and* to help
at least one or two others feel comfortable they understand how it
works -- if only Antoine understands it, it is too complex. If Antoine
does not succeed with this task he should roll it back. Or he can
choose to roll it back immediately, and then we can continue the
discussion under a lot less pressure, and come to a conclusion
amiably.

--Guido

On Fri, Mar 11, 2011 at 10:01 PM, Benjamin Peterson  wrote:
> 2011/3/11 Raymond Hettinger :
>> Today, there was a significant check-in to the peephole optimizer that I
>> think should be reverted:
>>                http://hg.python.org/cpython/rev/14205d0fee45/
>> The peephole optimizer pre-dated the introduction of the abstract syntax
>> tree.  Now that we have an AST, the preferred way to implement additional
>> optimizations is with AST manipulation, upstream from code generation.  This
>> approach is faster and much more reliable than the more brittle approach
>> of disassembling, analyzing, and rewriting the bytecode created by the
>> compiler.
>
> The problem is no such AST optimizer exists. It's one thing avoid
> changing old code because an improved version is in the works or
> available (say argparse in lieu of getopt) and quite another when no
> replacement code exists. At the moment, there is little reason not to
> accept progressive improvements (with sights on overall design as
> usual) to the code.
>
> IMO, Ast or not ast statically optimizing python in any meaningful way
> is a impossible task anyway. So, a better solution would be to just
> rip the thing out.
>
>> There should no longer be any significant changes to the existing
>> optimizer.  It took a good deal of care to get it right in the first place.
>>  The code is stable and has been proven successful by many years of
>> deployment.  The nature of the peephole optimizer  is that changes to it are
>> high risk, that the procedure is brittle, and that it is easily mucked-up in
>> ways that are hard to detect. Accordingly, we've kept to simple conservative
>> transformations and have assiduously avoided more complex (and interesting)
>> optimizations.
>
> Can you point to examples where changes introduced long-term
> instability? ISTM, that the peepholer is one of the better tested
> parts of the interpreter if not directly from its own tests, from
> running the entire stdlib tests.
>
>> FWIW, I've been the maintainer (as well as the original designer and
>> implementer) of the peephole optimizer from the beginning; however, today's
>> committer did not accept my advice to be very conservative with changes to
>> it and to strongly prefer AST transformations instead.  Please consider
>> reverting this change.
>
>
>
> --
> Regards,
> Benjamin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@pyt

Re: [Python-Dev] Callable, non-descriptor class attributes.

[Guido van Rossum]

On Fri, Mar 11, 2011 at 10:09 PM, Steven D'Aprano  wrote:
> Thomas Wouters wrote:
>>
>> One of the things brought up at the language summit (and I believe at the
>> VM
>> summit, although I wasn't there) was the unpredictable behaviour of
>> callables turning into methods when they're class attributes.
>
> [...]
>>
>>  1. Make staticmethod a callable object directly (it isn't, currently) and
>> apply it to any Python function that replaces a (in CPython) CFunction.
>> The
>> change to staticmethod may be a good idea regardless, but the policy of
>> making other implementations comply to this quirk in CPython seems (to me)
>> like unnecessary descrimination[*].
>
> I've been bitten by the (to me) surprising fact that staticmethod objects
> are not directly callable. It's hard to write a method/function inside a
> class which is callable both when the class is being created, and
> afterwards, even though naively staticmethod seems like it should do the
> job. A toy example:
>
>
> class Toy(object):
>    @staticmethod
>    def spam(n):
>        return ' '.join(['spam']*n)
>    lunch = spam(5) + "with a fried egg on top"
>
>
> +1 on making staticmethods callable. I would have found that useful in the
> past.

IIUC Thomas found that this breaks some current use of staticmethod.

>>  4. Make it an error to have a callable class attribute that isn't a
>> descriptor (although maybe we only discussed this one in my head.)
>
> Do you mean to make one, or both, of these an error?
>
> class C:
>    spam = len

Making this illegal is what #4 means, yes. But I find it unacceptable.

> C.__dict__['ham'] = lambda: None

This is already illegal (except for classic classes in Python 2). Also
lambda creates a standard function object so it *is* (or *has*?) a
descriptor.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Eugene Toder]

Experience shows that optimizations are always error prone, no matter
what framework or internal representation you use. I don't think we
should assume that simply rewriting all optimizations to work on AST
will make them bug free once and for all. On the contrary, I think
such a rewrite will introduce some bugs of it's own.

The remedy against this is well known. Instead of being afraid to
touch the code we should add more tests and verifiers. If had written
tests are not enough, test generator that produces python programs
with different structures can be written. Such generators are used by
many compiler writers. For verifiers, a function that checks that
bytecode is sane (doesn't reference invalid names or consts, doesn't
jump between instructions, all joins have same stack depth? etc) that
runs after optimizer in debug builds can save a lot of time.

Eugene
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Raymond Hettinger]

On Mar 11, 2011, at 11:11 PM, Eugene Toder wrote:

> Experience shows that optimizations are always error prone, no matter
> what framework or internal representation you use.

On that basis, I believe that we ought to declare peephole.c as being
somewhat off-limits for further development (except for small
adaptations if the underlying opcodes change meaning).

The code is stable and most of the low-hanging fruit has already been taken.

The bright ideas for other enhancements aren't actually new ideas.
They were considered at the outset and not included for a reason.
We were intentionally very conservative with the optimizer.


> I don't think we
> should assume that simply rewriting all optimizations to work on AST
> will make them bug free once and for all. On the contrary, I think
> such a rewrite will introduce some bugs of it's own.

You make an automatic truth when say, "I don't think we should assume X".

It would be more correct to say that peepholing is the wrong technique
for constant folding because:

1) much of the semantic information available the AST has been lost
by the time the bytecode is generated

2) peephole.c is having to disassemble the bytecode, reconstruct
semantic relationships as well as it can, recognize transformation
patterns and then regenerate bytecode.
Moving constant folding to the AST means that the disassembly and reassembly
don't have to be done and that more semantic information is available.

In other words, the two techniques are equal just because it is always
possible to make a mistake using any technique.


> The remedy against this is well known.
> Instead of being afraid to
> touch the code we should add more tests and verifiers.

Not really.  It is darned difficult to design tests to catch all the corner 
cases.
You've got wishful thinking if you think a handful of tests can catch
errors in code that sophisticated.  Look at GCC's bug history to get
a sense for what can go wrong with optimization.

I admire your and Antoine's enthusiasm for wanting to optimize,
but think your efforts are misdirected and that you should be
taking some advice from the guy who wrote thing in the first place.


Raymond

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Eugene Toder]

>> Experience shows that optimizations are always error prone, no matter
>> what framework or internal representation you use.
>
> On that basis, I believe that we ought to declare peephole.c as being
> somewhat off-limits for further development (except for small
> adaptations if the underlying opcodes change meaning).

With this attitude, it would be more logical to remove the whole thing
completely. Even if you don't touch the optimizer, any changes in the
compiler can start producing code that exposes a previously hidden
bug.

> The bright ideas for other enhancements aren't actually new ideas.
> They were considered at the outset and not included for a reason.

It would help if these were documented.

> It would be more correct

How can you be more correct than tautological truth? :)

> 1) much of the semantic information available the AST has been lost
> by the time the bytecode is generated

Not really. Many systems use stack based bytecode as an intermediate
form and reconstruct different IRs from it. Examples are JVM, CLR and
PyPy.
The real problem with bytecode is that it's a representation optimized
for storage and interpretation. Transforming it is harder, less
efficient and more error-prone than transforming a tree structure. But
transformations of trees are in no way automatically bug free.

> 2) peephole.c is having to disassemble the bytecode, reconstruct
> semantic relationships as well as it can, recognize transformation
> patterns and then regenerate bytecode.

v8 does peephole directly on x86 machine code. No kittens were harmed.

Now, I do agree with you that moving optimizations to AST is a step
forward. Moving them to a specialized IR in SSA form would be even
better.
If there was a framework for AST-based optimizations in the current
code, I'd write my code to use it. However, I couldn't find it.

What I don't understand is why doing 90% of the work and refuse to do
the last 10%. Looking at the actual patches, I do not see significant
increase in complexity or code size -- all the complex cases are
already handled by existing code. In some cases the code became
cleaner and simpler.

> Not really.  It is darned difficult to design tests to catch all the corner 
> cases.

You either do that or do not write optimizations. No one said it should be easy.

> You've got wishful thinking if you think a handful of tests can catch
> errors in code that sophisticated.

Why limit yourself with a handful of tests? Python is widespread,
there's *a lot* of code in Python. Unlike with libraries, any code you
run tests the optimizer, so just run a lot of code. And, as I've said,
write a test generator.

Cheers,
Eugene
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Cesare Di Mauro]

2011/3/12 Benjamin Peterson 

> 2011/3/11 Raymond Hettinger :
> > Today, there was a significant check-in to the peephole optimizer that I
> > think should be reverted:
> >http://hg.python.org/cpython/rev/14205d0fee45/
> > The peephole optimizer pre-dated the introduction of the abstract syntax
> > tree.  Now that we have an AST, the preferred way to implement additional
> > optimizations is with AST manipulation, upstream from code generation.
>  This
> > approach is faster and much more reliable than the more brittle approach
> > of disassembling, analyzing, and rewriting the bytecode created by the
> > compiler.
>
> The problem is no such AST optimizer exists. It's one thing avoid
> changing old code because an improved version is in the works or
> available (say argparse in lieu of getopt) and quite another when no
> replacement code exists. At the moment, there is little reason not to
> accept progressive improvements (with sights on overall design as
> usual) to the code.
>
> IMO, Ast or not ast statically optimizing python in any meaningful way
> is a impossible task anyway. So, a better solution would be to just
>  rip the thing out.
>
> --
> Regards,
> Benjamin


It's not true. I already moved almost all peephole optimizations
(introducing others, as well) from peephole.c to
ast.cand
compiler.cin
WPython 1.1.

Take a look at pages 21-23 of
this
.

Also, optimizations can be done not only for numbers, but even for tuples,
lists, dictionaries, and... slices (pag. 22). See pages 21-24 of
this
.

Cesare
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Callable, non-descriptor class attributes.

[Nick Coghlan]

On Fri, Mar 11, 2011 at 10:48 PM, Guido van Rossum  wrote:
>> +1 on making staticmethods callable. I would have found that useful in the
>> past.
>
> IIUC Thomas found that this breaks some current use of staticmethod.

>From his first post, I understood the compatibility issue to more be
the fact that even if we make it callable, it still needs to return
the underlying function from __get__ rather than itself. So making it
callable didn't break anything, but changing the behaviour of __get__
did.

So long as people follow the policy of applying staticmethod() when
stashing things they don't want to turn into methods in class
namespaces, that should be fine.

The difference between @staticmethod and a "@nondescriptor" decorator
that mimics CFunction behaviour would then largely be in whether or
not they implemented __get__ at all.

>>> hasattr(len, "__get__")
False
>>> hasattr(staticmethod(len), "__get__")
True
>>> staticmethod(len).__get__(type) is len
True
>>> set(dir(len)) - set(dir(staticmethod(len)))
{'__call__', '__name__', '__module__', '__self__'}
>>> set(dir(staticmethod(len))) - set(dir(len))
{'__func__', '__get__'}

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Callable, non-descriptor class attributes.

[Steven D'Aprano]

Guido van Rossum wrote:


+1 on making staticmethods callable. I would have found that useful in the
past.


IIUC Thomas found that this breaks some current use of staticmethod.



As I understand it, Thomas found that having staticmethod callable AND 
have staticmethod.__get__ return self breaks code. His exact words were:


"With regards to changing staticmethod, in an earlier iteration of the 
patch I modified staticmethod to be directly callable *and* to return 
self in its tp_descr_get, and this had a side-effect I hadn't 
considered: code that used a staticmethod-wrapped object in any way 
other than calling it, broke."


I am suggesting that staticmethod should leave __get__ as is, but add a 
__call__ method equivalent to this:


def __call__(self, *args, **kwargs):
func = self.__get__(self)
return func(*args, **kwargs)

This won't fix the problem with inconsistent behaviour between C and 
Python callables in classes, but it will reduce the surprising fact that 
staticmethods aren't callable. Sure, it's easy enough to subclass 
staticmethod, but the fact that you need to is a gotcha.





This is already illegal (except for classic classes in Python 2). Also
lambda creates a standard function object so it *is* (or *has*?) a
descriptor.


So it is. I learn something new.



--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Eugene Toder]

> One note on the patch: it allocates an extra stack which is dynamically grown;
> but there is no unittest to exercise the stack-growing code.

Isn't this doing it?

1.20 +# Long tuples should be folded too.
1.21 +asm = dis_single(repr(tuple(range(1
1.22 +# One LOAD_CONST for the tuple, one for the None return value
1.23 +self.assertEqual(asm.count('LOAD_CONST'), 2)
1.24 +self.assertNotIn('BUILD_TUPLE', asm)

Eugene
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Steven D'Aprano]

Cesare Di Mauro wrote:


Also, optimizations can be done not only for numbers, but even for tuples,
lists, dictionaries, and... slices (pag. 22). See pages 21-24 of
this


For the record, constant-folding, and the lack of it, does seem to come 
up in users' requests. Whether they *need* constant-folding or not, it 
seems that some people have come to expect it. E.g. see this recent 
thread on python-list:


http://code.activestate.com/lists/python-list/596506/



--
Steven

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] forward-porting from 3.1 to 3.2 to 3.3

[Eli Bendersky]

The devguide's recommendation is to "forward-port" changes withing a major
release line, i.e. if I need something in all 3.[123], then start with 3.1
and forward-port (by "hg merge ") to 3.2 and then 3.3

Just to clarify - does this mean that all changesets that are applied to 3.2
eventually get to 3.3 as well? Since we just do "hg merge 3.2" in the 3.3
clone, I guess this must be true. But then what happens if there's a change
in 3.2 that shouldn't get into 3.3? (say a bug fix for a feature that has
disappeared)?

Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com