Re: [Python-Dev] __getattr__ and new style classes

[Nick Coghlan]

Christian Heimes wrote:
> Steven D'Aprano wrote:
>> Not only don't I observe the same results as you, I'm afraid I can't
>> even get your code to run. I get a SyntaxError from the funny quotes
>> you're using: ´d.foo´ instead of 'd.foo' or "d.foo".
> 
> Kristján is using the old style and alternative syntax for repr().
> Somehow the `` got screwed up by either his mailer or the mailing list.
> Don't be ashamed that you aren't aware of the alternative syntax. We
> keep it locked up in the cellar and it has been removed from the new,
> shiny Python 3 world.

I think it's actually some single quotes that got mangled by the mailer.
Either way, something else is going on for Kristján to see such wildly
different results between old-style and new-style attribute access, when
the differences are in the noise for the other folks checking it.

I was able to get the old-style class to be consistently faster by going
back and trying the example on 2.4, but the change still wasn't even
close to the dramatic difference Kristján is seeing.

Kristján, is there a chance CCP optimised something in the old-style
class attribute lookup code and didn't get around to submitting the
patch back to python.org?

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] effect of "exec" on local scope

[Nick Coghlan]

Guido van Rossum wrote:
> Well, I don't recall what CLPython is, but I believe it is broken and
> that code should work -- there are (or used to be) examples of using
> exec to populate classes in the standard library so while it may look
> dodgy it really is exected to work...

I think this behaviour (modifying locals() at class scope) was actually
implicitly blessed in PEP 3115, even though that PEP doesn't explicitly
state locals() in a class body is required to grant access to the
namespace returned by __prepare__().

Perhaps the time is right for the locals() documentation to be more
explicit regarding when modifying its contents will affect the current
namespace?
- yes at module scope (effectively the same as globals())
- yes at class scope
- maybe (but typically not) at function scope

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Nick Coghlan]

Kristján Valur Jónsson wrote:
> Running regular python code through a profiler, and especially code that 
> relies much on the use of
> __getattr__() to emulate attribute access, will show hideous amounts of time 
> spent formatting
> attribute exceptions that get thrown away.
> 
> Any thoughts on how to do this better?

If the time is being spent in PyErr_Format, how far could you get adding
a dedicated function for creating AttributeErrors? Something along the
lines of:

PyErr_AttributeError(PyObject *object, PyObject *attr_name)

It would then be possible to modify AttributeError to have obj_type and
attr_name attributes (holding the type name and the name of the missing
attribute), so that any string formatting could be deferred until repr()
was called on the AttributeError instance.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subversion access down?

[Nick Coghlan]

Ulrich Eckhardt wrote:
> Hi!
> 
> Is it only me or does it fail for other people, too? I'm getting
> 
> | Server sent unexpected return value (503 Service
> | Unavailable) in response to OPTIONS request 
> | for 'http://svn.python.org/projects/python/trunk'

svn+ssh access is working for me - it looks like the web gateway may
just need a kick to get it going again.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subversion access down?

[A.M. Kuchling]

On Thu, Oct 09, 2008 at 10:50:24AM +0200, Ulrich Eckhardt wrote:
> Is it only me or does it fail for other people, too? I'm getting
> 
> | Server sent unexpected return value (503 Service
> | Unavailable) in response to OPTIONS request 
> | for 'http://svn.python.org/projects/python/trunk'  

svn.python.org is working again now.

--amk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Kristján Valur Jónsson]

Thanks for trying this out.
When I saw your comments, I realized that the difference is due to a speed 
patch that we have her at CCP.
The test was run using current 2.5 python, with a patch applied, as attached 
this message.
The patch was written by me a few years back when I noticed that python spends 
enormous time during regular
run creating exceptions that then get discarded internally.  Most of the cost 
comes from FormatException
when it is generating AttributeErrors.
So, when I thought I was complaining about slow new objects, I was really 
showing off my cool optimization.

The problem is, it is difficult to generalize this to python in general.  I 
spent some time last year to
try to improve new-style classes, by adding ts fields and telling the exception 
system to "expect a certain
type of exception, and if it is raised, use PyErr_SetNone() instead of the 
PyErr_Format because I will
clear it anyway".  I had macros such as
PyErr_Expect(AttributeError);
attr = PyObject_GetAttr(o, a);
PyErr_Expect(0);
if (!attr)
PyErr_Clear()

The problem was, I wasn't able to come up with a simple patch that showed 
consistent speed improvements
in the performance testsuite without showing some slowdowns in other places.

Running regular python code through a profiler, and especially code that relies 
much on the use of
__getattr__() to emulate attribute access, will show hideous amounts of time 
spent formatting
attribute exceptions that get thrown away.

Any thoughts on how to do this better?

And, for completeness, a new test run of this, using python25, first using the 
neutered patch version
(where I have set softfail=0 in the relevant places) and then with the patch 
active:

import timeit
s = """
class dude:
def bar(self):pass
def __getattr__(self, a): return a
class dude2(object):
def bar(self):pass
def __getattr__(self, a): return a
d = dude()
d2 = dude2()
d.a = d2.a = 1
"""
print timeit.Timer("d.foo", s).timeit()
print timeit.Timer("d.bar", s).timeit()
print timeit.Timer("d.a", s).timeit()

print timeit.Timer("d2.foo", s).timeit()
print timeit.Timer("d2.bar", s).timeit()
print timeit.Timer("d2.a", s).timeit()

patch neutered
1.35734336404
0.157773452422
0.0937950608722
1.48494915604
0.240154539405
0.186524222345

patch active:
0.352850669641
0.147599760073
0.0910020300097
1.4453737036
0.212842069748
0.203442097864


Cheers,

Kristján
> -Original Message-
> From: Nick Coghlan [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, October 08, 2008 21:37
> To: Kristján Valur Jónsson
> Cc: Python-Dev
> Subject: Re: [Python-Dev] __getattr__ and new style classes
>
> Kristján Valur Jónsson wrote:
> > Hello there.
> >
> > I‘ve just noticed what I consider a performance problem:
> >
> > Using new style classes to provide attribute-like access using
> > __getattr__ is considerably slower than old style classes:  Observe:
>
> I can't reproduce those relative numbers using SVN trunk. Using your
> setup code (copied and pasted directly from your message to my
> interpreter session) I got the following numbers:
>
>



speedpatch.rar
Description: speedpatch.rar
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Nick Coghlan]

Christian Heimes wrote:
> Nick Coghlan wrote:
>> I think it's actually some single quotes that got mangled by the mailer.
>> Either way, something else is going on for Kristján to see such wildly
>> different results between old-style and new-style attribute access, when
>> the differences are in the noise for the other folks checking it.
> 
> I still think they are back ticks. Maybe the repr() function of new
> style classes is slower than the old style repr(). It might explain the
> difference.

Nope - if they were backticks, you'd either get a NameError (if d and d2
aren't defined in the scope where the timeit calls are being made), or
you'd end up timing the evaluation of some string literals.

Mailer issues aside though, we still don't know where that initial set
of strange numbers is coming from.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Kristján Valur Jónsson]

It's an interesting idea...  But it seems hard to forge the magic that
does this (creating it in args[0] on demand) without some heavy work.

In my opinion, the problem is that we are raising all these errors
at all, with object creation and all that it entails.  We are using
exceptions very much as program logic to test for the existence of
attributes.

Python itself has the getattr(obj, attr, default) thing, encouraging
you to not use exceptions where the absence of an attribute is
fully excpected.  But the C layer doesn't have this.

In my opinion, the C api should have this capability, which is why
I bothered with implementing PyObject_GetAttrSoft() there.
Ideally, it should have a signature such as:
int PyObject_GetAttrSoft(PyObject **result, PyObject *obj, PyObject *attr, int 
raise);
where the return value indicates success (0) or an exception (-1) and the
"result" gets either NULL or something else in case of success.
"raise" would indicate whether we want the absence of an attibute to raise
an exception or not.

Now this is all well and good, but the big problem is that we invariably
end up calling the tp->tp_getattr slots that have no such functionality.

I'd actually be more inclined to try to go for a full solution using a
tp_gettattrc (c for conditional) with a default implementation
Maybe I'll play with that over the weekend, see where it takes us.

K

> -Original Message-
> From: Nick Coghlan [mailto:[EMAIL PROTECTED]
> Sent: Thursday, October 09, 2008 11:54
> To: Kristján Valur Jónsson
> Cc: Python-Dev
> Subject: Re: [Python-Dev] __getattr__ and new style classes
>
> Kristján Valur Jónsson wrote:
> > Running regular python code through a profiler, and especially code
> that relies much on the use of
> > __getattr__() to emulate attribute access, will show hideous amounts
> of time spent formatting
> > attribute exceptions that get thrown away.
> >
> > Any thoughts on how to do this better?
>
> If the time is being spent in PyErr_Format, how far could you get
> adding
> a dedicated function for creating AttributeErrors? Something along the
> lines of:
>
> PyErr_AttributeError(PyObject *object, PyObject *attr_name)
>
> It would then be possible to modify AttributeError to have obj_type and
> attr_name attributes (holding the type name and the name of the missing
> attribute), so that any string formatting could be deferred until
> repr()
> was called on the AttributeError instance.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
> ---
> http://www.boredomandlaziness.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] python dll no longer in system directory?

[Amaury Forgeot d'Arc]

2008/10/8 "Martin v. Löwis" <[EMAIL PROTECTED]>:
> Thomas Heller wrote:
>> Is it intended that python30.dll and python26.dll are not longer
>> installed in the \windows\system32 directory?
>
> No, it's not. Please create a bug report (or, better, study the
> msiexec logs, and msi.py, to find out why this happens).
>
> I might not have time to look into this before the next release
> candidate.

I confirm this. The following lines in msi.py seem to be the cause of
the change:

#dlldir = PyDirectory(db, cab, root, srcdir, "DLLDIR", ".")
#install python30.dll into root dir for now
dlldir = root

They were added by r61109: " Bundle msvcr90.dll as a "private assembly". "
but I don't know if simply restoring the previous value will work in every case:
If the C Run-Time is installed "privately", then python26.dll must
stay in c:\python26.

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Subversion access down?

[Ulrich Eckhardt]

Hi!

Is it only me or does it fail for other people, too? I'm getting

| Server sent unexpected return value (503 Service
| Unavailable) in response to OPTIONS request 
| for 'http://svn.python.org/projects/python/trunk'  


Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Christian Heimes]

Nick Coghlan wrote:

I think it's actually some single quotes that got mangled by the mailer.
Either way, something else is going on for Kristján to see such wildly
different results between old-style and new-style attribute access, when
the differences are in the noise for the other folks checking it.


I still think they are back ticks. Maybe the repr() function of new
style classes is slower than the old style repr(). It might explain the
difference.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Kristján Valur Jónsson]

No, it was really me being sloppy using outlook and fighting the editor trying 
to insert smart quotes :)
Sorry for the confusion.
K

> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf
> Of Christian Heimes
> Kristján is using the old style and alternative syntax for repr().
> Somehow the `` got screwed up by either his mailer or the mailing list.
> Don't be ashamed that you aren't aware of the alternative syntax. We
> keep it locked up in the cellar and it has been removed from the new,
> shiny Python 3 world.
>

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subversion access down?

[A.M. Kuchling]

On Thu, Oct 09, 2008 at 10:50:24AM +0200, Ulrich Eckhardt wrote:
> Is it only me or does it fail for other people, too? I'm getting
> 
> | Server sent unexpected return value (503 Service
> | Unavailable) in response to OPTIONS request 
> | for 'http://svn.python.org/projects/python/trunk'  

python.org seems to be having some network weirdness right now.
I'm looking into it.  (SVN access over SSH seems to be fine.)

--amk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Guido van Rossum wrote:
> On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen <[EMAIL PROTECTED]> wrote:
>> Josiah Carlson <[EMAIL PROTECTED]> wrote:
>>
>>> But yes, zope needs to be changed to reflect the updated
>>> asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
>>> easier to use now.
>> Just for completeness, I built a fresh 2.6, installed Medusa (from
>> http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
>> as well as it does on 2.5, anyway).  I think this is a Zope issue.
> 
> Way back in the day, Zope monkeypatched whole parts of asyncore,
> replacing them with some of its own code. If that's still the case,
> this breakage should be no surprise.

Zope has been hooking the 'asyncore.loop' function (wrapping it in order
to add a "clean shutdown' flog) since 2001 at least (the 2.5 branch is
the earliest checkout I have, and it was branched in early January 2002).

Zope also began patched asyncore's logging functions in 2.7, and later
updated that patch to make the logger use the stdlib 'logging' module.

I think the change we need to make (in ZServer/medusa/http_server.py) is
to emulate the new 'writeable' implementation in asynchat, at least when
running under 2.6.

Zope's 'ZServer.medusa.http_server.http_request.writable()'
implementation is::

def writable (self):
# this is just the normal async_chat 'writable',
# here for comparison
return self.ac_out_buffer or len(self.producer_fifo)


The Python 2.5 asynchat.asynchat.writable does::

def writable (self):
"predicate for inclusion in the writable for select()"
# return len(self.ac_out_buffer) or len(self.producer_fifo) or
# (not self.connected)
# this is about twice as fast, though not as clear.
return not (
(self.ac_out_buffer == '') and
self.producer_fifo.is_empty() and
self.connected
)

The Python 2.6 asynchat.asynchat.writable now does::

def writable (self):
"predicate for inclusion in the writable for select()"
return self.producer_fifo or (not self.connected)


medusa 0.5.4's medusa.http_server.http_request doesn't override
'writable', but it does have a broken 'writeable_for_proxy':


def writable_for_proxy (self):
# this version of writable supports the idea of a 'stalled'
# producer
# [i.e., it's not ready to produce any output yet] This is
# needed by
# the proxy, which will be waiting for the magic combination of
# 1) hostname resolved
# 2) connection made
# 3) data available.
if self.ac_out_buffer:
return 1
elif len(self.producer_fifo):
p = self.producer_fifo.first()
if hasattr (p, 'stalled'):
return not p.stalled()
else:
return 1


Tres.
- --
===
Tres Seaver  +1 540-429-0999  [EMAIL PROTECTED]
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFI7ix3+gerLs4ltQ4RAldnAKC/QLJHmdE9dxInkWuIGja0gtSXYwCcCJcH
6XooEwW/AkJ1ntmGyxi8urM=
=1kps
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] python dll no longer in system directory?

[Thomas Heller]

Amaury Forgeot d'Arc schrieb:
> 2008/10/8 "Martin v. Löwis" <[EMAIL PROTECTED]>:
>> Thomas Heller wrote:
>>> Is it intended that python30.dll and python26.dll are not longer
>>> installed in the \windows\system32 directory?
>>
>> No, it's not. Please create a bug report (or, better, study the
>> msiexec logs, and msi.py, to find out why this happens).

Done.  http://bugs.python.org/issue4091

>> I might not have time to look into this before the next release
>> candidate.
> 
> I confirm this. The following lines in msi.py seem to be the cause of
> the change:
> 
> #dlldir = PyDirectory(db, cab, root, srcdir, "DLLDIR", ".")
> #install python30.dll into root dir for now
> dlldir = root
> 
> They were added by r61109: " Bundle msvcr90.dll as a "private assembly". "
> but I don't know if simply restoring the previous value will work in every 
> case:
> If the C Run-Time is installed "privately", then python26.dll must
> stay in c:\python26.
> 

Amaury, can you add your analysis to the tracker, please?

-- 
Thanks,
Thomas

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] python dll no longer in system directory?

[Martin v. Löwis]

> I confirm this. The following lines in msi.py seem to be the cause of
> the change:
> 
> #dlldir = PyDirectory(db, cab, root, srcdir, "DLLDIR", ".")
> #install python30.dll into root dir for now
> dlldir = root
> 
> They were added by r61109: " Bundle msvcr90.dll as a "private assembly". "
> but I don't know if simply restoring the previous value will work in every 
> case:
> If the C Run-Time is installed "privately", then python26.dll must
> stay in c:\python26.

Ah, ok. Maybe I can find some time next week to look into this, but I
wouldn't mind if anybody else did.

In any case, thanks for this research so far (and too bad that nobody
noticed throughout all the beta releases).

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Distutils/packaging sprint this weekend

[A.M. Kuchling]

Tarek Zidae' is organizing a sprint on general
distutils/setuptools/packaging this weekend.  Physically it's in
Arlington VA, but participants will be hanging out in #distutils on
freenode's IRC.

More information at
.

--am"If you're in Fairfax County and need a lift, let me know"k
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[Martin v. Löwis]

[switching to python-dev]

Georg Brandl wrote:
> Martin v. Löwis schrieb:
>> Raymond Hettinger wrote:
 Merges should be handled by the original committer.
>>> Respectfully disagree.  Some people here having been taking
>>> responsibility for keeping the branches in-sync 
>> Which people specifically?
> 
> Specifically, Christian, Benjamin and myself have done larger merges
> to the 3k branch in the past, and if svnmerge is used, I suspect will
> do the same for 2.6.

That's different, though. Does any of you has actually taken
*responsibility* to do so, either unlimited, or with some limitation?
(e.g. for a year, or until 2.7 is released, or for all changes
but bsddb and Windows).
I would be (somewhat) happy to hear that, but I wouldn't really expect
it - we are all volunteers, and we typically consider taking
responsibility (e.g. as a release manager) very carefully.

Please don't get me wrong: I very much appreciate that you volunteer,
but I don't want to discharge any committer from merging on the
assumption that someone has formally taken responsibility.

I would be skeptical relying on such a commitment, knowing that RL
can get in the way too easily. E.g. Christian disappeared for some
time, and I completely sympathize with that - but it also tells
me that I can't trust on somebody doing something unless that someone
has explicitly said that he will do that, hoping that he will tell
me when the commitment is no longer valid (the same happened, e.g.,
in the Python job board, and happens all the time in other projects -
it took me about a year before I stepped back as the PyXML maintainer).

I can *also* sympathize with committers that say "we don't want to
backport, because we either don't have the time, or the expertise
(say, to install and run svnmerge on Windows)". I just accept that
not all patches that deserve backporting actually do get backported
(just as not all patches that deserve acceptance do get accepted,
in the first place).

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[Amaury Forgeot d'Arc]

Hello,

Concerning the management of this particular change / development, I
see three additional issues:

- First, I think that the answer given here:
http://mail.python.org/pipermail/python-checkins/2008-October/074659.html
does not match the question.
It seems to me that Skip was asking whether the "memory leak" impacted
the 2.6 branch, and the answer should have been "No": the change that
introduced the memory leak had just been committed 10 minutes before.

- Because of this misunderstanding, the changes to this
GetCurrentDirectoryW were backported to the release2.6 branch, despite
the fact that it's not a regression from a previous version, the NEWS
entry explicitly expresses doubts about the correction (which I happen
to share), there is no unit test and no item in the issue tracker.

- The backport to release26-maint http://svn.python.org/view?rev=66865&view=rev
also merged other changes (new unrelated unit tests). IMO unrelated
changes should be committed separately: different commit messages help
to understand the motivation of each backport.

I'm not blaming anyone; my feelings are certainly biased by the
somewhat strict procedures that we recently followed when the trunk
was in "release candidate" mode (submit a patch, ask for a review, add
the reviewer's name in the commit message).
I think that some of these rules should still apply to the maintenance branches.
On the other hand, I am all for a trunk branch with few restrictions:
it's here to share code with others and experiment with the next
python release. The need for stability is much lower.

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Documentation idea

[Raymond Hettinger]

Background
--
In the itertools module docs, I included pure python equivalents for each of the C functions.  Necessarily, some of those 
equivalents are only approximate but they seem to have greatly enhanced the docs.  Something similar is in the builtin docs for 
any() and all().  The new collections.namedtuple() factory function also includes a verbose option that prints a pure python 
equivalent for the generated class. And in the decimal module, I took examples directly from the spec and included them in 
doc-testable docstrings.  This assured compliance with the spec while providing clear examples to anyone who bothers to look at the 
docstrings.


For itertools docs, I combined those best practices and included sample calls in the pure-python code (see the current docs for 
itertools to see what I mean -- perhaps look at the docs for a new tool like itertools.product() or itertools.izip_longest() to see 
how useful it is).


Bright idea
--
Let's go one step further and do this just about everywhere and instead of putting it in the docs, attach an exec-able string as an 
attribute to our C functions.  Further, those pure python examples should include doctests so that the user can see a typical 
invocation and calling pattern.


Say we decide to call the attribute something like ".python", then you could 
write something like:

   >>> print(all.python)
  def all(iterable):
   '''Return True if all elements of the iterable are true.

   >>> all(isinstance(x, int) for x in [2, 4, 6.13, 8])
   False
   >>> all(isinstance(x, int) for x in [2, 4, 6, 8])
   True
   '''

   for element in iterable:
   if not element:
return False
   return True

There you have it, a docstring, doctestable examples, and pure python equivalent all in one place.  And since the attribute is 
distinguished from __doc__, we can insist that the string be exec-able (something we can't insist on for arbitrary docstrings).


Benefits


* I think this will greatly improve the understanding of tools like str.split() which have proven to be difficult to document with 
straight prose.  Even with simple things like any() and all(), it makes it self-evident that the functions have early-out behavior 
upon hitting the first mismatch.


* The exec-able definitions and docstrings will be testable

* It will assist pypy style projects and other python implementations when they 
have to build equivalents to CPython.

* We've gotten good benefits from doctests for pure python functions, why not 
extend this best practice to our C functions.

* The whole language will become more self-explanatory and self-documenting.

* Will eliminate confusion about what functions were exactly intended to do.

* Will confer benefits similar to test driven development where the documentation and  pure python version are developed first and 
doctests gotten to pass, then the C version is created to match.


* For existing code, this is a perfect project for people who want to start contributing to the language but aren't ready to start 
writing C (the should be able to read C however so that the equivalent really does match the C code).



Limits
-

* In some cases, there may be no pure python equivalent (i.e. sys.getsize()).

* Sometimes the equivalent can only be approximate because the actual C 
function is too complex (i.e. itertools.tee()).

* Some cases, like int(), are useful as a type, have multiple functions, and 
are hard to write as pure python equivalents.

* For starters, it probably only makes to do this for things that are more "algorithmic" like any() and all() or things that have a 
straight-forward equivalent like property() written using descriptors.



Premise
---

Sometimes pure python is more expressive, precise, and easy to read than English prose. 


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[Martin v. Löwis]

> It seems to me that Skip was asking whether the "memory leak" impacted
> the 2.6 branch, and the answer should have been "No": the change that
> introduced the memory leak had just been committed 10 minutes before.

You are probably right (although it's not quite clear from Skip's question).

> - Because of this misunderstanding, the changes to this
> GetCurrentDirectoryW were backported to the release2.6 branch, despite
> the fact that it's not a regression from a previous version, the NEWS
> entry explicitly expresses doubts about the correction (which I happen
> to share), there is no unit test and no item in the issue tracker.

I think it is fine that this fix was backported (assuming, without
review, that the fix is actually correct).

It is a bugfix, and it shouldn't realistically break existing applications.

IOW, PEP 6 was followed (except that there is no Patch Czar).

> - The backport to release26-maint 
> http://svn.python.org/view?rev=66865&view=rev
> also merged other changes (new unrelated unit tests). IMO unrelated
> changes should be committed separately: different commit messages help
> to understand the motivation of each backport.

Yes, that is unfortunate.

I'm skeptical that new tests actually need backporting at all. Python
doesn't really get better by new tests being added to an old branch.
Near-term, it might get worse because the new tests might cause false
positives, making users worried for no reason.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[glyph]

On 11:12 pm, [EMAIL PROTECTED] wrote:
- The backport to release26-maint 
http://svn.python.org/view?rev=66865&view=rev

also merged other changes (new unrelated unit tests).



IMO unrelated
changes should be committed separately: different commit messages help
to understand the motivation of each backport.


This I agree with completely, but...

I'm skeptical that new tests actually need backporting at all. Python
doesn't really get better by new tests being added to an old branch.


Presumably if the new tests cover functionality that somebody cares 
about, it would be valuable to make sure that maintenance bugfixes don't 
break this functionality in maintenance branches either?  In fact, I 
would think that maintenance branches would want to be much _more_ 
paranoid about making sure that changes don't break anything.


(Again: not specific to these unrelated tests that are getting merged, 
it sounds like accidentally - it just seems strange, as a general 
principle or policy, to say that maintenance branches shouldn't have new 
tests added to them.)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation idea

[Christian Heimes]

Raymond Hettinger wrote: lots of cool stuff!

The idea sounds great!

Are you planing to embed the pure python code in C code? That's going to 
increase the data segment of the executable. It should be possible to 
disable and remove the pure python example with a simple ./configure 
option and some macro magic. File size and in memory size is still 
critical for embedders.


Christian

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation idea

[Raymond Hettinger]

[Christian Heimes]

The idea sounds great!

Are you planing to embed the pure python code in C code?


Am experimenting with a descriptor that fetches the attribute string from a separate text file.  This keeps the C build from getting 
fat.  More importantly, it let's us write the execable string in a more natural way (it bites to write C style docstrings using \n 
and trailing backslashes).  The best part is that people without C compilers can still submit patches to the text files.



Raymond 


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Greg Ewing]

Nick Coghlan wrote:


If the time is being spent in PyErr_Format, how far could you get adding
a dedicated function for creating AttributeErrors? Something along the
lines of:

PyErr_AttributeError(PyObject *object, PyObject *attr_name)


More generally, it might be useful to have some mechanism for
deferred instantiation of exceptions, so you can do something
like indicate what type of exception you want to raise, and
specify a function and some arguments to call to instantiate
the exception, but the instantiation itself doesn't happen
unless the exception object is actually needed by Python
code.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Amaury Forgeot d'Arc]

Hello,

On Fri, Oct 10, 2008 at 2:15 AM, Greg Ewing <[EMAIL PROTECTED]> wrote:
> Nick Coghlan wrote:
>
>> If the time is being spent in PyErr_Format, how far could you get adding
>> a dedicated function for creating AttributeErrors? Something along the
>> lines of:
>>
>> PyErr_AttributeError(PyObject *object, PyObject *attr_name)
>
> More generally, it might be useful to have some mechanism for
> deferred instantiation of exceptions, so you can do something
> like indicate what type of exception you want to raise, and
> specify a function and some arguments to call to instantiate
> the exception, but the instantiation itself doesn't happen
> unless the exception object is actually needed by Python
> code.

But this is already the case, and the reason why there are three
variable to describe an exception: type, value and traceback.
At the C level, the value is often a string (when using PyErr_Format,
for example), or a tuple.
Normalization (=creation of the exception object) only occurs when
needed, e.g when entering an "except:" handler in python code, or when
the exception is printed.
However, the "value" string object must be created anyway, and this
could be avoided in most Getattr cases.

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Greg Ewing]

Amaury Forgeot d'Arc wrote:


But this is already the case, and the reason why there are three
variable to describe an exception: type, value and traceback.


Yes, but you only get one object for the value, which means
at least allocating a tuple if you want to be able to report
something like "AttributeError: 'fooblat' object has no
attribute 'asdf'".

I'm thinking about a C api that would let you supply multiple
pre-existing values without having to allocate anything.
Not sure exactly how it would work, though.

Also, what about Py3? Has the type/value separation gone away
completely, or is it still there at the C level?

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[Stephen J. Turnbull]

"Martin v. Löwis" writes:

 > I'm skeptical that new tests actually need backporting at all. Python
 > doesn't really get better by new tests being added to an old branch.
 > Near-term, it might get worse because the new tests might cause false
 > positives, making users worried for no reason.

If they do fail, they're not "false" positives.  If they're "false",
then the test is broken, no?  So find a way to label them as tests
added ex-post, with the failures *not* being regressions but rather
latent bugs newly detected, and (presumably) as "wont-fix".

>From a QA point of view one would like to be able to assess how many
latent bugs are making it through to end-of-life.  The new tests will
help in that.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation idea

[Brett Cannon]

On Thu, Oct 9, 2008 at 4:12 PM, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
[SNIP]
> Bright idea
> --
> Let's go one step further and do this just about everywhere and instead of
> putting it in the docs, attach an exec-able string as an attribute to our C
> functions.  Further, those pure python examples should include doctests so
> that the user can see a typical invocation and calling pattern.
>
> Say we decide to call the attribute something like ".python", then you could
> write something like:
>
>   >>> print(all.python)
>  def all(iterable):
>   '''Return True if all elements of the iterable are true.
>
>   >>> all(isinstance(x, int) for x in [2, 4, 6.13, 8])
>   False
>   >>> all(isinstance(x, int) for x in [2, 4, 6, 8])
>   True
>   '''
>
>   for element in iterable:
>   if not element:
>return False
>   return True
>
> There you have it, a docstring, doctestable examples, and pure python
> equivalent all in one place.  And since the attribute is distinguished from
> __doc__, we can insist that the string be exec-able (something we can't
> insist on for arbitrary docstrings).
>

The idea is great. I assume the special file support is mostly for the
built-ins since extension modules can do what heapq does; have a pure
Python version people import and that code pulls in any supporting C
code.

As for an implementation, you could go as far as to have a flag in the
extension module that says, "look for Python equivalents" and during
module initialization find the file and pull it in. Although doing it
that way would not necessarily encourage people as much to start with
the pure Python version and then only do C equivalents when
performance or design requires it.

> Benefits
> 
>
> * I think this will greatly improve the understanding of tools like
> str.split() which have proven to be difficult to document with straight
> prose.  Even with simple things like any() and all(), it makes it
> self-evident that the functions have early-out behavior upon hitting the
> first mismatch.
>
> * The exec-able definitions and docstrings will be testable
>

And have some way to test both the Python and C version with the same
tests (when possible)?

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattr__ and new style classes

[Brett Cannon]

On Thu, Oct 9, 2008 at 5:37 PM, Greg Ewing <[EMAIL PROTECTED]> wrote:
> Amaury Forgeot d'Arc wrote:
>
>> But this is already the case, and the reason why there are three
>> variable to describe an exception: type, value and traceback.
>
> Yes, but you only get one object for the value, which means
> at least allocating a tuple if you want to be able to report
> something like "AttributeError: 'fooblat' object has no
> attribute 'asdf'".
>
> I'm thinking about a C api that would let you supply multiple
> pre-existing values without having to allocate anything.
> Not sure exactly how it would work, though.
>
> Also, what about Py3? Has the type/value separation gone away
> completely, or is it still there at the C level?
>

It's still there. I think the major C changes for exceptions was
adding support for the three new attributes (__traceback__,
__context__, and __cause__).

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation idea

[Lisandro Dalcin]

On Thu, Oct 9, 2008 at 8:50 PM, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> [Christian Heimes]
>>
>> The idea sounds great!
>>
>> Are you planing to embed the pure python code in C code?
>
> Am experimenting with a descriptor that fetches the attribute string from a
> separate text file.

Have you ever considered the same approach for docstrings in C code?
As reference, NumPy already has some trickery for maintaining
docstrings outside C sources. Of course, descriptors would be a far
better for implementing and support this in core Python and other
projects...

>  This keeps the C build from getting fat.  More
> importantly, it let's us write the execable string in a more natural way (it
> bites to write C style docstrings using \n and trailing backslashes).  The
> best part is that people without C compilers can still submit patches to the
> text files.
>
>
> Raymond
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com
>



-- 
Lisandro Dalcín
---
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation idea

[Raymond Hettinger]

Yes, I'm looking a couple of different approaches to loading the strings.
For now though, I want to focus on the idea itself, not the implementation.
The important thing is to gather widespread support before getting into
the details of how the strings get loaded.


Raymond



- Original Message - 
From: "Lisandro Dalcin" <[EMAIL PROTECTED]>


Have you ever considered the same approach for docstrings in C code?
As reference, NumPy already has some trickery for maintaining
docstrings outside C sources. Of course, descriptors would be a far
better for implementing and support this in core Python and other
projects...


 This keeps the C build from getting fat.  More


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] test message - please ignore

[skip]

(messing with the python.org spam filter - please ignore)

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] another test message - please ignore

[skip]

still working on spam filter...

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation idea

[glyph]

On 9 Oct, 11:12 pm, [EMAIL PROTECTED] wrote:

Background
--
In the itertools module docs, I included pure python equivalents for 
each of the C functions.  Necessarily, some of those equivalents are 
only approximate but they seem to have greatly enhanced the docs.


Why not go the other direction?

Ostensibly the reason for writing a module like 'itertools' in C is 
purely for performance.  There's nothing that I'm aware of in that 
module which couldn't be in Python.


Similarly, cStringIO, cPickle, etc.  Everywhere these diverge, it is (if 
not a flat-out bug) not optimal.  External projects are encouraged by a 
wealth of documentation to solve performance problems in a similar way: 
implement in Python, once you've got the interface right, optimize into 
C.


So rather than have a C implementation, which points to Python, why not 
have a Python implementation that points at C?  'itertools' (and 
similar) can actually be Python modules, and use a decorator, let's call 
it "C", to do this:


   @C("_c_itertools.count")
   class count(object):
   """
   This is the documentation for both the C version of 
itertools.count

   and the Python version - since they should be the same, right?
   """

In Python itself, the Python module would mostly be for documentation, 
and therefore solve the problem that Raymond is talking about, but it 
could also be a handy fallback for sanity checking, testing, and use in 
other Python runtimes (ironpython, jython, pypy).  Many third-party 
projects already use ad-hoc mechanisms for doing this same thing, but an 
officially-supported way of saying "this works, but the optimized 
version is over here" would be a very useful convention.


In those modules which absolutely require some C stuff to work, the 
python module could still serve as documentation.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation idea

[Jared Grubb]

This is a really interesting idea. If extra memory/lookup overhead is  
a concern, you could enable this new feature by default when the  
interactive interpreter is started (where it's more likely to be  
invoked), and turn it off by default when running scripts/modules.


Jared

On 9 Oct 2008, at 20:37, [EMAIL PROTECTED] wrote:



On 9 Oct, 11:12 pm, [EMAIL PROTECTED] wrote:

Background
--
In the itertools module docs, I included pure python equivalents  
for each of the C functions.  Necessarily, some of those  
equivalents are only approximate but they seem to have greatly  
enhanced the docs.


Why not go the other direction?

Ostensibly the reason for writing a module like 'itertools' in C is  
purely for performance.  There's nothing that I'm aware of in that  
module which couldn't be in Python.


Similarly, cStringIO, cPickle, etc.  Everywhere these diverge, it is  
(if not a flat-out bug) not optimal.  External projects are  
encouraged by a wealth of documentation to solve performance  
problems in a similar way: implement in Python, once you've got the  
interface right, optimize into C.


So rather than have a C implementation, which points to Python, why  
not have a Python implementation that points at C?  'itertools' (and  
similar) can actually be Python modules, and use a decorator, let's  
call it "C", to do this:


  @C("_c_itertools.count")
  class count(object):
  """
  This is the documentation for both the C version of  
itertools.count

  and the Python version - since they should be the same, right?
  """

In Python itself, the Python module would mostly be for  
documentation, and therefore solve the problem that Raymond is  
talking about, but it could also be a handy fallback for sanity  
checking, testing, and use in other Python runtimes (ironpython,  
jython, pypy).  Many third-party projects already use ad-hoc  
mechanisms for doing this same thing, but an officially-supported  
way of saying "this works, but the optimized version is over here"  
would be a very useful convention.


In those modules which absolutely require some C stuff to work, the  
python module could still serve as documentation.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/jared.grubb%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[Martin v. Löwis]

> If they do fail, they're not "false" positives.  If they're "false",
> then the test is broken, no?

Correct. But they might well be broken, no?

> So find a way to label them as tests
> added ex-post, with the failures *not* being regressions but rather
> latent bugs newly detected, and (presumably) as "wont-fix".

No such way exists, hence they can't be labeled that way.
No such labeling can be added, since that would be a new feature.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[Martin v. Löwis]

> Presumably if the new tests cover functionality that somebody cares
> about, it would be valuable to make sure that maintenance bugfixes don't
> break this functionality in maintenance branches either?

Yes. At the same time, there is also a risk that the new tests just fail
because they are not correctly formulated. Users don't report that
against the trunk, because they are not running the trunk. Instead,
they download the next maintenance release, run the tests, see that it
fails, get frightened, and return to the prior release (which didn't
have any failing tests).

There is a certain prevention already that later maintenance fixes don't
break the earlier ones: those fixes typically get checked into the trunk
also, where the tests do exist. So the committer would find out even
before the patch gets to the maintenance branch.

All that said, I don't want to state a policy about whether new tests
should or should not get added to maintenance branches. I'd leave this
up to the committers, and just wanted to caution about this risk.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[Andrew Bennetts]

"Martin v. Löwis" wrote:
[...]
> There is a certain prevention already that later maintenance fixes don't
> break the earlier ones: those fixes typically get checked into the trunk
> also, where the tests do exist. So the committer would find out even
> before the patch gets to the maintenance branch.

By this logic, the maintenance branch would need no tests at all,
because they are all in trunk already!

-Andrew.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[Martin v. Löwis]

Andrew Bennetts wrote:
> "Martin v. Löwis" wrote:
> [...]
>> There is a certain prevention already that later maintenance fixes don't
>> break the earlier ones: those fixes typically get checked into the trunk
>> also, where the tests do exist. So the committer would find out even
>> before the patch gets to the maintenance branch.
> 
> By this logic, the maintenance branch would need no tests at all,
> because they are all in trunk already!

No, this is not the logic. The tests in the maintenance branch have gone
through alpha and beta releases, so end users have seen them already (at
least, some of them). Of course, it might still be possible that there
is an incorrect test in the released version; those can get fixed in
later maintenance releases.

So 2.6.0 will contain a lot of tests that have never been tested in
a wide variety of systems. Some are incorrect, and get fixed in 2.6.1,
and stay fixed afterwards. This is completely different from somebody
introducing a new test in 2.6.4. It means that there are more failures
in a maintenance release, not less as in the first case.

Of course, it might be that a developer deliberately wants to see
a certain test be run in the field, because there is a uncertainty
whether the fix actually works. However, it is questionable whether
such a fix would belong in the maintenance release.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r66863 - python/trunk/Modules/posixmodule.c

[Stephen J. Turnbull]

"Martin v. Löwis" writes:

 > > If they do fail, they're not "false" positives.  If they're "false",
 > > then the test is broken, no?
 > 
 > Correct. But they might well be broken, no?

I would hope some effort is made that they not be.  If they generate a
positive, I would expect that the contributor would try to fix that
before committing, no?  If they discover that it's "false", they fix
or remove the test; otherwise they document it.

 > > So find a way to label them as tests
 > > added ex-post, with the failures *not* being regressions but rather
 > > latent bugs newly detected, and (presumably) as "wont-fix".
 > 
 > No such way exists,

Add a documentation file called "README.expected-test-failures".
AFAIK documentation is always acceptable, right?

Whether that is an acceptable solution to the "latent bug" problem is
a different question.  I'd rather know that Python has unexpected
behavior, and have a sample program (== test) to demonstrate it, than
not.  YMMV.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com