Re: [Python-Dev] remaining issues from Klocwork static analysis

[Georg Brandl]

Neal Norwitz wrote:
> On 7/25/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
>> Neal Norwitz wrote:
>> > # 74 Object/funcobject.c:143Suspicious deref of ptr before NULL check
>>
>> Not quite sure what it is complaining about, but
>>
>> else if (PyTuple_Check(closure)) {
>> Py_XINCREF(closure);
>> }
>>
>> looks indeed suspicious: Why do we check for NULL (XINCREF) when
>> we know closure can't be NULL (Tuple_Check). Drop the X, and see
>> if the warning goes away
> 
> Yes, I definitely think dropping the X would make the warning go away.
>  Do we want to check for a NULL pointer and raise an exception?  The
> docs don't address the issue, so I think if we added a check, ie:  if
> (closure && PyTuple_Check(closure)) and got rid of the X that would be
> fine as well.

You'll have to do something about the error message, then, since it
uses closure->ob_type.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Strategy for converting the decimal module to C

[Greg Ewing]

Nick Maclaren wrote:

> The compiled code has made a data structure temporarily inconsistent
> because the operation is safe (say, list insertion), and then gets an
> asynchronous interrupt (e.g. SIGINT).  The SIGINT handler does some
> operation (e.g. I/O) that implicitly uses floating-point, which then
> interrupts.

Well, of course anything can be made to happen asynchronously
by calling it from something asynchronous, such as a SIGINT
handler. That doesn't change the fact that the floating
point operation itself is deterministic, including whether
it causes an exception.

Well-written programs don't do any more in a signal handler
than is absolutely necessary, for reasons which apply equally
well whether floating point is involved or not. I'd say
the mistake was made right at the beginning by assuming
that the data structure in question was safe while allowing
a SIGINT to occur to a handler that's not careful enough
about what it does.

BTW, it seems to me you could get exactly the same problem
if FP exceptions were handled entirely in user mode, as
you suggest. Not that I don't agree that would be a good
idea -- I do -- but it wouldn't prevent this particular
kind of mistake.

And all of this is getting rather far away from where we
started, which was simply instrumenting a piece of code
to count floating point exceptions. Such a program isn't
going to be doing I/O in SIGINT handlers or installing
FP exception handlers that mess with unrelated critical
data structures.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] remaining issues from Klocwork static analysis

[Gerhard Häring]

Martin v. Löwis wrote:
> Neal Norwitz wrote:
>> # 61 Modules/_sqlite/cursor.c:599  Null pointer may be dereferenced
>>
>> Null pointer 'self->statement' that comes from line 674 may be
>> dereferenced by passing argument 1 to function
>> 'statement_mark_dirty' at line 599.
> 
> Looks like a problem. Maybe a break is missing after line 674?

The code is a bit complicated here, and admittedly not the nicest one, 
but I verified it and there is no problem here.

_query_execute() is designed to handle both executemany() and execute().

multiple is a local variable that is 1 if there is a set of tuples of 
SQL parameters (executemany) instead of a single tuple of SQL parameters 
(execute), in which case it's 0.

Before the while loop, the code makes sure that parameters_iter is an 
iterator that returns SQL parameter tuples. So if there is only a single 
parameter tuple, a temporary list is created an parameters_iter is an 
iterator over it.

So, if !multiple (referenced code in line 674), the while-loop will only 
be executed once, and in the second loop, the while-loop will exit 
because then the following code will break:

 parameters = PyIter_Next(parameters_iter);
 if (!parameters) {
 break;
 }

Code in line 599 is thus not executed with self->statement = 0.

-- Gerhard
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] remaining issues from Klocwork static analysis

[Michael Hudson]

"Neal Norwitz" <[EMAIL PROTECTED]> writes:

> On 7/25/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
>> >
>> > Yes, I definitely think dropping the X would make the warning go away.
>> > Do we want to check for a NULL pointer and raise an exception?  The
>> > docs don't address the issue, so I think if we added a check, ie:  if
>> > (closure && PyTuple_Check(closure)) and got rid of the X that would be
>> > fine as well.
>>
>> The docs do address the issue:
>>
>> \var{closure} must be \var{Py_None} or a tuple of cell objects.
>>
>> It doesn't allow for NULL, and None indicates that the closure
>> should become NULL. The only caller of it in the core will never
>> pass NULL.
>>
>> If you want to check that this is not NULL on the grounds that
>> somebody may call it incorrectly, then you should also check that
>> op is not NULL, because somebody may call it incorrectly.
>
> We never really did address this issue did?  A while back we talked
> about whether to assert vs check and do PyErr_BadInternalCall().  I
> don't remember a clear resolution (though my memory).  I vaguely
> remember a preference towards asserting, but I don't know if that was
> in all cases or maybe it was just my preference. :-)
>
> I'm happy to assert here too.  But it's really a broader question.  I
> guess I'm even happy to just remove the X.  It would be nice to handle
> this consistently going forward.

I think I'm rather in favour of assert()ing this sort of thing.  If
you're programming in C, you can cause crashes any which way and
removing one doesn't seem worth making correct usage pay any kind of
(admittedly miniscule) performance penalty.  It would be nice if API
docs explicitly stated which pointer arguments could be NULL, and then
it would be a programming error to pass a NULL pointer argument in any
other place.  I have no idea how far away from this we are already :-)

Cheers,
mwh

-- 
  Gullible editorial staff continues to post links to any and all
  articles that vaguely criticize Linux in any way.
 -- Reason #4 for quitting slashdot today, from
http://www.cs.washington.edu/homes/klee/misc/slashdot.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Strategy for converting the decimal module to C

[Raymond Hettinger]

Greg Ewing
> And all of this is getting rather far away from where we
> started, which was simply instrumenting a piece of code
> to count floating point exceptions.

I'm thinking of adding a note to the Py2.5 docs that the counting feature is 
not 
part of the standard and should not be expected to work on other 
implementations 
of the standard (including a planned CPython extension module).


Raymond

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Strategy for converting the decimal module to C

[Facundo Batista]

2006/7/26, Raymond Hettinger <[EMAIL PROTECTED]>:

> Greg Ewing
> > And all of this is getting rather far away from where we
> > started, which was simply instrumenting a piece of code
> > to count floating point exceptions.
>
> I'm thinking of adding a note to the Py2.5 docs that the counting feature is 
> not
> part of the standard and should not be expected to work on other 
> implementations
> of the standard (including a planned CPython extension module).

+1

-- 
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] New miniconf module

[Sylvain Fourmanoit]

I wrote a data persistence module called miniconf, aimed at making 
easy to create and safely retrieve configuration info from external, 
human-readable sources using Python syntax. I feel it would eventually 
make a nice addition to the standard library.

The code was only newly refactored in this form, but it as been 
broadly distributed and used as a part of the adesklets project for over a 
year by a significant user base on multiple platforms.

Here it is, as a patch against Python 2.5 SVN tree[1], or as a 
stand-alone module hosted on the Python Cheese Shop[2]; any feedback is 
welcomed.

-- 
Sylvain <[EMAIL PROTECTED]>

Hackers are just a migratory lifeform with a tropism for computers.

[1]http://sourceforge.net/tracker/index.php?func=detail&aid=1527597&group_id=5470&atid=355470
[2]http://cheeseshop.python.org/pypi?:action=display&name=miniconf&version=1.0.1
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] 2.5: uses of sys.exc_type, exc_value

[A.M. Kuchling]

http://www.python.org/sf/1525469 reports that SimpleXMLRPCServer.py
still uses sys.exc_type and sys.exc_value when handling exceptions.
These variables aren't thread-safe and sys.exc_info() is the better
way.  I have a patch attached to the bug that fixes the problem.

Question 1: is this worth fixing for 2.5?  (It's not really a bugfix,
more of a style cleanup.)

Question 2: I searched for uses of the old variables and found these:

Lib/idlelib/WindowList.py:  sys.exc_type, ":", sys.exc_value
Lib/logging/__init__.py:return sys.exc_traceback.tb_frame.f_back
Lib/lib-tk/Tkinter.py:exc, val, tb = sys.exc_type, sys.exc_value, 
sys.exc_traceback
Lib/plat-mac/cfmfile.py:raise Res.Error, "no 'cfrg' 
resource found", sys.exc_traceback
Lib/SocketServer.py:sys.exc_traceback = None# Help garbage 
collection

Plus some references in the test suite, the demos, and faqwizard.py.

SocketServer should use sys.exc_clear() instead.  Tkinter.py could
just call exc_info(), but I wonder if the usage of the variables is
intentional here.  sys.exc_info() was introduced in Python 1.5, so
logging/__init__.py could be fixed without affecting 1.5.2
compatibility.

Should the above uses be fixed, too?  

--amk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.5: uses of sys.exc_type, exc_value

[Guido van Rossum]

Clearly they should be fixed. Whether in 2.5 or 2.6 I'll leave up to
Neal and Anthony.

On 7/26/06, A.M. Kuchling <[EMAIL PROTECTED]> wrote:
> http://www.python.org/sf/1525469 reports that SimpleXMLRPCServer.py
> still uses sys.exc_type and sys.exc_value when handling exceptions.
> These variables aren't thread-safe and sys.exc_info() is the better
> way.  I have a patch attached to the bug that fixes the problem.
>
> Question 1: is this worth fixing for 2.5?  (It's not really a bugfix,
> more of a style cleanup.)
>
> Question 2: I searched for uses of the old variables and found these:
>
> Lib/idlelib/WindowList.py:  sys.exc_type, ":", 
> sys.exc_value
> Lib/logging/__init__.py:return sys.exc_traceback.tb_frame.f_back
> Lib/lib-tk/Tkinter.py:exc, val, tb = sys.exc_type, sys.exc_value, 
> sys.exc_traceback
> Lib/plat-mac/cfmfile.py:raise Res.Error, "no 'cfrg' 
> resource found", sys.exc_traceback
> Lib/SocketServer.py:sys.exc_traceback = None# Help garbage 
> collection
>
> Plus some references in the test suite, the demos, and faqwizard.py.
>
> SocketServer should use sys.exc_clear() instead.  Tkinter.py could
> just call exc_info(), but I wonder if the usage of the variables is
> intentional here.  sys.exc_info() was introduced in Python 1.5, so
> logging/__init__.py could be fixed without affecting 1.5.2
> compatibility.
>
> Should the above uses be fixed, too?
>
> --amk
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Release manager pronouncement needed: PEP 302 Fix

[Phillip J. Eby]

I posted last week about a need-for-speed patch that broke PEP 302 
compliance, and asked if it should be fixed or reverted.  I got exactly one 
response which said "yes, it should be fixed or reverted", which 
unfortunately didn't answer my question as to which one we should do.  :)

If we don't revert it, there are two ways to fix it.  One is to just change 
PEP 302 so that the behavior is unbroken by definition.  :)  The other is 
to actually go ahead and fix it by adding PathImporter and NullImporter 
types to import.c, along with a factory function on sys.path_hooks to 
create them.  (This would've been the PEP-compliant way to implement the 
need-for-speed patch.)

So, "fix" by documentation, fix by fixing, or fix by reverting?  Which 
should it be?

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New miniconf module

[David Hopwood]

Sylvain Fourmanoit wrote:
> I wrote a data persistence module called miniconf, aimed at making 
> easy to create and safely retrieve configuration info from external, 
> human-readable sources using Python syntax. I feel it would eventually 
> make a nice addition to the standard library.

>From a security point of view, this is a great improvement on the existing
pickle, marshal, and shelve modules. Those modules could not be safely
imported from restricted code.

miniconf, OTOH, appears to have an interface compatible with capability
security. (I have not checked that the compiler.ast module used in its
implementation is safe.) However:

+Limitations
+===
+
+miniconf has a few limitations one should be aware of:
[...]
+- It is not preemptiple: concurrent calls to dump() or load() will
+  have unpredictable results and must be avoided.

This limitation should be fixed before the module is added to the standard
library, IMHO.

-- 
David Hopwood <[EMAIL PROTECTED]>


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New miniconf module

[Phillip J. Eby]

At 07:47 PM 7/26/2006 +0100, David Hopwood wrote:
>Sylvain Fourmanoit wrote:
> > I wrote a data persistence module called miniconf, aimed at making
> > easy to create and safely retrieve configuration info from external,
> > human-readable sources using Python syntax. I feel it would eventually
> > make a nice addition to the standard library.
>
> >From a security point of view, this is a great improvement on the existing
>pickle, marshal, and shelve modules. Those modules could not be safely
>imported from restricted code.
>
>miniconf, OTOH, appears to have an interface compatible with capability
>security. (I have not checked that the compiler.ast module used in its
>implementation is safe.) However:
>
>+Limitations
>+===
>+
>+miniconf has a few limitations one should be aware of:
>[...]
>+- It is not preemptiple: concurrent calls to dump() or load() will
>+  have unpredictable results and must be avoided.
>
>This limitation should be fixed before the module is added to the standard
>library, IMHO.

It looks like it's trivial to fix; the code uses a strange and unnecessary 
complication of creating nested classes and nested singleton instances 
thereof.  Getting rid of the singletons to create a new instance for each 
dump/load call would suffice to make the implementation re-entrant, 
although de-nesting the classes would also be a good idea.  :)

The loading code could also be made a lot faster by using a dictionary 
mapping AST node types to functions, instead of doing string manipulation 
for each node.  Each function could take 'pedantic' as a parameter, which 
would eliminate the need to have an object at all, let alone a singleton.

Finally, there is an interesting characteristic of the code's 
interpretation of names: any name other than 'True' is interpreted as 'False'!

On the whole, though, I don't see a lot of difference between this format 
and say, JavaScript Object Notation (JSON), which can be parsed and 
generated by many other languages as well as multiple Python libraries already.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New miniconf module

[Sylvain Fourmanoit]

> miniconf, OTOH, appears to have an interface compatible with capability
> security (I have not checked that the compiler.ast module used in its
> implementation is safe.)

I woudn't be 100% sure either (obviously, I didn't write this nice piece 
of code, let alone the underlying parser), but I read it and tried to 
abuse it without success (I haven't found obvious buffer overflow and 
such)... As far as I know, the abstract  syntax tree generation exposed 
via compiler.ast is a safe operation, in the sense that it doesn't allow 
execution of code when feeded from  arbitrary strings via 
compiler.parse(); in the worst case scenario, it raises a SyntaxError or 
similar exceptions, as documented... If anybody know more on this issue, I 
will be happy to hear about it.

> miniconf has a few limitations one should be aware of:
>
> - It is not preemptiple: concurrent calls to dump() or load() will
>   have unpredictable results and must be avoided.
>
> This limitation should be fixed before the module is added to the 
> standard library, IMHO.

If this is the general opinion, I will be glad to change this... 
The only reason miniconf is not thread-safe for the moment is that I 
chose to re-use over and over a single instance of each of my two 
processing classes to reduce resources usage, but this seems pretty 
pointless (and overly complicated) now that I look at it. Yours,

-- 
Sylvain <[EMAIL PROTECTED]>

Your files are now being encrypted and thrown into the bit bucket.
EOF
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New miniconf module

[Sylvain Fourmanoit]

> It looks like it's trivial to fix; the code uses a strange and 
> unnecessary complication of creating nested classes and nested 
> singleton instances thereof.  Getting rid of the singletons to create a 
> new instance for each dump/load call would suffice to make the 
> implementation re-entrant, although de-nesting the classes would also be 
> a good idea.  :)

OK then, I will change this.

> The loading code could also be made a lot faster by using a dictionary 
> mapping AST node types to functions, instead of doing string 
> manipulation for each node.  Each function could take 'pedantic' as a 
> parameter, which would eliminate the need to have an object at all, let 
> alone a singleton.
> 
I am not convinced the current string manipulation for mapping the nodes 
types to the methods of the _Load class has such a significant impact on 
performance, but I will test your suggestion... The only difference with 
current code is that we use a dynamically computed string as the 
dictionary key to locate the function instead of the node type themselves 
as keys.

>  Finally, there is an interesting characteristic of the code's
>  interpretation of names: any name other than 'True' is interpreted as
>  'False'!

;-) It will be corrected in the next release.

>  On the whole, though, I don't see a lot of difference between this format
>  and say, JavaScript Object Notation (JSON), which can be parsed and
>  generated by many other languages as well as multiple Python libraries
>  already.

The difference is that this is Python code, already familiar to all Python 
coders... Besides, it sits directly on top of the real Python parser, 
mitigating the need of a new one, and keeping the added code complexity 
to a strict minimum.

But I agree this looks a lot like JSON, since ecmascript syntax for 
literals looks a lot like the one of Python... For the same reasons there 
is a need for JSON, I think having something like miniconf in the 
standard lib would benefit the users.

--
Sylvain <[EMAIL PROTECTED]>

If you think the system is working, ask someone who's waiting for a 
prompt.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New miniconf module

[Phillip J. Eby]

At 05:43 PM 7/26/2006 -0400, Sylvain Fourmanoit wrote:
> > The loading code could also be made a lot faster by using a dictionary
> > mapping AST node types to functions, instead of doing string
> > manipulation for each node.  Each function could take 'pedantic' as a
> > parameter, which would eliminate the need to have an object at all, let
> > alone a singleton.
> >
>I am not convinced the current string manipulation for mapping the nodes
>types to the methods of the _Load class has such a significant impact on
>performance, but I will test your suggestion...

I haven't tested this with your code specifically, but I know that in the 
past I have nearly tripled the speed of AST-visiting code by doing this; 
string manipulation plus attribute lookup is a lot more expensive than 
direct dictionary lookups.


>  The only difference with
>current code is that we use a dynamically computed string as the
>dictionary key to locate the function instead of the node type themselves
>as keys.

Actually you're doing string manipulation plus an *attribute* lookup, and 
attribute lookups can involve multiple dictionary lookups.  But anyway, 
test and see what you get.  :)


>But I agree this looks a lot like JSON, since ecmascript syntax for
>literals looks a lot like the one of Python... For the same reasons there
>is a need for JSON, I think having something like miniconf in the
>standard lib would benefit the users.

Actually, I would see more reason to include JSON in the standard library, 
since it's at least something approaching an internet protocol these days.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New miniconf module

[John J Lee]

On Wed, 26 Jul 2006, Phillip J. Eby wrote:
[...]
> Actually, I would see more reason to include JSON in the standard library,
> since it's at least something approaching an internet protocol these days.

+1


John

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] JSON implementation in Python 2.6

[Bob Ippolito]

On Jul 26, 2006, at 3:18 PM, John J Lee wrote:

> On Wed, 26 Jul 2006, Phillip J. Eby wrote:
> [...]
>> Actually, I would see more reason to include JSON in the standard  
>> library,
>> since it's at least something approaching an internet protocol  
>> these days.
>
> +1

If there's a consensus on that, my simplejson [1] implementation  
could migrate to the stdlib for 2.6.

The API is modeled after marshal and pickle, the code should be PEP 8  
compliant, its test suite has pretty good coverage, it's already used  
by (at least) TurboGears and Django, and it's the implementation  
currently "endorsed by" json.org.

The work that would be required would be:

- LaTeX docs (currently reST in docstrings)
- Move the tests around and make them run from the suite rather than  
via nose
- Possible module rename (jsonlib?)

[1] http://undefined.org/python/#simplejson

-bob

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] DRAFT: python-dev summary for 2006-06-16 to 2006-06-30

[Steven Bethard]

Here's the draft for the second half of June.  As always, comments and
corrections are greatly appreciated.


=
Announcements
=

---
Python 2.5 schedule
---

A number of bugs are being squashed as Python 2.5 moves towards its
next release.  See `PEP 356`_ for more details and the full schedule.

.. _PEP 356: http://www.python.org/dev/peps/pep-0356/

Contributing threads:

- `Beta 1 schedule ? (Bug in stringobject?)
`__
- `Adding winerror module (Beta 1 schedule ?)
`__
- `current 2.5 issues
`__
- `TRUNK FREEZE IMMINENT FOR 2.5 BETA 1 - 00:00 UTC, 20-JUNE-2006
`__
- `beta1 coming real soon
`__
- `RELEASED Python 2.5 (beta 1)
`__
- `TRUNK is UNFROZEN, but in FEATURE FREEZE
`__
- `2.5 and beyond
`__

-
Checkins for betas and release candidates
-

Anthony Baxter announced some guidelines for checkins for the beta and
release candidate releases. For all beta releases:

* All checkins must have an entry for Misc/NEWS, a test and docs
* All checkins that add features must have approval from a release manager
* All checkins must not break any of the buildbots

For all release candidates:

* All checkins must have approval from a release manager

Approval from a release manager (Anthony or Neal) should preferably be
obtained in public (e.g. the python-dev list) and should be noted in
the commit message.

Contributing threads:

- `When to branch release25-maint?
`__
- `RFC: trunk checkins between now and 2.5 final
`__

---
FishEye on the Python Subversion Repository
---

FishEye is once again `available for the Python repository`_.

.. _available for the Python repository:
http://fisheye3.cenqua.com/browse/python

Contributing thread:

- `FishEye on Python CVS Repository
`__


=
Summaries
=

-
PEP 3103: A Switch/Case Statement
-

After Thomas Lee provided a `simple patch implementing a switch
statement`_ for Python, there was a massive discussion about it and
how `PEP 275`_ should best be implemented. After much discussion,
basically three camps arose:

* School I: The switch statement should just be syntactic sugar for
the corresponding if/elif chain.
* School II: The switch statement should dispatch on a precomputed
dict of values.
* School III: The switch statement should correspond to an if/elif
chain but require all expressions to be hashable (to allow for better
optimizations).

School I was primarily concerned with the repetition of the ``x ==``
in something like::

if x == ...:
   ...
elif x == ...:
   ...
elif x == ...:
   ...
else:
   ...

School II seemed to feel that just aiding DRY was not enough to
introduce a new construct, and that the switch statement should also
be able to avoid the function definitions in dispatching code like::

def f(...):
...
def g(...):
...
def h(...):
...
dispatch_dict = {x:f, y:g, z:h}
dispatch_dict[value](*args)

In order to optimize this kind of code, School II wanted to be able to
compute the dispatch dict ahead of time, so that it wouldn't have be
recomputed each time the switch statement was executed. There was a
lot of discussion as to exactly when this freezing should occur, with
some voting for module compilation time (allowing only constants in
the cases), some voting for function definition time (allowing only
constants and non-local names in the cases) and some voting for the
first time the switch statement is executed (allowing only constants
and both local and non-local names). Guido put together a thorough
summary of the options in `PEP 3103`_.

There was some discussion of introducing a ``static`` keyword which
would cause an expression to be evaluated at function definition time,
so that, for example, the following code would create a list of
functions returning each of 0, 1, ... 9::

funcs = [lambda: (static i) for i in xrange(10)]

The intention was that switch statement cases would then allow only
constants or static expressions. Guido requested a separate PEP on th

[Python-Dev] [Windows, buildbot] kill_python.c mystery

[Tim Peters]

Rarely I'll be running the Python tests in my sandbox from a DOS box,
and the test run will just end.  Like so:

C:\Code\python\PCbuild>python  -E -tt ../lib/test/regrtest.py -uall -rw
test_softspace
test_codecmaps_kr
...
test_float
test_userdict

C:\Code\python\PCbuild>

No indication of success or failure -- the process just vanishes mid-stream.

Today I noticed this happened when the buildbot started to run tests,
and I'm 100% sure it's due to this code in
Tools/buildbot/kill_python.c (the buildbot log files showed that
kill_python.c killed /some/ Python process, and the Python running
release-build tests in my sandbox was the only plausible candidate):

if ((strstr(path, "build\\pcbuild\\python_d.exe") != NULL) ||
(strstr(path, "build\\python.exe") != NULL)) {
printf("Terminating %s (pid %d)\n", path, pids[i]);
if (!TerminateProcess(hProcess, 1)) {

The second clause in the first `if` looks for a substring match on:

build\python.exe

and that just happens to match a suffix of:

C:\Code\python\PCbuild\python.exe

which is the release-build Python I happen to be running in my sandbox.

Why is the second clause there?  That is, are we /trying/ to kill a
release-build Python running from the user's sandbox, and if so why?
Introducing the second clause was the sole change in rev 46817, and
the checkin comment doesn't really explain it:

Port cygwin kill_python changes from 2.4 branch.

Since I don't know what it's trying to accomplish, I hesitate to
change it.  It's quite clear what the first clause is trying to
accomplish, and that one hasn't caused any problems.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Improving unit tests for the standard library

[Matt Fleming]

Hi, after speaking with Neal off-list about writing tests for the
pkgutil module, we agreed it would be a good idea to start a page on
http://wiki.python.org/moin/ stating any tests for the standard
library that either,

a) need to be written
b) can be improved

I've started the page http://wiki.python.org/moin/ImprovingLibTests
that lists all the test files I could think of that need to be
written. Ive also included a table for improvements to existing tests,
along with a column that allows you to specify exactly what needs
improving.

I hope this will be of use to people, and I hope people will find time
to modify the page approriately. When I get some spare time from my
SoC project, I'll be working my way through the list.

Thanks, Matt

-- 
http://mattssanctuary.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] patch for mbcs codec (again)

[H.Yamamoto]

Hello.

I noticed mbcs codec still has problem when calls StreamReader.
Can anyone commit the patch "fix.patch version2" on
http://python.org/sf/1455898 ?

# Very sorry about this... I thought I checked this, but I definitely looked
at something
# wrong.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Which version of distutils to ship with Python 2.5?

[Collin Winter]

Is it intentional that Python 2.5 is (currently) shipping with
distutils 2.4.0, while Python 2.4 (at least 2.4.1, 2.4.2 and 2.4.3)
shipped with distutils 2.4.1? Judging from my own tests, distutils
2.4.1 fixed several bugs that some of my test suites depend on (the
fixes, not the bugs ; ).

Thanks,
Collin Winter
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Weekly Python Patch/Bug Summary

[Kurt B. Kaiser]

Patch / Bug Summary
___

Patches :  401 open ( +3) /  3342 closed ( +8) /  3743 total (+11)
Bugs:  896 open ( -8) /  6035 closed (+24) /  6931 total (+16)
RFE :  224 open ( +2) /   233 closed ( +2) /   457 total ( +4)

New / Reopened Patches
__

pkgutil.walk_packages ignores onerror arg  (2006-07-20)
CLOSED http://python.org/sf/1525766  opened by  James Y Knight

Tkdnd mouse cursor handling patch  (2006-07-20)
   http://python.org/sf/1525806  opened by  klappnase

calltip awkwardly shortened  (2006-07-20)
CLOSED http://python.org/sf/1525817  opened by  Lingl

str.__iter__ and unicode.__iter__  (2006-07-21)
   http://python.org/sf/1526367  opened by  Walter Dörwald

Fix socketmodule compile on NetBSD  (2006-07-21)
   http://python.org/sf/1526460  opened by  Matt Fleming

New module: miniconf  (2006-07-24)
   http://python.org/sf/1527597  opened by  S.Fourmanoit

winsound - probably win9x port is not working  (2006-07-24)
CLOSED http://python.org/sf/1527744  opened by  Hirokazu Yamamoto

Expose case-insensitivity of string.Template  (2006-07-25)
   http://python.org/sf/1528167  opened by  Chad Whitacre

PyShell.recall - fix indentation logic  (2006-07-25)
   http://python.org/sf/1528468  opened by  Tal Einat

patch for mbcs codecs  (2006-03-22)
   http://python.org/sf/1455898  reopened by  ocean-city

Patches Closed
__

pkgutil.walk_packages ignores onerror arg  (2006-07-20)
   http://python.org/sf/1525766  closed by  gbrandl

calltip awkwardly shortened  (2006-07-20)
   http://python.org/sf/1525817  closed by  loewis

ParenMatch: workaround for misinterpreting of closing parens  (2006-01-16)
   http://python.org/sf/1407280  closed by  kbk

Syntax-related improvements to IDLE  (2004-02-28)
   http://python.org/sf/906702  closed by  kbk

(partial?) fix for Misc/python-config.in  (2006-07-16)
   http://python.org/sf/1523356  closed by  gbrandl

ConnectRegistry blocks all threads  (2006-03-12)
   http://python.org/sf/1448199  closed by  loewis

MS Windows - module search path fix  (2005-07-04)
   http://python.org/sf/1232023  closed by  loewis

winsound - probably win9x port is not working  (2006-07-24)
   http://python.org/sf/1527744  closed by  gbrandl

1515163 fix - traceback and str exc  (2006-06-30)
   http://python.org/sf/1515343  closed by  gbrandl

New / Reopened Bugs
___

inspect.py: still infinite recursion inspecting frames  (2006-07-03)
CLOSED http://python.org/sf/1516184  reopened by  nnorwitz

Bug in shutil.copytree on Windows  (2006-07-20)
   http://python.org/sf/1525866  opened by  Mike Foord

email package quoted printable behaviour changed  (2006-07-20)
   http://python.org/sf/1525919  opened by  Thomas Arendsen Hein

Win32: subprocess.Popen() w/o "pipe" throws an exception  (2006-07-21)
CLOSED http://python.org/sf/1526203  opened by  Larry Hastings

Concatenation on a long string breaks  (2006-07-21)
   http://python.org/sf/1526585  opened by  Jp Calderone

current directory added to sys.path on win32  (2006-07-22)
CLOSED http://python.org/sf/1526785  opened by  John Ehresman

unbalanced parentheses  from command line crash pdb  (2006-07-21)
   http://python.org/sf/1526834  opened by  Ilya Sandler

PythonLauncher uses incorrect working directory  (2006-07-23)
   http://python.org/sf/1527397  opened by  Bob Ippolito

optparse should support arbitrary number of arguments  (2006-07-24)
   http://python.org/sf/1527705  reopened by  riteshsarraf

optparse should support arbitrary number of arguments  (2006-07-24)
   http://python.org/sf/1527705  opened by  Ritesh Raj Sarraf

tarfile chokes on ipython archive on Windows  (2006-07-24)
   http://python.org/sf/1527974  opened by  Arve Knudsen

difflib.SequenceMatcher.find_longest_match()  wrong result  (2006-07-25)
   http://python.org/sf/1528074  opened by  John Machin

urllib2 data argument  (2006-07-25)
   http://python.org/sf/1528258  opened by  paul rubin

forward in turtle module may cause incorrect display  (2006-07-25)
   http://python.org/sf/1528363  opened by  NatureMage

IDLE: printing issue on OSX  (2006-07-25)
   http://python.org/sf/1528593  opened by  Ronald Oussoren

Python 2.5b2 fails to build on Solaris 10  (2006-07-25)
   http://python.org/sf/1528620  opened by  Guido Ostkamp

Python 2.5b2 fails to build (GCC) on Solaris 10  (2006-07-26)
   http://python.org/sf/1529269  opened by  Guido Ostkamp

Bugs Closed
___

inspect.py: still infinite recursion inspecting frames  (2006-07-03)
   http://python.org/sf/1516184  closed by  pje

logging using the SysLog handler fails if locale is set  (2006-07-17)
   http://python.org/sf/1524081  closed by  vsajip

Win32: subprocess.Popen() w/o "pipe" throws an exception  (2006-07-21)
   http://python.org/sf/1526203  closed by  gbrandl

Malloc, memory error, failmalloc, low memory.  (2006-07

[Python-Dev] Internal namespace proposal

[David Hopwood]

[This message is cc:d to the e-lang list, but please take any replies to
[EMAIL PROTECTED]

Brett Cannon wrote:
> On 7/19/06, Ka-Ping Yee <[EMAIL PROTECTED]> wrote:
> 
>> OMG!!!  Is all i can say at the moment.  Very excited.

This is very encouraging. Thanks to ?!ng, Michael Chermside and others for 
making
the case for capabilities.

> Also realize that I am using object-capabilities to secure the interpreter,
> not objects.  That will be enough of a challenge to do for now.  Who knows,
> maybe some day Python can support object-capabilities at the object level,
> but for now I am just trying to isolate and protect individual interpreters
> in the same process.

I think that the alternative of providing object-granularity protection domains
straight away is more practical than you suggest, and I'd like to at least make
sure that this possibility has been thoroughly explored.

Below is a first-cut proposal for enforcing namespace restrictions, i.e. support
for non-public attributes and methods, on Python objects and modules. It is not
sufficient by itself to provide capability security, but it could be the
basis for doing that at object granularity.

(Note that this proposal would only affect sandboxed/restricted interpreters,
at least for the time being. The encapsulation it provides is also useful
for reasons other than security, and I think there is nothing about it that
would be unreasonable to apply to an unrestricted interpreter, but for
compatibility, that would have to be enabled by a __future__ option or similar.)


Internal namespace proposal
===

Existing Python code tends to use a convention where the names of attributes
and methods intended only for internal use are prefixed by '_'. This convention
comes from PEP 8 , which says:

#  In addition, the following special forms using leading or trailing
#  underscores are recognized (these can generally be combined with any case
#  convention):
#
#   - _single_leading_underscore: weak "internal use" indicator.  E.g. "from M
# import *" does not import objects whose name starts with an underscore.
#
#   - single_trailing_underscore_: used by convention to avoid conflicts with
# Python keyword, e.g.
#
# Tkinter.Toplevel(master, class_='ClassName')
#
#   - __double_leading_underscore: when naming a class attribute, invokes name
# mangling (inside class FooBar, __boo becomes _FooBar__boo; see below).
#
#   - __double_leading_and_trailing_underscore__: "magic" objects or
# attributes that live in user-controlled namespaces.  E.g. __init__,
# __import__ or __file__.  Never invent such names; only use them
# as documented.

I propose that the "internal" status of names beginning with _ (including
those beginning with __) should be enforced in restricted interpreters. This
is better than introducing a new annotation, because it will do the right
thing for existing code that follows this part of PEP 8.

More precisely:

  A restricted interpreter refuses access to any object attribute or method
  with a name beginning with '_' (by throwing a new exception type
  'InternalAccessException'), unless the access is from a method and its
  static target is that method's first argument variable.

  Also, a restricted interpreter refuses access to any module-global
  variable or module-global function with a name beginning with '_' (by
  throwing 'InternalAccessException'), unless the access is statically from
  the same module.


(A method's first argument is usually called 'self', but that's just a 
convention.
By "static target", I mean that to access an internal attribute _foo in a
method with first argument 'self', you must write "self._foo"; attempting to
access "x._foo" will fail even if 'x' happens to be the same object as 'self'.
This allows such accesses to be reported at compile-time, rather than only at
run-time.)

I am using the term "internal" rather than "private" or "protected", because
these semantics are not the same as either "private" or "protected" in C++ or
Java. In Python with this change, an object can only access its own internal
methods and attributes. In C++ and Java, an object can access private and 
protected
members of other objects of the same class. The rationale for this difference is
explained below.

The use of _single vs __double underscores encodes a useful distinction that
would not change. Ignoring the point in the previous paragraph, a _single
underscore is similar to "protected" in languages like C++ and Java, while a
__double underscore is similar to "private". This is purely a consequence of
the name mangling: if a class X and its subclass Y both name an attribute
__foo, then we will end up with two attributes _X__foo and _Y__foo in instances
of Y, which is the desired behaviour for private attributes. In the case of an
attribute called _foo, OTOH, there can be only one such attribute per object,
which is the desired behaviour fo

Re: [Python-Dev] [Windows, buildbot] kill_python.c mystery

[Neal Norwitz]

On 7/26/06, Tim Peters <[EMAIL PROTECTED]> wrote:
>
> Today I noticed this happened when the buildbot started to run tests,
> and I'm 100% sure it's due to this code in
> Tools/buildbot/kill_python.c (the buildbot log files showed that
> kill_python.c killed /some/ Python process, and the Python running
> release-build tests in my sandbox was the only plausible candidate):
>
> if ((strstr(path, "build\\pcbuild\\python_d.exe") != NULL) ||
> (strstr(path, "build\\python.exe") != NULL)) {
> printf("Terminating %s (pid %d)\n", path, pids[i]);
> if (!TerminateProcess(hProcess, 1)) {
>
> The second clause in the first `if` looks for a substring match on:
>
> build\python.exe
>
> and that just happens to match a suffix of:
>
> C:\Code\python\PCbuild\python.exe
>
> which is the release-build Python I happen to be running in my sandbox.
>
> Why is the second clause there?  That is, are we /trying/ to kill a
> release-build Python running from the user's sandbox, and if so why?

No, I don't believe that was the intent.  The exe on cygwin uses the
unix convention, not the Windows convention for the filename.  ie,
either a debug
or release build on cygwin are both called python.exe.

So the second clause is there to kill the process when it's running
under cygwin.
It's interesting that the process appears to be running as
./python.exe, but build shows up in filename.  From that I deduce that
it must contain the complete path.  Assuming that is true, we can
change the code to ensure that build is a directory since that's what
buildbot does (add the leading \\):

> (strstr(path, "\\build\\python.exe") != NULL)) {

I tested this change with a different path, so I believe it will work
fine and not catch PCbuild.  I'll check in this change and add some
comments.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Release manager pronouncement needed: PEP 302 Fix

[Neal Norwitz]

What is the behaviour that was added which broke compliance?  What is
the benefit of the behaviour?

>From your description of fixing the problem, it seems there's some
risk invovled as it's modiyfing import.c, plus adding new features.
What is your recommendation?

n
--

On 7/26/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> I posted last week about a need-for-speed patch that broke PEP 302
> compliance, and asked if it should be fixed or reverted.  I got exactly one
> response which said "yes, it should be fixed or reverted", which
> unfortunately didn't answer my question as to which one we should do.  :)
>
> If we don't revert it, there are two ways to fix it.  One is to just change
> PEP 302 so that the behavior is unbroken by definition.  :)  The other is
> to actually go ahead and fix it by adding PathImporter and NullImporter
> types to import.c, along with a factory function on sys.path_hooks to
> create them.  (This would've been the PEP-compliant way to implement the
> need-for-speed patch.)
>
> So, "fix" by documentation, fix by fixing, or fix by reverting?  Which
> should it be?
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Windows, buildbot] kill_python.c mystery

[Martin v. Löwis]

Tim Peters wrote:
> Today I noticed this happened when the buildbot started to run tests,
> and I'm 100% sure it's due to this code in
> Tools/buildbot/kill_python.c 

Didn't you know that you signed in to run arbitrary viruses, worms, and
trojan horses when you added your machine to the buildbot infrastructure
:-? You just haven't seen buildbot erasing your hard disk and filling
your coffee machine with tea, yet.

>   (strstr(path, "build\\python.exe") != NULL)) {
> Why is the second clause there?

That's for Cygwin (i.e. Anthony Baxter's machine). As Neal suggests,
preceding the executable path with another backslash should solve
this problem.

As a related note, this entire approach will also manage to kill
python.exe from an unrelated buildbot installation, e.g. a 2.4
build job might kill python.exe from the trunk. This actually helped
when I tried to get the Cygwin slave to get unstuck, and shouldn't
do harm since we currently don't run to builds on the same slave
simultaneously, but could be surprising when parallel builds
are activated some day.

Sorry for messing with your machine,

Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Which version of distutils to ship with Python 2.5?

[Martin v. Löwis]

Collin Winter wrote:
> Is it intentional that Python 2.5 is (currently) shipping with
> distutils 2.4.0, while Python 2.4 (at least 2.4.1, 2.4.2 and 2.4.3)
> shipped with distutils 2.4.1? Judging from my own tests, distutils
> 2.4.1 fixed several bugs that some of my test suites depend on (the
> fixes, not the bugs ; ).

Are these bugs not fixed in the distutils that shipped with Python 2.5b2?

In any case, I bumped the version number to 2.5, according to the policy
discussed in

http://mail.python.org/pipermail/distutils-sig/2005-January/004368.html

Thanks for pointing this out.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com