Re: [Python-Dev] Preserving the blamelist

[Tim Peters]

[Neal]
>> ...
>> I'm concerned about the negative ref leak in test_contextlib.  I
>> wonder if this was a result of PJE's fix for generators?

[Tim]
> I don't know, but if you do
>
> while 1:
> test_contextlib.test_main()
> gc.collect()
>
> under a debug build it eventually (> 500 iterations) crashes with
>
> Fatal Python error: deallocating None
> ...

OK, I fixed that (incorrect decref in gen_throw()).

Neal, that should also repair test_contextlib's ref leak oddities. 
The reason you didn't see this before is that test_contextlib didn't
actually run any tests until recently.

Phillip, when eyeballing gen_dealloc(), I didn't understand two things:

1. Why doesn't

if (gen->gi_frame->f_stacktop!=NULL) {

   check first to be sure that gen->gi_frame != Py_None?  Is that impossible
   here for some reason?

2. It _looks_ like "gi_frame != NULL" is an (undocumented) invariant. 
Right?  If so,

Py_XDECREF(gen->gi_frame);

   sends a confusing message (because of the "X", implying that NULL is OK).
   Regardless, it would be good to add comments to genobject.h explaining
   the possible values gi_frame can hold.  For example, what does it mean
   when gi_frame is Py_None?  Can it ever be NULL?  It's very hard to reverse-
   engineer invariants and "special value" intents from staring at code.  Not to
   say that isn't fun ;-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyObject_REPR()

[Martin v. Löwis]

Jeremy Hylton wrote:
> It's intended as an internal debugging API.  I find it very convenient
> for adding with a printf() here and there, which is how it got added
> long ago.  It should really have a comment mentioning that it leaks
> the repr object, and starting with an _ wouldn't be bad either.

I'm still in favour of removing it. If you need it in debugging code,
writing PyObject_AsString(PyObject_Repr(o)) doesn't seem too hard,
IMO. If you write it so often that it hurts, you can put it at the
top of your file (and perhaps also give it a shorter name, such as
P(o)).

Nobody but you seemed to know about its existence, and, given that
it is flawed, neither its definition nor its use belong into Python
(IMO, of course).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Checking assigned bugs/patches

[Neal Norwitz]

Can all developers try to logon to SF and take a look at the
bugs/patches assigned to you?  Sometimes we were assigned items and we
may not have been notified (or forgot that we have outstanding items).
 This link should take you to your page:

  http://sourceforge.net/my/tracker.php

If you aren't going to work on the items assigned to you, try to find
someone else to work on it or at least unassign the item.  Now would
be a great time for volunteers to try to help fix some bugs as we try
to stabilize 2.5.

I suppose now is the time to admit I've got 12 bugs and 7 patches
assigned to me. :-(

Thanks,
n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Preserving the blamelist

[Martin v. Löwis]

Tim Peters wrote:
> All the trunk buildbots started failing about 5 hours ago, in
> test_parser.  There have been enough checkins since then that the
> boundary between passing and failing is about to scroll off forever. 

It's not lost, though; it's just not displayed anymore. It would be
possible to lengthen the waterfall from 12 hours to a larger period
of time.

> While I'm spreading guilt, a bunch of filepath-related tests started
> failing Monday on the "g4 osx.4 trunk" buildbot, but the blamelist for
> that one is long gone.

You can go back in time yourself, by editing the build number in

http://www.python.org/dev/buildbot/trunk/g4%20osx.4%20trunk/builds/361

The builds from Monday are around 330. The first failure of test_filecmp
seems to have been in build 341, which has Georg (45259) and
Anthony (45261) on the blame list.

The failures might have nothing to with the changes, though: It appears
that some files are still left from earlier tests, and now the setUp
code fails because /tmp/dir already exists...

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Preserving the blamelist

[Neal Norwitz]

On 4/12/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
>
> The failures might have nothing to with the changes, though: It appears
> that some files are still left from earlier tests, and now the setUp
> code fails because /tmp/dir already exists...

That was my guess.  I went in and cleaned up all the stray files I
could find.  I was able to run the tests manually and they passed.  I
just forced a build and we'll see if they all pass this time.  At
least some people will, I'm going to sleep.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: expose PEP 302 facilities via 'imp' and 'pkgutil'

[Nick Coghlan]

Phillip J. Eby wrote:
> At 07:43 AM 4/12/2006 +1000, Nick Coghlan wrote:
>> Phillip J. Eby wrote:
>> > I propose to create a new API, 'imp.find_loader()' and have it 
>> return a PEP
>> > 302-compatible loader object, even for cases that would normally be 
>> handled
>> > via 'imp.load_module()'.  In such cases, the loader returned would 
>> be an
>> > instance of one of a loader class similar to those in runpy,
>> > test_importhooks, and setuptools (which also has similar code).
>>
>> runpy tries to retrieve "imp.get_loader()" before falling back to its own
>> emulation - switching that to look for "find_loader()" instead would 
>> be fine
>> (although I'd probably leave the emulation in place, since it allows the
>> module to be used on 2.4 as well as 2.5).
> 
> I was proposing to move the emulation features (at least the classes) to 
> pkgutil, or in the alternative, making them a 
> published/documented/supported API, rather than private.

Either works for me - I managed to momentarily forget that you're planning to 
write this in Python.

>> Would it be worth the effort to switch 'imp' to being a hybrid module?
> 
> I have no idea; what's a hybrid module?

In this particular case, it would be a Python module "imp.py" that included 
the line "from _imp import *" (where _imp is the current C-only module with a 
different name).

The Python module could then provide access to the bits which don't need to be 
blazingly fast and would be painful to write in C. It's proved to be a 
convenient approach in a few other places.

>> runpy needs a get_filename() method, so it knows what to set __file__ 
>> too -
>> currently its emulation supports that, but it isn't officially part of 
>> the PEP
>> 302 API.
> 
> It sounds like maybe a new PEP is needed to document all the extensions 
> to the importer/loader protocols.  :(
> 
> Interestingly, these are all more examples of where adaptation or 
> generic functions would be handy to have in the stdlib.  Duck-typing 
> protocols based on attribute names are hard to extend and have to wait 
> for new library releases for upgrades.  :(

Yep. For example, one limitation of runpy.py is that it doesn't get the 
__file__ attribute right for module's inside a zip package - the zipimporter 
objects don't expose the information that run_module() needs in order to set 
the value correctly.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] String initialization (was: The "i" string-prefix: I18n'ed strings)

[Jason Orendorff]

A compiler hook on string initialization, eh?

I have a distantly related story--this isn't important, just another
random Python use case for the file.  (The i"xyzzy" proposal wouldn't
help this case.)

In scons, your SConscripts (makefiles, essentially) are Python source
code.  You typically have SConscripts throughout your source tree. 
Any SConscript could have something like this:

  sort_exe = Program('sort', ['main.c', 'timsort.c'])

The problem is dealing with relative filenames.  The only sane way to
resolve "main.c" to an abspath is relative to the source file that
physically contains that string literal token.[1]  But that's
impossible to determine at run time.

scons uses some cleverness to guess the directory.  It's always right,
except when it's wrong.  Maddening.

So, what does this have to do with string initialization hooks?  If
scons could "decorate" string constants as part of SConscript
compilation/execution, this problem could actually be solved.

-j

[1] Well, this is my opinion, but it's the right one.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] building with C++

[skip]

Anthony> I've done a lot of the work to get Python to build with g++ -
Anthony> ... I can't spend much more time on this now, but if someone
Anthony> wants to take over and finish it off, that'd be great.

Is this on a branch or available as a patch somewhere?

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] building with C++

[Martin v. Löwis]

[EMAIL PROTECTED] wrote:
> Anthony> I've done a lot of the work to get Python to build with g++ -
> Anthony> ... I can't spend much more time on this now, but if someone
> Anthony> wants to take over and finish it off, that'd be great.
> 
> Is this on a branch or available as a patch somewhere?

It's the trunk.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Preserving the blamelist

[Phillip J. Eby]

At 03:00 AM 4/12/2006 -0400, Tim Peters wrote:
>Phillip, when eyeballing gen_dealloc(), I didn't understand two things:
>
>1. Why doesn't
>
> if (gen->gi_frame->f_stacktop!=NULL) {
>
>check first to be sure that gen->gi_frame != Py_None?

Apparently, it's because I'm an idiot, and because nobody else realized 
this during the initial review of the patch.  :)


>Is that impossible here for some reason?

No, I goofed.  It's amazing that this doesn't dump core whenever a 
generator exits.  :(



>2. It _looks_ like "gi_frame != NULL" is an (undocumented) invariant.
>Right?  If so,
>
> Py_XDECREF(gen->gi_frame);
>
>sends a confusing message (because of the "X", implying that NULL is OK).
>Regardless, it would be good to add comments to genobject.h explaining
>the possible values gi_frame can hold.  For example, what does it mean
>when gi_frame is Py_None?  Can it ever be NULL?

I think what happened is that at one point I thought I was going to set 
gi_frame=NULL when there's no active frame, in order to speed up 
reclamation of the frame.  However, I think I then thought that it would 
break the operation of the generator's 'gi_frame' attribute, which is 
defined as T_OBJECT.  Or else I thought that the tp_visit was screwed up in 
that case.

So, it looks to me like what this *should* do is simply allow gi_frame to 
be NULL instead of Py_None, which would get rid of all the silly casting, 
and retroactively make the XDECREF correct.  :)

Does gen_traverse() need to do anything special to visit a null 
pointer?  Should it only conditionally visit it?

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: expose PEP 302 facilities via 'imp' and 'pkgutil'

[Phillip J. Eby]

At 08:39 PM 4/12/2006 +1000, Nick Coghlan wrote:
>In this particular case, it would be a Python module "imp.py" that 
>included the line "from _imp import *" (where _imp is the current C-only 
>module with a different name).

I don't think that's going to work, since importing won't work until _imp 
itself is imported...  :)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] refleaks in 2.4

[Armin Rigo]

Hi all,

On Sun, Mar 26, 2006 at 11:39:50PM -0800, Neal Norwitz wrote:
> There are 5 tests that leak references that are present in 2.4.3c1,
> but not on HEAD.  It would be great if someone can diagnose these and
> suggest a fix.
> 
> test_doctest leaked [1, 1, 1] references
> test_pkg leaked [10, 10, 10] references
> test_pkgimport leaked [2, 2, 2] references
> test_traceback leaked [11, 11, 11] references
> test_unicode leaked [7, 7, 7] references
> 
> test_traceback leaks due to test_bug737473.

A follow-up on this: all the tests apart from test_traceback are due to
the dummy object in dictionaries.  I modified the code to ignore exactly
these references and all the 4 other tests no longer leak.  I'm about to
check in this nice time saver :-)

For information, the 2.5 HEAD now reports the following remaining leaks:

test_generators leaked [1, 1, 1, 1] references
test_threadedtempfile leaked [-85, 85, -85, 85] references
test_threading_local leaked [34, 40, 26, 28] references
test_urllib2 leaked [-66, 143, -77, -66] references


A bientot,

Armin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Preserving the blamelist

[Tim Peters]

[Tim ]
>> I'd whine about not checking buildbot health after a code change,
>> except that it's much more tempting to point out that Thomas couldn't
>> have run the test suite successfully on his own box in this case :-)

[Thomas]
> Not with -uall, yes. Without -uall it ran fine (by chance, I admit.)

That shouldn't be:  test_parser doesn't require (or check for) any
resources.  test_tokenize is the one that runs a random sample in the
absence of `-u compiler`, and that one was passing even with -uall.

> I didn't feel like running the rather lengthy -uall test on my box right
> before bedtime; I won't be skipping that again anytime soon ;)

It sounds like you're letting your life interfere with Python
development.  Stop that ;-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Preserving the blamelist

[Tim Peters]

[Tim]
>> Phillip, when eyeballing gen_dealloc(), I didn't understand two things:
>>
>> 1. Why doesn't
>>
>> if (gen->gi_frame->f_stacktop!=NULL) {
>>
>>check first to be sure that gen->gi_frame != Py_None?

[Phillip]
> Apparently, it's because I'm an idiot, and because nobody else realized
> this during the initial review of the patch.  :)

Then you were the bold adventurer, and the reviewers were the idiots ;-)

>> Is that impossible here for some reason?

> No, I goofed.  It's amazing that this doesn't dump core whenever a
> generator exits.  :(

Well, it's extremely likely that &((PyGenObject *)Py_None)->f_stacktop
is a legit part of the process address space, so that dereferencing is
non-problematic. We just read up nonsense then, test against it, and
the conditional

gen->ob_type->tp_del(self);

is harmless as gen_del() returns at once (because it does check for
Py_None right away -- we never try to treat gi_frame->f_stacktop _as_
a frame pointer here, we just check it against NULL).  If we have to
have insane code, harmlessly insane is the best kind :--)

>> 2. It _looks_ like "gi_frame != NULL" is an (undocumented) invariant.
>>Right?  If so,
>>
>> Py_XDECREF(gen->gi_frame);
>>
>>sends a confusing message (because of the "X", implying that NULL is OK).
>>Regardless, it would be good to add comments to genobject.h explaining
>>the possible values gi_frame can hold.  For example, what does it mean
>>when gi_frame is Py_None?  Can it ever be NULL?

> I think what happened is that at one point I thought I was going to set
> gi_frame=NULL when there's no active frame, in order to speed up
> reclamation of the frame.  However, I think I then thought that it would
> break the operation of the generator's 'gi_frame' attribute, which is
> defined as T_OBJECT.

That shouldn't be a problem:  when PyMember_GetOne() fetches a
T_OBJECT that's NULL, it returns (a properly incref'ed) Py_None
instead.

> Or else I thought that the tp_visit was screwed up in that case.
>
> So, it looks to me like what this *should* do is simply allow gi_frame to
> be NULL instead of Py_None, which would get rid of all the silly casting,
> and retroactively make the XDECREF correct.  :)

That indeed sounds better to me, although you still don't get out of
writing gi_frame comments for genobject.h :-)

> Does gen_traverse() need to do anything special to visit a null
> pointer?  Should it only conditionally visit it?

The body of gen_traverse() is best written:

Py_VISIT(gen->gi_frame);
return 0;

Py_VISIT is defined in objimpl.h, and takes care of NULLs and
exceptional returns.

BTW, someone looking for an easy task might enjoy rewriting other
tp_traverse slots to use Py_VISIT.  We even have cases now (like
super_traverse) where modules define their own workalike
traverse-visit macros, which has become confusing since a standard
macro was defined for this purpose.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Request for review

[Seo Sanghyeon]

Can someone have a look at #860326? I got bitten by it today, and I can
see no reason not to apply suggested patch.

Thanks!

Seo Sanghyeon
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Preserving the blamelist

[A.M. Kuchling]

On Wed, Apr 12, 2006 at 01:43:43PM -0400, Tim Peters wrote:
> BTW, someone looking for an easy task might enjoy rewriting other
> tp_traverse slots to use Py_VISIT.  We even have cases now (like
> super_traverse) where modules define their own workalike
> traverse-visit macros, which has become confusing since a standard
> macro was defined for this purpose.

Do we need a list of CPython cleanup projects?
(Compare to the Linux kernel janitors:  
.)

--amk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Preserving the blamelist

[Neal Norwitz]

On 4/12/06, A.M. Kuchling <[EMAIL PROTECTED]> wrote:
> On Wed, Apr 12, 2006 at 01:43:43PM -0400, Tim Peters wrote:
> > BTW, someone looking for an easy task might enjoy rewriting other
> > tp_traverse slots to use Py_VISIT.  We even have cases now (like
> > super_traverse) where modules define their own workalike
> > traverse-visit macros, which has become confusing since a standard
> > macro was defined for this purpose.
>
> Do we need a list of CPython cleanup projects?
> (Compare to the Linux kernel janitors:
> .)

+1.  Could add a page to the wiki and link to it from the dev faq.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] cleanup list

[Jim Jewett]

On Wed, Apr 12, 2006 at 01:43:43PM -0400, Tim Peters wrote:
> BTW, someone looking for an easy task might enjoy rewriting other
> tp_traverse slots to use Py_VISIT.  We even have cases now (like
> super_traverse) where modules define their own workalike
> traverse-visit macros, which has become confusing since a standard
> macro was defined for this purpose.

A.M. Kuchling amk asked:

> Do we need a list of CPython cleanup projects?
> (Compare to the Linux kernel janitors:
> .)

It would probably be useful, if there were someone ready to verify and
commit the cleanups.

A similar list for python modules might be even more useful; I doubt
I'm the only one often without a working C setup.  (Common on windows,
I've seen it even on unix boxes that did have python preinstalled.)

-jJ
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] need info for externally maintained modules PEP

[Gregory P. Smith]

On Sat, Apr 08, 2006 at 02:47:28PM -0700, Brett Cannon wrote:
> OK, I am going to write the PEP I proposed a week or so ago, listing
> all modules and packages within the stdlib that are maintained
> externally so we have a central place to go for contact info or where
> to report bugs on issues.  This should only apply to modules that want
> bugs reported outside of the Python tracker and have a separate dev
> track.  People who just use the Python repository as their mainline
> version can just be left out.
> 
> For each package I need the name of the module, the name of the
> maintainer, homepage of the module outside of Python, and where to
> report bugs.  Do people think we need to document the version that has
> been imported into Python and when that was done as well?
> 
> Anyway, here is a list of the packages that I think have outside
> maintenance (or at least have been at some point).  Anyone who has
> info on them that I need, please let me know the details.  Also, if I
> missed any, obviously speak up:
> 
> - bsddb (still external?)

the python repository code is the mainline version.

http://pybsddb.sf.net/

feel free to list me as the maintainer, but i'd prefer
[EMAIL PROTECTED] be listed as the contact.

the pybsddb project exists only to supply a more up to date module to
people on older pythons or to supply new versions of BerkeleyDB to
people on current python.

bug reports sometimes do go to the pybsddb bug+patch tracker and
mailing list where I eventually take care of them.  i'd be happier
with them going to the python tracker (some have) but haven't
attempted to make that the only way.  using the python tracker means
that i'm not on the critical path to look at a bsddb bug report.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for review

[Georg Brandl]

Seo Sanghyeon wrote:
> Can someone have a look at #860326? I got bitten by it today, and I can
> see no reason not to apply suggested patch.

I've reviewed it and checked it in.

Cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for review

[Georg Brandl]

Georg Brandl wrote:
> Seo Sanghyeon wrote:
>> Can someone have a look at #860326? I got bitten by it today, and I can
>> see no reason not to apply suggested patch.
> 
> I've reviewed it and checked it in.

Hm. This broke a few doctests. I can fix them, but I wonder if
doctest should accept a bare exception name if the exception
is defined in the current module. Or should it ignore the module
name altogether?

(Background:

In normal exception tracebacks, non-builtin exceptions are printed with
their module name prepended:

Traceback:
[...]
decimal.InvalidOperation: ...

When formatted by traceback.format_exception_only, the module name was
omitted, which the patch mentioned above corrected. Since doctest relies
on that behavior, three stdlib doctests broke.)

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: expose PEP 302 facilities via 'imp' and 'pkgutil'

[Nick Coghlan]

Phillip J. Eby wrote:
> At 08:39 PM 4/12/2006 +1000, Nick Coghlan wrote:
>> In this particular case, it would be a Python module "imp.py" that 
>> included the line "from _imp import *" (where _imp is the current 
>> C-only module with a different name).
> 
> I don't think that's going to work, since importing won't work until 
> _imp itself is imported...  :)
> 
> 
Details, details. . .

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] building with C++

[Skip Montanaro]

> > Anthony> I've done a lot of the work to get Python to build with g++ -

> > Is this on a branch or available as a patch somewhere?
> 
> It's the trunk.

Is there a primer that will get me to where Anthony is?  I tried the obvious

CC=g++ ./configure --with-cxx=g++

and the build fails trying to compile Objects/genobject.c.  From the sounds
of Anthony's email he was at the point where it built and was having test
problems.

Skip


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] building with C++

[Anthony Baxter]

On Thursday 13 April 2006 10:59, Skip Montanaro wrote:
> > > Anthony> I've done a lot of the work to get Python to build
> > > with g++ -
> > >
> > > Is this on a branch or available as a patch somewhere?
> >
> > It's the trunk.
>
> Is there a primer that will get me to where Anthony is?  I tried
> the obvious
>
> CC=g++ ./configure --with-cxx=g++

That's what I've been doing.

> and the build fails trying to compile Objects/genobject.c.  From
> the sounds of Anthony's email he was at the point where it built
> and was having test problems.

The genobject.c error is new - I just fixed it, it was shallow.

The code is _nearly_ building fine. there's an issue in _sre.c with 
some code that either returns a Py_UNICODE* or an SRE_CHAR* (unsigned 
char*) in a void*. The code probably needs a refactoring to deal with 
that. There's also 
Python/compile.c: In function ‘int compiler_compare(compiler*, 
_expr*)’:
Python/compile.c:3065: error: invalid cast from type ‘void*’ to type 
‘cmpop_ty’
Python/compile.c:3075: error: invalid cast from type ‘void*’ to type 
‘cmpop_ty’
which I haven't looked at yet. Anyone else is welcome to fix these.
To get past those two, I've been building just those two files with 
gcc, with "make CC=gcc Python/compile.o Modules/_sre.o"



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r45321 - in python/trunk: Lib/test/test_traceback.py Lib/traceback.py Misc/NEWS

[Tim Peters]

[georg.brandl]
> Author: georg.brandl
> Date: Wed Apr 12 23:14:09 2006
> New Revision: 45321
>
> Modified:
>python/trunk/Lib/test/test_traceback.py
>python/trunk/Lib/traceback.py
>python/trunk/Misc/NEWS
> Log:
> Patch #860326: traceback.format_exception_only() now prepends the
> exception's module name to non-builtin exceptions, like the interpreter
> itself does.

And all the trunk buildbot runs have failed since, in at least
test_decimal, test_doctest and test_unpack.  Please run tests before
checking in.

The 2.4 backport of this patch should be reverted, since it changes
visible behavior (for example, all the 2.4 branch buildbot runs also
fail now).

Fine by me if we change the failing tests on the trunk to pass (noting
that should have been done before checking in).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r45321 - in python/trunk: Lib/test/test_traceback.py Lib/traceback.py Misc/NEWS

[Phillip J. Eby]

At 09:14 PM 4/12/2006 -0400, Tim Peters wrote:
>The 2.4 backport of this patch should be reverted, since it changes
>visible behavior (for example, all the 2.4 branch buildbot runs also
>fail now).
>
>Fine by me if we change the failing tests on the trunk to pass (noting
>that should have been done before checking in).

It's not fine by me if the reason for the failure is that doctests trapping 
specific exceptions no longer work with the patch.

If that's the case, the patch should be reverted entirely, absent any 
compelling reasoning for breaking *everyone else's* doctests in 2.5.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for review

[Tim Peters]

[Georg Brandl]
> Hm. This broke a few doctests. I can fix them, but I wonder if
> doctest should accept a bare exception name if the exception
> is defined in the current module.

No.

> Or should it ignore the module name altogether?

No.  doctest strives to be magic-free WYSIWYG.  If someone _intends_
the module name to be optional in a doctest, they should explicitly
use doctest's ELLIPSIS option.

> (Background:
>
> In normal exception tracebacks, non-builtin exceptions are printed with
> their module name prepended:
>
> Traceback:
> [...]
> decimal.InvalidOperation: ...
>
> When formatted by traceback.format_exception_only, the module name was
> omitted, which the patch mentioned above corrected. Since doctest relies
> on that behavior, three stdlib doctests broke.)

Changes to visible behavior should not be introduced in bugfix
releases, unless that happens as an unavoidable consequence of
repairing a critical bug, so this should be yanked from 2.4.  I agree
the traceback formatting inconsistency was a bug, but it was hardly
critical (for example, nobody noticed it for 15 years <0.5 wink>).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r45321 - in python/trunk: Lib/test/test_traceback.py Lib/traceback.py Misc/NEWS

[Anthony Baxter]

On Thursday 13 April 2006 11:14, Tim Peters wrote:
> > Patch #860326: traceback.format_exception_only() now prepends the
> > exception's module name to non-builtin exceptions, like the
> > interpreter itself does.
>
> And all the trunk buildbot runs have failed since, in at least
> test_decimal, test_doctest and test_unpack.  Please run tests
> before checking in.
>
> The 2.4 backport of this patch should be reverted, since it changes
> visible behavior (for example, all the 2.4 branch buildbot runs
> also fail now).

Correct. I have reverted this on the release24-maint branch.

> Fine by me if we change the failing tests on the trunk to pass
> (noting that should have been done before checking in).

I'm reverting on the trunk, too. Per PJE's email as well, I think 
this needs discussion before committing (and it needs the tests 
updated, first!)

Anthony
-- 
Anthony Baxter <[EMAIL PROTECTED]>
It's never too late to have a happy childhood.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r45321 - in python/trunk: Lib/test/test_traceback.py Lib/traceback.py Misc/NEWS

[Tim Peters]

[Tim]
>> ...
>> Fine by me if we change the failing tests on the trunk to pass (noting
>> that should have been done before checking in).

[Phillip]
> It's not fine by me if the reason for the failure is that doctests trapping
> specific exceptions no longer work with the patch.>
> If that's the case, the patch should be reverted entirely, absent any
> compelling reasoning for breaking *everyone else's* doctests in 2.5.

I agree with the patch author's position that "it's a bug" that the
traceback module's functions' output doesn't match the interpreter's
traceback output.  That certainly wasn't intended; it's most likely
left over from the days when only strings could be exceptions, and
nobody noticed that the old traceback-module code "didn't work exactly
right" anymore when the notion of exceptions was generalized.

WRT doctests, it's a mixed bag.  Yes, some tests will fail.  doctests
are always vulnerable to changes "like this".  OTOH, the change also
simplifies writing new doctests.  I remember being baffled at doing
things like:

>>> import decimal
>>> 1 / decimal.Decimal(0)
Traceback (most recent call last):
  File "", line 1, in ?
  File "C:\Python24\lib\decimal.py", line 1314, in __rdiv__
return other.__div__(self, context=context)
  File "C:\Python24\lib\decimal.py", line 1140, in __div__
return self._divide(other, context=context)
  File "C:\Python24\lib\decimal.py", line 1222, in _divide
return context._raise_error(DivisionByZero, 'x / 0', sign)
  File "C:\Python24\lib\decimal.py", line 2267, in _raise_error
raise error, explanation
decimal.DivisionByZero: x / 0

in an interactive shell, pasting it into a doctest, and then seeing
the doctest fail because the exception name "somehow" magically
changed to just "DivisionByZero".  I got used to that before I found
time to understand it, though.  Future generations wouldn't suffer
this cognitive dissonance.

A less-desirable (IMO) alternative is to change the Python core to
display bare (non-dotted) exception names when it produces a
traceback.  The fundamental bug here is that the core and the
traceback module produce different tracebacks given the same exception
info, and changing either could repair that.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] building with C++

[Jeremy Hylton]

On 4/12/06, Anthony Baxter <[EMAIL PROTECTED]> wrote:
> The code is _nearly_ building fine. there's an issue in _sre.c with
> some code that either returns a Py_UNICODE* or an SRE_CHAR* (unsigned
> char*) in a void*. The code probably needs a refactoring to deal with
> that. There's also
> Python/compile.c: In function 'int compiler_compare(compiler*,
> _expr*)':
> Python/compile.c:3065: error: invalid cast from type 'void*' to type
> 'cmpop_ty'
> Python/compile.c:3075: error: invalid cast from type 'void*' to type
> 'cmpop_ty'
> which I haven't looked at yet. Anyone else is welcome to fix these.

The code in compile.c is pretty dodgy.  I'd like to think of a better
way to represent an array of cmpop_ty objects than casting ints to
void* and then back.

Jeremy

> To get past those two, I've been building just those two files with
> gcc, with "make CC=gcc Python/compile.o Modules/_sre.o"
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] building with C++

[Anthony Baxter]

On Thursday 13 April 2006 12:03, Jeremy Hylton wrote:
> The code in compile.c is pretty dodgy.  I'd like to think of a
> better way to represent an array of cmpop_ty objects than casting
> ints to void* and then back.

Well, it's compiling and working with C++ now, at least. But yeah, 
that's pretty nasty. 

Anthony
-- 
Anthony Baxter <[EMAIL PROTECTED]>
It's never too late to have a happy childhood.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] building with C++

[Anthony Baxter]

So I lied - I found the time today to spread the magic pixie dust of 
extern "C" {} around to get the Python core to build and link with 
g++. (It still builds with gcc). There's still an issue with 
Modules/_sre.c, and now that we can run setup.py, there's lots and 
lots of errors from the various code in Modules that's not C++ safe.

Anthony
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] building with C++

[skip]

Jeremy> The code in compile.c is pretty dodgy.  I'd like to think of a
Jeremy> better way to represent an array of cmpop_ty objects than
Jeremy> casting ints to void* and then back.

http://python.org/sf/1469594

might be a step in the right direction.

I assigned it to Anthony.  Feel free to assign it back to me.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] building with C++

[Jeremy Hylton]

Looks good to me.  Why don't you check it in.

Jeremy

On 4/12/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
> Jeremy> The code in compile.c is pretty dodgy.  I'd like to think of a
> Jeremy> better way to represent an array of cmpop_ty objects than
> Jeremy> casting ints to void* and then back.
>
> http://python.org/sf/1469594
>
> might be a step in the right direction.
>
> I assigned it to Anthony.  Feel free to assign it back to me.
>
> Skip
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r45321 - in python/trunk: Lib/test/test_traceback.py Lib/traceback.py Misc/NEWS

[Greg Ewing]

Tim Peters wrote:

> A less-desirable (IMO) alternative is to change the Python core to
> display bare (non-dotted) exception names when it produces a
> traceback.

Much less desirable, considering that there are modules
which define an exception just called "error" on the
assumption that you're going to see the module name
as well.

-- 
Greg Ewing, Computer Science Dept, +--+
University of Canterbury,  | Carpe post meridiam! |
Christchurch, New Zealand  | (I'm not a morning person.)  |
[EMAIL PROTECTED]  +--+
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] TODO Wiki (was: Preserving the blamelist)

[Tim Peters]

[A.M. Kuchling]
>> Do we need a list of CPython cleanup projects?
>> (Compare to the Linux kernel janitors:
>> .)

[Neal Norwitz]
> +1.  Could add a page to the wiki and link to it from the dev faq.

Not the same thing, but I just added:

http://wiki.python.org/moin/SimpleTodo

and linked to it from

http://wiki.python.org/moin/CodingProjectIdeas

and populated it with the Py_VISIT task.  Feel free to add MessyTodo
and FantasyTodo pages too ;-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] TODO Wiki (was: Preserving the blamelist)

[Neal Norwitz]

On 4/12/06, Tim Peters <[EMAIL PROTECTED]> wrote:
> [A.M. Kuchling]
> >> Do we need a list of CPython cleanup projects?
> >> (Compare to the Linux kernel janitors:
> >> .)
>
> [Neal Norwitz]
> > +1.  Could add a page to the wiki and link to it from the dev faq.
>
> Not the same thing, but I just added:
>
> http://wiki.python.org/moin/SimpleTodo
>
> and linked to it from
>
> http://wiki.python.org/moin/CodingProjectIdeas
>
> and populated it with the Py_VISIT task.  Feel free to add MessyTodo
> and FantasyTodo pages too ;-)

Cool, now can we also get a SummerOfCodeTodo? :-)

Google should be doing another Summer Of Code soon.  We should start
thinking about any projects we would like to see done.  Also, we will
need to find mentors.  I hope everyone who was a mentor last year will
mentor again.  I plan to mentor and I'll find a way to rope Guido in
too. :-)

Cheers,
n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode vs buffer (array) design issue can crash interpreter

[Neal Norwitz]

On 3/31/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote:
> Martin v. Löwis wrote:
> > Neal Norwitz wrote:
> >> See http://python.org/sf/1454485 for the gory details.  Basically if
> >> you create a unicode array (array.array('u')) and try to append an
> >> 8-bit string (ie, not unicode), you can crash the interpreter.
> >>
> >> The problem is that the string is converted without question to a
> >> unicode buffer.  Within unicode, it assumes the data to be valid, but
> >> this isn't necessarily the case.  We wind up accessing an array with a
> >> negative index and boom.
> >
> > There are several problems combined here, which might need discussion:
> >
> > - why does the 'u#' converter use the buffer interface if available?
> >   it should just support Unicode objects. The buffer object makes
> >   no promise that the buffer actually is meaningful UCS-2/UCS-4, so
> >   u# shouldn't guess that it is.
> >   (FWIW, it currently truncates the buffer size to the next-smaller
> >multiple of sizeof(Py_UNICODE), and silently so)
> >
> >   I think that part should just go: u# should be restricted to unicode
> >   objects.
>
> 'u#' is intended to match 's#' which also uses the buffer
> interface. It expects the buffer returned by the object
> to a be a Py_UNICODE* buffer, hence the calculation of the
> length.
>
> However, we already have 'es#' which is a lot safer to use
> in this respect: you can explicity define the encoding you
> want to see, e.g. 'unicode-internal' and the associated
> codec also takes care of range checks, etc.
>
> So, I'm +1 on restricting 'u#' to Unicode objects.

Note:  2.5 no longer crashes, 2.4 does.

Does this mean you would like to see this patch checked in to 2.5? 
What should we do about 2.4?

Index: Python/getargs.c
===
--- Python/getargs.c(revision 45333)
+++ Python/getargs.c(working copy)
@@ -1042,11 +1042,8 @@
STORE_SIZE(PyUnicode_GET_SIZE(arg));
}
else {
-   char *buf;
-   Py_ssize_t count = convertbuffer(arg, p, &buf);
-   if (count < 0)
-   return converterr(buf, arg, msgbuf, bufsize);
-   STORE_SIZE(count/(sizeof(Py_UNICODE)));
+   return converterr("cannot convert raw buffers"",
+ arg, msgbuf, bufsize);
}
format++;
} else {

> > - should Python guarantee that all characters in a Unicode object
> >   are between 0 and sys.maxunicode? Currently, it is possible to
> >   create Unicode strings with either negative or very large Py_UNICODE
> >   elements.
> >
> > - if the answer to the last question is no (i.e. if it is intentional
> >   that a unicode object can contain arbitrary Py_UNICODE values): should
> >   Python then guarantee that Py_UNICODE is an unsigned type?
>
> Py_UNICODE must always be unsigned. The whole implementation
> relies on this and has been designed with this in mind (see
> PEP 100). AFAICT, the configure does check that Py_UNICODE
> is always unsigned.

Martin fixed the crashing problem in 2.5 by making wchar_t unsigned
which was a bug.  (A configure test was reversed IIRC.)  Can this
change to wchar_t be made in 2.4?  That technically changes all the
interfaces even though it was a mistake.  What should be done for 2.4?

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] int vs ssize_t in unicode

[Neal Norwitz]

Martin,

In Include/ucnhash.h I notice some integers and wonder if those should
be Py_ssize_t.  It looks like they are just names so they should be
pretty short.

But in Objects/unicodeobject.c, I notice a bunch of ints and casts to
int and wonder if they should be changed to Py_ssize_t/removed:

235:
assert(lengthlength = (int)length;

376, 404:
  int i;


1366: (seems like this could be a 64-bit value)
  int nneeded;

(i stopped at this point, there are probably more)


Modules/unicodedata.c (lots of ints, not sure if they are a problem)

494:
isize = PyUnicode_GET_SIZE(input);

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] int vs ssize_t in unicode

[Martin v. Löwis]

Neal Norwitz wrote:
> In Include/ucnhash.h I notice some integers and wonder if those should
> be Py_ssize_t.  It looks like they are just names so they should be
> pretty short.

Right: int size is below 100; the name length of a Unicode character
is below 40 (I believe). OTOH, changing them to Py_ssize_t wouldn't
hurt, either.

> 235:
> assert(length unicode->length = (int)length;

Right: I just changed it. It may date back to a version of the patch
where I only changed the signatures of the functions, but not the
object layout.

> 376, 404:
>   int i;

Right, changed.

> 1366: (seems like this could be a 64-bit value)
>   int nneeded;

Right; also, the safe downcast can go away.

> (i stopped at this point, there are probably more)

I looked through the entire file, and fixed all I could find.

> Modules/unicodedata.c (lots of ints, not sure if they are a problem)

Most of them are not, since most functions deal only with Unicode
characters, character names, and character numbers (which are below 2**21).

> 494:
> isize = PyUnicode_GET_SIZE(input);

This is a problem, though; I fixed it.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com