Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Ronald Oussoren
On 17-nov-2005, at 3:15, Travis Oliphant wrote: > Jim Jewett wrote: > >> > >> (2) Is he allocating new _types_, which I think don't get properly >> >> collected. >> >> > > Bingo. Yes, definitely allocating new _types_ (an awful lot of > them...) > --- that's what the "array scalars" are: new

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread JustFillBug
On 2005-11-16, Travis Oliphant <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: >>I seemed to have misunderstood the discussion. Was the original user >>accessing and saving copies of many millions of these doubles? >> > He *was* accessing them (therefore generating a call to an array-scalar

[Python-Dev] DRAFT: python-dev Summary for 2005-10-16 to 2005-10-31

2005-11-16 Thread Tony Meyer
And this one brings us up-to-date (apart from the fortnight ending yesterday). Again, if you have the time, please send any comments/ corrections to us. Once again thanks to Steve for covering me and getting this all out on his own. = Announcements = --

[Python-Dev] DRAFT: python-dev Summary for 2005-10-01 to 2005-10-15

2005-11-16 Thread Tony Meyer
As you have noticed, there has been a summary delay recently. This is my fault (insert your favourite thesis/work/leisure excuse here). Steve has generously covered my slackness by doing all of the October summaries himself (thanks!). Anyway, if you have some moments to spare, cast your

[Python-Dev] DRAFT: python-dev Summary for 2005-09-16 to 2005-09-30

2005-11-16 Thread Tony Meyer
It's been some time (all that concurrency discussion didn't help ;) but here's the second half of September. Many apologies for the delay; hopefully you agree with Guido's 'better late than never', and I promise to try harder in the future. Note that the delay is all my bad, and epithets

[Python-Dev] Patch Req. # 1351020 & 1351036: PythonD modifications

2005-11-16 Thread decker
Hello, I would appreciate feedback concerning these patches before the next "PythonD" (for DOS/DJGPP) is released. Thanks in advance. Regards, Ben Decker Systems Integrator http://www.caddit.net - Stay ahead of the information curve. Receive MCAD n

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Travis Oliphant
Jim Jewett wrote: >Do you have the code that caused problems? > > Yes. I was able to reproduce his trouble and was trying to debug it. >The things I would check first are > >(1) Is he allocating (peak usage) a type (such as integers) that >never gets returned to the free pool, in case you nee

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread skip
>> [1] There *is* an array type for general PyObjects in scipy_core, but >> that's not being used in the code that blows up and has nothing to do >> with the problem Travis is talking about. Josiah> I seemed to have misunderstood the discussion. I'm sorry, but I'm confused as w

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Neal Norwitz
On 11/16/05, Travis Oliphant <[EMAIL PROTECTED]> wrote: > > As verified by removing usage of the Python PyObject_MALLOC function, it > was the Python memory manager that was performing poorly. Even though > the array-scalar objects were deleted, the memory manager would not > re-use their memory

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Travis Oliphant
Josiah Carlson wrote: >Robert Kern <[EMAIL PROTECTED]> wrote: > > >>[1] There *is* an array type for general PyObjects in scipy_core, but >>that's not being used in the code that blows up and has nothing to do >>with the problem Travis is talking about. >> >> > >I seemed to have misunderstoo

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Josiah Carlson
Robert Kern <[EMAIL PROTECTED]> wrote: > > [1] There *is* an array type for general PyObjects in scipy_core, but > that's not being used in the code that blows up and has nothing to do > with the problem Travis is talking about. I seemed to have misunderstood the discussion. Was the original us

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Robert Kern
Josiah Carlson wrote: > Travis Oliphant <[EMAIL PROTECTED]> wrote: >>I think definitely, his usage pattern represented a "bad" corner case. >>An unusable "corner" case in fact. At any rate, moving to use the >>system free and malloc fixed the immediate problem. I mainly wanted to >>report t

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Josiah Carlson
Travis Oliphant <[EMAIL PROTECTED]> wrote: > > [EMAIL PROTECTED] wrote: > > >Travis> More to the point, however, these scalar objects were allocated > >Travis> using the standard PyObject_New and PyObject_Del functions which > >Travis> of course use the Python memory manager. One us

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Travis Oliphant
[EMAIL PROTECTED] wrote: >Travis> More to the point, however, these scalar objects were allocated >Travis> using the standard PyObject_New and PyObject_Del functions which >Travis> of course use the Python memory manager. One user ported his >Travis> (long-running) code to the new

Re: [Python-Dev] Is some magic required to check out new files from svn?

2005-11-16 Thread Armin Rigo
Hi, On Sun, Nov 13, 2005 at 07:08:15AM -0600, [EMAIL PROTECTED] wrote: > The full svn status output is > > % svn status > ! . > ! Python The "!" definitely mean that these items are missing, or for directories, incomplete in some way. You need to play around until the "!"

[Python-Dev] Conclusion: Event loops, PyOS_InputHook, and Tkinter

2005-11-16 Thread Jim Jewett
Phillip J. Eby: > did you ever try using IPython, and confirm whether it > does or does not address the issue As I understand it, using IPython (or otherwise changing the interactive mode) works fine *if* you just want a point solution -- get something up in some environment chosen by the develop

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Thomas Lee
Just messing around with some ideas. I was trying to avoid the ugly macros (note my earlier whinge about a learning curve) but they're the cleanest way I could think of to get around the problem without resorting to a mass deallocation right at the end of the AST run. Which may not be all that

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Nick Coghlan
Thomas Lee wrote: > As the writer of the crappy code that sparked this conversation, I feel > I should say something :) Don't feel bad about it. It turned out the 'helpful' review comments from Neal and I didn't originally work out very well either ;) With the AST compiler being so new, this is

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Collin Winter
On 11/16/05, Niko Matsakis <[EMAIL PROTECTED]> wrote: > - Another idea is to have trees of arenas: the idea is that when an > arena is created, it is assigned a parent. When an arena is freed, > an arenas in its subtree are also freed. This way you can have one > master arena for exception handli

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Fredrik Lundh
Thomas Lee wrote: > Even if it meant we had just one function call - one, safe function call > that deallocated all the memory allocated within a function - that we > had to put before each and every return, that's better than what we > have. alloca? (duck)

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Thomas Lee
By the way, I liked the sound of the arena/pool tree - really good idea. Thomas Lee wrote: >Niko Matsakis wrote: > > > >>>Boy am I wanting RAII from C++ for automatic freeing when scope is >>>left. Maybe we need to come up with a similar thing, like all memory >>>that should be freed once a sc

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Thomas Lee
Niko Matsakis wrote: >>Boy am I wanting RAII from C++ for automatic freeing when scope is >>left. Maybe we need to come up with a similar thing, like all memory >>that should be freed once a scope is left must use some special struct >>that stores references to all created memory locally and then

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Niko Matsakis
> Boy am I wanting RAII from C++ for automatic freeing when scope is > left. Maybe we need to come up with a similar thing, like all memory > that should be freed once a scope is left must use some special struct > that stores references to all created memory locally and then a free > call must be

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread skip
Travis> More to the point, however, these scalar objects were allocated Travis> using the standard PyObject_New and PyObject_Del functions which Travis> of course use the Python memory manager. One user ported his Travis> (long-running) code to the new scipy core and found much to

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Nick Coghlan
Nick Coghlan wrote: > Marek Baczek BaczyƄski wrote: >> 2005/11/15, Nick Coghlan <[EMAIL PROTECTED]>: >>> It avoids the potential for labelling problems that arises when goto's are >>> used for resource cleanup. It's a far cry from real exception handling, but >>> it's the best solution I've seen wi

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Thomas Lee
As the writer of the crappy code that sparked this conversation, I feel I should say something :) Brett Cannon wrote: >On 11/15/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > > >>On 11/15/05, Jeremy Hylton <[EMAIL PROTECTED]> wrote: >> >> >>>Thanks for the message. I was going to suggest t

Re: [Python-Dev] Memory management in the AST parser & compiler

2005-11-16 Thread Brett Cannon
On 11/15/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > On 11/15/05, Jeremy Hylton <[EMAIL PROTECTED]> wrote: > > > > Thanks for the message. I was going to suggest the same thing. I > > think it's primarily a question of how to add an arena layer. The AST > > phase has a mixture of malloc/free a