|
Hi
I use Activestate Python (2.4.3) in a Windows 32
bit environment.
I have a problem with a large programme that uses a
lot of memory (numerous large arrays which are dynamically extended). I
keep getting unpredictable crashes (on a machine with 1/2 GB memory) whereas on
my laptop (1 GB memory) it seems OK.
The portion of the code which crashes varies a bit
but curiously is NEVER a part where any memory allocation is
happening.
There are no debugger messages it just crashes (and
reboots the machine more often than not).
I have so far tried:
Using the gc module: can't seem to locate any
circular references and garbage collection works OK.
Surrounding all memory allocation code with
Try...Except but no MemoryError ever gets reported.
Running under various versions of Python (standard
distrib from Python Org, Stackless Python etc)
This kind of crash is classically a stack/heap
collision problem (and/or segmentation fault) but if I monitor memory usage
during execution (with Windows Monitor) I usually have 200 or so MB free at the
point of crash.
I have to say I have noticed (the programme is
basically a batch-factoring programme for integers) that no matter how I tune gc
I can't get it to reliably free memory in between factoring each
integer.
Because this is a development programme (and for
performance reasons) I use global variables some of which refer to the large
arrays.
My questions are:
1) Does the mere fact that a
function cites a variable as global create a reference which prevents it being
collected by gc?
2) I notice that in Unix I would
have access to a resource module which might help - is there nothing similar in
Python for Windows?
3) Is there any way to allocate a
pool of available memory to my programme at the outset from which I can allocate
my large arrays?
4) Is there any module / function
which would enable me to access the BYTE size of Python objects such as arrays
etc or to track memory usage from within Python.
5) Does anyone have ANY
suggestions please?
I'm keen to solve this because I would like to make
my programme generally available - in every other respect its near complete and
massively outperforms the only other comparable pure python module (nzmath)
which does the same job.
Thanks in anticipation.
Phil
|
_______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
