That amount of memory does sound way fishy. I'd suggest that you make sure
there are no leaks in your importer. The easiest way to find out if there
are would be to create a script that looks like this:
a = MyImporter(...);
Print(a);
Usage("memory");
Executive("flush dictionary");
Usage("memory");
With those five lines duplicated four or five times in a row. Then
dx -script < filename
You'll get a sequence of lines looking like:
Starting DX executive
Open Visualization Data Explorer
More Info at www.research.ibm.com/dx
and www.opendx.org
Version - 4.2.0
Memory cache will use 100 MB (6 for small items, 94 for large)
0: worker here [5787]
0: PRINT:
Field. 4 components.
0: 104857600 bytes total: 5174104 in use, 99683496 free
0: 104857600 bytes total: 1101272 in use, 103756328 free
0: PRINT:
Field. 4 components.
0: 104857600 bytes total: 5156264 in use, 99701336 free
0: 104857600 bytes total: 1085192 in use, 103772408 free
0: PRINT:
Field. 4 components.
0: 104857600 bytes total: 5140184 in use, 99717416 free
0: 104857600 bytes total: 1069112 in use, 103788488 free
0: PRINT:
Field. 4 components.
0: 104857600 bytes total: 5124104 in use, 99733496 free
0: 104857600 bytes total: 1053032 in use, 103804568 free
0: PRINT:
Field. 4 components.
0: 104857600 bytes total: 5108024 in use, 99749576 free
0: 104857600 bytes total: 1036952 in use, 103820648 free
0: PRINT:
Field. 4 components.
0: 104857600 bytes total: 5091944 in use, 99765656 free
0: 104857600 bytes total: 1020872 in use, 103836728 free
0: PRINT:
Field. 4 components.
0: 104857600 bytes total: 5075864 in use, 99781736 free
0: 104857600 bytes total: 1004792 in use, 103852808 free
0: cleaning up and exiting
You notice that there'll always be a meg or so in use. However, after
that, the first in each pair represents the amount of memory in use before
and after the cache is flushed. The usage will vary a little bit, but it
should not increase substantially.
Greg
"Mike H. Miller"
<[EMAIL PROTECTED]> To:
[email protected]
Sent by: cc:
[EMAIL PROTECTED] Subject: [opendx-users] Memory
Usage
son.ibm.com
08/23/2002 11:26 AM
Please respond to
opendx-users
Hi,
I have a dataset that's about 2 million points (floats), and I'm trying
to figure out why DX is taking up so much memory to process it. I'm
importing it with some custom modules, so maybe my mistake is there...
or maybe I'm missing something else.
When I first read in the (unconnected) dataset, DX uses about 30
megabytes of RAM. After I connect it, memory usage for the dataset
(unstructured hexahedra) is over 380 megabytes, and when I try and
isosurface it, it usually fails with an out of memory error. I have a
gig of memory, but I need to do some other things with it, and I don't
want to give DX 600 meg of memory to handle 2 million data points. I
suppose I'm not deleting/freeing some of the interim objects my module
creates-- but I am free'ing all the malloc arrays, and I can't DXDelete
anything that becomes a component of the final output. Am I missing
something?
--
Mike Miller
[EMAIL PROTECTED] ->
[EMAIL PROTECTED]
#### signature.asc has been removed from this note on August 23 2002 by
Gregory D Abram
