Mike,
You should not have to cache anything explicitly here, it should be
handled by OpenDX' standard caching (assuming you haven't disabled it via
either the -cache off command line argument or the options in the module's
configuration dialog box). If you run the net with a parameter=1, then
all partial results computed for that set of inputs will be cached. If
you go to parameter=2, then the parts of the network that are dependent on
that parameter will be recalculated, resulting in more memory use. If you
run out, it'll throw away the partial results from the first execution, buf
if not, and you return to parameter =1, then partial results will be
available and no further memory will be used. If thats not the case,
there's a problem somewhere else.
With regard to the second question, think ownership. Here are the rules:
When you create an object - eg. DXNewField or DXCopy, you receive a
(fake) ownership of it - see below.
When you add an object to another - eg. DXSetComponentValue or
DXSetEnumeratedMember, the parent object receives an ownership of it
When you DXDelete an object, the object's ownership count is
decremented. If its the only ownership of that object, the object is
really deleted. In that case, every object owned by the object being
deleted is itself DXDeleted, and the process recurs.
When an object is the output of a module - eg. out[k], the ownership of
the object is transferred to the executive, and it will be referenced.
When you ask for information from an object - DXGetComponentValue,
DXGetArrayData etc., you do notreceive an ownership. You can use
pointers you receive (eg. pointers to the data contained in an Array
object or pointers to a child member of a Group object) only as long as
the parent object itself does not get deleted, and you do not have to
clear or delete the pointers.
Seems simple, but there are a couple subtleties.
This is all implemented via ref counts (in a debugger, you can see the ref
count on an object by dumping the fourth 32-bit word found in the block of
memory pointed to by the object - eg.
int *ptr = (int *)DXGetComponentValue(field, "positions");
ptr[3] is the ref count on the positions of the field, which is an Array
object.
The original ownership of an object is not reflected in the ref count -
objects are create with a ref count of 0. Objects are deleted whenever the
result of decrementing their ref count is <= 0. So you have the following
scenarios:
1. Create an object, fool around with it, but don't add it to another
object. Then you have a (fake) ownership of it, and you must DXDelete it
to make it go away. Ref count goes from 0 to -1, and the object really
gets deleted.
2. Create an object and hand it to another object - eg. DXNewArray, then
DXSetComponentValue that array into a Field object. The ref count goes
from 0 to 1 with the DXSetComponentValue, and you have transfered the
(fake) ownership to the Field object, where it becomes a (non-fake)
ownership - eg. the ref count goes from 0 to 1. Now, if you were to
DXDelete the Array, its ref count would go from 1 to 0, and the Array would
really be deleted leaving the field with a dangling reference. So you don't
do that.
If this seems odd (it sure does to me) its based on perceived likelihood of
creating memory leaks. If objects were created with a ref count of 1, then
you would always have to delete them, and since scenario #2 way
predominates, it was thought that this would be likely to produce memory
leaks.
Finallt, there are a couple cases (Arrays and Strings, I think) where
DXCopy doesn't create a new object, but adds a reference to the original
one and returns the same pointer its given. This really leads to probems -
suppose you have an Array that is owned by precisely one Field.
DXArray a = DXGetComponentValue(...)
ref count of a is 1
DXArray b = DXCopy(a)
you might think a != b and the ref count of b is 0, but a = b, and the
ref count is 2
...
out[0] = (Object)b;
the ref count is incremented to 3 , but there are only 2 owners - the
original field and the executive. Memory leak!
Hope this helps,
Greg
"Mike H. Miller"
<[EMAIL PROTECTED]> To:
[email protected]
Sent by: cc:
[EMAIL PROTECTED] Subject: [opendx-users] Memory
Leaks / Caching
son.ibm.com
08/27/2002 09:26 AM
Please respond to
opendx-users
Hi,
I have some modules that appear to be creating giant memory leaks due to
what I think is my faulty implementation of caching. Based on a module I
found somewhere, I put the following string at the end of a module named
Map3Space, which takes two inputs, a field, and a slicenumber. I wanted
to set it up to cache the output, so I did:
sprintf (str, "Map3Space%d", slicenumber);
out[0] = DXGetCacheEntryV (str, 0, 0, NULL);
if (out[0] == NULL)
{
out[0] = (Object) outfield;
DXSetCacheEntryV (out[0], 0, str, 0, 0, NULL);
}
The problem is that this has a memory leak of 263613 bytes when run with
a 32K point dataset (floats-- seems about right). If I eliminate the
cache lines, no such leak occurs. However, eliminating the lines also
means that if I run the network with slicenumber = 1, then go to 2, it
uses the same 20meg that the whole net requires. Going back to 1 uses
_another_ 20 meg... that doesn't seem right.
On another note-- I do a lot of memory management stuff in my modules.
Obviously, anything that's malloc'ed has to be free'd. When I do
DXNewFields for outputs, I know I can't DXDelete that field, nor any of
its components. What about when I do something like this:
posarray = (Array) DXGetComponentValue (infield, "positions");
pos_float = DXGetArrayData (posarray);
In theory, I assume all that should be created here are two pointers--
but do I need to do something to clear them?
--
Mike Miller
[EMAIL PROTECTED] ->
[EMAIL PROTECTED]
#### signature.asc has been removed from this note on August 27 2002 by
Gregory D Abram
