By default pointer_to_array will not cause the Julia runtime take ownership
of the array, but you can optionally pass `own=true` to cause this to
happen, and then the memory will be garbage collected when it is no longer
referenced. You can also find this information at the REPL by typing
`help(ponter_to_array)`
On Tuesday, May 6, 2014 3:18:19 PM UTC-5, Gustavo Camilo wrote:
>
> Am I supposed to run c_free(pointer) after using pointer_to_array (and
> before returning my c wrapper function)?
>
> On Tuesday, 6 May 2014 15:13:08 UTC-4, Gustavo Camilo wrote:
>>
>> But if the C code returns a pointer to the malloced data, and I
>> pointer_to_array(pointer,size,true), it should take ownership of it, and
>> properly garbage it?
>>
>> On Tuesday, 6 May 2014 15:10:04 UTC-4, James Porter wrote:
>>>
>>> The Julia garbage collector does not know about or manage memory you
>>> allocate in external C calls, if you free that memory yourself in your C
>>> code you should see your memory usage improve dramatically.
>>>
>>> On Tuesday, May 6, 2014 1:33:30 PM UTC-5, Gustavo Camilo wrote:
>>>>
>>>> Hi all, I have a related question so I thought I would bump it up:
>>>>
>>>> I call a C function to do a lot of work using pmap in my code. Inside
>>>> my C function I malloc and return arrays of floats. These then are used to
>>>> perform some other computations.
>>>>
>>>> However I've put this inside an optimizer so now these functions are
>>>> called over and over again. However my memory usage blows up, as if after
>>>> ending the
>>>> function call the data that was malloc'ed is not released.
>>>>
>>>> *What I am wondering is if every time the optimizer calls this, since
>>>> several variables were malloc'ed in C,*
>>>> *does the Julia garbage collector get rid of them, or should I pass
>>>> empty matrices the the C function.*
>>>>
>>>> Thanks for your help, first is a sketch of each C call:
>>>>
>>>> for i in 1:length(parameterSpace)
>>>> results[i] = pmap(cFunction,parameters[i])
>>>> end
>>>>
>>>> (inside of every cFunction call there was a malloc)
>>>>
>>>> and then linear indexing to recuperate the results
>>>> matrix = Array(Float64,size1,size2,...)
>>>> for s1 in 1:size1, s2 in 1:size2
>>>> matrix[s1,s2] = results[s1 + s2]
>>>> end
>>>>
>>>>
>>>>
>>>> Here is the objective function of the optimization
>>>>
>>>> function computeModel(parameters::Vector)
>>>> theta1 = parameters[1]
>>>> theta2 = parameters[2]
>>>> betaG = parameters[3]
>>>> chi = parameters[4]
>>>> alpha1 = parameters[5]
>>>> alpha2 = parameters[6]
>>>> rho1 = parameters[7]
>>>> rho2 = parameters[8]
>>>> sigma1 = parameters[9]
>>>> sigma2 = parameters[10]
>>>> k0 = parameters[11]
>>>> g0 = parameters[12]
>>>>
>>>> sizeV = 20
>>>> nsim = 1000
>>>> age = 40
>>>>
>>>> println("-----------------------------------------------------")
>>>> println("New Iteration. Parameters: ",parameters)
>>>> println("Compute Priv")
>>>> privBase =
>>>> ModelBase(0.96,betaG,0.05,alpha1,chi,theta1,sigma1,rho1,g0,sizeV,sizeV,sizeV,sizeV,sizeV,k0)
>>>> privModel = genModelPrivate(privBase)
>>>> newBounds,guess,capitals = findBoundsPrivateInC(privModel)
>>>> vf,pk,pq1,pq2,pq3 =
>>>> runVFIPrivInC(newBounds,guess,tol=.003,capitals=capitals)
>>>> sk,sq1,sq2,sq3,sd,sw,sz,sq =
>>>> simulatePriv(newBounds,pk,pq1,pq2,pq3,capitals,nsim,age)
>>>> momentsPriv = computeMoments(sk,sw,sd,sz,sq,newBounds)
>>>>
>>>>
>>>> println("Compute SOE")
>>>> soeBase =
>>>> ModelBase(0.96,betaG,0.05,alpha2,chi,theta2,sigma2,rho2,g0,sizeV,sizeV,sizeV,sizeV,sizeV,k0)
>>>> soeModel = genModelSOE(soeBase)
>>>>
>>>> newBoundsSoe,guessSoe,capitalsSoe = findBoundsSOEInC(soeModel)
>>>> newValueFPareto,pK,pQ1,pQ2,pQ3 =
>>>> runVFISOEInC(newBoundsSoe,guess=guessSoe,tol=.003,capitals=capitalsSoe)
>>>> nVF,promiseMap =
>>>> findPromiseSpace(newValueFPareto,newBoundsSoe,pK,pQ1,pQ2,pQ3)
>>>> sk,sq1,sq2,sq3,sd,sw,sz,sq =
>>>> simulateSOE(newBoundsSoe,pK,pQ1,pQ2,pQ3,capitalsSoe,promiseMap,nsim,age)
>>>> momentsSoe = computeMoments(sk,sw,sd,sz,sq,newBoundsSoe)
>>>>
>>>> popMoments =
>>>> [0.0830,0.1215,0.0599,0.1243,0.0039,0.016,0.3432,0.6291,0.0834,0.1542,0.0651,0.2109,0.0079,0.02,0.6705,1.2212]
>>>>
>>>> weight = eye(length(popMoments))
>>>> g = float64(vcat([momentsPriv[i] for
>>>> i=1:length(momentsPriv)],[momentsSoe[i] for i=1:length(momentsSoe)])) -
>>>> popMoments
>>>>
>>>> value = (g'*weight*g)[1]
>>>> println("f(x) = ", value)
>>>>
>>>> return value
>>>> end
>>>>
>>>>
>>>> On Wednesday, 16 April 2014 19:21:29 UTC-4, Gustavo Camilo wrote:
>>>>>
>>>>> Hi all, I'm currently using pmap to distribute to a bunch of cores
>>>>> some work to do. I was just wondering how you guys are structuring this.
>>>>>
>>>>> Basically I have a function written in C that I need to pass a bunch
>>>>> of parameters to so that it can do its work, so I first create a tuple
>>>>> with
>>>>> all the parameters needed
>>>>> for one worker to find results, for every combination of parameters:
>>>>>
>>>>> (I'm solving an infinite dimensional programming problem using bellman
>>>>> equations and brute force to maximize, the C code searches through the
>>>>> control space
>>>>> and finds the best choices to maximize the objective)
>>>>>
>>>>> input =
>>>>> [(w,z,mu,model,currentValue[:,:,nextPromise[mu]],squeeze(squeeze(capitals[w,z,:,:,:,:],2),1))
>>>>>
>>>>> for w in 1:nW, z in 1:nZ, mu in 1:nMu];
>>>>>
>>>>> And then I simply pass it with pmap to the wrapper of my C function:
>>>>>
>>>>> results = pmap(findPolicyC,input)
>>>>>
>>>>> And use linear indexing to recuperate the answers:
>>>>>
>>>>> anArray = results[w + nW*((z-1) + nZ*(mu-1))]
>>>>>
>>>>>
>>>>> Is this how you guys would structure this parallel computation? I've
>>>>> attached the code for any that are interested. You need GSL to run the C
>>>>> part:
>>>>> Eg. compile with, where the directories point to GSL: gcc
>>>>> -I/home/gcam/china/gsl -L/home/gcam/china/gsl/.libs -shared -fPIC
>>>>> findPolicySOE.c -lgsl -lm -o findPSoe.so
>>>>>
>>>>>
>>>>> Thanks for the input!
>>>>>
>>>>
