Am I supposed to run c_free(pointer) after using pointer_to_array (and
before returning my c wrapper function)?
On Tuesday, 6 May 2014 15:13:08 UTC-4, Gustavo Camilo wrote:
>
> But if the C code returns a pointer to the malloced data, and I
> pointer_to_array(pointer,size,true), it should take ownership of it, and
> properly garbage it?
>
> On Tuesday, 6 May 2014 15:10:04 UTC-4, James Porter wrote:
>>
>> The Julia garbage collector does not know about or manage memory you
>> allocate in external C calls, if you free that memory yourself in your C
>> code you should see your memory usage improve dramatically.
>>
>> On Tuesday, May 6, 2014 1:33:30 PM UTC-5, Gustavo Camilo wrote:
>>>
>>> Hi all, I have a related question so I thought I would bump it up:
>>>
>>> I call a C function to do a lot of work using pmap in my code. Inside my
>>> C function I malloc and return arrays of floats. These then are used to
>>> perform some other computations.
>>>
>>> However I've put this inside an optimizer so now these functions are
>>> called over and over again. However my memory usage blows up, as if after
>>> ending the
>>> function call the data that was malloc'ed is not released.
>>>
>>> *What I am wondering is if every time the optimizer calls this, since
>>> several variables were malloc'ed in C,*
>>> *does the Julia garbage collector get rid of them, or should I pass
>>> empty matrices the the C function.*
>>>
>>> Thanks for your help, first is a sketch of each C call:
>>>
>>> for i in 1:length(parameterSpace)
>>> results[i] = pmap(cFunction,parameters[i])
>>> end
>>>
>>> (inside of every cFunction call there was a malloc)
>>>
>>> and then linear indexing to recuperate the results
>>> matrix = Array(Float64,size1,size2,...)
>>> for s1 in 1:size1, s2 in 1:size2
>>> matrix[s1,s2] = results[s1 + s2]
>>> end
>>>
>>>
>>>
>>> Here is the objective function of the optimization
>>>
>>> function computeModel(parameters::Vector)
>>> theta1 = parameters[1]
>>> theta2 = parameters[2]
>>> betaG = parameters[3]
>>> chi = parameters[4]
>>> alpha1 = parameters[5]
>>> alpha2 = parameters[6]
>>> rho1 = parameters[7]
>>> rho2 = parameters[8]
>>> sigma1 = parameters[9]
>>> sigma2 = parameters[10]
>>> k0 = parameters[11]
>>> g0 = parameters[12]
>>>
>>> sizeV = 20
>>> nsim = 1000
>>> age = 40
>>>
>>> println("-----------------------------------------------------")
>>> println("New Iteration. Parameters: ",parameters)
>>> println("Compute Priv")
>>> privBase =
>>> ModelBase(0.96,betaG,0.05,alpha1,chi,theta1,sigma1,rho1,g0,sizeV,sizeV,sizeV,sizeV,sizeV,k0)
>>> privModel = genModelPrivate(privBase)
>>> newBounds,guess,capitals = findBoundsPrivateInC(privModel)
>>> vf,pk,pq1,pq2,pq3 =
>>> runVFIPrivInC(newBounds,guess,tol=.003,capitals=capitals)
>>> sk,sq1,sq2,sq3,sd,sw,sz,sq =
>>> simulatePriv(newBounds,pk,pq1,pq2,pq3,capitals,nsim,age)
>>> momentsPriv = computeMoments(sk,sw,sd,sz,sq,newBounds)
>>>
>>>
>>> println("Compute SOE")
>>> soeBase =
>>> ModelBase(0.96,betaG,0.05,alpha2,chi,theta2,sigma2,rho2,g0,sizeV,sizeV,sizeV,sizeV,sizeV,k0)
>>> soeModel = genModelSOE(soeBase)
>>>
>>> newBoundsSoe,guessSoe,capitalsSoe = findBoundsSOEInC(soeModel)
>>> newValueFPareto,pK,pQ1,pQ2,pQ3 =
>>> runVFISOEInC(newBoundsSoe,guess=guessSoe,tol=.003,capitals=capitalsSoe)
>>> nVF,promiseMap =
>>> findPromiseSpace(newValueFPareto,newBoundsSoe,pK,pQ1,pQ2,pQ3)
>>> sk,sq1,sq2,sq3,sd,sw,sz,sq =
>>> simulateSOE(newBoundsSoe,pK,pQ1,pQ2,pQ3,capitalsSoe,promiseMap,nsim,age)
>>> momentsSoe = computeMoments(sk,sw,sd,sz,sq,newBoundsSoe)
>>>
>>> popMoments =
>>> [0.0830,0.1215,0.0599,0.1243,0.0039,0.016,0.3432,0.6291,0.0834,0.1542,0.0651,0.2109,0.0079,0.02,0.6705,1.2212]
>>>
>>> weight = eye(length(popMoments))
>>> g = float64(vcat([momentsPriv[i] for
>>> i=1:length(momentsPriv)],[momentsSoe[i] for i=1:length(momentsSoe)])) -
>>> popMoments
>>>
>>> value = (g'*weight*g)[1]
>>> println("f(x) = ", value)
>>>
>>> return value
>>> end
>>>
>>>
>>> On Wednesday, 16 April 2014 19:21:29 UTC-4, Gustavo Camilo wrote:
>>>>
>>>> Hi all, I'm currently using pmap to distribute to a bunch of cores some
>>>> work to do. I was just wondering how you guys are structuring this.
>>>>
>>>> Basically I have a function written in C that I need to pass a bunch of
>>>> parameters to so that it can do its work, so I first create a tuple with
>>>> all the parameters needed
>>>> for one worker to find results, for every combination of parameters:
>>>>
>>>> (I'm solving an infinite dimensional programming problem using bellman
>>>> equations and brute force to maximize, the C code searches through the
>>>> control space
>>>> and finds the best choices to maximize the objective)
>>>>
>>>> input =
>>>> [(w,z,mu,model,currentValue[:,:,nextPromise[mu]],squeeze(squeeze(capitals[w,z,:,:,:,:],2),1))
>>>>
>>>> for w in 1:nW, z in 1:nZ, mu in 1:nMu];
>>>>
>>>> And then I simply pass it with pmap to the wrapper of my C function:
>>>>
>>>> results = pmap(findPolicyC,input)
>>>>
>>>> And use linear indexing to recuperate the answers:
>>>>
>>>> anArray = results[w + nW*((z-1) + nZ*(mu-1))]
>>>>
>>>>
>>>> Is this how you guys would structure this parallel computation? I've
>>>> attached the code for any that are interested. You need GSL to run the C
>>>> part:
>>>> Eg. compile with, where the directories point to GSL: gcc
>>>> -I/home/gcam/china/gsl -L/home/gcam/china/gsl/.libs -shared -fPIC
>>>> findPolicySOE.c -lgsl -lm -o findPSoe.so
>>>>
>>>>
>>>> Thanks for the input!
>>>>
>>>
