Hi Daniel, Thank you very much for your information! Really helpful. I have a few questions and comments inserted below.
On Fri, Nov 14, 2008 at 12:05 PM, Daniel Wheeler <[EMAIL PROTECTED]>wrote: > > A few things: > > 1 ) I assume you are running up against the limit of the machines > memory. You might want to check this, by looking at top while the job > is running to make sure. Sometimes I've had memory limited for other > reasons to make sure I don't crash the computer, so make sure you are > getting all the resources. > >From the "top", the program takes a vsize of around 2G while the available physical RAM is >20G for the server. So I think I have enough resource. > 2) I am not sure what type of problem you are running, but fipy can > use a lot of extra memory through creation of binops and caching. One > thing you might want to try is to put all the variable dependencies in > one long expression. For example, > > E = A * B > > F = C * D > > G = E * F > > probably uses more memory than > > G = A * B * C * D > Good point! I probably have some redundancy like this in my code. I have to check carefully. Thanks for reminding. > > 3) Take a look at your equations and figure out the order i.e. do you > have second order or fourth order diffusion. From that you can figure > out the bandwidth. Also figure out how many variables you are solving > for. From this information you should get a pretty good idea of the > minimum memory requirement that any simulation will need. You have to > store old and new values of the variables and the largest matrix. The > matrices are destroyed after being used in fipy. > I got the memory error when defining one of the source terms for the equation. I have a second order diffusion term and several coupled source terms (You probably might have seen those long long equations I sent you before). I don't know if the source terms cause the trouble. > 4) Make sure you are actually using the uniform grid classes. These > have massive savings in memory. At the moment non uniform meshes use a > lot more memory, but this will possibly be rectified when the new > variable mesh branch is merged. > Yes I am using the uniform grid. All the meshes I use so far are created by Grid3D. I am thinking to use GMSH though. > > 5) The direct solvers use a lot more memory then just the size of the > matrix. I am now using the GMRES solver from pysparse. What do you recommend? > > 6) If things just don't add up you can step through the program by > putting in raw_inputs and then looking at top and figure out where the > memory is allocated. You may want to print the variable values after > each deceleration to make them evaluate and allocate memory if they > are cached. Step though each equation as well as at least a few time > steps. > When I look at the top, the memory usage for the program doesn't change much throughout the definition of the equations, and it crashes somehow in between. And I am solving for an equilibrium state without any time evolution (just one time step). But I will check the variables. > Cheers > > On Thu, Nov 13, 2008 at 12:47 PM, Zhiwen Liang <[EMAIL PROTECTED]> wrote: > > Hi all, > > I am working on a 3D model that requires a very large mesh. When I tried > a > > uniform grid of 80*80*80 cells, I got the following attached memory > error. I > > wonder if there are ways to overcome this limit. Thank you in advance for > > any advice. > > Regards, > > Zhiwen > > python(12494) malloc: *** mmap(size=73728000) failed (error code=12) > > *** error: can't allocate region > > *** set a breakpoint in malloc_error_break to debug > > Traceback (most recent call last): > > File "run_pyramid.py", line 55, in <module> > > > > > eq_u,eq_v,eq_w=getEquations(u=u,v=v,w=w,c11=c11,c12=c12,c13=c13,c33=c33,c44= > > c44,beta11=beta11,beta22=beta22,beta33=beta33,cinn=cinn) > > File > "/Volumes/RAID2/Users/liangz/DEV/Elastic/Solve/elasticEquations.py", > > line > > 41, in getEquations > > > > > graduvw_w=Gradc(c13,Zhat)*Gradc(u,Xhat)+Gradc(c44,Xhat)*Gradc(u,Zhat)+Gradc( > > c13,Zhat)*Gradc(v,Yhat)+Gradc(c44,Yhat)*Gradc(v,Zhat) > > File "/Volumes/RAID2/Users/liangz/DEV/Elastic/Solve/myGrads.py", line > 15, > > in G > > radc > > return _Gradc(var).dot(hat) > > File > > "/Volumes/RAID2/usr/local/stow/fipy-trunk/lib/python2.5/site-packages/fip > > y/variables/meshVariable.py", line 193, in dot > > return _MeshVariable.__dot(self, other, > > self._OperatorVariableClass(baseClas > > s)) > > File > > "/Volumes/RAID2/usr/local/stow/fipy-trunk/lib/python2.5/site-packages/fip > > y/variables/meshVariable.py", line 165, in __dot > > opShape = numerix._broadcastShape(A[index].shape, B.shape) > > File > > "/Volumes/RAID2/usr/local/stow/fipy-trunk/lib/python2.5/site-packages/fip > > y/variables/variable.py", line 1287, in __getitem__ > > unit=self.getUnit(), > > File > > "/Volumes/RAID2/usr/local/stow/fipy-trunk/lib/python2.5/site-packages/fip > > y/variables/variable.py", line 249, in getUnit > > return self._extractUnit(self.getValue()) > > File > > "/Volumes/RAID2/usr/local/stow/fipy-trunk/lib/python2.5/site-packages/fip > > y/variables/variable.py", line 453, in getValue > > value = self._calcValue() > > File > > "/Volumes/RAID2/usr/local/stow/fipy-trunk/lib/python2.5/site-packages/fip > > y/variables/gaussCellGradVariable.py", line 93, in _calcValue > > return inline._optionalInline(self._calcValueIn, self._calcValuePy, > N, > > M, id > > s, orientations, volumes) > > File > > "/Volumes/RAID2/usr/local/stow/fipy-trunk/lib/python2.5/site-packages/fip > > y/tools/inline/inline.py", line 9, in _optionalInline > > return pythonFn(*args) > > File > > "/Volumes/RAID2/usr/local/stow/fipy-trunk/lib/python2.5/site-packages/fip > > y/variables/gaussCellGradVariable.py", line 80, in _calcValuePy > > grad = numerix.array(numerix.sum(orientations * contributions, 1)) > > File "/sw/lib/python2.5/site-packages/numpy/ma/core.py", line 1712, in > > __mul__ > > return multiply(self, other) > > File "/sw/lib/python2.5/site-packages/numpy/ma/core.py", line 514, in > > __call__ > > (d1, d2) = (get_data(a), get_data(b)) > > File "/sw/lib/python2.5/site-packages/numpy/ma/core.py", line 317, in > > get_data > > data = getattr(a, '_data', np.array(a, subok=subok)) > > MemoryError > > > > > > > > -- > Daniel Wheeler > Thanks a million! I learned a lot from the discussion. Have a nice weekend! Regards, Zhiwen
