Yes, the runs were in serial. hmm. I just remembered that I also supply my CouplingMatrix for the system matrix and so the nnz should probably be divided by sum(CouplingMatrix(ivar,:)) where ivar would be the variable number. I hope that makes sense. I'll check this again tomorrow or later tonight and see if that fixes it.
Or I could also be creating an expanded stencil in my physics based preconditioning that is triggering the extra mallocs. This should be evident if I spy the matrix in matlab and compare with my sparsity for matrix in the code. On Fri, Dec 4, 2009 at 8:14 PM, Kirk, Benjamin (JSC-EG311) <[email protected]> wrote: > That will be interesting. Is this in serial? In parallel you have > 'on-processor' and 'off-processor' nonzeros, the total number in a given row > is the sum of the two... > > > > ----- Original Message ----- > From: Vijay S. Mahadevan <[email protected]> > To: Kirk, Benjamin (JSC-EG311) > Cc: [email protected] <[email protected]>; > [email protected] <[email protected]> > Sent: Fri Dec 04 20:06:29 2009 > Subject: Re: [Libmesh-users] Block matrix nnz's > > Yes they are. I did divide the total number of nonzeros in the row by > the number of variables but I still get lots of malloc hits during the > first assembly. And that's why I posed this question. I'll have to > print out the matrix and load it in matlab to see how my nnz > allocation differs from the actual matrix sparsity. I'll do this > tomorrow and let you know if there was something wrong in my > assumption. > > On Fri, Dec 4, 2009 at 7:24 PM, Kirk, Benjamin (JSC-EG311) > <[email protected]> wrote: >> Are all your variables of the same type? If not, the concept of a block is >> more tedious... If so, can't you just divide the total number of nonzeros >> in the row by the number of variables in the system? >> >> -Ben >> >> >> ----- Original Message ----- >> From: Vijay S. Mahadevan <[email protected]> >> To: Derek Gaston <[email protected]> >> Cc: [email protected] <[email protected]> >> Sent: Fri Dec 04 14:32:04 2009 >> Subject: Re: [Libmesh-users] Block matrix nnz's >> >>> Believe it or not... Dr. Ben Kirk has thought of everything... and there is >>> no reason for us lesser beings to burn brain cycles over this kind of thing >>> ;-) >> >> I have come to the same conclusion several times before too :-) >> >> Derek, I have used the CouplingMatrix option before but was thinking >> along the lines of say calling DofMap with CouplingMatrix twice. One >> with the right coupling and one with a reduced coupling strategy >> (store only one variable block). This way, all true matvecs happen >> using the true coupled matrix while the reduced size matrix is used >> for performing preconditioning operations. There are some neat physics >> based preconditioners that can use these single blocks effectively >> (reduced storage and sometimes symmetric even if system matrix is >> block unsymmetric) and depending on the problem, can work better than >> ILU or some other numerical preconditioner. >> >> I got a reply from Rahul Sampath and his suggestion was to add a new >> ImplicitSystem with one variable to get the right nnz distribution for >> the single block. That seems reasonable and clean for now. Are there >> other alternatives ? >> >> Vijay >> >> On Fri, Dec 4, 2009 at 1:32 PM, Derek Gaston <[email protected]> wrote: >>> Believe it or not... Dr. Ben Kirk has thought of everything... and there is >>> no reason for us lesser beings to burn brain cycles over this kind of thing >>> ;-) >>> >>> In this case what you need to do is create a CouplingMatrix >>> (http://libmesh.sourceforge.net/doxygen/classCouplingMatrix.php) and attach >>> it to the DofMap _before_ compute_sparsity() gets called... >>> >>> You attach it to the DofMap by setting a public member: >>> >>> http://libmesh.sourceforge.net/doxygen/classDofMap.php#4d50b4c013a93b8036c126ced15ecbe6 >>> >>> Derek >>> >>> On Dec 4, 2009, at 12:27 AM, Vijay S. Mahadevan wrote: >>> >>>> Hi, >>>> >>>> I'm trying to figure out a clean way to find the nnz sparsity >>>> structure for a given variable. That is given an EquationSystem and >>>> its corresponding Dofmap, can I find the correct nnz structure to >>>> create a matrix ignoring the coupling between variables. It is useful >>>> for creating block preconditioners where the entire matrix (with all >>>> coupling between variables) is not needed but just individual blocks >>>> would be enough to perform Block-Jacobi type operations for the >>>> preconditioner. >>>> >>>> I can always sweep through all the elements, compute the dofs for the >>>> variable of interest and create this nnz but that seems hackish. This >>>> information should definitely be derived from DofMap but looking at >>>> the DofMap code, I could not figure out how this can be done correctly >>>> either. Is there a simpler and efficient way to do this ? >>>> >>>> Vijay >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Join us December 9, 2009 for the Red Hat Virtual Experience, >>>> a free event focused on virtualization and cloud computing. >>>> Attend in-depth sessions from your desk. Your couch. Anywhere. >>>> http://p.sf.net/sfu/redhat-sfdev2dev >>>> _______________________________________________ >>>> Libmesh-users mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/libmesh-users >>> >>> >> >> ------------------------------------------------------------------------------ >> Join us December 9, 2009 for the Red Hat Virtual Experience, >> a free event focused on virtualization and cloud computing. >> Attend in-depth sessions from your desk. Your couch. Anywhere. >> http://p.sf.net/sfu/redhat-sfdev2dev >> _______________________________________________ >> Libmesh-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/libmesh-users >> > ------------------------------------------------------------------------------ Join us December 9, 2009 for the Red Hat Virtual Experience, a free event focused on virtualization and cloud computing. Attend in-depth sessions from your desk. Your couch. Anywhere. http://p.sf.net/sfu/redhat-sfdev2dev _______________________________________________ Libmesh-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-users
