On Fri, Jan 27, 2012 at 12:04 PM, Tabrez Ali <stali at geology.wisc.edu> wrote:
> Matt > > Thanks. Yes I can wait for a few weeks/months (its only a local memory > footprint issue and in no way hampers my ability to solve problems). > > Btw just for curiosity are you using the same technique discussed by Barry? > Basically. You have to store indices while counting if you don't want to overcount. This is the part he glosses over. Matt > Maybe I am overlooking something trivial in his post. > > Tabrez > > > On Jan 27, 2012, at 10:26 AM, Matthew Knepley wrote: > > On Fri, Jan 27, 2012 at 10:32 AM, Tabrez Ali <stali at geology.wisc.edu>wrote: > >> PETSc Gurus >> >> First I want to thank you for patiently answering questions that I have >> asked in past regarding preallocation. Unfortunately I am still having >> problems. >> >> I have a small unstructured FE (elasticity) code that uses PETSc. >> >> Unfortunately I am not yet able to find an _efficient_ way of calculating >> the non-zero structure so I simply overestimate the o_nz and d_nz values (= >> dimension x max_number_of nodes _a_node_contributes_to) which means that my >> stiffness matrix consumes at least 2X more memory. For example, for a >> perfectly structured mesh of linear hexes this number is 3*27=81=o_nz=d_nz. >> In complicated 3D linear tet/hex meshes that I have generated I rarely need >> to set a value greater than 150. In general I have found that as long as I >> keep DOF/core between 100-200K (assuming 1GB/per core) there is enough >> local RAM left even when memory for stiffness matrix is overestimated by >> 2X-6X. >> >> In any case now I do want to preallocate exactly for better memory >> performance and the ambiguity involved in choosing a reasonable o_nz/d_nz. >> The way I am trying to do it involves loops like ... >> >> do i=1, num_local_elements >> do j=1, num_total_nodes >> ... >> end do >> end do >> >> or using lists that involve searches. >> >> This unfortunately takes much time (much more than assembly and solve) >> due to the second loop. I am aware of many posts/slides by PETSc authors >> that mention that the non-zero structure can be found simply by looping >> once through elements but what to do next is not quite clear. >> >> I am also aware of this post by Barry [ http://lists.mcs.anl.gov/** >> pipermail/petsc-users/2008-**May/003020.html<http://lists.mcs.anl.gov/pipermail/petsc-users/2008-May/003020.html>] >> but I cannot get it to work for a simple 4 element problem. Can some one >> please expand on the strategy being suggested by Barry specially where the >> loops are discussed (I understand the arrays and the vecscatter part). I am >> also not sure how does using a t of 0.5 and 1 prevents double counting. >> Shouldn't it be 0 and 1? >> >> I am also aware of some capabilities in DMMESH but right now I dont >> understand it well enough to utilize it. >> > > I agree with you here. Its not easy enough to use. However, I have > rewritten the basics of it completely in C, which > will all have easy Fortran bindings. You can specify your mesh with > adjacency lists using Fortran arrays. The last part > I am working on is matrix preallocation. I would not normally ask you to > wait, but I think I am pretty close. Mail me > directly if you want more info. I think it should only take me a few weeks > to finish. > > Thanks, > > Matt > > Alternatively does PETSc support MATMPIAIJ of size=integer(1)/logical >> instead of real(8) which I can use to put zeros/ones (when I loop over >> elements the first time) to get the non-zero structure? >> >> Thanks in advance. >> >> Tabrez >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120127/ce1b5ba6/attachment.htm>
