On Thu, 14 Jan 2010 14:11:04 -0600, "Kirk, Benjamin (JSC-EG311)" <[email protected]> wrote: > From a PETSc point of view that is certainly true, but for libMesh > applications we allocate the sparsity exactly, so there is no allocation > inside the MatSetValues. Eventhough MatSetValues is not thread-safe, when > the allocation is correct putting a mutex around the element matrix > insertion has suffered no noticeable performance degradation out to 16 > threads...
Interesting, but this just means that your integration is very expensive. If the problem is simple so that integration is very cheap, then the cost of insertion will matter. I know that the Fenics group has seen some cases where insertion costs are significant compared to integration and solves, but I think it was only an issue for heterogeneous Helmholtz operators (which show up in many semi-implicit schemes). > Depending on the architecture, we've seen 4-socket 4-core nodes perform > *better* (lower wall-clock time) using 4 MPI tasks on the node that > 16. This isn't uncommon, the cores end up just competing for bandwidth. Also, the extra cores mean smaller subdomains which increases your iteration counts with most preconditioners. > Of course, as previously discussed, this only has a real impact on runtime > if you spend a nontrivial amount of time in the matrix assembly. This is > true in some of my applications, where the matrix assembly time is > comparable to (or sometimes greater than) the linear solve time. Yeah, I try to avoid assembly in these cases, by lagging the preconditioner and doing more things matrix-free (the matrix-free operations can certainly benefit from threading, you'll probably realize a greater speedup this way since you can parallelize the mat-vecs, which are CPU-bound instead of memory-bound when done unassembled -- usually by storing the nonlinearity at quadrature points). Jed ------------------------------------------------------------------------------ Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev _______________________________________________ Libmesh-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-users
