Hi, I digging for some more info;
The first thing I realize is trilinos solver is about 2 slower than PySparse. But this doesn't explain all problem. I try to solve a mesh with 40,000 cells (200x200) on 1 core with trilinos and it takes about 17 secs. Than I give 160,000 cells to 4 core (40,000 cells on each core) and result is 46 secs. There should be some communication overhead but this couldn't explain 2.5 time slower solution. I still couldn't find a way to debug or profile parallel solution. Thanks, Serbulent 2014-09-30 18:41 GMT+03:00 Serbulent UNSAL <[email protected]>: > Thanks for your interest, > > I already shutdown preconditioner as shown in notebook with; > > solvers.linearPCGSolver.LinearPCGSolver(precon=None, iterations=500, > tolerance=1e-15) > > I think this should turn preconditioner off but still code run completed > at around 55 sec with 8 cores (serial was around 41 sec). So I think > problem is beyond preconditioner. I paste last version of the code below if > you'd like to try. > > http://pastebin.com/VL4dWCJr > > I also would like to know is there a decent way to profile parallel run > may be it can gave me a clue on problem. > > Serbulent > > > 2014-09-30 17:43 GMT+03:00 Daniel Wheeler <[email protected]>: > >> On Mon, Sep 29, 2014 at 4:09 PM, Serbulent UNSAL <[email protected]> >> wrote: >> >>> Thanks for help. >>> >>> Using LinearPCGSolver gives some more speed but it it still slower than >>> serial version ( 41 sec vs. 55 sec ). >>> >> >> Serbulent, >> >> It seems that in this notebook, >> http://nbviewer.ipython.org/github/wd15/fipy-efficiency/blob/master/notebooks/FiPy-IPython.ipynb, >> the serial version PCG using Trilinos is faster than the serial version of >> PySparse without preconditioning. Preconditioning with Trilinos can often >> impact efficiency quite heavily. >> >> >> >>> >>> I tried to use Gmsh mesh with using workaround defined at >>> http://wd15.github.io/2014/01/30/fipy-trilinos-anaconda/ >>> But Gmsh is even slower than normal mesh. >>> >> >> I don't think I factored that in when writing up the notebooks. I might >> look into that. >> >> >>> >>> Since I use a big mesh (400x400) I think, I will have communication >>> issue that defined at the end of the first notebook. >>> >> >> Take a look at the second notebook, >> http://nbviewer.ipython.org/github/wd15/fipy-efficiency/blob/master/notebooks/cluster.ipynb, >> 400x400 should be large enough to get some reasonable scaling. >> >> >> >>> >>> I'm open for any alternative strategies and suggestions. >>> >> >> I am not sure what to suggest. Certainly disable the preconditioners in >> Trilinos and see what happens. It may be that we need to use >> preconditioning in a less naive way in FiPy (not precondition at every >> sweep, there may be ways to do that with Trilinos). >> >> -- >> Daniel Wheeler >> >> _______________________________________________ >> fipy mailing list >> [email protected] >> http://www.ctcms.nist.gov/fipy >> [ NIST internal ONLY: https://email.nist.gov/mailman/listinfo/fipy ] >> >> >
_______________________________________________ fipy mailing list [email protected] http://www.ctcms.nist.gov/fipy [ NIST internal ONLY: https://email.nist.gov/mailman/listinfo/fipy ]
