90% of the time is spent in your Jacobian evaluation routine which is clearly a bottleneck.
On Aug 13, 2013, at 12:34 PM, Jin, Shuangshuang wrote: > Hello, Jed and Barry, thanks for your reply. > > We are solving a power system dynamic simulation problem. We set up the DAE > equations and its Jacobian matrix, and would like to use the Trapezoid method > to solve it. > > That's also the reason why we chose TSTHETA. From the PETSc manual, we read > that: > > "-ts_type theta -ts_theta_theta 0.5 -ts_theta_endpoint corresponds to > Crank-Nicholson (TSCN). This method can be applied to DAE. > For the default Theta=0.5, this is the trapezoid rule (also known as > Crank-Nicolson, see TSCN)." > > I haven't heard of ARKIMEX or ROSW before. Are they some external packages or > DAE solvers that implement the Trapezoid method? > > I have also tried the -ksp_type preonly -pc_type lu option you indicated but > failed. The PETSC ERROR messages are: No support for this operation for this > object type! Matrix format mpiaij does not have a built-in PETSc LU! PETSc does not have a native parallel direct solver. You can use MUMPS (--download-mumps) or superlu_dist (--download_superlu_dist) > > Attached please see the log_summary for running the TSTHETA with > -ts_theta_theta 0.5 and its default ksp solver. Please help me to evaluate > the performance and see what's the bottleneck of the slow computation speed. > > Thanks a lot! > > Shuangshuang > > > > > > > -----Original Message----- > From: Barry Smith [mailto:[email protected]] > Sent: Monday, August 12, 2013 6:39 PM > To: Jed Brown > Cc: Jin, Shuangshuang; [email protected] > Subject: Re: [petsc-users] Performance of PETSc TS solver > > > Also always send the output from running with -log_summary whenever you ask > performance questions so we know what kind of performance it is getting. > > Barry > > On Aug 12, 2013, at 8:14 PM, Jed Brown <[email protected]> wrote: > >> "Jin, Shuangshuang" <[email protected]> writes: >> >>> Hello, PETSc developers, >>> I have a question regarding the performance of PETSc TS solver >>> expecially the TSTHETA. I used it to solve my DAE equations. >> >> TSTHETA is not L-stable and not stiffly accurate, so it's not normally >> something that you'd want to use for a DAE. Make sure you're getting >> meaningful results and try switching to something like an ARKIMEX or >> ROSW since those are likely better for your problem. >> >>> I have recorded the solution times when different numbers of processors are >>> used: >>> >>> 2 processors: 1021 seconds, >>> 4 processors: 587.244 seconds, >>> 8 processors: 421.565 seconds, >>> 16 processors: 355.594 seconds, >>> 32 processors: 322.28 seconds, >>> 64 processors: 382.967 seconds. >>> >>> It seems like with 32 processors, it reaches the best performance. >>> However, 322.28 seconds to solve such DAE equations is too slow than >>> I expected. >> >> The number of equations (1152) is quite small, so I'm not surprised >> there is no further speedup. Can you explain more about your equations? >> >>> >>> I have the following questions based on the above results: >>> 1. Is this the usual DAE solving time in PETSc to for the problem with >>> this dimension? >> >> That depends what your function is. >> >>> 2. I was told that in TS, by default, ksp uses GMRES, and the >>> preconditioner is ILU(0), is there any other alterative ksp solver or >>> options I should use in the command line to solve the problem much >>> faster? >> >> I would use -ksp_type preonly -pc_type lu for such small problems. Is >> the system dense? >> >>> 3. Do you have any other suggestion for me to speed up the DAE >>> computation in PETSc? >> >> Can you describe what sort of problem you're dealing with, what causes >> the stiffness in your equations, what accuracy you want, etc. > > <job.out.summary>
