Hi Stefano,

On Feb 13, 2018, at 5:40 AM, Stefano Zampini 
<stefano.zamp...@gmail.com<mailto:stefano.zamp...@gmail.com>> wrote:


what are the performances you get with MATSELL in PFLOTRAN?

For PFLOTRAN and many other examples, the SELL-based MatMult is usually 2X 
faster than AIJ (non-inode version).

Today, I run the SELL matrix for the first time on a KNL with different 
problems (using PetIGA)  and got the results attached.
It seems that SELL is faster then AIJ for 2D Poisson, and slightly faster for 
3D Poisson. However, for multi-component problems (i.e. Elasticity and 
Cahn-Hilliard in mixed formulation) it has comparable performances or slower.
in these cases AIJ is faster since it uses the inodes routines.

Is this expected?

This is expected since currently SELL hasn't been optimized for multi-component 
problem (there is no inode version for SELL yet).

These are the flags I used to compile PETSc on KNL --COPTFLAGS=-xMIC-AVX512 -O3 

If you are using intel compiler 18, the flag -mP2OPT_hpo_vec_remainder=F is not 
needed. But it is necessary for older versions of intel compiler.

Hong (Mr.)

2018-02-12 21:04 GMT+03:00 Richard Tran Mills 
On Mon, Feb 12, 2018 at 8:47 AM, Smith, Barry F. 
<bsm...@mcs.anl.gov<mailto:bsm...@mcs.anl.gov>> wrote:

> On Feb 12, 2018, at 10:25 AM, Stefano Zampini 
> <stefano.zamp...@gmail.com<mailto:stefano.zamp...@gmail.com>> wrote:
> Barry,
> for sure Amat,Pmat is the right approach; however, with complicated user 
> codes, we are not always in control of having a different Jacobian matrix.
> Since Mat*SELL does not currently support any preconditioning except PCSOR 
> and PCJACOBI, we ask the user to put codes like
> if (type is SELL)
>  create two matrices (and maybe modify the code in many other parts)
> else
>   ok with the previous code

   I don't disagree with what you are saying and am not opposed to the proposed 

   Perhaps we need to do a better job with making the mat,pmat approach simpler 
or better documented so more people use it naturally in their applications.

I wrote some code like that in some of the Jacobian/function routines in 
PFLOTRAN to experiment with MATSELL, and it works, but looks and feels pretty 
hacky. And if I wanted to support it for all of the different systems that 
PFLOTRAN can model, then I'd have to reproduce that it in many different 
Jacobian and function evaluation routines. I also don't like that it makes it 
awkward to play with the many combinations of matrix types and preconditioners 
that PETSc allows: The above pseudocode should really say "if (type is SELL) 
and (preconditioner is not PCSOR or PCJACOBI)". I do think that Amat,Pmat is a 
good approach in many situations, but it's easy to construct scenarios in which 
it falls short.

In some situations, what I'd like to have happen is what Stefano is talking 
about, with an automatic conversion to AIJ happening if SELL doesn't support an 
operation. But, ideally, I think this sort of implicit format conversion 
shouldn't be something hard-coded into the workings of SELL. Instead, there 
should be some general mechanism by which PETSc recognizes that a particular 
operation is unsupported for a given matrix format, and then it can 
(optionally) copy/convert to a different matrix type (probably default to AIJ, 
but it shouldn't have to be AIJ) that supports the operation. This sort of 
implicit data rearrangement game may actually become more important if future 
computer architectures strongly prefer different data layouts different types 
of operations (though let's not get ahead of ourselves).



> Just my two cents.
> 2018-02-12 19:10 GMT+03:00 Smith, Barry F. 
> <bsm...@mcs.anl.gov<mailto:bsm...@mcs.anl.gov>>:
> > On Feb 12, 2018, at 9:59 AM, Stefano Zampini 
> > <stefano.zamp...@gmail.com<mailto:stefano.zamp...@gmail.com>> wrote:
> >
> > FYI, I just checked and MatSOR_*SELL does not use any vectorized 
> > instruction.
> > Why just not converting to SeqAIJ, factor and then use the AIJ 
> > implementation for MatSolve for the moment?
>   Why not use the mat, pmat feature of the solvers to pass in both matrices 
> and have the solvers handle using two formats simultaneously instead of 
> burdening the MatSELL code with tons of special code for automatically 
> converting to AIJ for solvers etc?
> >
> > 2018-02-12 18:06 GMT+03:00 Stefano Zampini 
> > <stefano.zamp...@gmail.com<mailto:stefano.zamp...@gmail.com>>:
> >
> >
> > 2018-02-12 17:36 GMT+03:00 Jed Brown 
> > <j...@jedbrown.org<mailto:j...@jedbrown.org>>:
> > Karl Rupp <r...@iue.tuwien.ac.at<mailto:r...@iue.tuwien.ac.at>> writes:
> >
> > > Hi Stefano,
> > >
> > >> Is there any plan to write code for native ILU/ICC etc for SeqSELL, at 
> > >> least to have BJACOBI in parallel?
> > >
> > > (imho) ILU/ICC is a pain to do with SeqSELL. Point-Jacobi should be
> > > possible, yes. SELL is really just tailored to MatMults and a pain for
> > > anything that is not very similar to a MatMult...
> >
> > There is already MatSOR_*SELL.  MatSolve_SeqSELL wouldn't be any harder.
> > I think it would be acceptable to convert to SeqAIJ, factor, and convert
> > the factors back to SELL.
> >
> > Yes, this was my idea. Today I have started coding something. I'll push the 
> > branch whenever I have anything working
> >
> >
> >
> > --
> > Stefano
> >
> >
> >
> > --
> > Stefano
> --
> Stefano


Reply via email to