On Thu, 11 Mar 2021 at 09:27, Mathieu Dutour <mathieu.dut...@gmail.com> wrote:
> Dear all, > > I would like to work with a special kind of linear system that ought to be > very common but I am not sure that it is possible in PETSC. > > What we have is an unstructured grid with say 3.10^5 nodes in it. > At each node, we have a number of frequency/direction and together > this makes about 1000 values at the node. So, in total the linear system > has say 3.10^8 values. > > We managed to implement this system with Petsc but the performance > was unsatisfactory. > I think part of the reason the answers you are getting aren't helpful to you is that you have not identified "what" exactly you find to be unsatisfactory. Nor is it obvious what you consider to be satisfactory. For example, does "unsatisfactory" relate to any of these items? * memory usage of the matrix * time taken to assemble the matrix * time taken to perform MatMult() * solve time If it does, providing the output from -log_view (from an optimized build of petsc) would be helpful, and moreover it would provide developers with a baseline result with which they could compare to should any implementation changes be made. Having established what functionality is causing you concern, it would then be help for you to explain why you think it should be better, e.g. based on a performance model, prior experience with other software, etc. More information would help. Thanks, Dave > We think that Petsc is not exploiting the special > structure of the matrix and we wonder if this structure can be implemented > in Petsc. > > By special structure we mean the following. An entry in the linear system > is of the form (i, j) with 1<=i<=1000 and 1<=j<=N with N = 3.10^5. > The node (i , j) is adjacent to all the nodes (i' , j) and thus they make > a block > diagonal entry. But the node (i , j) is also adjacent to some nodes (i , > j') > [About 6 such nodes, but it varies]. > > Would there be a way to exploit this special structure in Petsc? I think > this should be fairly common and significant speedup could be obtained. > > Best, > > Mathieu >