[petsc-users] Recursive fieldsplit PCs

2013-12-02 Thread Lawrence Mitchell
Dear petsc-users, I have a 3x3 block system built as a single MatNest (with 9 Mats in it). I'd like to treat this as a 2x2 system: [ A B C D ] where A is 2x2 and precondition the A block with a schur complement. Since I have a mat nest, the ISes for the three fields are just

Re: [petsc-users] Recursive fieldsplit PCs

2013-12-02 Thread Lawrence Mitchell
On 2 Dec 2013, at 12:23, Matthew Knepley knep...@gmail.com wrote: On Mon, Dec 2, 2013 at 2:26 AM, Lawrence Mitchell lawrence.mitch...@imperial.ac.uk wrote: Dear petsc-users, I have a 3x3 block system built as a single MatNest (with 9 Mats in it). I'd like to treat this as a 2x2 system

Re: [petsc-users] Recursive fieldsplit PCs

2013-12-02 Thread Lawrence Mitchell
On 2 Dec 2013, at 15:20, Matthew Knepley knep...@gmail.com wrote: On Mon, Dec 2, 2013 at 6:31 AM, Lawrence Mitchell lawrence.mitch...@imperial.ac.uk wrote: On 2 Dec 2013, at 12:23, Matthew Knepley knep...@gmail.com wrote: On Mon, Dec 2, 2013 at 2:26 AM, Lawrence Mitchell

[petsc-users] Null spaces with fieldsplit preconditioning

2014-03-12 Thread Lawrence Mitchell
Hello, I have a mixed FEM system, for which the null space is constant functions in one of the sub spaces. That is, I have W = V * Q and the null space is a constant function in Q (and zero in V). If I assemble the operator, I get a block structure: X = [A B, C D] When I solve this

[petsc-users] Poor multigrid convergence in parallel

2014-07-21 Thread Lawrence Mitchell
Hello all, I'm implementing a multigrid solver using PCMG, starting with a simple Poisson equation (with strong boundary conditions) to ensure I'm doing things right. Everything works fine in serial, but when running on two processes, with the default chebyshev smoother, convergence goes to

Re: [petsc-users] Poor multigrid convergence in parallel

2014-07-21 Thread Lawrence Mitchell
On 21 Jul 2014, at 11:50, Dave May dave.mayhe...@gmail.com wrote: Hi Lawrence, Hmm this sounds odd. The convergence obtained with chebyshev should be essentially identical in serial and parallel when using a jacobi preconditioner So I was maybe a bit unclear in my previous mail: If I

Re: [petsc-users] Poor multigrid convergence in parallel

2014-07-21 Thread Lawrence Mitchell
On 21 Jul 2014, at 12:52, Dave May dave.mayhe...@gmail.com wrote: -pc_type mg -mg_levels_ksp_type richardson -mg_levels_pc_type jacobi -mg_levels_ksp_max_it 2 then I get identical convergence in serial and parallel Good. That's the correct result. if, however, I run with

Re: [petsc-users] Poor multigrid convergence in parallel

2014-07-21 Thread Lawrence Mitchell
To follow up, On 21 Jul 2014, at 13:11, Lawrence Mitchell lawrence.mitch...@imperial.ac.uk wrote: On 21 Jul 2014, at 12:52, Dave May dave.mayhe...@gmail.com wrote: -pc_type mg -mg_levels_ksp_type richardson -mg_levels_pc_type jacobi -mg_levels_ksp_max_it 2 then I get identical

Re: [petsc-users] Poor multigrid convergence in parallel

2014-07-22 Thread Lawrence Mitchell
On 21 Jul 2014, at 18:29, Jed Brown j...@jedbrown.org wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: Below I show output from a run on 1 process and then two (along with ksp_view) for the following options: -pc_type mg -ksp_rtol 1e-8 -ksp_max_it 6 -pc_mg_levels 2

Re: [petsc-users] Poor multigrid convergence in parallel

2014-07-26 Thread Lawrence Mitchell
On 25 Jul 2014, at 21:28, Jed Brown j...@jedbrown.org wrote: Sorry about not following up. I also find these results peculiar. Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: So I'm sort of none-the-wiser. I'm a little bit at a loss as to why this occurs, but either switching

Re: [petsc-users] Poor multigrid convergence in parallel

2014-07-29 Thread Lawrence Mitchell
On 29 Jul 2014, at 13:37, Jed Brown j...@jedbrown.org wrote: Please always use reply-all so that your messages go to the list. Sorry, fat-fingered the buttons. Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: On 28 Jul 2014, at 23:27, Jed Brown j...@jedbrown.org wrote

Re: [petsc-users] Poor multigrid convergence in parallel

2014-07-29 Thread Lawrence Mitchell
On 29/07/14 14:35, Jed Brown wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: So my coarse space is spanned by the fine one, so I copy coarse dofs to the corresponding fine ones and then linearly interpolate to get the coefficient value at the missing fine dofs. Good

Re: [petsc-users] Poor multigrid convergence in parallel

2014-07-29 Thread Lawrence Mitchell
On 29 Jul 2014, at 16:58, Jed Brown j...@jedbrown.org wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: No, I'm L2-projecting (with mass-lumping) for the restriction. So if I weren't lumping, I think this is the dual of the prolongation. A true L2 projection is a dense

Re: [petsc-users] Set matrix column to vector

2014-09-02 Thread Lawrence Mitchell
On 02/09/14 09:41, Florian Lindner wrote: Am 01.09.2014 16:11, schrieb Matthew Knepley: I recommend running using the debugger so you can get a stack trace, and perhaps see exactly what the problem is. You can also run under valgrind as the error says. This of course I tried, but had no

[petsc-users] Nullspaces for schur complement PCs

2014-11-07 Thread Lawrence Mitchell
Hi petsc-dev, I'm solving a pure Neumann mixed Poisson-like problem, preconditioning with a schur complement. The pressure space has a nullspace of the constant functions and so I attach the appropriate nullspace to the krylov solver, and compose the constant nullspace with the IS defining

Re: [petsc-users] Nullspaces for schur complement PCs

2014-11-10 Thread Lawrence Mitchell
On 10 Nov 2014, at 21:48, Matthew Knepley knep...@gmail.com wrote: On Fri, Nov 7, 2014 at 11:04 AM, Lawrence Mitchell lawrence.mitch...@imperial.ac.ukwrote: Hi petsc-dev, I'm solving a pure Neumann mixed Poisson-like problem, preconditioning with a schur complement. The pressure space

Re: [petsc-users] Nullspaces for schur complement PCs

2014-11-11 Thread Lawrence Mitchell
On 11 Nov 2014, at 04:28, Jed Brown j...@jedbrown.org wrote: Matthew Knepley knep...@gmail.com writes: at which point the nullspace is not attached, no? Crap, that is right. I think we should propagate the nullspace from pmat[1]. Uh, there is no reason pmat[1] can be expected to have

Re: [petsc-users] Nullspaces for schur complement PCs

2014-11-12 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/11/14 15:18, Jed Brown wrote: ... Matt and I have a long-running disagreement about nullspace handling in PETSc. He added the code that hangs it on an IS despite it not logically belonging on IS, simply because the implementation in

Re: [petsc-users] Error while calling a fieldsplit preconditioner containing a mg/gamg preconditioner

2014-11-21 Thread Lawrence Mitchell
On 21 Nov 2014, at 16:55, Luc Berger-Vergiat lb2...@columbia.edu wrote: Sure, so with gamg (which was my first choice too), I get the following: [0]PETSC ERROR: - Error Message -- [0]PETSC ERROR:

[petsc-users] Confusion with MatGetLocalSubMatrix

2014-12-19 Thread Lawrence Mitchell
Dear petsc-users, I'm trying to setup matrices and data structures for use with MatGetLocalSubMatrix, but I'm rather lost in a world of ISes and block sizes. I have the layouts and so forth correct where all my fields have block size 1, but am struggling to follow how things fit together for

Re: [petsc-users] Confusion with MatGetLocalSubMatrix

2014-12-19 Thread Lawrence Mitchell
On 19 Dec 2014, at 17:12, Jed Brown j...@jedbrown.org wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: Dear petsc-users, I'm trying to setup matrices and data structures for use with MatGetLocalSubMatrix, but I'm rather lost in a world of ISes and block sizes. I have

Re: [petsc-users] Confusion with MatGetLocalSubMatrix

2014-12-19 Thread Lawrence Mitchell
... Calling MatGetLocalSubMatrix results in an error: [0]PETSC ERROR: - Error Message -- [0]PETSC ERROR: Petsc has generated inconsistent data [0]PETSC ERROR: Blocksize of localtoglobalmapping 1 must match

Re: [petsc-users] Confusion with MatGetLocalSubMatrix

2014-12-19 Thread Lawrence Mitchell
On 19 Dec 2014, at 18:01, Jed Brown j...@jedbrown.org wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: What is ISSetBlockSize for then? Just hanging information on the IS for use elsewhere? The index set would need to be contiguous: So given my field layout, I think I

Re: [petsc-users] Confusion with MatGetLocalSubMatrix

2014-12-19 Thread Lawrence Mitchell
On 19 Dec 2014, at 18:19, Lawrence Mitchell lawrence.mitch...@imperial.ac.uk wrote: ... Calling MatGetLocalSubMatrix results in an error: [0]PETSC ERROR: - Error Message -- [0]PETSC ERROR: Petsc has

Re: [petsc-users] Multiple in-flight communications with PetscSFs

2015-02-07 Thread Lawrence Mitchell
On 6 Feb 2015, at 18:09, Jed Brown j...@jedbrown.org wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: ... Or is this not something that is supposed to work at all and I'm just lucky. It's supposed to work, but I think not tested. Can you test the above (have

Re: [petsc-users] Issue with window SF type using derived datatypes for reduction

2015-02-17 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/02/15 01:31, Jed Brown wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: Having just tried a build with --download-mpich, I notice this problem does not occur. So should I shout at the OpenMPI team? Open MPI has many long

Re: [petsc-users] Confusion with MatGetLocalSubMatrix

2015-01-12 Thread Lawrence Mitchell
Hi folks, On 19 Dec 2014, at 18:29, Lawrence Mitchell lawrence.mitch...@imperial.ac.uk wrote: On 19 Dec 2014, at 18:19, Lawrence Mitchell lawrence.mitch...@imperial.ac.uk wrote: ... Calling MatGetLocalSubMatrix results in an error: [0]PETSC ERROR: - Error

Re: [petsc-users] Issue with window SF type using derived datatypes for reduction

2015-02-10 Thread Lawrence Mitchell
On 10 Feb 2015, at 01:31, Jed Brown j...@jedbrown.org wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: Having just tried a build with --download-mpich, I notice this problem does not occur. So should I shout at the OpenMPI team? Open MPI has many long-standing bugs

[petsc-users] Unable to extract submatrix from DMComposite Mat if block size 1

2015-02-25 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, I have multi-field operators that have the same layout as given by the matrices created by DMComposite and would like to be able to switch between a MatNest representation (if I'm using fieldsplit PCs) or AIJ (if not). This works fine if the

[petsc-users] Poor FAS convergence for linear problems when not monitoring on levels

2015-02-24 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, I'm observing poor convergence of FAS on linear problems when I use a snes_type of ksponly on each level, unless I also monitor the snes on those levels. This is reproducible with snes ex35.c: Using one iteration of newton on each level: $

Re: [petsc-users] Multiple in-flight communications with PetscSFs

2015-02-24 Thread Lawrence Mitchell
YYYSetFromOptions() that the user called. We do violate this rule occasionally but we don't want new XXXSetFromOptions() put into the code randomly. Barry On Feb 7, 2015, at 7:57 AM, Jed Brown j...@jedbrown.org wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: + ierr

[petsc-users] DMs not transferred into PCCOMPOSITE?

2015-06-08 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, I have a multi-field system where the splits are defined by a DM. I can use a fieldsplit PC on its own with no trouble, but cannot do so inside a composite PC. My matrices do not have a blocksize set (because the data layout doesn't match)

Re: [petsc-users] Set PC PythonContext within NPC

2015-06-25 Thread Lawrence Mitchell
On 24 Jun 2015, at 17:02, Asbjørn Nilsen Riseth ris...@maths.ox.ac.uk wrote: Hi all, I'm currently trying to set up a nonlinear solver that uses NGMRES with NEWTONLS as a right preconditioner. The NEWTONLS has a custom linear preconditioner. Everything is accessed through petsc4py.

Re: [petsc-users] Modified Taylor-Hood elements with piece-wise constant pressure for Stokes equation

2015-06-02 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/06/15 13:37, Matthew Knepley wrote: This construction appears to throw away unisolvence. Yes, it's normally used for building spaces where the resulting enriched space is unisolvent (e.g. MINI). In the Boffi paper, they use QR to solve

Re: [petsc-users] Modified Taylor-Hood elements with piece-wise constant pressure for Stokes equation

2015-06-02 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/06/15 17:20, Jed Brown wrote: Lawrence Mitchell lawrence.mitch...@imperial.ac.uk writes: So the mass matrix for CG1+DG0 is singular? I believe so, yes. Fabulous. Now let's take a one element domain. What is the norm of the vector u

Re: [petsc-users] Null space/Near null space

2015-07-23 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 22/07/15 23:57, Barry Smith wrote: If your matrix has a null space you should always be setting MatSetNullSpace. If you know the near null space then you should also always set it (it cannot do any harm) because some preconditioners can

Re: [petsc-users] A combined petsc4py, FEniCS, and Firedrake issue

2015-07-14 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Justin, On 14/07/15 09:24, Justin Chang wrote: Hi everyone, As the title says, I have an issue regarding all three. Not sure which mailing to post this to but i figured somebody here may help. I was attempting to install Firedrake on my

[petsc-users] DMPlex natural to global mappings

2015-10-23 Thread Lawrence Mitchell
istributed coordinates, and get an error because on rank-0 the block size is 2, not 1. I fix this in the following way: commit aef309e0b49bf6e55e6f970bc0bdabf021b346f6 Author: Lawrence Mitchell <lawrence.mitch...@imperial.ac.uk> Date: Fri Oct 23 09:32:36 2015 +0100 plexdistribute:

Re: [petsc-users] Partitioning a mesh with petsc4py

2015-07-08 Thread Lawrence Mitchell
On 8 Jul 2015, at 12:53, Javier Quinteros jav...@gfz-potsdam.de wrote: Signed PGP part Hi Matthew. Thanks for the answer. We've checked again and actually it is working with chaco. But we haven't found any example/way to use it with metis, that it's what we wanted. I've seen that there

Re: [petsc-users] block matrix in serial

2015-09-16 Thread Lawrence Mitchell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 16/09/15 12:40, Matthew Knepley wrote: > On Tue, Sep 15, 2015 at 9:05 PM, Adrian Croucher > > > wrote: > > hi > > I have a test code (attached) that sets up a finite volume mesh >

Re: [petsc-users] grid sequencing & DMComposite

2015-09-11 Thread Lawrence Mitchell
> On 11 Sep 2015, at 14:37, Gideon Simpson wrote: > > There doesn’t seem to be a global analog of DMCompositeGetLocalVectors. Do I > need to do manual indexing? I think you use DMCompositeGetAccess. Lawrence signature.asc Description: Message signed with OpenPGP

Re: [petsc-users] mpi_aij MatGetSubMatrix with mat_block_size!=1

2015-11-30 Thread Lawrence Mitchell
On 30/11/15 16:14, Eric Chamberland wrote: > Hi, > > Using PETSc 3.5.3. > > We have a "A" matrix, mpi_aij with block_size=3. > > We create a IS with ISCreateStride, then extract A_00 with > MatGetSubMatrix(..., MAT_INITIAL_MATRIX,...). > > We know that A_00 is block_size = 3 and mpi_aij,

Re: [petsc-users] Providing context to DMShell

2016-01-12 Thread Lawrence Mitchell
On 12/01/16 15:37, Boris Boutkov wrote: > Hello All, > > I'm trying to attach a context to the DMShell similarly to how a context > is passed into SNES through the SetFunction routine. Specifically, I'm > looking to provide my own interpolation routine to both >

Re: [petsc-users] Parallel assembly FEM matrix

2016-06-06 Thread Lawrence Mitchell
> On 6 Jun 2016, at 01:43, Nestor Cerpa wrote: > > Hello, > > I am learning to use Petsc in order to modify my sequential FEM code to work > in parallel. First, I am trying to understand how a FEM assembly loop would > work in parallel using Petsc but I run into some

Re: [petsc-users] How to refine the Vertex Sets in DMPlex

2016-06-08 Thread Lawrence Mitchell
> On 8 Jun 2016, at 06:45, Rongliang Chen wrote: > > Hi Matt, > > I tried your suggestions but I still can not get the refined Vertex Sets. > What I am doing looks like this: > --- > if (!rank) { >

Re: [petsc-users] Providing Interpolation/Injections to Multigrid infrastructure

2016-02-12 Thread Lawrence Mitchell
> On 12 Feb 2016, at 15:20, Paul T. Bauman wrote: > > Hi Barry, > > On Thu, Feb 11, 2016 at 2:45 PM, Barry Smith wrote: > > > On Feb 11, 2016, at 1:36 PM, Boris Boutkov wrote: > > > > Hello All, > > > > I'm currently working on

Re: [petsc-users] DMRefine fails

2016-02-25 Thread Lawrence Mitchell
On 25/02/16 06:19, Rongliang Chen wrote: > Dear All, > > In my application, I need an overlapping partition of the mesh, that is > to set the overlapping size in DMPlexDistribute(*dm, partitioner, > overlap, NULL, ) to be nonzero (overlap=1). > > At the same time, I want to use the DMRefine

Re: [petsc-users] Strange GAMG performance for mixed FE formulation

2016-03-04 Thread Lawrence Mitchell
> On 4 Mar 2016, at 15:24, Justin Chang wrote: > > So with -pc_gamg_square_graph 10 I get the following: Because you're using gamg inside the fieldsplit, I think you need: -fieldsplit_1_pc_gamg_square_graph 10 > [0] PCSetUp_GAMG(): level 0) N=48000, n data rows=1, n

Re: [petsc-users] Strange GAMG performance for mixed FE formulation

2016-03-03 Thread Lawrence Mitchell
On 02/03/16 22:28, Justin Chang wrote: ... > Down solver (pre-smoother) on level 3 > > KSP Object: (solver_fieldsplit_1_mg_levels_3_) > linear system matrix = precond matrix: ... > Mat Object: 1 MPI processes > >

Re: [petsc-users] multigrid preconditioning and adaptivity

2016-03-07 Thread Lawrence Mitchell
On 07/03/16 14:16, Lukasz Kaczmarczyk wrote: > >> On 7 Mar 2016, at 13:50, Matthew Knepley > > wrote: >> >> On Mon, Mar 7, 2016 at 6:58 AM, Lukasz Kaczmarczyk >> > > wrote: >>

Re: [petsc-users] Nested Fieldsplit for custom index sets

2016-07-27 Thread Lawrence Mitchell
> On 27 Jul 2016, at 03:54, Safin, Artur wrote: > > Barry, > >> Do you have a call to KSPSetFromOptions() before the call >> PCFieldSplitGetSubKSP()? I am guessing not which means that the PC does not >> yet know that it is of type fieldplit. > > Yes, I call

Re: [petsc-users] Nested Fieldsplit for custom index sets

2016-07-28 Thread Lawrence Mitchell
Dear Artur, On 28/07/16 02:20, Safin, Artur wrote: > Barry, Lawrence, > >> I think the SubKSPs (and therefore SubPCs) are not set up until you call >> KSPSetUp(ksp) which your code does not do explicitly and is therefore done >> in KSPSolve. > > I added KSPSetUp(), but unfortunately the issue

Re: [petsc-users] GAMG with PETSc

2016-08-12 Thread Lawrence Mitchell
[Added petsc-maint to cc, since I think this is an actual bug] > On 12 Aug 2016, at 01:16, Gaetan Kenway wrote: > > Hi > > I'm attempting to try out using GAMG for a preconditioner for my compressible > CFD problem. However, I'm getting segfaults when trying to run the

Re: [petsc-users] Regarding ksp ex42 - Citations

2016-07-21 Thread Lawrence Mitchell
> On 21 Jul 2016, at 09:41, domenico lahaye wrote: > > Thanks. > > KSPSetOperators() allows to precondition A^h with M^h. > This is lovely and great as it allows to implement the shifted Laplace > preconditioner for the Helmholtz equation. > > Recently I managed to

Re: [petsc-users] Regarding ksp ex42 - Citations

2016-07-21 Thread Lawrence Mitchell
[Reintroducing petsc-users in cc] On 21/07/16 10:18, domenico lahaye wrote: > Thanks Lauwrence. > > Does the fact that the coarse level preconditioner M^H should be > constructed > by Galerkin coarse (rather then rediscretization) cause additional > wrinkles? Do you want to rediscretise A,

Re: [petsc-users] Regarding ksp ex42 - Citations

2016-07-21 Thread Lawrence Mitchell
> On 21 Jul 2016, at 10:55, domenico lahaye wrote: > > Apologies for being not sufficient clear in my previous message. > > I would like to be able to Galerkin coarsen A^h to obtain A^H > and to separately Galerkin coarsen M^h to obtain M^H. > > So, yes, the way in

Re: [petsc-users] SNESFAS for an unstructured mesh

2016-06-29 Thread Lawrence Mitchell
> On 29 Jun 2016, at 19:47, Khai Pham wrote: > > Thanks for your response. Is DMMoad is other special kind of data structure > in Petsc? I'd like to use the petsc FAS for the finite element code in our > group and all the data structure for mesh topology is all set. Thus

Re: [petsc-users] GAMG huge hash being requested

2017-02-21 Thread Lawrence Mitchell
Hi Justin, On 21/02/17 06:01, Justin Chang wrote: > Okay thanks Now done. Cheers, Lawrence signature.asc Description: OpenPGP digital signature

Re: [petsc-users] GAMG huge hash being requested

2017-02-19 Thread Lawrence Mitchell
> On 19 Feb 2017, at 18:55, Justin Chang wrote: > > Okay, it doesn't seem like the Firedrake fork (which is what I am using) has > this latest fix. Lawrence, when do you think it's possible you folks can > incorporate these fixes I'll fast forward our branch pointer on

Re: [petsc-users] ksp solve error with nested matrix

2017-02-09 Thread Lawrence Mitchell
On 09/02/17 17:29, Barry Smith wrote: > >> On Feb 8, 2017, at 11:39 PM, Dave May wrote: >> >> Any time you modify one of the submats, you need to call assembly begin/end >> on that sub matrix AND on the outer matnest object. > >Weird, and prone to errors it seems

Re: [petsc-users] monitoring the convergence of fieldsplit 0 and 1

2017-01-18 Thread Lawrence Mitchell
On 18/01/17 08:40, Klaij, Christiaan wrote: > Barry, > > I've managed to replicate the problem with 3.7.4 > snes/examples/tutorials/ex70.c. Basically I've added > KSPGetTotalIterations to main (file is attached): PCFieldSplitGetSubKSP returns, in the Schur case: MatSchurComplementGet(pc->schur,

Re: [petsc-users] strong-scaling vs weak-scaling

2016-08-31 Thread Lawrence Mitchell
On 31/08/16 16:46, Mark Adams wrote: > And you can't get much more detail with hypre because it does not > record performance data. Or can you get hypre to print its performance > data? -pc_hypre_boomeramg_print_statistics -pc_hypre_boomeramg_print_debug Should give you some info, not sure it's

Re: [petsc-users] A question of DMPlexGetAdjacency

2016-11-14 Thread Lawrence Mitchell
> On 14 Nov 2016, at 14:59, leejearl wrote: > > Hi all: > >I am not sure the using of the function "DMPlexGetAdjacency". > >My codes are as follows: > >PetscInt adj, *adjSize=NULL; > >ierr = DMPlexGetAdjacency(dm, p, , ); Your calling sequence is wrong:

Re: [petsc-users] pcfieldsplitting not working for VI

2016-11-28 Thread Lawrence Mitchell
> On 29 Nov 2016, at 02:02, Justin Chang wrote: > > Why is this happening? It seems to me VINEWTONRSLS is not understanding that > I have a two-field formulation. For the built in solvers, we set up a dmshell on the snes that knows how to do field splitting. Because

Re: [petsc-users] pcfieldsplitting not working for VI

2016-11-29 Thread Lawrence Mitchell
>> >> I added the following line: >> >> rs_solver.setDM(a._dm) >> >> And it gives this error: >> >> Traceback (most recent call last): >> File "sphere.py", line 128, in >> rs_solver.setDM(a._dm) >> AttributeEr

Re: [petsc-users] PetscFE questions

2016-10-21 Thread Lawrence Mitchell
> On 21 Oct 2016, at 08:26, Julian Andrej wrote: > > On Thu, Oct 20, 2016 at 5:18 PM, Matthew Knepley wrote: >> On Thu, Oct 20, 2016 at 9:42 AM, Julian Andrej wrote: >>> >>> Thanks for the suggestion. I guess DMCreateSubDM can

Re: [petsc-users] FieldSplit and Biot's poroelasticity

2016-12-13 Thread Lawrence Mitchell
On 13/12/16 16:50, Karin wrote: > Dear Petsc-gurus, > > I am solving Biot's poroelasticity problem : > Images intégrées 1 > > I am using a mixed P2-P1 finite element discretization. > > I am using the fieldsplit framework to solve the linear systems. Here > are the options I am using : >

Re: [petsc-users] FieldSplit, multigrid and blocksize

2016-12-07 Thread Lawrence Mitchell
On 07/12/16 13:43, Karin wrote: > Thanks Barry. > I must emphasize that my unknowns are not numbered in a regular way : > I am using a P2-P1 finite element and the middle nodes do not carry a > pressure DOF. So the global numbering is somewhat like : >

Re: [petsc-users] petsc4py --with-64-bit-indices

2016-12-17 Thread Lawrence Mitchell
> On 17 Dec 2016, at 19:19, Barry Smith wrote: > > Looks like --install-option= are options for pip not the underlying package. > > Lisandro, how does one do what seems to be a simple request? Set PETSC_CONFIGURE_OPTIONS to any additional flags you want to pass to

Re: [petsc-users] left and right preconditioning with a constant null space

2017-03-23 Thread Lawrence Mitchell
On 23/03/17 08:42, Klaij, Christiaan wrote: > Matt, Lawrence > > > The same problem happens when using gmres with rtol 1e-6 in the > schur complement (attachment "left_schur"). I'm not sure what > this tells us. If I understand Lawrence correctly, the null space > may be attached to the wrong

Re: [petsc-users] left and right preconditioning with a constant null space

2017-03-22 Thread Lawrence Mitchell
> On 22 Mar 2017, at 16:39, Matthew Knepley wrote: > > Hmm, I wonder if the problem is that A11 has the nullspace, but the PC pmat > is actually B^T diag(A)^{-1} B. I have > to look where the solver is actually taking the nullspace from. We need to > improve the -ksp_view

Re: [petsc-users] left and right preconditioning with a constant null space

2017-03-23 Thread Lawrence Mitchell
On 23/03/17 15:37, Klaij, Christiaan wrote: > Yes, that's clearer, thanks! I do have is0 and is1 so I can try > PetscObjectCompose and let you know. > > Note though that the viewer reports that both S and A11 have a > null space attached... My matrix is a matnest and I've attached a > null

Re: [petsc-users] left and right preconditioning with a constant null space

2017-03-24 Thread Lawrence Mitchell
> On 24 Mar 2017, at 15:11, Klaij, Christiaan wrote: > > I've written a small PETSc program that loads the four blocks, > constructs Sp, attaches the null space and solves with a random > rhs vector. > > This small program replicates the same behaviour as the real > code:

Re: [petsc-users] Configure nested PCFIELDSPLIT with general index sets

2017-03-21 Thread Lawrence Mitchell
> On 21 Mar 2017, at 13:24, Matthew Knepley wrote: > > I think the remedy is as easy as specifying a DMShell that has a PetscSection > (DMSetDefaultSection) with your ordering, and > I think this is how Firedrake (http://www.firedrakeproject.org/) does it. We actually don't

Re: [petsc-users] Configuring PETSc for KNL

2017-04-06 Thread Lawrence Mitchell
On 06/04/17 12:25, Matthew Knepley wrote: > I'm not sure whether getting the Intel acronyms mixed up (KNL vs MKL) > makes the quote above better or worse. > > > Too cryptic. Are you saying that this cannot be what is happening? How > would you explain > the drop off in performance? >

Re: [petsc-users] Partitioning does not work

2017-04-18 Thread Lawrence Mitchell
> On 15 Apr 2017, at 22:31, Barry Smith wrote: > > > The DMPlex experts and point you in the correct direction. > > >> On Apr 15, 2017, at 4:28 PM, Orxan Shibliyev wrote: >> >> About partitioning of DMPlex, manual says that, >> >> In exactly

Re: [petsc-users] Is OpenMP still available for PETSc?

2017-07-03 Thread Lawrence Mitchell
> On 3 Jul 2017, at 15:23, Damian Kaliszan wrote: > > Hi, > > >>> 1) You can call Bcast on PETSC_COMM_WORLD > > To be honest I can't find Bcast method in petsc4py.PETSc.Comm (I'm > using petsc4py) Use PETSc.COMM_WORLD.tompi4py() to get an mpi4py communicator that

Re: [petsc-users] DMPlex distribution with custom adjacency

2017-08-04 Thread Lawrence Mitchell
> On 6 Jun 2017, at 09:01, Lawrence Mitchell <lawrence.mitch...@imperial.ac.uk> > wrote: > >> If you do really want to prune them, then I guess overriding the >> DMPlexGetAdjacency() as you propose is probably the best way. I would >> be willing to put it in

Re: [petsc-users] Petsc ILU PC Change between 3.6.4 and 3.7.x?

2017-08-22 Thread Lawrence Mitchell
On 22/08/17 14:49, Klaij, Christiaan wrote: > CALL > PetscOptionsSetValue(PETSC_NULL_OBJECT,"-sub_pc_factor_shift_type","nonzero",ierr) > > since there seems to be no setter function for this (correct me > if I'm wrong). I think you want PCFactorSetShiftType. Lawrence

Re: [petsc-users] Petsc ILU PC Change between 3.6.4 and 3.7.x?

2017-08-23 Thread Lawrence Mitchell
On 23/08/17 07:51, Klaij, Christiaan wrote: > Lawrence, > > Well, I was looking for something like SubPCFactorSetShiftType, > or is it supposed to somehow "trickle down" to the sub pc's? Ah, I see. No, the approach below is what you need. Imagine that you used a sub_pc inside a multigrid

Re: [petsc-users] how to get the vertices belongs to a control volume in 2D?

2017-05-12 Thread Lawrence Mitchell
> On 12 May 2017, at 04:57, 李季 wrote: > > Hi developers: > I have such a question that I want to get the vertices of a cell. I know > I can get the points by > 1. Getting the faces of a cell such as "DMPlexGetCone(dm, c, "; > 2. Getting the vertices of every face

Re: [petsc-users] a question about PetscSectionCreate

2017-05-28 Thread Lawrence Mitchell
> On 28 May 2017, at 09:16, leejearl wrote: > > Hi, Dave: I want to store a PetscInt tag for every cell of the dmplex with > the struct. Thanks, You probably want to use a DMLabel to store these ids. Unless you have a different I'd for every cell. Lawrence

Re: [petsc-users] DMPlex distribution with FVM adjacency

2017-05-25 Thread Lawrence Mitchell
> On 25 May 2017, at 18:05, Matthew Knepley wrote: > > If you want that, is there a reason you cannot use the FEM style FALSE+TRUE? > If you already want the closure, usually the star is not really adding > anything new. Ok, let me clarify. Given shared facets, I'd like

[petsc-users] DMPlex distribution with FVM adjacency

2017-05-25 Thread Lawrence Mitchell
Dear petsc-users, I am trying to distribute a triangle mesh with a cell halo defined by FVM adjacency (i.e. if I have a facet in the initial (0-overlap) distribution, I want the cell on the other side of it). Reading the documentation, I think I do: DMPlexSetAdjacencyUseCone(PETSC_TRUE)

Re: [petsc-users] DMPlex distribution with custom adjacency

2017-06-05 Thread Lawrence Mitchell
> On 2 Jun 2017, at 05:09, Matthew Knepley wrote: > > > Coming back to this, I think I understand the problem a little better. > > Consider this mesh: > > ++ > |\ 3 | > | \ | > |2 \ | > | \| > ++ > |\ 1 | > | \ | > |0 \ | > | \| > ++ > > Let's say I run

Re: [petsc-users] DMPlex distribution with FVM adjacency

2017-06-01 Thread Lawrence Mitchell
On 25/05/17 21:00, Matthew Knepley wrote: > On Thu, May 25, 2017 at 2:22 PM, Lawrence Mitchell > <lawrence.mitch...@imperial.ac.uk > <mailto:lawrence.mitch...@imperial.ac.uk>> wrote: > > > > On 25 May 2017, at 20:03, Matthew Knepley <knep...@gmail.com

Re: [petsc-users] DMPlex distribution with FVM adjacency

2017-06-01 Thread Lawrence Mitchell
On 01/06/17 16:51, Lawrence Mitchell wrote: ... > So I can write a "DMPlexGetAdjacency" information that only returns > non-empty adjacencies for facets. But it's sort of lying about what > it does now. Proposed a PR that allows the user to specify what they want adjacency t

Re: [petsc-users] DMPlex distribution with FVM adjacency

2017-06-01 Thread Lawrence Mitchell
On 1 Jun 2017, at 19:12, Stefano Zampini wrote: >> Now I'd like to grow the overlap such that any cell I can see through >> a facet (and its closure) lives in the overlap. >> > > Lawrence, why do you need the closure here? Why facet adjacency is not enough? Sorry,

Re: [petsc-users] DMPlex distribution with custom adjacency

2017-06-06 Thread Lawrence Mitchell
> On 5 Jun 2017, at 23:01, Matthew Knepley wrote: > > To do a FEM integral on a facet f I need: > > i) to evaluate coefficients at quadrature points (on the facet) > ii) to evaluate basis functions at quadrature points (on the facet) > > for (i), I need all the dofs in

Re: [petsc-users] DMPlex distribution with FVM adjacency

2017-05-25 Thread Lawrence Mitchell
On 25/05/17 16:25, Matthew Knepley wrote: > On Thu, May 25, 2017 at 9:27 AM, Lawrence Mitchell > <lawrence.mitch...@imperial.ac.uk > <mailto:lawrence.mitch...@imperial.ac.uk>> wrote: > > Dear petsc-users, > > I am trying to distribute a trian

Re: [petsc-users] DMPlex distribution with FVM adjacency

2017-05-25 Thread Lawrence Mitchell
> On 25 May 2017, at 19:23, Matthew Knepley wrote: > > Ok, let me clarify. > > Given shared facets, I'd like closure(support(facet)) this is a subset of the > fem adjacency. "Add in the cell and its closure from the remote rank". This > doesn't include remote cells I can

Re: [petsc-users] DMPlex distribution with FVM adjacency

2017-05-25 Thread Lawrence Mitchell
> On 25 May 2017, at 19:46, Matthew Knepley wrote: > > Sounds like DG. I will get out my dead chicken for the incantation Actually no! Mixed H(div)-L2 for Stokes. Which has facet integrals for partially discontinuous fields. If you do redundant compute for such terms,

Re: [petsc-users] DMPlex distribution with FVM adjacency

2017-05-25 Thread Lawrence Mitchell
> On 25 May 2017, at 20:03, Matthew Knepley wrote: > > > Hmm, I thought I made adjacency per field. I have to look. That way, no > problem with the Stokes example. DG is still weird. You might, we don't right now. We just make the topological adjacency that is "large

[petsc-users] PETSc user meeting 2018, June 4-6, London

2017-10-12 Thread Lawrence Mitchell
, and abstract submission will follow in early 2018. We look forward to seeing you in London's famous London. Please contact petsc2...@mcs.anl.gov if you have any questions, suggestions, or comments. Lawrence Mitchell (for the organising committee)

Re: [petsc-users] Pulling submeshes out of a DMPlex object

2017-09-06 Thread Lawrence Mitchell
On 6 Sep 2017, at 20:58, Matthew Knepley wrote: >> 1. The locally visible part of a distributed mesh (so that I can >> discretise and solve independent problems on each local part) > > This is already a standalone Plex. If you delete the PetscSF, it never knows > anyone

[petsc-users] Pulling submeshes out of a DMPlex object

2017-09-06 Thread Lawrence Mitchell
Dear all, for various reasons I'd like, given an existing DMPlex mesh, to be able to pull submeshes out. These might take various forms: 1. The locally visible part of a distributed mesh (so that I can discretise and solve independent problems on each local part) 2. Some subset of entities of

[petsc-users] Understanding matmult memory performance

2017-09-29 Thread Lawrence Mitchell
Dear all, I'm attempting to understand some results I'm getting for matmult performance. In particular, it looks like I'm obtaining timings that suggest that I'm getting more main memory bandwidth than I think is possible. The run setup is using 2 24 core (dual socket) ivybridge nodes (Xeon

Re: [petsc-users] Understanding matmult memory performance

2017-09-29 Thread Lawrence Mitchell
> On 29 Sep 2017, at 15:05, Tobin Isaac <tis...@cc.gatech.edu> wrote: > > On Fri, Sep 29, 2017 at 09:04:47AM -0400, Tobin Isaac wrote: >> On Fri, Sep 29, 2017 at 12:19:54PM +0100, Lawrence Mitchell wrote: >>> Dear all, >>> >>> I'm attempting to

Re: [petsc-users] Understanding matmult memory performance

2017-09-29 Thread Lawrence Mitchell
> On 29 Sep 2017, at 15:24, Lawrence Mitchell > <lawrence.mitch...@imperial.ac.uk> wrote: > >> according to >> https://ark.intel.com/products/75283/Intel-Xeon-Processor-E5-2697-v2-30M-Cache-2_70-GHz >> you get 59.7 GB/sec of peak memory bandwidth per CPU, so yo

Re: [petsc-users] MatSetValues with openMP and no data races

2017-10-12 Thread Lawrence Mitchell
On 12/10/17 14:40, Guillaume Emond wrote: > Goodmorning, > > I would like to clarify a point about the insertion of values with > MatSetValues in a openMP loop. I know these routines are not thread > safe. But, in our situation, we used a graph coloring algorithm on our > mesh to make

  1   2   >