Re: [petsc-users] Increasing norm with finer mesh

[Smith, Barry F.]

   Please send your version of the example that computes the mean norm of the 
grid; I suspect we are talking apples and oranges

   Barry



> On Oct 1, 2018, at 7:51 PM, Weizhuo Wang  wrote:
> 
> I also tried to divide the norm by m*n , which is the number of grids, the 
> trend of norm still increases.
> 
> Thanks!
> 
> Weizhuo
> 
> Matthew Knepley  于2018年10月1日周一 下午7:45写道：
> On Mon, Oct 1, 2018 at 6:31 PM Weizhuo Wang  wrote:
> Hi!
> 
> I'm recently trying out the example code provided with the KSP solver 
> (ex12.c). I noticed that the mean norm of the grid increases as I use finer 
> meshes. For example, the mean norm is 5.72e-8 at m=10 n=10. However at m=100, 
> n=100, mean norm increases to 9.55e-6. This seems counter intuitive, since 
> most of the time error should decreases when using finer grid. Am I doing 
> this wrong?
> 
> The norm is misleading in that it is the l_2 norm, meaning just the sqrt of 
> the sum of the squares of
> the vector entries. It should be scaled by the volume element to approximate 
> a scale-independent
> norm (like the L_2 norm).
> 
>   Thanks,
> 
>  Matt
>  
> Thanks! 
> -- 
> Wang Weizhuo
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments 
> is infinitely more interesting than any results to which their experiments 
> lead.
> -- Norbert Wiener
> 
> https://www.cse.buffalo.edu/~knepley/
> 
> 
> -- 
> Wang Weizhuo

Re: [petsc-users] Increasing norm with finer mesh

[Matthew Knepley]

On Mon, Oct 1, 2018 at 8:51 PM Weizhuo Wang  wrote:

> I also tried to divide the norm by m*n , which is the number of grids, the
> trend of norm still increases.
>

We need to be precise. First, look at the initial residual, because that is
what you control with the initial
guess. You are saying that the initial residual does not asymptote? I would
be reluctant to believe that.

  Thanks,

 Matt


> Thanks!
>
> Weizhuo
>
> Matthew Knepley  于2018年10月1日周一 下午7:45写道：
>
>> On Mon, Oct 1, 2018 at 6:31 PM Weizhuo Wang 
>> wrote:
>>
>>> Hi!
>>>
>>> I'm recently trying out the example code provided with the KSP solver
>>> (ex12.c). I noticed that the mean norm of the grid increases as I use finer
>>> meshes. For example, the mean norm is 5.72e-8 at m=10 n=10. However at
>>> m=100, n=100, mean norm increases to 9.55e-6. This seems counter intuitive,
>>> since most of the time error should decreases when using finer grid. Am I
>>> doing this wrong?
>>>
>>
>> The norm is misleading in that it is the l_2 norm, meaning just the sqrt
>> of the sum of the squares of
>> the vector entries. It should be scaled by the volume element to
>> approximate a scale-independent
>> norm (like the L_2 norm).
>>
>>   Thanks,
>>
>>  Matt
>>
>>
>>> Thanks!
>>> --
>>> Wang Weizhuo
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> 
>>
>
>
> --
> Wang Weizhuo
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] Increasing norm with finer mesh

[Weizhuo Wang]

I also tried to divide the norm by m*n , which is the number of grids, the
trend of norm still increases.

Thanks!

Weizhuo

Matthew Knepley  于2018年10月1日周一 下午7:45写道：

> On Mon, Oct 1, 2018 at 6:31 PM Weizhuo Wang  wrote:
>
>> Hi!
>>
>> I'm recently trying out the example code provided with the KSP solver
>> (ex12.c). I noticed that the mean norm of the grid increases as I use finer
>> meshes. For example, the mean norm is 5.72e-8 at m=10 n=10. However at
>> m=100, n=100, mean norm increases to 9.55e-6. This seems counter intuitive,
>> since most of the time error should decreases when using finer grid. Am I
>> doing this wrong?
>>
>
> The norm is misleading in that it is the l_2 norm, meaning just the sqrt
> of the sum of the squares of
> the vector entries. It should be scaled by the volume element to
> approximate a scale-independent
> norm (like the L_2 norm).
>
>   Thanks,
>
>  Matt
>
>
>> Thanks!
>> --
>> Wang Weizhuo
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> 
>


-- 
Wang Weizhuo

Re: [petsc-users] Increasing norm with finer mesh

[Matthew Knepley]

On Mon, Oct 1, 2018 at 6:31 PM Weizhuo Wang  wrote:

> Hi!
>
> I'm recently trying out the example code provided with the KSP solver
> (ex12.c). I noticed that the mean norm of the grid increases as I use finer
> meshes. For example, the mean norm is 5.72e-8 at m=10 n=10. However at
> m=100, n=100, mean norm increases to 9.55e-6. This seems counter intuitive,
> since most of the time error should decreases when using finer grid. Am I
> doing this wrong?
>

The norm is misleading in that it is the l_2 norm, meaning just the sqrt of
the sum of the squares of
the vector entries. It should be scaled by the volume element to
approximate a scale-independent
norm (like the L_2 norm).

  Thanks,

 Matt


> Thanks!
> --
> Wang Weizhuo
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

[petsc-users] Increasing norm with finer mesh

[Weizhuo Wang]

Hi!

I'm recently trying out the example code provided with the KSP solver
(ex12.c). I noticed that the mean norm of the grid increases as I use finer
meshes. For example, the mean norm is 5.72e-8 at m=10 n=10. However at
m=100, n=100, mean norm increases to 9.55e-6. This seems counter intuitive,
since most of the time error should decreases when using finer grid. Am I
doing this wrong?

Thanks!
-- 
Wang Weizhuo

Re: [petsc-users] Fortran undefined reference

[Matthew Knepley]

On Mon, Oct 1, 2018 at 2:44 PM Josh L  wrote:

> Hi,
>
> I am using PETSc makefile to compile my code. few DMPlex function ,
> e.g. PetscPartitionerSetType, DMPlexGetCellNumbering etc, fails the
> linking(undefined reference to x).  I am using PETSc/3.9.
> Anyone knows what is causing it ?
>

Yes. We have to write Fortran bindings by hand, so not all functions has
them yet. Could you make
a list of all the ones you want fixed, and I will do it?

  Thanks,

 Matt


> Thanks,
> Josh
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

[petsc-users] Fortran undefined reference

[Josh L]

Hi,

I am using PETSc makefile to compile my code. few DMPlex function ,
e.g. PetscPartitionerSetType, DMPlexGetCellNumbering etc, fails the
linking(undefined reference to x).  I am using PETSc/3.9.
Anyone knows what is causing it ?


Thanks,
Josh

Re: [petsc-users] DMDA with dimension of size 1

[Phil Tooley]

Hi Patrick,

You are spot on, I was copying in a vector and not taking care to pad
with 1s to length 3.

Thanks

Phil

On 01/10/2018 15:57, Patrick Sanan wrote:
> Whoops, sent that patch too fast (forgot to update SETERRQ3 to
> SETERRQ4). Updated patch for maint attached.
>
> Am Mo., 1. Okt. 2018 um 16:55 Uhr schrieb Patrick Sanan
> mailto:patrick.sa...@gmail.com>>:
>
> Meshes of size 1 should work.
>
> Looks like there is a bug in the code to produce this error
> message (patch for maint attached). It's not outputting the "P"
> (size in the 3rd dimension) component.
>
> This is just speculation without a full error message or code to
> reproduce, but perhaps the size in the third dimension is an
> uninitialized value which is triggering this warning with a 10 x
> 10 x (huge garbage) x 1 (dof) mesh.
>
> Am Mo., 1. Okt. 2018 um 15:59 Uhr schrieb Phil Tooley
> mailto:phil.too...@sheffield.ac.uk>>:
>
> Hi all,
>
> Is it valid to have a DMDA with one of the dimensions of size
> 1.  I was
> hoping to avoid having to write explicit logic to handle the
> case that I
> am working on a 2D rather than a 3D image for my current
> application.  
> Unfortunately when I try to construct such a DMDA I get an error:
>
> [0]PETSC ERROR: - Error Message
> --
> [0]PETSC ERROR: Overflow in integer operation:
> [0]PETSC ERROR: Mesh of 10 by 10 by 1 (dof) is too large for
> 32 bit indices
>
> Is there a workaround other than "write everything twice"?
>
> Thanks
>
> Phil
>
> -- 
> Phil Tooley
> Research Software Engineering
> University of Sheffield
>
-- 
Phil Tooley
Research Software Engineering
University of Sheffield

Re: [petsc-users] MatPtAPNumeric_MPIAIJ_MPIAIJ_scalable

[Fande Kong]

Thanks, Jed. I figured out. I just simply made P more sparse, then the
product would not take too much memory.

Fande,

On Fri, Sep 28, 2018 at 9:59 PM Jed Brown  wrote:

> It depends entirely on your matrices.  For example, if A is an arrowhead
> matrix (graph of a star -- one hub and many leaves) then A^2 is dense.
> If you have particular stencils for A and P, then we could tell you the
> fill ratio.
>
> Fande Kong  writes:
>
> > Hi All,
> >
> > I was wondering how much memory is required to get PtAP done? Do you have
> > any simple formula to this? So that I can have an  estimate.
> >
> >
> > Fande,
> >
> >
> > [132]PETSC ERROR: - Error Message
> > --
> > [132]PETSC ERROR: Out of memory. This could be due to allocating
> > [132]PETSC ERROR: too large an object or bleeding by not properly
> > [132]PETSC ERROR: destroying unneeded objects.
> > [132]PETSC ERROR: Memory allocated 0 Memory used by process 3249920
> > [132]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
> > [132]PETSC ERROR: Memory requested 89148704
> > [132]PETSC ERROR: See
> http://www.mcs.anl.gov/petsc/documentation/faq.html
> > for trouble shooting.
> > [132]PETSC ERROR: Petsc Release Version 3.9.4, unknown
> > [132]PETSC ERROR: ../../rattlesnake-opt on a arch-theta-avx512-64-opt
> named
> > nid03830 by fdkong Fri Sep 28 22:43:45 2018
> > [132]PETSC ERROR: Configure options --LIBS=-lstdc++
> > --known-64-bit-blas-indices=0 --known-bits-per-byte=8
> > --known-has-attribute-aligned=1 --known-level1-dcache-assoc=8
> > --known-level1-dcache-linesize=64 --known-level1-dcache-size=32768
> > --known-memcmp-ok=1 --known-mklspblas-supports-zero-based=0
> > --known-mpi-c-double-complex=1 --known-mpi-int64_t=1
> > --known-mpi-long-double=1 --known-mpi-shared-libraries=0
> > --known-sdot-returns-double=0 --known-sizeof-MPI_Comm=4
> > --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8
> > --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8
> > --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8
> > --known-sizeof-void-p=8 --known-snrm2-returns-double=0 --with-batch=1
> > --with-blaslapack-lib="-mkl
> > -L/opt/intel/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64"
> > --with-cc=cc --with-clib-autodetect=0 --with-cxx=CC
> > --with-cxxlib-autodetect=0 --with-debugging=0 --with-fc=ftn
> > --with-fortranlib-autodetect=0 --with-hdf5=0 --with-memalign=64
> > --with-mpiexec=aprun --with-shared-libraries=0 --download-metis=1
> > --download-parmetis=1 --download-superlu_dist=1 --download-hypre=1
> > --download-ptscotch=1 COPTFLAGS="-O3 -xMIC-AVX512" CXXOPTFLAGS="-O3
> > -xMIC-AVX512" FOPTFLAGS="-O3 -xMIC-AVX512"
> > PETSC_ARCH=arch-theta-avx512-64-opt --with-64-bit-indices=1
> > [132]PETSC ERROR: #1 PetscSegBufferCreate() line 64 in
> > /gpfs/mira-home/fdkong/petsc/src/sys/utils/segbuffer.c
> > [132]PETSC ERROR: #2 PetscSegBufferCreate() line 64 in
> > /gpfs/mira-home/fdkong/petsc/src/sys/utils/segbuffer.c
> > [132]PETSC ERROR: #3 PetscSegBufferExtractInPlace() line 227 in
> > /gpfs/mira-home/fdkong/petsc/src/sys/utils/segbuffer.c
> > [132]PETSC ERROR: #4 MatStashScatterBegin_BTS() line 854 in
> > /gpfs/mira-home/fdkong/petsc/src/mat/utils/matstash.c
> > [132]PETSC ERROR: #5 MatStashScatterBegin_Private() line 461 in
> > /gpfs/mira-home/fdkong/petsc/src/mat/utils/matstash.c
> > [132]PETSC ERROR: #6 MatAssemblyBegin_MPIAIJ() line 683 in
> > /gpfs/mira-home/fdkong/petsc/src/mat/impls/aij/mpi/mpiaij.c
> > [132]PETSC ERROR: #7 MatAssemblyBegin() line 5158 in
> > /gpfs/mira-home/fdkong/petsc/src/mat/interface/matrix.c
> > [132]PETSC ERROR: #8 MatPtAPNumeric_MPIAIJ_MPIAIJ_scalable() line 262 in
> > /gpfs/mira-home/fdkong/petsc/src/mat/impls/aij/mpi/mpiptap.c
> > [132]PETSC ERROR: #9 MatPtAP_MPIAIJ_MPIAIJ() line 172 in
> > /gpfs/mira-home/fdkong/petsc/src/mat/impls/aij/mpi/mpiptap.c
> > [132]PETSC ERROR: #10 MatPtAP() line 9182 in
> > /gpfs/mira-home/fdkong/petsc/src/mat/interface/matrix.c
> > [132]PETSC ERROR: #11 MatGalerkin() line 10615 in
> > /gpfs/mira-home/fdkong/petsc/src/mat/interface/matrix.c
> > [132]PETSC ERROR: #12 PCSetUp_MG() line 730 in
> > /gpfs/mira-home/fdkong/petsc/src/ksp/pc/impls/mg/mg.c
> > [132]PETSC ERROR: #13 PCSetUp_HMG() line 336 in
> > /gpfs/mira-home/fdkong/petsc/src/ksp/pc/impls/hmg/hmg.c
> > [132]PETSC ERROR: #14 PCSetUp() line 923 in
> > /gpfs/mira-home/fdkong/petsc/src/ksp/pc/interface/precon.c
> > [132]PETSC ERROR: #15 KSPSetUp() line 381 in
> > /gpfs/mira-home/fdkong/petsc/src/ksp/ksp/interface/itfunc.c
> > [136]PETSC ERROR: - Error Message
> > --
>

Re: [petsc-users] DMDA with dimension of size 1

[Patrick Sanan]

Whoops, sent that patch too fast (forgot to update SETERRQ3 to SETERRQ4).
Updated patch for maint attached.

Am Mo., 1. Okt. 2018 um 16:55 Uhr schrieb Patrick Sanan <
patrick.sa...@gmail.com>:

> Meshes of size 1 should work.
>
> Looks like there is a bug in the code to produce this error message (patch
> for maint attached). It's not outputting the "P" (size in the 3rd
> dimension) component.
>
> This is just speculation without a full error message or code to
> reproduce, but perhaps the size in the third dimension is an uninitialized
> value which is triggering this warning with a 10 x 10 x (huge garbage) x 1
> (dof) mesh.
>
> Am Mo., 1. Okt. 2018 um 15:59 Uhr schrieb Phil Tooley <
> phil.too...@sheffield.ac.uk>:
>
>> Hi all,
>>
>> Is it valid to have a DMDA with one of the dimensions of size 1.  I was
>> hoping to avoid having to write explicit logic to handle the case that I
>> am working on a 2D rather than a 3D image for my current application.
>> Unfortunately when I try to construct such a DMDA I get an error:
>>
>> [0]PETSC ERROR: - Error Message
>> --
>> [0]PETSC ERROR: Overflow in integer operation:
>> [0]PETSC ERROR: Mesh of 10 by 10 by 1 (dof) is too large for 32 bit
>> indices
>>
>> Is there a workaround other than "write everything twice"?
>>
>> Thanks
>>
>> Phil
>>
>> --
>> Phil Tooley
>> Research Software Engineering
>> University of Sheffield
>>
>>


0001-DMSetUp_DA_3D-fix-warning-message-for-int32-mesh-siz.patch
Description: Binary data

Re: [petsc-users] DMDA with dimension of size 1

[Patrick Sanan]

Meshes of size 1 should work.

Looks like there is a bug in the code to produce this error message (patch
for maint attached). It's not outputting the "P" (size in the 3rd
dimension) component.

This is just speculation without a full error message or code to reproduce,
but perhaps the size in the third dimension is an uninitialized value which
is triggering this warning with a 10 x 10 x (huge garbage) x 1 (dof) mesh.

Am Mo., 1. Okt. 2018 um 15:59 Uhr schrieb Phil Tooley <
phil.too...@sheffield.ac.uk>:

> Hi all,
>
> Is it valid to have a DMDA with one of the dimensions of size 1.  I was
> hoping to avoid having to write explicit logic to handle the case that I
> am working on a 2D rather than a 3D image for my current application.
> Unfortunately when I try to construct such a DMDA I get an error:
>
> [0]PETSC ERROR: - Error Message
> --
> [0]PETSC ERROR: Overflow in integer operation:
> [0]PETSC ERROR: Mesh of 10 by 10 by 1 (dof) is too large for 32 bit indices
>
> Is there a workaround other than "write everything twice"?
>
> Thanks
>
> Phil
>
> --
> Phil Tooley
> Research Software Engineering
> University of Sheffield
>
>


0001-DMSetUp_DA_3D-fix-warning-message-for-int32-mesh-siz.patch
Description: Binary data

[petsc-users] DMDA with dimension of size 1

[Phil Tooley]

Hi all,

Is it valid to have a DMDA with one of the dimensions of size 1.  I was
hoping to avoid having to write explicit logic to handle the case that I
am working on a 2D rather than a 3D image for my current application.  
Unfortunately when I try to construct such a DMDA I get an error:

[0]PETSC ERROR: - Error Message
--
[0]PETSC ERROR: Overflow in integer operation:
[0]PETSC ERROR: Mesh of 10 by 10 by 1 (dof) is too large for 32 bit indices

Is there a workaround other than "write everything twice"?

Thanks

Phil

-- 
Phil Tooley
Research Software Engineering
University of Sheffield

Re: [petsc-users] Fwd: Implementing a homotopy solver

[Matthew Knepley]

On Sun, Sep 30, 2018 at 6:06 PM zakaryah  wrote:

> OK, thanks.
>
> I'm using a composite DM, DM_packer.  To make a separate KSP, I did the
> following in the FormJacobian() routine, after assembling the Jacobian
> matrix A and the RHS for the tangent vector, b:
>
>- KSPCreate(PETSC_COMM_WORLD,)
>- KSPSetDM(ksp,DM_packer)
>- KSPSetDMActive(ksp,PETSC_FALSE) - because I want to set the operators
>- KSPSetOperators(ksp,A,P)
>- VecSet(n,0) - set initial guess to zero
>- KSPSolve(ksp,b,n) - this solve works correctly
>- VecNormalize(n,NULL)
>-
>
> MatNullSpaceCreate(PetscObjectComm((PetscObject)A),PETSC_FALSE,1,,)
>- MatSetNullSpace(A,nullsp)
>
> Then, with -snes_type newtonls -pc_type none -ksp_monitor
> -ksp_monitor_true_residual -ksp_view, the output for the KSPSolve described
> above, i.e. for the tangent vector, looks correct:
>
>   0 KSP preconditioned resid norm 1.e+03 true resid norm
> 1.e+03 ||r(i)||/||b|| 1.e+00
> ...
>
> 185 KSP preconditioned resid norm 9.900713131874e-03 true resid norm
> 9.900713131904e-03 ||r(i)||/||b|| 9.900713131904e-06
>
> Linear solve converged due to CONVERGED_RTOL iterations 185
>
> KSP Object: 1 MPI processes
>
>   type: gmres
>
> restart=30, using Classical (unmodified) Gram-Schmidt
> Orthogonalization with no iterative refinement
>
> happy breakdown tolerance 1e-30
>
>   maximum iterations=1, initial guess is zero
>
>   tolerances:  relative=1e-05, absolute=1e-50, divergence=1.
>
>   left preconditioning
>
>   using PRECONDITIONED norm type for convergence test
>
> PC Object: 1 MPI processes
>
>   type: none
>
>   linear system matrix = precond matrix:
>
>   Mat Object: 1 MPI processes
>
> type: seqaij
>
> rows=78247, cols=78247
>
> total: nonzeros=6063481, allocated nonzeros=6063481
>
> total number of mallocs used during MatSetValues calls =0
>
>   using I-node routines: found 26083 nodes, limit used is 5
>
>
> But the next KSP, i.e. the one in the SNES, doesn't converge in the true
> residual:
>
>
>   0 KSP preconditioned resid norm 2.045599092896e-04 true resid norm
> 2.803828296212e-04 ||r(i)||/||b|| 1.e+00
>
> 191 KSP preconditioned resid norm 2.009941278534e-09 true resid norm
> 1.636010142734e-04 ||r(i)||/||b|| 5.834915586465e-01
>
>
> KSP Object: 1 MPI processes
>
>   type: gmres
>
> restart=30, using Classical (unmodified) Gram-Schmidt
> Orthogonalization with no iterative refinement
>
> happy breakdown tolerance 1e-30
>
>   maximum iterations=1, initial guess is zero
>
>   tolerances:  relative=1e-05, absolute=1e-50, divergence=1.
>
>   left preconditioning
>
>   using PRECONDITIONED norm type for convergence test
>
> PC Object: 1 MPI processes
>
>   type: none
>
>   linear system matrix = precond matrix:
>
>   Mat Object: 1 MPI processes
>
> type: seqaij
>
> rows=78247, cols=78247
>
> total: nonzeros=6063481, allocated nonzeros=6063481
>
> total number of mallocs used during MatSetValues calls =0
>
>   has attached null space
>
>   using I-node routines: found 26083 nodes, limit used is 5
>
>
> My guess about what's going on is that the tangent vector n isn't really
> in the nullspace of A.
>

I do not understand the rest, but this is the problem. Maybe it would make
more sense if you wrote things
in linear algebraic notation.

  Thanks,

 Matt


> Rather, it's in the nullspace of the m x (m+1) submatrix of A.  So, An=c
> e_{m+1}, where c is an arbitrary constant and e_{m+1} is the m+1 th basis
> vector.  The nonlinear function also has an added row, F_{m+1}, which is
> set to zero in the FormFunction() routine.  I don't care about the value of
> F_{m+1}, but I suppose that if it's included in the true residual, and NOT
> in the "preconditioned" residual, even with pc_type none, then it will be
> tricky to diagnose the performance.  Should I be using my own routine to
> evaluate the residual, so that F_{m+1} is not included?
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/