Re: [petsc-users] dmplex face normals orientation

[Ingo Gaertner]

Thank you, Matt, this answers my question.

Ingo

2017-04-18 22:21 GMT+02:00 Matthew Knepley :

> On Tue, Apr 18, 2017 at 2:33 PM, Ingo Gaertner  > wrote:
>
>> The part that does not make sense is:
>> The code calculates that
>> face 11 (or edge 11 if you call the boundary of a 2D cell an edge) has
>> the centroid c=(0.75 1.00 0.00) and the normal n=(0.00
>> 0.50 0.00) and that
>> face 12 (or edge 12) has the centroid c=(0.00 0.50 0.00) and
>> the normal n=(-1.00 -0.00 0.00).
>> I understood your previous answer ("The normal should be outward, unless
>> the face has orientation < 0") such that I have to reverse these normals to
>> get an outward normal, because faces 11 and 12 have orientation=-2. But the
>> normals n=(0.00 0.50 0.00) and n=(-1.00 -0.00
>> 0.00) do already point outward. If I reverse them, they point inward.
>>
>> I need a CONSISTENT rule how to make use of the normals obtained from
>> DMPlexComputeGeometryFVM(dm, ,) to calculate OUTWARD
>> pointing normals with respect to EACH INDIVIDUAL cell.
>> Or do I have to iterate over each face of each cell and use a geometric
>> check to see if the calculated normals are pointing inward or outward?
>>
>
> I apologize. I did not understand the question before. The convention I
> have chosen might not be the best one,
> but it seems appropriate for FVM.
>
> The orientation of a face normal is chosen such that
>
>   n . r > 0
>
> where r is the vector from the centroid of the left face to
> the centroid of the right face. If we do GetSupport() for
> a face we get back {l, r}, where r could be empty, meaning
> the cell at infinity.
>
> This convention means that I never have to check orientations
> when I am using the facegeom[] stuff in FVM, I just need to have
> {l, r} which I generally have in the loop.
>
>   Thanks,
>
>  Matt
>
>
>> Thanks
>> Ingo
>>
>>
>>
>> 2017-04-18 20:20 GMT+02:00 Matthew Knepley :
>>
>>> On Tue, Apr 18, 2017 at 12:46 AM, Ingo Gaertner <
>>> ingogaertner@gmail.com> wrote:
>>>
 Dear Matt,
 please explain your previous answer ("The normal should be outward,
 unless the face has orientation < 0") with respect to my example.
 As I think, the example shows that the face normals for faces 11 and 12
 are pointing outward, although they have orientation=-2. For all other
 faces the normal direction and their orientation sign agree with what you
 said.

>>>
>>> 1) You should destroy the objects at the end
>>>
>>>   ierr = VecDestroy();CHKERRQ(ierr);
>>>   ierr = VecDestroy();CHKERRQ(ierr);
>>>   ierr = DMDestroy();CHKERRQ(ierr);
>>>
>>> 2) You should call
>>>
>>>   ierr = DMSetFromOptions(dm);CHKERRQ(ierr);
>>>   ierr = DMViewFromOptions(dm, NULL, "-dm_view");CHKERRQ(ierr);
>>>
>>> after DM creation to make it easier to customize and debug. Then we can
>>> use
>>>
>>>   -dm_view ::ascii_info_detail
>>>
>>> to look at the DM.
>>>
>>> 3) Lets look at Cell 0
>>>
>>> [0]: 0 < 8 (0)
>>> [0]: 0 < 13 (0)
>>> [0]: 0 < 10 (-2)
>>> [0]: 0 < 12 (-2)
>>>
>>> There are two edges reversed. The edges themselves {8, 13, 10, 12}
>>> should proceed counter-clockwise from the bottom, and have vertices
>>>
>>> [0]: 8 < 2 (0)
>>> [0]: 8 < 3 (0)
>>> [0]: 13 < 3 (0)
>>> [0]: 13 < 6 (0)
>>> [0]: 10 < 5 (0)
>>> [0]: 10 < 6 (0)
>>> [0]: 12 < 2 (0)
>>> [0]: 12 < 5 (0)
>>>
>>> so we get as we expect
>>>
>>>   2 --> 3 --> 6 --> 5
>>>
>>> which agrees with the coordinates
>>>
>>>   (   2) dim  2 offset   0 0. 0.
>>>   (   3) dim  2 offset   2 0.5 0.
>>>   (   4) dim  2 offset   4 1. 0.
>>>   (   5) dim  2 offset   6 0. 1.
>>>   (   6) dim  2 offset   8 0.5 1.
>>>   (   7) dim  2 offset  10 1. 1.
>>>
>>> Which part does not make sense?
>>>
>>>   Thanks,
>>>
>>>  Matt
>>>
>>>
 Thanks
 Ingo

 2017-04-14 11:28 GMT+02:00 Ingo Gaertner :

> Thank you, Matt,
> as you say, the face orientations do change sign when switching
> between the two adjacent cells. (I confused my program output. You are
> right.) But it seems not to be always correct to keep the normal direction
> for orientation>=0 and invert it for orientation<0:
>
> I include sample code below to make my question more clear. The
> program creates a HexBoxMesh split horizontally in two cells (at x=0.5) It
> produces this output:
>
> "Face centroids (c) and normals(n):
> face #008 c=(0.25 0.00 0.00) n=(-0.00 -0.50
> 0.00)
> face #009 c=(0.75 0.00 0.00) n=(-0.00 -0.50
> 0.00)
> face #010 c=(0.25 1.00 0.00) n=(0.00 0.50 0.00)
> face #011 c=(0.75 1.00 0.00) n=(0.00 0.50 0.00)
> face #012 c=(0.00 0.50 0.00) n=(-1.00 -0.00
> 0.00)
> face #013 

Re: [petsc-users] Installation question

[Satish Balay]

Sorry - should have mentioned:

do 'rm -rf arch-linux-cxx-opt' and rerun configure again.

The mpich install from previous build [that is currently in arch-linux-cxx-opt/]
is conflicting with --with-mpi-dir=/app1/centos6.3/gnu/mvapich2-1.9/

Satish


On Wed, 19 Apr 2017, Pham Pham wrote:

> I reconfigured PETSs with installed MPI, however, I got serous error:
> 
> **ERROR*
>   Error during compile, check arch-linux-cxx-opt/lib/petsc/conf/make.log
>   Send it and arch-linux-cxx-opt/lib/petsc/conf/configure.log to
> petsc-ma...@mcs.anl.gov
> 
> 
> Please explain what is happening?
> 
> Thank you very much.
> 
> 
> 
> 
> On Wed, Apr 19, 2017 at 11:43 PM, Satish Balay  wrote:
> 
> > Presumably your cluster already has a recommended MPI to use [which is
> > already installed. So you should use that - instead of
> > --download-mpich=1
> >
> > Satish
> >
> > On Wed, 19 Apr 2017, Pham Pham wrote:
> >
> > > Hi,
> > >
> > > I just installed petsc-3.7.5 into my university cluster. When evaluating
> > > the computer system, PETSc reports "It appears you have 1 node(s)", I
> > donot
> > > understand this, since the system is a multinodes system. Could you
> > please
> > > explain this to me?
> > >
> > > Thank you very much.
> > >
> > > S.
> > >
> > > Output:
> > > =
> > > Now to evaluate the computer systems you plan use - do:
> > > make PETSC_DIR=/home/svu/mpepvs/petsc/petsc-3.7.5
> > > PETSC_ARCH=arch-linux-cxx-opt streams
> > > [mpepvs@atlas7-c10 petsc-3.7.5]$ make
> > > PETSC_DIR=/home/svu/mpepvs/petsc/petsc-3.7.5
> > PETSC_ARCH=arch-linux-cxx-opt
> > > streams
> > > cd src/benchmarks/streams; /usr/bin/gmake  --no-print-directory
> > > PETSC_DIR=/home/svu/mpepvs/petsc/petsc-3.7.5
> > PETSC_ARCH=arch-linux-cxx-opt
> > > streams
> > > /home/svu/mpepvs/petsc/petsc-3.7.5/arch-linux-cxx-opt/bin/mpicxx -o
> > > MPIVersion.o -c -Wall -Wwrite-strings -Wno-strict-aliasing
> > > -Wno-unknown-pragmas -fvisibility=hidden -g -O
> > > -I/home/svu/mpepvs/petsc/petsc-3.7.5/include
> > > -I/home/svu/mpepvs/petsc/petsc-3.7.5/arch-linux-cxx-opt/include
> > > `pwd`/MPIVersion.c
> > > Running streams with
> > > '/home/svu/mpepvs/petsc/petsc-3.7.5/arch-linux-cxx-opt/bin/mpiexec '
> > using
> > > 'NPMAX=12'
> > > Number of MPI processes 1 Processor names  atlas7-c10
> > > Triad: 9137.5025   Rate (MB/s)
> > > Number of MPI processes 2 Processor names  atlas7-c10 atlas7-c10
> > > Triad: 9707.2815   Rate (MB/s)
> > > Number of MPI processes 3 Processor names  atlas7-c10 atlas7-c10
> > atlas7-c10
> > > Triad:13559.5275   Rate (MB/s)
> > > Number of MPI processes 4 Processor names  atlas7-c10 atlas7-c10
> > atlas7-c10
> > > atlas7-c10
> > > Triad:14193.0597   Rate (MB/s)
> > > Number of MPI processes 5 Processor names  atlas7-c10 atlas7-c10
> > atlas7-c10
> > > atlas7-c10 atlas7-c10
> > > Triad:14492.9234   Rate (MB/s)
> > > Number of MPI processes 6 Processor names  atlas7-c10 atlas7-c10
> > atlas7-c10
> > > atlas7-c10 atlas7-c10 atlas7-c10
> > > Triad:15476.5912   Rate (MB/s)
> > > Number of MPI processes 7 Processor names  atlas7-c10 atlas7-c10
> > atlas7-c10
> > > atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> > > Triad:15148.7388   Rate (MB/s)
> > > Number of MPI processes 8 Processor names  atlas7-c10 atlas7-c10
> > atlas7-c10
> > > atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> > > Triad:15799.1290   Rate (MB/s)
> > > Number of MPI processes 9 Processor names  atlas7-c10 atlas7-c10
> > atlas7-c10
> > > atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> > > Triad:15671.3104   Rate (MB/s)
> > > Number of MPI processes 10 Processor names  atlas7-c10 atlas7-c10
> > > atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> > > atlas7-c10 atlas7-c10
> > > Triad:15601.4754   Rate (MB/s)
> > > Number of MPI processes 11 Processor names  atlas7-c10 atlas7-c10
> > > atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> > > atlas7-c10 atlas7-c10 atlas7-c10
> > > Triad:15434.5790   Rate (MB/s)
> > > Number of MPI processes 12 Processor names  atlas7-c10 atlas7-c10
> > > atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> > > atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> > > Triad:15134.1263   Rate (MB/s)
> > > 
> > > np  speedup
> > > 1 1.0
> > > 2 1.06
> > > 3 1.48
> > > 4 1.55
> > > 5 1.59
> > > 6 1.69
> > > 7 1.66
> > > 8 1.73
> > > 9 1.72
> > > 10 1.71
> > > 11 1.69
> > > 12 1.66
> > > Estimation of possible speedup of MPI programs based on Streams
> > benchmark.
> > > It appears you have 1 node(s)
> > > Unable to plot speedup to a file
> > > Unable to open matplotlib to plot speedup
> > > [mpepvs@atlas7-c10 petsc-3.7.5]$
> > > [mpepvs@atlas7-c10 

Re: [petsc-users] Installation question

[Satish Balay]

Presumably your cluster already has a recommended MPI to use [which is
already installed. So you should use that - instead of
--download-mpich=1

Satish

On Wed, 19 Apr 2017, Pham Pham wrote:

> Hi,
> 
> I just installed petsc-3.7.5 into my university cluster. When evaluating
> the computer system, PETSc reports "It appears you have 1 node(s)", I donot
> understand this, since the system is a multinodes system. Could you please
> explain this to me?
> 
> Thank you very much.
> 
> S.
> 
> Output:
> =
> Now to evaluate the computer systems you plan use - do:
> make PETSC_DIR=/home/svu/mpepvs/petsc/petsc-3.7.5
> PETSC_ARCH=arch-linux-cxx-opt streams
> [mpepvs@atlas7-c10 petsc-3.7.5]$ make
> PETSC_DIR=/home/svu/mpepvs/petsc/petsc-3.7.5 PETSC_ARCH=arch-linux-cxx-opt
> streams
> cd src/benchmarks/streams; /usr/bin/gmake  --no-print-directory
> PETSC_DIR=/home/svu/mpepvs/petsc/petsc-3.7.5 PETSC_ARCH=arch-linux-cxx-opt
> streams
> /home/svu/mpepvs/petsc/petsc-3.7.5/arch-linux-cxx-opt/bin/mpicxx -o
> MPIVersion.o -c -Wall -Wwrite-strings -Wno-strict-aliasing
> -Wno-unknown-pragmas -fvisibility=hidden -g -O
> -I/home/svu/mpepvs/petsc/petsc-3.7.5/include
> -I/home/svu/mpepvs/petsc/petsc-3.7.5/arch-linux-cxx-opt/include
> `pwd`/MPIVersion.c
> Running streams with
> '/home/svu/mpepvs/petsc/petsc-3.7.5/arch-linux-cxx-opt/bin/mpiexec ' using
> 'NPMAX=12'
> Number of MPI processes 1 Processor names  atlas7-c10
> Triad: 9137.5025   Rate (MB/s)
> Number of MPI processes 2 Processor names  atlas7-c10 atlas7-c10
> Triad: 9707.2815   Rate (MB/s)
> Number of MPI processes 3 Processor names  atlas7-c10 atlas7-c10 atlas7-c10
> Triad:13559.5275   Rate (MB/s)
> Number of MPI processes 4 Processor names  atlas7-c10 atlas7-c10 atlas7-c10
> atlas7-c10
> Triad:14193.0597   Rate (MB/s)
> Number of MPI processes 5 Processor names  atlas7-c10 atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10
> Triad:14492.9234   Rate (MB/s)
> Number of MPI processes 6 Processor names  atlas7-c10 atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10 atlas7-c10
> Triad:15476.5912   Rate (MB/s)
> Number of MPI processes 7 Processor names  atlas7-c10 atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> Triad:15148.7388   Rate (MB/s)
> Number of MPI processes 8 Processor names  atlas7-c10 atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> Triad:15799.1290   Rate (MB/s)
> Number of MPI processes 9 Processor names  atlas7-c10 atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> Triad:15671.3104   Rate (MB/s)
> Number of MPI processes 10 Processor names  atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10
> Triad:15601.4754   Rate (MB/s)
> Number of MPI processes 11 Processor names  atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10 atlas7-c10
> Triad:15434.5790   Rate (MB/s)
> Number of MPI processes 12 Processor names  atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> atlas7-c10 atlas7-c10 atlas7-c10 atlas7-c10
> Triad:15134.1263   Rate (MB/s)
> 
> np  speedup
> 1 1.0
> 2 1.06
> 3 1.48
> 4 1.55
> 5 1.59
> 6 1.69
> 7 1.66
> 8 1.73
> 9 1.72
> 10 1.71
> 11 1.69
> 12 1.66
> Estimation of possible speedup of MPI programs based on Streams benchmark.
> It appears you have 1 node(s)
> Unable to plot speedup to a file
> Unable to open matplotlib to plot speedup
> [mpepvs@atlas7-c10 petsc-3.7.5]$
> [mpepvs@atlas7-c10 petsc-3.7.5]$
> 

Re: [petsc-users] GAMG for the unsymmetrical matrix

[Kong, Fande]

Thanks, Mark,

Now, the total compute time using GAMG is competitive with ASM.  Looks like
I could not use something like: "-mg_level_1_ksp_type gmres" because this
option makes the compute time much worse.

Fande,

On Thu, Apr 13, 2017 at 9:14 AM, Mark Adams  wrote:

>
>
> On Wed, Apr 12, 2017 at 7:04 PM, Kong, Fande  wrote:
>
>>
>>
>> On Sun, Apr 9, 2017 at 6:04 AM, Mark Adams  wrote:
>>
>>> You seem to have two levels here and 3M eqs on the fine grid and 37 on
>>> the coarse grid. I don't understand that.
>>>
>>> You are also calling the AMG setup a lot, but not spending much time
>>> in it. Try running with -info and grep on "GAMG".
>>>
>>
>> I got the following output:
>>
>> [0] PCSetUp_GAMG(): level 0) N=3020875, n data rows=1, n data cols=1,
>> nnz/row (ave)=71, np=384
>> [0] PCGAMGFilterGraph():  100.% nnz after filtering, with threshold
>> 0., 73.6364 nnz ave. (N=3020875)
>> [0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square
>> [0] PCGAMGProlongator_AGG(): New grid 18162 nodes
>> [0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.978702e+00
>> min=2.559747e-02 PC=jacobi
>> [0] PCGAMGCreateLevel_GAMG(): Aggregate processors noop: new_size=384,
>> neq(loc)=40
>> [0] PCSetUp_GAMG(): 1) N=18162, n data cols=1, nnz/row (ave)=94, 384
>> active pes
>> [0] PCSetUp_GAMG(): 2 levels, grid complexity = 1.00795
>> [0] PCSetUp_GAMG(): level 0) N=3020875, n data rows=1, n data cols=1,
>> nnz/row (ave)=71, np=384
>> [0] PCGAMGFilterGraph():  100.% nnz after filtering, with threshold
>> 0., 73.6364 nnz ave. (N=3020875)
>> [0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square
>> [0] PCGAMGProlongator_AGG(): New grid 18145 nodes
>> [0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.978584e+00
>> min=2.557887e-02 PC=jacobi
>> [0] PCGAMGCreateLevel_GAMG(): Aggregate processors noop: new_size=384,
>> neq(loc)=37
>> [0] PCSetUp_GAMG(): 1) N=18145, n data cols=1, nnz/row (ave)=94, 384
>> active pes
>>
>
> You are still doing two levels. Just use the parameters that I told you
> and you should see that 1) this coarsest (last) grid has "1 active pes" and
> 2) the overall solve time and overall convergence rate is much better.
>
>
>> [0] PCSetUp_GAMG(): 2 levels, grid complexity = 1.00792
>> GAMG specific options
>> PCGAMGGraph_AGG   40 1.0 8.0759e+00 1.0 3.56e+07 2.3 1.6e+06 1.9e+04
>> 7.6e+02  2  0  2  4  2   2  0  2  4  2  1170
>> PCGAMGCoarse_AGG  40 1.0 7.1698e+01 1.0 4.05e+09 2.3 4.0e+06 5.1e+04
>> 1.2e+03 18 37  5 27  3  18 37  5 27  3 14632
>> PCGAMGProl_AGG40 1.0 9.2650e-01 1.2 0.00e+00 0.0 9.8e+05 2.9e+03
>> 9.6e+02  0  0  1  0  2   0  0  1  0  2 0
>> PCGAMGPOpt_AGG40 1.0 2.4484e+00 1.0 4.72e+08 2.3 3.1e+06 2.3e+03
>> 1.9e+03  1  4  4  1  4   1  4  4  1  4 51328
>> GAMG: createProl  40 1.0 8.3786e+01 1.0 4.56e+09 2.3 9.6e+06 2.5e+04
>> 4.8e+03 21 42 12 32 10  21 42 12 32 10 14134
>> GAMG: partLevel   40 1.0 6.7755e+00 1.1 2.59e+08 2.3 2.9e+06 2.5e+03
>> 1.5e+03  2  2  4  1  3   2  2  4  1  3  9431
>>
>>
>>
>>
>>
>>
>>
>>
>>>
>>>
>>> On Fri, Apr 7, 2017 at 5:29 PM, Kong, Fande  wrote:
>>> > Thanks, Barry.
>>> >
>>> > It works.
>>> >
>>> > GAMG is three times better than ASM in terms of the number of linear
>>> > iterations, but it is five times slower than ASM. Any suggestions to
>>> improve
>>> > the performance of GAMG? Log files are attached.
>>> >
>>> > Fande,
>>> >
>>> > On Thu, Apr 6, 2017 at 3:39 PM, Barry Smith 
>>> wrote:
>>> >>
>>> >>
>>> >> > On Apr 6, 2017, at 9:39 AM, Kong, Fande  wrote:
>>> >> >
>>> >> > Thanks, Mark and Barry,
>>> >> >
>>> >> > It works pretty wells in terms of the number of linear iterations
>>> (using
>>> >> > "-pc_gamg_sym_graph true"), but it is horrible in the compute time.
>>> I am
>>> >> > using the two-level method via "-pc_mg_levels 2". The reason why
>>> the compute
>>> >> > time is larger than other preconditioning options is that a matrix
>>> free
>>> >> > method is used in the fine level and in my particular problem the
>>> function
>>> >> > evaluation is expensive.
>>> >> >
>>> >> > I am using "-snes_mf_operator 1" to turn on the Jacobian-free
>>> Newton,
>>> >> > but I do not think I want to make the preconditioning part
>>> matrix-free.  Do
>>> >> > you guys know how to turn off the matrix-free method for GAMG?
>>> >>
>>> >>-pc_use_amat false
>>> >>
>>> >> >
>>> >> > Here is the detailed solver:
>>> >> >
>>> >> > SNES Object: 384 MPI processes
>>> >> >   type: newtonls
>>> >> >   maximum iterations=200, maximum function evaluations=1
>>> >> >   tolerances: relative=1e-08, absolute=1e-08, solution=1e-50
>>> >> >   total number of linear solver iterations=20
>>> >> >   total number of function evaluations=166
>>> >> >   norm schedule ALWAYS
>>> >> >   SNESLineSearch Object:   384 MPI processes
>>> >> > type: bt
>>> >> >   interpolation: cubic
>>> >> >   

Re: [petsc-users] VecAssembly gives segmentation fault with MPI

[Karl Rupp]

Hi Francesco,

please don't drop petsc-users from the communication. This will likely 
provide you with better and faster answers.


Since your current build is with debugging turned off, please 
reconfigure with debugging turned on, as the error message says. Chances 
are good that you will get much more precise information about what went 
wrong.


Best regards,
Karli



On 04/19/2017 03:03 PM, Francesco Migliorini wrote:

Hi, thank you for your answer!

Unfortunately I cannot use Valgrind on the machine I am using, but I am
sure than the error is in using VecAssembly. Here's the error message
from PETSc:

[1]PETSC ERROR:

[1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
probably memory access out of range
[1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[1]PETSC ERROR: or see
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS
X to find memory corruption errors
[1]PETSC ERROR: configure using --with-debugging=yes, recompile, link,
and run
[1]PETSC ERROR: to get more information on the crash.
[1]PETSC ERROR: - Error Message
--
[1]PETSC ERROR: Signal received
[1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
for trouble shooting.
[1]PETSC ERROR: Petsc Release Version 3.6.3, Dec, 03, 2015
[1]PETSC ERROR: /u/migliorini/SPEED/SPEED on a arch-linux-opt named
idra116 by migliorini Wed Apr 19 10:20:48 2017
[1]PETSC ERROR: Configure options
--prefix=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/petsc/3.6.3
--with-petsc-arch=arch-linux-opt --with-fortran=1 --with-pic=1
--with-debugging=0 --with-x=0 --with-blas-lapack=1
--with-blas-lib=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/openblas/0.2.17/lib/libopenblas.so
--with-lapack-lib=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/openblas/0.2.17/lib/libopenblas.so
--with-boost=1
--with-boost-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/boost/1.60.0
--with-fftw=1
--with-fftw-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/fftw/3.3.4
--with-hdf5=1
--with-hdf5-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/hdf5/1.8.16
--with-hypre=1
--with-hypre-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/hypre/2.11.0
--with-metis=1
--with-metis-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/metis/5
--with-mumps=1
--with-mumps-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/mumps/5.0.1
--with-netcdf=1
--with-netcdf-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/netcdf/4.4.0
--with-p4est=1
--with-p4est-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/p4est/1.1
--with-parmetis=1
--with-parmetis-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/metis/5
--with-ptscotch=1
--with-ptscotch-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/scotch/6.0.4
--with-scalapack=1
--with-scalapack-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/scalapack/2.0.2
--with-suitesparse=1
--with-suitesparse-dir=/u/sw/pkgs/toolchains/gcc-glibc/5/pkgs/suitesparse/4.5.1
[1]PETSC ERROR: #1 User provided function() line 0 in  unknown file
--
MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
with errorcode 59.

do in=1,nnod_loc is a loop over the nodes contained in the local vector
because the program arrives to Petsc initialization with already
multiple processes. Then I thought Petsc was applied to all the
processes separately and therefore the global dimensions of the system
were the local ones of the MPI processes. Maybe it does not work in this
way...

2017-04-19 13:25 GMT+02:00 Karl Rupp >:

Hi Francesco,

please consider the following:

 a) run your code through valgrind to locate the segmentation fault.
Maybe there is already a memory access problem in the sequential
version.

 b) send any error messages as well as the stack trace.

 c) what is you intent with "do in = nnod_loc"? Isn't nnoc_loc the
number of local elements?

Best regards,
Karli




On 04/19/2017 12:26 PM, Francesco Migliorini wrote:

Hello!

I have an MPI code in which a linear system is created and
solved with
PETSc. It works in sequential run but when I use multiple cores the
VecAssemblyBegin/End give segmentation fault. Here's a sample of
my code:

call PetscInitialize(PETSC_NULL_CHARACTER,perr)

  ind(1) = 3*nnod_loc*max_time_deg
  call VecCreate(PETSC_COMM_WORLD,feP,perr)
  call VecSetSizes(feP,PETSC_DECIDE,ind,perr)
  call VecSetFromOptions(feP,perr)

  do in = nnod_loc
do jt = 1,mm
ind(1) = 3*((in -1)*max_time_deg + (jt-1))
fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +1)
call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
ind(1) = 3*((in -1)*max_time_deg + (jt-1)) +1
 

Re: [petsc-users] VecAssembly gives segmentation fault with MPI

[Jed Brown]

Please always use "reply-all" so that your messages go to the list.
This is standard mailing list etiquette.  It is important to preserve
threading for people who find this discussion later and so that we do
not waste our time re-answering the same questions that have already
been answered in private side-conversations.  You'll likely get an
answer faster that way too.

Francesco Migliorini  writes:

> Hi, thank you for your answer!
>
> Yes xxx_loc means local but it is referred to the MPI processes, so each
> process has different xxx_loc values. 

Always use Debug mode PETSc so we can check when you pass inconsistent
information to Vec.  The global size needs to be the same on every
process.

> Indeed, the program arrives to Petsc initialization with already
> multiple processes. Then I thought Petsc was applied to all the
> processes separately and therefore the global dimensions of the system
> were the local ones of the MPI processes. Maybe it does not work in
> this way... However mm is parameter equal for all the processes (in
> particular it is 3) and the processes do not have exactly the same
> number of nodes.
>
> 2017-04-19 13:20 GMT+02:00 Jed Brown :
>
>> Francesco Migliorini  writes:
>>
>> > Hello!
>> >
>> > I have an MPI code in which a linear system is created and solved with
>> > PETSc. It works in sequential run but when I use multiple cores the
>> > VecAssemblyBegin/End give segmentation fault. Here's a sample of my code:
>> >
>> > call PetscInitialize(PETSC_NULL_CHARACTER,perr)
>> >
>> >   ind(1) = 3*nnod_loc*max_time_deg
>> >   call VecCreate(PETSC_COMM_WORLD,feP,perr)
>> >   call VecSetSizes(feP,PETSC_DECIDE,ind,perr)
>>
>> You set the global size here (does "nnod_loc" mean local? and is it the
>> same size on every process?), but then set values for all of these
>> below.
>>
>> >   call VecSetFromOptions(feP,perr)
>> >
>> >   do in = nnod_loc
>> > do jt = 1,mm
>>
>> What is mm?
>>
>> > ind(1) = 3*((in -1)*max_time_deg + (jt-1))
>> > fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +1)
>> > call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
>> > ind(1) = 3*((in -1)*max_time_deg + (jt-1)) +1
>> > fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +2)
>> > call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
>> > ind(1) = 3*((in -1)*max_time_deg + (jt-1)) +2
>> > fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +3)
>> > call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
>> >  enddo
>> >  enddo
>> >   enddo
>> >   call VecAssemblyBegin(feP,perr)
>> >   call VecAssemblyEnd(feP,perr)
>> >
>> > The vector has 640.000 elements more or less but I am running on a high
>> > performing computer so there shouldn't be memory issues. Does anyone know
>> > where is the problem and how can I fix it?
>> >
>> > Thank you,
>> > Francesco Migliorini
>>


signature.asc
Description: PGP signature

Re: [petsc-users] VecAssembly gives segmentation fault with MPI

[Karl Rupp]

Hi Francesco,

please consider the following:

 a) run your code through valgrind to locate the segmentation fault. 
Maybe there is already a memory access problem in the sequential version.


 b) send any error messages as well as the stack trace.

 c) what is you intent with "do in = nnod_loc"? Isn't nnoc_loc the 
number of local elements?


Best regards,
Karli



On 04/19/2017 12:26 PM, Francesco Migliorini wrote:

Hello!

I have an MPI code in which a linear system is created and solved with
PETSc. It works in sequential run but when I use multiple cores the
VecAssemblyBegin/End give segmentation fault. Here's a sample of my code:

call PetscInitialize(PETSC_NULL_CHARACTER,perr)

  ind(1) = 3*nnod_loc*max_time_deg
  call VecCreate(PETSC_COMM_WORLD,feP,perr)
  call VecSetSizes(feP,PETSC_DECIDE,ind,perr)
  call VecSetFromOptions(feP,perr)

  do in = nnod_loc
do jt = 1,mm
ind(1) = 3*((in -1)*max_time_deg + (jt-1))
fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +1)
call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
ind(1) = 3*((in -1)*max_time_deg + (jt-1)) +1
fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +2)
call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
ind(1) = 3*((in -1)*max_time_deg + (jt-1)) +2
fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +3)
call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
 enddo
 enddo
  enddo
  call VecAssemblyBegin(feP,perr)
  call VecAssemblyEnd(feP,perr)

The vector has 640.000 elements more or less but I am running on a high
performing computer so there shouldn't be memory issues. Does anyone
know where is the problem and how can I fix it?

Thank you,
Francesco Migliorini

[petsc-users] VecAssembly gives segmentation fault with MPI

[Francesco Migliorini]

Hello!

I have an MPI code in which a linear system is created and solved with
PETSc. It works in sequential run but when I use multiple cores the
VecAssemblyBegin/End give segmentation fault. Here's a sample of my code:

call PetscInitialize(PETSC_NULL_CHARACTER,perr)

  ind(1) = 3*nnod_loc*max_time_deg
  call VecCreate(PETSC_COMM_WORLD,feP,perr)
  call VecSetSizes(feP,PETSC_DECIDE,ind,perr)
  call VecSetFromOptions(feP,perr)

  do in = nnod_loc
do jt = 1,mm
ind(1) = 3*((in -1)*max_time_deg + (jt-1))
fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +1)
call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
ind(1) = 3*((in -1)*max_time_deg + (jt-1)) +1
fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +2)
call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
ind(1) = 3*((in -1)*max_time_deg + (jt-1)) +2
fval(1) = fe(3*((in -1)*max_time_deg + (jt-1)) +3)
call VecSetValues(feP,1,ind,fval(1),INSERT_VALUES,perr)
 enddo
 enddo
  enddo
  call VecAssemblyBegin(feP,perr)
  call VecAssemblyEnd(feP,perr)

The vector has 640.000 elements more or less but I am running on a high
performing computer so there shouldn't be memory issues. Does anyone know
where is the problem and how can I fix it?

Thank you,
Francesco Migliorini