[petsc-users] O3 versus O2
Dear community, We are in the middle of testing a simulator where the main computational bottleneck is solving a linear problem. We do this by calling GMRES+BoomerAMG through PETSc. This is a commercial code, pretended to serve clients with workstations or with access to clusters. Would you recommend O3 versus O2 optimizations? Maybe just to compile the linear algebra libraries? Some years ago, I worked on another project where going back to O2 solved a weird runtime error that I was never able to solve. This triggers my untrust. Thank you for your time! Alfredo
Re: [petsc-users] error when trying to compile with HPDDM
Hi Pierre, no, I don't really need that flag. I removed it and the
installation process went well. I just noticed a "minor" detail when
building SLEPc:
"gmake[3]: warning: jobserver unavailable: using -j1. Add '+' to parent
make rule"
so the compilation of that library went slow.
Thanks,
Alfredo
On Thu, Jan 5, 2023 at 12:39 PM Pierre Jolivet wrote:
>
>
> On 5 Jan 2023, at 7:06 PM, Matthew Knepley wrote:
>
> On Thu, Jan 5, 2023 at 11:36 AM Alfredo Jaramillo <
> ajaramillopa...@gmail.com> wrote:
>
>> Dear developers,
>> I'm trying to compile petsc together with the HPDDM library. A series on
>> errors appeared:
>>
>> /home/ajaramillo/petsc/x64-openmpi-aldaas2021/include/HPDDM_specifications.hpp:
>> In static member function ‘static constexpr __float128
>> std::numeric_limits<__float128>::min()’:
>> /home/ajaramillo/petsc/x64-openmpi-aldaas2021/include/HPDDM_specifications.hpp:54:57:
>> error: unable to find numeric literal operator ‘operator""Q’
>>54 | static constexpr __float128 min() noexcept { return
>> FLT128_MIN; }
>>
>> I'm attaching the log files to this email.
>>
>
> Pierre,
>
> It looks like we may need to test for FLT_MIN and FLT_MAX in configure
> since it looks like Alfredo's headers do not have them.
> Is this correct?
>
>
> We could do that, but I bet this is a side effect of the fact that Alfredo
> is using --with-cxx-dialect=C++11.
> Alfredo, did you got that flag from someone else’s configure, or do you
> know what that flag is doing?
> - If yes, do you really need to stick to -std=c++11?
> - If no, please look at
> https://gitlab.com/petsc/petsc/-/issues/1284#note_1173803107 and consider
> removing that flag, or at least changing the option to
> --with-cxx-dialect=11. If compilation still fails, please send the
> up-to-date configure.log/make.log
>
> Thanks,
> Pierre
>
> Thanks,
>
> Matt
>
>
>> Could you please help me with this?
>>
>> bests regards
>> Alfredo
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
>
>
Re: [petsc-users] setting a vector with VecSetValue versus VecSetValues
omg... for some reason I was thinking it takes local indices. Yes..
modifying that line the code works well.
thank you!
Alfredo
On Thu, Jan 5, 2023 at 10:38 AM Junchao Zhang
wrote:
> VecSetValue() also needs global indices, so you need PetscInt gl_row = (
> PetscInt)(i)+rstart;
>
> --Junchao Zhang
>
>
> On Thu, Jan 5, 2023 at 11:25 AM Alfredo Jaramillo <
> ajaramillopa...@gmail.com> wrote:
>
>> dear PETSc developers,
>>
>> I have a code where I copy an array to a distributed petsc vector with
>> the next lines:
>>
>> 1 for (int i = 0; i < ndof_local; i++) {
>> 2 PetscInt gl_row = (PetscInt)(i)+rstart;
>> 3 PetscScalar val = (PetscScalar)u[i];
>> 4 VecSetValues(x,1,_row,,INSERT_VALUES);
>> 5 }
>>
>> // for (int i = 0; i < ndof_local; i++) {
>> // PetscInt gl_row = (PetscInt)(i);
>> // PetscScalar val = (PetscScalar)u[i];
>> // VecSetValue(x,gl_row,val,INSERT_VALUES);
>> // }
>>
>> VecAssemblyBegin(x);
>> VecAssemblyEnd(x);
>>
>> This works as expected. If, instead of using lines 1-5, I use the lines
>> where VecSetValue is used with local indices, then the vector is null on
>> all the processes but rank 0, and the piece of information at rank zero is
>> incorrect.
>>
>> What could I be doing wrong?
>>
>> bests regards
>> Alfredo
>>
>
[petsc-users] setting a vector with VecSetValue versus VecSetValues
dear PETSc developers,
I have a code where I copy an array to a distributed petsc vector with the
next lines:
1 for (int i = 0; i < ndof_local; i++) {
2 PetscInt gl_row = (PetscInt)(i)+rstart;
3 PetscScalar val = (PetscScalar)u[i];
4 VecSetValues(x,1,_row,,INSERT_VALUES);
5 }
// for (int i = 0; i < ndof_local; i++) {
// PetscInt gl_row = (PetscInt)(i);
// PetscScalar val = (PetscScalar)u[i];
// VecSetValue(x,gl_row,val,INSERT_VALUES);
// }
VecAssemblyBegin(x);
VecAssemblyEnd(x);
This works as expected. If, instead of using lines 1-5, I use the lines
where VecSetValue is used with local indices, then the vector is null on
all the processes but rank 0, and the piece of information at rank zero is
incorrect.
What could I be doing wrong?
bests regards
Alfredo
Re: [petsc-users] MatView to file
Hello Mark,
But why should this depend on the number of processes?
thanks
Alfredo
On Fri, Aug 12, 2022 at 1:42 PM Mark Adams wrote:
> With 4 million elements you are nowhere near the 32 but integer limit of
> 2B or 32Gb of memory.
>
> See the https://petsc.org/main/docs/manualpages/Mat/MatView
> You should go to binary format when doing large matrices.
>
> Mark
>
> On Fri, Aug 12, 2022 at 1:00 PM Alfredo Jaramillo <
> ajaramillopa...@gmail.com> wrote:
>
>> Hello Mark,
>> Thank you, I added the lines that you sent.
>> This only happens when running the code with more than 1 process. With
>> only 1 MPI process the matrix is printed out.
>> With 2 processes or more I observed the program begins to allocate RAM
>> until it exceeds the computer capacity (32GB) so I wasn't able to get the
>> stack trace.
>>
>> However, I was able to reproduce the problem by compiling
>> src/ksp/ksp/tutorials/ex54.c.html
>> <https://petsc.org/release/src/ksp/ksp/tutorials/ex54.c.html> (modifying
>> line 144) and running it with
>>
>> mpirun -np 2 ex54 -ne 1000
>>
>> This gives a sparse matrix of order ~1 million. When running ex54 with
>> only one MPI process I don't observe this excessive allocation and the
>> matrix is printed out.
>>
>> Thanks,
>> Alfredo
>>
>>
>>
>> On Fri, Aug 12, 2022 at 10:02 AM Mark Adams wrote:
>>
>>> You also want:
>>>
>>> PetscCall(PetscViewerPopFormat(viewer));
>>> PetscCall(PetscViewerDestroy());
>>>
>>> This should not be a problem.
>>> If this is a segv and you configure it with '--with-debugging=1', you
>>> should get a stack trace, which would help immensely.
>>> Or run in a debugger to get a stack trace.
>>>
>>> Thanks,
>>> Mark
>>>
>>>
>>>
>>> On Fri, Aug 12, 2022 at 11:26 AM Alfredo Jaramillo <
>>> ajaramillopa...@gmail.com> wrote:
>>>
>>>> Dear developers,
>>>>
>>>> I'm writing a sparse matrix into a file by doing
>>>>
>>>> if (dump_mat) {
>>>> PetscViewer viewer;
>>>> PetscViewerASCIIOpen(PETSC_COMM_WORLD,"mat-par-aux.m",);
>>>> PetscViewerPushFormat(viewer, PETSC_VIEWER_ASCII_MATLAB);
>>>> MatView(A,viewer);
>>>> }
>>>>
>>>> This works perfectly for small cases.
>>>> The program crashes for a case where the matrix A is of order 1 million
>>>> but with only 4 million non-zero elements.
>>>>
>>>> Maybe at some point petsc is full-sizing A?
>>>>
>>>> Thank you,
>>>> Alfredo
>>>>
>>>
Re: [petsc-users] MatView to file
Hello Mark,
Thank you, I added the lines that you sent.
This only happens when running the code with more than 1 process. With only
1 MPI process the matrix is printed out.
With 2 processes or more I observed the program begins to allocate RAM
until it exceeds the computer capacity (32GB) so I wasn't able to get the
stack trace.
However, I was able to reproduce the problem by compiling
src/ksp/ksp/tutorials/ex54.c.html
<https://petsc.org/release/src/ksp/ksp/tutorials/ex54.c.html> (modifying
line 144) and running it with
mpirun -np 2 ex54 -ne 1000
This gives a sparse matrix of order ~1 million. When running ex54 with only
one MPI process I don't observe this excessive allocation and the matrix is
printed out.
Thanks,
Alfredo
On Fri, Aug 12, 2022 at 10:02 AM Mark Adams wrote:
> You also want:
>
> PetscCall(PetscViewerPopFormat(viewer));
> PetscCall(PetscViewerDestroy());
>
> This should not be a problem.
> If this is a segv and you configure it with '--with-debugging=1', you
> should get a stack trace, which would help immensely.
> Or run in a debugger to get a stack trace.
>
> Thanks,
> Mark
>
>
>
> On Fri, Aug 12, 2022 at 11:26 AM Alfredo Jaramillo <
> ajaramillopa...@gmail.com> wrote:
>
>> Dear developers,
>>
>> I'm writing a sparse matrix into a file by doing
>>
>> if (dump_mat) {
>> PetscViewer viewer;
>> PetscViewerASCIIOpen(PETSC_COMM_WORLD,"mat-par-aux.m",);
>> PetscViewerPushFormat(viewer, PETSC_VIEWER_ASCII_MATLAB);
>> MatView(A,viewer);
>> }
>>
>> This works perfectly for small cases.
>> The program crashes for a case where the matrix A is of order 1 million
>> but with only 4 million non-zero elements.
>>
>> Maybe at some point petsc is full-sizing A?
>>
>> Thank you,
>> Alfredo
>>
>
[petsc-users] MatView to file
Dear developers,
I'm writing a sparse matrix into a file by doing
if (dump_mat) {
PetscViewer viewer;
PetscViewerASCIIOpen(PETSC_COMM_WORLD,"mat-par-aux.m",);
PetscViewerPushFormat(viewer, PETSC_VIEWER_ASCII_MATLAB);
MatView(A,viewer);
}
This works perfectly for small cases.
The program crashes for a case where the matrix A is of order 1 million but
with only 4 million non-zero elements.
Maybe at some point petsc is full-sizing A?
Thank you,
Alfredo
Re: [petsc-users] error when solving a linear system with gmres + pilut/euclid
thank you, Barry, I wasn't able to reproduce the error on my computer, neither on a second cluster. On the first cluster, I requested to activate X11 at some node for attaching a debugger, and that activation (if possible) should take some time. I will inform you of any news on that. kind regards Alfredo On Tue, Aug 25, 2020 at 6:46 PM Barry Smith wrote: > > I have submitted a merge request > https://gitlab.com/petsc/petsc/-/merge_requests/3096 that will make the > error handling and message clearer in the future. > > Barry > > > On Aug 25, 2020, at 8:55 AM, Alfredo Jaramillo > wrote: > > In fact, on my machine the code is compiled with gnu, and on the cluster > it is compiled with intel (2015) compilers. I just run the program with > "-fp_trap" and got: > > === >|> Assembling interface problem. Unk # 56 >|> Solving interface problem > Residual norms for interp_ solve. > 0 KSP Residual norm 3.642615470862e+03 > [0]PETSC ERROR: *** unknown floating point error occurred *** > [0]PETSC ERROR: The specific exception can be determined by running in a > debugger. When the > [0]PETSC ERROR: debugger traps the signal, the exception can be found with > fetestexcept(0x3f) > [0]PETSC ERROR: where the result is a bitwise OR of the following flags: > [0]PETSC ERROR: FE_INVALID=0x1 FE_DIVBYZERO=0x4 FE_OVERFLOW=0x8 > FE_UNDERFLOW=0x10 FE_INEXACT=0x20 > [0]PETSC ERROR: Try option -start_in_debugger > [0]PETSC ERROR: likely location of problem given in stack below > [0]PETSC ERROR: - Stack Frames > > [1]PETSC ERROR: [2]PETSC ERROR: *** unknown floating point error occurred > *** > [3]PETSC ERROR: *** unknown floating point error occurred *** > [3]PETSC ERROR: The specific exception can be determined by running in a > debugger. When the > [4]PETSC ERROR: *** unknown floating point error occurred *** > [4]PETSC ERROR: The specific exception can be determined by running in a > debugger. When the > [4]PETSC ERROR: [5]PETSC ERROR: *** unknown floating point error occurred > *** > [5]PETSC ERROR: The specific exception can be determined by running in a > debugger. When the > [5]PETSC ERROR: debugger traps the signal, the exception can be found with > fetestexcept(0x3f) > [5]PETSC ERROR: where the result is a bitwise OR of the following flags: > [6]PETSC ERROR: *** unknown floating point error occurred *** > [6]PETSC ERROR: The specific exception can be determined by running in a > debugger. When the > [6]PETSC ERROR: debugger traps the signal, the exception can be found with > fetestexcept(0x3f) > [6]PETSC ERROR: where the result is a bitwise OR of the following flags: > [6]PETSC ERROR: FE_INVALID=0x1 FE_DIVBYZERO=0x4 FE_OVERFLOW=0x8 > FE_UNDERFLOW=0x10 FE_INEXACT=0x20 > [7]PETSC ERROR: *** unknown floating point error occurred *** > [7]PETSC ERROR: The specific exception can be determined by running in a > debugger. When the > [7]PETSC ERROR: debugger traps the signal, the exception can be found with > fetestexcept(0x3f) > [7]PETSC ERROR: where the result is a bitwise OR of the following flags: > [7]PETSC ERROR: FE_INVALID=0x1 FE_DIVBYZERO=0x4 FE_OVERFLOW=0x8 > FE_UNDERFLOW=0x10 FE_INEXACT=0x20 > [7]PETSC ERROR: Try option -start_in_debugger > [7]PETSC ERROR: likely location of problem given in stack below > [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not > available, > [0]PETSC ERROR: INSTEAD the line number of the start of the function > [0]PETSC ERROR: is given. > [0]PETSC ERROR: [0] PetscDefaultFPTrap line 355 > /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/sys/error/fp.c > [0]PETSC ERROR: [0] VecMDot line 1154 > /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/vec/vec/interface/rvector.c > [0]PETSC ERROR: [0] KSPGMRESClassicalGramSchmidtOrthogonalization line 44 > /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/ksp/ksp/impls/gmres/borthog2.c > [0]PETSC ERROR: [0] KSPGMRESCycle line 122 > /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/ksp/ksp/impls/gmres/gmres.c > [0]PETSC ERROR: [0] KSPSolve_GMRES line 225 > /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/ksp/ksp/impls/gmres/gmres.c > [0]PETSC ERROR: [0] KSPSolve_Private line 590 > /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: *** unknown floating point error occurred *** > === > > So it seems that in fact a division by 0 is taking place. I will try to > run this in debug mode. > > thanks > Alfredo > > On Tue, Aug 25, 2020 at 10:23 AM Barry Smith wrote: > >> >> Sounds like it might be a compiler problem
Re: [petsc-users] error when solving a linear system with gmres + pilut/euclid
In fact, on my machine the code is compiled with gnu, and on the cluster it is compiled with intel (2015) compilers. I just run the program with "-fp_trap" and got: === |> Assembling interface problem. Unk # 56 |> Solving interface problem Residual norms for interp_ solve. 0 KSP Residual norm 3.642615470862e+03 [0]PETSC ERROR: *** unknown floating point error occurred *** [0]PETSC ERROR: The specific exception can be determined by running in a debugger. When the [0]PETSC ERROR: debugger traps the signal, the exception can be found with fetestexcept(0x3f) [0]PETSC ERROR: where the result is a bitwise OR of the following flags: [0]PETSC ERROR: FE_INVALID=0x1 FE_DIVBYZERO=0x4 FE_OVERFLOW=0x8 FE_UNDERFLOW=0x10 FE_INEXACT=0x20 [0]PETSC ERROR: Try option -start_in_debugger [0]PETSC ERROR: likely location of problem given in stack below [0]PETSC ERROR: - Stack Frames [1]PETSC ERROR: [2]PETSC ERROR: *** unknown floating point error occurred *** [3]PETSC ERROR: *** unknown floating point error occurred *** [3]PETSC ERROR: The specific exception can be determined by running in a debugger. When the [4]PETSC ERROR: *** unknown floating point error occurred *** [4]PETSC ERROR: The specific exception can be determined by running in a debugger. When the [4]PETSC ERROR: [5]PETSC ERROR: *** unknown floating point error occurred *** [5]PETSC ERROR: The specific exception can be determined by running in a debugger. When the [5]PETSC ERROR: debugger traps the signal, the exception can be found with fetestexcept(0x3f) [5]PETSC ERROR: where the result is a bitwise OR of the following flags: [6]PETSC ERROR: *** unknown floating point error occurred *** [6]PETSC ERROR: The specific exception can be determined by running in a debugger. When the [6]PETSC ERROR: debugger traps the signal, the exception can be found with fetestexcept(0x3f) [6]PETSC ERROR: where the result is a bitwise OR of the following flags: [6]PETSC ERROR: FE_INVALID=0x1 FE_DIVBYZERO=0x4 FE_OVERFLOW=0x8 FE_UNDERFLOW=0x10 FE_INEXACT=0x20 [7]PETSC ERROR: *** unknown floating point error occurred *** [7]PETSC ERROR: The specific exception can be determined by running in a debugger. When the [7]PETSC ERROR: debugger traps the signal, the exception can be found with fetestexcept(0x3f) [7]PETSC ERROR: where the result is a bitwise OR of the following flags: [7]PETSC ERROR: FE_INVALID=0x1 FE_DIVBYZERO=0x4 FE_OVERFLOW=0x8 FE_UNDERFLOW=0x10 FE_INEXACT=0x20 [7]PETSC ERROR: Try option -start_in_debugger [7]PETSC ERROR: likely location of problem given in stack below [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, [0]PETSC ERROR: INSTEAD the line number of the start of the function [0]PETSC ERROR: is given. [0]PETSC ERROR: [0] PetscDefaultFPTrap line 355 /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/sys/error/fp.c [0]PETSC ERROR: [0] VecMDot line 1154 /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/vec/vec/interface/rvector.c [0]PETSC ERROR: [0] KSPGMRESClassicalGramSchmidtOrthogonalization line 44 /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/ksp/ksp/impls/gmres/borthog2.c [0]PETSC ERROR: [0] KSPGMRESCycle line 122 /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: [0] KSPSolve_GMRES line 225 /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: [0] KSPSolve_Private line 590 /mnt/lustre/home/ajaramillo/petsc-3.13.0/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: *** unknown floating point error occurred *** === So it seems that in fact a division by 0 is taking place. I will try to run this in debug mode. thanks Alfredo On Tue, Aug 25, 2020 at 10:23 AM Barry Smith wrote: > > Sounds like it might be a compiler problem generating bad code. > > On the machine where it fails you can run with -fp_trap to have it error > out as soon as a Nan or Inf appears. If you can use the debugger on that > machine you can tell the debugger to catch floating point exceptions and > see the exact line an values of variables where a Nan or Inf appear. > >As Matt conjectured it is likely there is a divide by zero before PETSc > detects and it may be helpful to find out exactly where that happens. > > Barry > > > On Aug 25, 2020, at 8:03 AM, Alfredo Jaramillo > wrote: > > Yes, Barry, that is correct. > > > > On Tue, Aug 25, 2020 at 1:02 AM Barry Smith wrote: > >> >> On one system you get this error, on another system with the identical >> code and test case you do not get the error? >> >> You get it with three iterative methods but not with MUMPS? >> >> Barry >> >> >> On Aug 24, 2020, at 8:35 PM, Alfredo Jaramillo >> wrote: >
Re: [petsc-users] error when solving a linear system with gmres + pilut/euclid
Yes, Barry, that is correct. On Tue, Aug 25, 2020 at 1:02 AM Barry Smith wrote: > > On one system you get this error, on another system with the identical > code and test case you do not get the error? > > You get it with three iterative methods but not with MUMPS? > > Barry > > > On Aug 24, 2020, at 8:35 PM, Alfredo Jaramillo > wrote: > > Hello Barry, Matthew, thanks for the replies ! > > Yes, it is our custom code, and it also happens when setting -pc_type > bjacobi. Before testing an iterative solver, we were using MUMPS (-ksp_type > preonly -ksp_pc_type lu -pc_factor_mat_solver_type mumps) without issues. > > Running the ex19 (as "mpirun -n 4 ex19 -da_refine 5") did not produce any > problem. > > To reproduce the situation on my computer, I was able to reproduce the > error for a small case and -pc_type bjacobi. For that particular case, when > running in the cluster the error appears at the very last iteration: > > = > 27 KSP Residual norm 8.230378644666e-06 > [0]PETSC ERROR: - Error Message > -- > [0]PETSC ERROR: Invalid argument > [0]PETSC ERROR: Scalar value must be same on all processes, argument # 3 > > > whereas running on my computer the error is not launched and convergence > is reached instead: > > > Linear interp_ solve converged due to CONVERGED_RTOL iterations 27 > > > I will run valgrind to seek for possible memory corruptions. > > thank you > Alfredo > > On Mon, Aug 24, 2020 at 9:00 PM Barry Smith wrote: > >> >>Oh yes, it could happen with Nan. >> >>KSPGMRESClassicalGramSchmidtOrthogonalization() >> calls KSPCheckDot(ksp,lhh[j]); so should detect any NAN that appear and >> set ksp->convergedreason but the call to MAXPY() is still made before >> returning and hence producing the error message. >> >>We should circuit the orthogonalization as soon as it sees a Nan/Inf >> and return immediately for GMRES to cleanup and produce a very useful error >> message. >> >> Alfredo, >> >> It is also possible that the hypre preconditioners are producing a >> Nan because your matrix is too difficult for them to handle, but it would >> be odd to happen after many iterations. >> >>As I suggested before run with -pc_type bjacobi to see if you get the >> same problem. >> >> Barry >> >> >> On Aug 24, 2020, at 6:38 PM, Matthew Knepley wrote: >> >> On Mon, Aug 24, 2020 at 6:27 PM Barry Smith wrote: >> >>> >>>Alfredo, >>> >>> This should never happen. The input to the VecMAXPY in gmres is >>> computed via VMDot which produces the same result on all processes. >>> >>>If you run with -pc_type bjacobi does it also happen? >>> >>>Is this your custom code or does it happen in PETSc examples >>> also? Like src/snes/tutorials/ex19 -da_refine 5 >>> >>> Could be memory corruption, can you run under valgrind? >>> >> >> Couldn't it happen if something generates a NaN? That also should not >> happen, but I was allowing that pilut might do it. >> >> Thanks, >> >> Matt >> >> >>> Barry >>> >>> >>> > On Aug 24, 2020, at 4:05 PM, Alfredo Jaramillo < >>> ajaramillopa...@gmail.com> wrote: >>> > >>> > Dear PETSc developers, >>> > >>> > I'm trying to solve a linear problem with GMRES preconditioned with >>> pilut from HYPRE. For this I'm using the options: >>> > >>> > -ksp_type gmres -pc_type hypre -pc_hypre_type pilut -ksp_monitor >>> > >>> > If I use a single core, GMRES (+ pilut or euclid) converges. However, >>> when using multiple cores the next error appears after some number of >>> iterations: >>> > >>> > [0]PETSC ERROR: Scalar value must be same on all processes, argument # >>> 3 >>> > >>> > relative to the function VecMAXPY. I attached a screenshot with more >>> detailed output. The same happens when using euclid. Can you please give me >>> some insight on this? >>> > >>> > best regards >>> > Alfredo >>> > >>> >>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> <http://www.cse.buffalo.edu/~knepley/> >> >> >> >
Re: [petsc-users] error when solving a linear system with gmres + pilut/euclid
Hello Barry, Matthew, thanks for the replies ! Yes, it is our custom code, and it also happens when setting -pc_type bjacobi. Before testing an iterative solver, we were using MUMPS (-ksp_type preonly -ksp_pc_type lu -pc_factor_mat_solver_type mumps) without issues. Running the ex19 (as "mpirun -n 4 ex19 -da_refine 5") did not produce any problem. To reproduce the situation on my computer, I was able to reproduce the error for a small case and -pc_type bjacobi. For that particular case, when running in the cluster the error appears at the very last iteration: = 27 KSP Residual norm 8.230378644666e-06 [0]PETSC ERROR: - Error Message -- [0]PETSC ERROR: Invalid argument [0]PETSC ERROR: Scalar value must be same on all processes, argument # 3 whereas running on my computer the error is not launched and convergence is reached instead: Linear interp_ solve converged due to CONVERGED_RTOL iterations 27 I will run valgrind to seek for possible memory corruptions. thank you Alfredo On Mon, Aug 24, 2020 at 9:00 PM Barry Smith wrote: > >Oh yes, it could happen with Nan. > >KSPGMRESClassicalGramSchmidtOrthogonalization() > calls KSPCheckDot(ksp,lhh[j]); so should detect any NAN that appear and > set ksp->convergedreason but the call to MAXPY() is still made before > returning and hence producing the error message. > >We should circuit the orthogonalization as soon as it sees a Nan/Inf > and return immediately for GMRES to cleanup and produce a very useful error > message. > > Alfredo, > > It is also possible that the hypre preconditioners are producing a Nan > because your matrix is too difficult for them to handle, but it would be > odd to happen after many iterations. > >As I suggested before run with -pc_type bjacobi to see if you get the > same problem. > > Barry > > > On Aug 24, 2020, at 6:38 PM, Matthew Knepley wrote: > > On Mon, Aug 24, 2020 at 6:27 PM Barry Smith wrote: > >> >>Alfredo, >> >> This should never happen. The input to the VecMAXPY in gmres is >> computed via VMDot which produces the same result on all processes. >> >>If you run with -pc_type bjacobi does it also happen? >> >>Is this your custom code or does it happen in PETSc examples also? >> Like src/snes/tutorials/ex19 -da_refine 5 >> >> Could be memory corruption, can you run under valgrind? >> > > Couldn't it happen if something generates a NaN? That also should not > happen, but I was allowing that pilut might do it. > > Thanks, > > Matt > > >> Barry >> >> >> > On Aug 24, 2020, at 4:05 PM, Alfredo Jaramillo < >> ajaramillopa...@gmail.com> wrote: >> > >> > Dear PETSc developers, >> > >> > I'm trying to solve a linear problem with GMRES preconditioned with >> pilut from HYPRE. For this I'm using the options: >> > >> > -ksp_type gmres -pc_type hypre -pc_hypre_type pilut -ksp_monitor >> > >> > If I use a single core, GMRES (+ pilut or euclid) converges. However, >> when using multiple cores the next error appears after some number of >> iterations: >> > >> > [0]PETSC ERROR: Scalar value must be same on all processes, argument # 3 >> > >> > relative to the function VecMAXPY. I attached a screenshot with more >> detailed output. The same happens when using euclid. Can you please give me >> some insight on this? >> > >> > best regards >> > Alfredo >> > >> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> > > >
Re: [petsc-users] error when solving a linear system with gmres + pilut/euclid
complementing the previous message, the same error appears when using boomeramg On Mon, Aug 24, 2020 at 6:05 PM Alfredo Jaramillo wrote: > Dear PETSc developers, > > I'm trying to solve a linear problem with GMRES preconditioned with pilut > from HYPRE. For this I'm using the options: > > -ksp_type gmres -pc_type hypre -pc_hypre_type pilut -ksp_monitor > > If I use a single core, GMRES (+ pilut or euclid) converges. However, when > using multiple cores the next error appears after some number of iterations: > > [0]PETSC ERROR: Scalar value must be same on all processes, argument # 3 > > relative to the function VecMAXPY. I attached a screenshot with more > detailed output. The same happens when using euclid. Can you please give me > some insight on this? > > best regards > Alfredo >
Re: [petsc-users] error when configuring petsc with Intel Compilers 2019 update 3
Thank you Matthew!
I need this version to work on my computer in order to look for a bug that
appears in a cluster.
I'm not sure how to make it work, I will try with an older Linux
distribution.
regards
Alfredo
On Tue, May 26, 2020 at 1:02 PM Matthew Knepley wrote:
> On Tue, May 26, 2020 at 11:23 AM Alfredo Jaramillo <
> ajaramillopa...@gmail.com> wrote:
>
>> hello dear PETSc team,
>>
>> I'm trying to install PETSc with the 2019 update 3 Intel Parallel Studio.
>> When starting the configuration process there appears the next message:
>>
>>
>>
>>
>>
>>
>>
>> *TESTING: checkFortran90Array from
>> config.compilersFortran(/opt/petsc-3.13.0/config/BuildSystem/config/compilersFortran.py:211)***
>>UNABLE to CONFIGURE with GIVEN OPTIONS(see configure.log for
>> details):---Could
>> not check Fortran pointer
>> arguments
>>
>> the configuration log is attached to this message
>>
>> I would be very thankful of any kind help on this matter
>>
>
> It seems like headers in /usr/include are conflicting with your new
> compiler.
>
> Executing: mpiicc -c -o
> /tmp/petsc-n__j58ih/config.compilersFortran/conftest.o
> -I/tmp/petsc-n__j58ih/config.libraries
> -I/tmp/petsc-n__j58ih/config.setCompilers
> -I/tmp/petsc-n__j58ih/config.compilers
> -I/tmp/petsc-n__j58ih/config.utilities.closure
> -I/tmp/petsc-n__j58ih/config.compilersFortran -fPIC -O3 -march=native
> -mtune=native /tmp/petsc-n__j58ih/config.compilersFortran/conftest.c
> Possible ERROR while running compiler: exit code 2
> stderr:
> In file included from /usr/include/bits/floatn.h(119),
> from /usr/include/stdlib.h(55),
> from
> /tmp/petsc-n__j58ih/config.compilersFortran/conftest.c(4):
> /usr/include/bits/floatn-common.h(214): error: invalid combination of type
> specifiers
> typedef float _Float32;
> ^
>
> In file included from /usr/include/bits/floatn.h(119),
> from /usr/include/stdlib.h(55),
> from
> /tmp/petsc-n__j58ih/config.compilersFortran/conftest.c(4):
> /usr/include/bits/floatn-common.h(251): error: invalid combination of type
> specifiers
> typedef double _Float64;
> ^
>
> In file included from /usr/include/bits/floatn.h(119),
> from /usr/include/stdlib.h(55),
> from
> /tmp/petsc-n__j58ih/config.compilersFortran/conftest.c(4):
> /usr/include/bits/floatn-common.h(268): error: invalid combination of type
> specifiers
> typedef double _Float32x;
> ^
>
> In file included from /usr/include/bits/floatn.h(119),
> from /usr/include/stdlib.h(55),
> from
> /tmp/petsc-n__j58ih/config.compilersFortran/conftest.c(4):
> /usr/include/bits/floatn-common.h(285): error: invalid combination of type
> specifiers
> typedef long double _Float64x;
> ^
>
> compilation aborted for
> /tmp/petsc-n__j58ih/config.compilersFortran/conftest.c (code 2)
> Source:
> #include "confdefs.h"
> #include "conffix.h"
> #include
> #include
> void f90arraytest_(void* a1, void* a2,void* a3, void* i)
> {
> printf("arrays [%p %p %p]\n",a1,a2,a3);
> fflush(stdout);
> return;
> }
> void f90ptrtest_(void* a1, void* a2,void* a3, void* i, void* p1 ,void* p2,
> void* p3)
> {
> printf("arrays [%p %p %p]\n",a1,a2,a3);
> if ((p1 == p3) && (p1 != p2)) {
> printf("pointers match! [%p %p] [%p]\n",p1,p3,p2);
> fflush(stdout);
> } else {
> printf("pointers do not match! [%p %p] [%p]\n",p1,p3,p2);
> fflush(stdout);
> exit(111);
> }
> return;
> }
>
> Thanks,
>
> Matt
>
> Alfredo
>>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
Re: [petsc-users] multiple definition of `main' with intel compilers
Hello Satish and PETSc developers, I'm sending this email in case somebody else is having the same problem with the intel compilers. In my personal computer (with Fedora 32, gcc 10). When I tried to execute the resulting binary there appears the error message: build/linux_icc/partrans_linux_icc_opt: /opt/intel/clck/2019.8/lib/intel64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by build/linux_icc/partrans_linux_icc_opt) a workaround that worked for me was to delete the link /opt/intel/clck/2019.8/lib/intel64/libstdc++.so.6 that was doing: libstdc++.so.6 -> libstdc++.so.6.0.20 (in the folder /opt/intel/clck/2019.8/lib/intel64/) and replace it by a link to the default c++ library of my system: sudo ln -s /usr/lib64/libstdc++.so.6.0.20 libstdc++.so.6 the code is running so such replacement seems to work fine. best wishes Alfredo On Thu, May 21, 2020 at 10:25 PM Alfredo Jaramillo < ajaramillopa...@gmail.com> wrote: > I just read at the end of the section "compilers" here > https://www.mcs.anl.gov/petsc/documentation/installation.html that one > can indicate these libraries by doing > ./configure --LIBS='-Bstatic -lifcore -Bdynamic' > > thank you very much for your help, Satish > > alfredo > > On Thu, May 21, 2020 at 10:08 PM Satish Balay wrote: > >> Configure attempts to automatically determine the language compatible >> libraries by running 'icc -v' and 'ifort -v' - and uses what it says. [this >> list can be different based on compiler options used] >> >> And PETSc examples work fine - so its not clear why fcoremt_pic is >> causing issues with your app build. >> >> There are some differences in your build vs examples - so that could be >> one reason. So its best to build PETSc and your app with the exact same >> compilers and options and see if that works. >> >> Alternative is to specify to PETSc configure the compatibility libraries >> so that it doesn't try to figure this out. [and add extra un-needed >> libraries] >> >> With Intel compilers - its likely the additional configure option is: >> >> LIBS="-Bstatic -lifcore -Bdynamic" >> >> If this works - configure won't run 'ifort -v' and grab fcoremt_pic etc.. >> >> Satish >> >> On Thu, 21 May 2020, Alfredo Jaramillo wrote: >> >> > In fact, removing -lifcore_pic from >> PETSC_DIR/PETSC_ARCH/lib/petscvariables >> > solved the problem (this was the first I traied and it worked), it >> compiles >> > and the program runs fine. >> > >> > Also, -lpetsc was being listed some times when compiling .c to .o, I >> fixed >> > that in the scripts. >> > >> > Now, I'm a bit lost about when fcoremt_pic is being linked or if it is >> > necessary at all? >> > >> > thank you very much! >> > alfredo >> > >> > On Thu, May 21, 2020 at 7:21 PM Satish Balay wrote: >> > >> > > For one - PETSc is built without -qopenmp flag - but your makefile is >> > > using it. Intel compiler can link in with >> > > different [incompatible] compiler libraries when some flags change >> this >> > > way [causing conflict]. >> > > >> > > However - the issue could be: >> > > >> > > your makefile is listing PETSC_LIB [or however you are accessing petsc >> > > info from petsc makefiles] redundantly. >> > > >> > > i.e >> > > >> > > - its listing -lpetsc etc when compiling .c to .o files [this >> should not >> > > happen] >> > > - its listing -lpetsc etc twice [-lifcore_pic is listed once >> though] - >> > > don't know if this is the reason for the problem. >> > > >> > > You can try [manually] removing -lifcore_pic from >> > > PETSC_DIR/PETSC_ARCH/lib/petscvariables - and see if this problem >> goes away >> > > >> > > Satish >> > > >> > > On Thu, 21 May 2020, Alfredo Jaramillo wrote: >> > > >> > > > here is the output: >> > > > >> > > > alfredo.jaramillo@sdumont11 tutorials]$ make ex19 >> > > > mpiicc -fPIC -O3 -march=native -mtune=native -fPIC -O3 >> -march=native >> > > > -mtune=native -I/scratch/simulreserv/softwares/petsc-3.13.0/include >> > > > >> > > >> -I/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/include >> > > > -I/scratch/simulreserv/softwares/valgrind-3.15.0/include >> > > > -I/scratch/app/zlib/1.2.11/includeex19.c >>
Re: [petsc-users] multiple definition of `main' with intel compilers
I just read at the end of the section "compilers" here https://www.mcs.anl.gov/petsc/documentation/installation.html that one can indicate these libraries by doing ./configure --LIBS='-Bstatic -lifcore -Bdynamic' thank you very much for your help, Satish alfredo On Thu, May 21, 2020 at 10:08 PM Satish Balay wrote: > Configure attempts to automatically determine the language compatible > libraries by running 'icc -v' and 'ifort -v' - and uses what it says. [this > list can be different based on compiler options used] > > And PETSc examples work fine - so its not clear why fcoremt_pic is causing > issues with your app build. > > There are some differences in your build vs examples - so that could be > one reason. So its best to build PETSc and your app with the exact same > compilers and options and see if that works. > > Alternative is to specify to PETSc configure the compatibility libraries > so that it doesn't try to figure this out. [and add extra un-needed > libraries] > > With Intel compilers - its likely the additional configure option is: > > LIBS="-Bstatic -lifcore -Bdynamic" > > If this works - configure won't run 'ifort -v' and grab fcoremt_pic etc.. > > Satish > > On Thu, 21 May 2020, Alfredo Jaramillo wrote: > > > In fact, removing -lifcore_pic from > PETSC_DIR/PETSC_ARCH/lib/petscvariables > > solved the problem (this was the first I traied and it worked), it > compiles > > and the program runs fine. > > > > Also, -lpetsc was being listed some times when compiling .c to .o, I > fixed > > that in the scripts. > > > > Now, I'm a bit lost about when fcoremt_pic is being linked or if it is > > necessary at all? > > > > thank you very much! > > alfredo > > > > On Thu, May 21, 2020 at 7:21 PM Satish Balay wrote: > > > > > For one - PETSc is built without -qopenmp flag - but your makefile is > > > using it. Intel compiler can link in with > > > different [incompatible] compiler libraries when some flags change this > > > way [causing conflict]. > > > > > > However - the issue could be: > > > > > > your makefile is listing PETSC_LIB [or however you are accessing petsc > > > info from petsc makefiles] redundantly. > > > > > > i.e > > > > > > - its listing -lpetsc etc when compiling .c to .o files [this should > not > > > happen] > > > - its listing -lpetsc etc twice [-lifcore_pic is listed once though] > - > > > don't know if this is the reason for the problem. > > > > > > You can try [manually] removing -lifcore_pic from > > > PETSC_DIR/PETSC_ARCH/lib/petscvariables - and see if this problem goes > away > > > > > > Satish > > > > > > On Thu, 21 May 2020, Alfredo Jaramillo wrote: > > > > > > > here is the output: > > > > > > > > alfredo.jaramillo@sdumont11 tutorials]$ make ex19 > > > > mpiicc -fPIC -O3 -march=native -mtune=native -fPIC -O3 -march=native > > > > -mtune=native -I/scratch/simulreserv/softwares/petsc-3.13.0/include > > > > > > > > -I/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/include > > > > -I/scratch/simulreserv/softwares/valgrind-3.15.0/include > > > > -I/scratch/app/zlib/1.2.11/includeex19.c > > > > > > > > -Wl,-rpath,/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/lib > > > > > > > > -L/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/lib > > > > > > > > -Wl,-rpath,/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/lib > > > > > > > > -L/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/lib > > > > > > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/release_mt > > > > > > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/release_mt > > > > > > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib > > > > > > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib > > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/clck/ > > > > 3.1.2.006/lib/intel64 > -L/opt/intel/parallel_studio_xe_2016_update2/clck/ > > > > 3.1.2.006/lib/intel64 > > > > >
Re: [petsc-users] multiple definition of `main' with intel compilers
In fact, removing -lifcore_pic from PETSC_DIR/PETSC_ARCH/lib/petscvariables solved the problem (this was the first I traied and it worked), it compiles and the program runs fine. Also, -lpetsc was being listed some times when compiling .c to .o, I fixed that in the scripts. Now, I'm a bit lost about when fcoremt_pic is being linked or if it is necessary at all? thank you very much! alfredo On Thu, May 21, 2020 at 7:21 PM Satish Balay wrote: > For one - PETSc is built without -qopenmp flag - but your makefile is > using it. Intel compiler can link in with > different [incompatible] compiler libraries when some flags change this > way [causing conflict]. > > However - the issue could be: > > your makefile is listing PETSC_LIB [or however you are accessing petsc > info from petsc makefiles] redundantly. > > i.e > > - its listing -lpetsc etc when compiling .c to .o files [this should not > happen] > - its listing -lpetsc etc twice [-lifcore_pic is listed once though] - > don't know if this is the reason for the problem. > > You can try [manually] removing -lifcore_pic from > PETSC_DIR/PETSC_ARCH/lib/petscvariables - and see if this problem goes away > > Satish > > On Thu, 21 May 2020, Alfredo Jaramillo wrote: > > > here is the output: > > > > alfredo.jaramillo@sdumont11 tutorials]$ make ex19 > > mpiicc -fPIC -O3 -march=native -mtune=native -fPIC -O3 -march=native > > -mtune=native -I/scratch/simulreserv/softwares/petsc-3.13.0/include > > > -I/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/include > > -I/scratch/simulreserv/softwares/valgrind-3.15.0/include > > -I/scratch/app/zlib/1.2.11/includeex19.c > > > -Wl,-rpath,/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/lib > > > -L/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/lib > > > -Wl,-rpath,/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/lib > > > -L/scratch/simulreserv/softwares/petsc-3.13.0/x64-O3-3.13-intel2016-64/lib > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/release_mt > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/release_mt > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/clck/ > > 3.1.2.006/lib/intel64 -L/opt/intel/parallel_studio_xe_2016_update2/clck/ > > 3.1.2.006/lib/intel64 > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/ipp/lib/intel64 > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/ipp/lib/intel64 > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/compiler/lib/intel64 > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/compiler/lib/intel64 > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64 > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64 > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/tbb/lib/intel64/gcc4.4 > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/tbb/lib/intel64/gcc4.4 > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/daal/lib/intel64_lin > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/daal/lib/intel64_lin > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/tbb/lib/intel64_lin/gcc4.4 > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/tbb/lib/intel64_lin/gcc4.4 > > > -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/compiler/lib/intel64_lin > > > -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/compiler/lib/intel64_lin > > -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.8.5 > > -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 > > -Wl,-rpath,/opt/intel/mpi-rt/5.1/intel64/lib/release_mt > > -Wl,-rpath,/opt/intel/mpi-rt/5.1/intel64/lib -lpetsc -lHYPRE -lcmumps > > -ldmumps -lsmumps -lzmumps -lmumps_common -lpord -lscalapack -lopenblas > > -lstdc++ -ldl -l
Re: [petsc-users] multiple definition of `main' with intel compilers
,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64 -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64 -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/tbb/lib/intel64/gcc4.4 -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/tbb/lib/intel64/gcc4.4 -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/daal/lib/intel64_lin -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/daal/lib/intel64_lin -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/tbb/lib/intel64_lin/gcc4.4 -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/tbb/lib/intel64_lin/gcc4.4 -Wl,-rpath,/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/compiler/lib/intel64_lin -L/opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/compiler/lib/intel64_lin -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -Wl,-rpath,/opt/intel/mpi-rt/5.1/intel64/lib/release_mt -Wl,-rpath,/opt/intel/mpi-rt/5.1/intel64/lib -lpetsc -lHYPRE -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common -lpord -lscalapack -lopenblas -lstdc++ -ldl -lmpifort -lmpi -lmpigi -lrt -lpthread -lifport -lifcore_pic -limf -lsvml -lm -lipgo -lirc -lgcc_s -lirc_s -lquadmath -lstdc++ -ldl -o ex5f [alfredo.jaramillo@sdumont11 tutorials]$ ./ex5f Number of SNES iterations = 4 [alfredo.jaramillo@sdumont11 tutorials]$ ./ex19 lid velocity = 0.0625, prandtl # = 1., grashof # = 1. Number of SNES iterations = 2 On Thu, May 21, 2020 at 6:53 PM Satish Balay wrote: > Please copy/paste complete [compile] commands from: > > src/snes/tutorials/ > make clean > make ex19 > make ex5f > > Likely the link command used in your code is different than what is used > here - triggering errors. > > Satish > > On Thu, 21 May 2020, Alfredo Jaramillo wrote: > > > hello Satish, no the tests seem to be ok altough some error related to > mpd. > > > > ==THE TESTS=== > > > > Running check examples to verify correct installation > > Using PETSC_DIR=/scratch/simulreserv/softwares/petsc-3.13.0 and > > PETSC_ARCH=x64-O3-3.13-intel2016-64 > > Possible error running C/C++ src/snes/tutorials/ex19 with 1 MPI process > > See http://www.mcs.anl.gov/petsc/documentation/faq.html > > mpiexec_sdumont11: cannot connect to local mpd > > (/tmp/mpd2.console_alfredo.jaramillo); possible causes: > > 1. no mpd is running on this host > > 2. an mpd is running but was started without a "console" (-n option) > > Possible error running C/C++ src/snes/tutorials/ex19 with 2 MPI processes > > See http://www.mcs.anl.gov/petsc/documentation/faq.html > > mpiexec_sdumont11: cannot connect to local mpd > > (/tmp/mpd2.console_alfredo.jaramillo); possible causes: > > 1. no mpd is running on this host > > 2. an mpd is running but was started without a "console" (-n option) > > 1,5c1,3 > > < lid velocity = 0.0016, prandtl # = 1., grashof # = 1. > > < 0 SNES Function norm 0.0406612 > > < 1 SNES Function norm 4.12227e-06 > > < 2 SNES Function norm 6.098e-11 > > < Number of SNES iterations = 2 > > --- > > > mpiexec_sdumont11: cannot connect to local mpd > > (/tmp/mpd2.console_alfredo.jaramillo); possible causes: > > > 1. no mpd is running on this host > > > 2. an mpd is running but was started without a "console" (-n option) > > /scratch/simulreserv/softwares/petsc-3.13.0/src/snes/tutorials > > Possible problem with ex19 running with hypre, diffs above > > = > > 1,9c1,3 > > < lid velocity = 0.0625, prandtl # = 1., grashof # = 1. > > < 0 SNES Function norm 0.239155 > > < 0 KSP Residual norm 0.239155 > > < 1 KSP Residual norm < 1.e-11 > > < 1 SNES Function norm 6.81968e-05 > > < 0 KSP Residual norm 6.81968e-05 > > < 1 KSP Residual norm < 1.e-11 > > < 2 SNES Function norm < 1.e-11 > > < Number of SNES iterations = 2 > > --- > > > mpiexec_sdumont11: cannot connect to local mpd > > (/tmp/mpd2.console_alfredo.jaramillo); possible causes: > > > 1. no mpd is running on this host > > > 2. an mpd is running but was started without a "console" (-n option) > > /scratch/simulreserv/softwares/petsc-3.13.0/src/snes/tutorials > > Possible problem with ex19 running with mumps, diffs a
Re: [petsc-users] multiple definition of `main' with intel compilers
hello Satish, no the tests seem to be ok altough some error related to mpd. ==THE TESTS=== Running check examples to verify correct installation Using PETSC_DIR=/scratch/simulreserv/softwares/petsc-3.13.0 and PETSC_ARCH=x64-O3-3.13-intel2016-64 Possible error running C/C++ src/snes/tutorials/ex19 with 1 MPI process See http://www.mcs.anl.gov/petsc/documentation/faq.html mpiexec_sdumont11: cannot connect to local mpd (/tmp/mpd2.console_alfredo.jaramillo); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option) Possible error running C/C++ src/snes/tutorials/ex19 with 2 MPI processes See http://www.mcs.anl.gov/petsc/documentation/faq.html mpiexec_sdumont11: cannot connect to local mpd (/tmp/mpd2.console_alfredo.jaramillo); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option) 1,5c1,3 < lid velocity = 0.0016, prandtl # = 1., grashof # = 1. < 0 SNES Function norm 0.0406612 < 1 SNES Function norm 4.12227e-06 < 2 SNES Function norm 6.098e-11 < Number of SNES iterations = 2 --- > mpiexec_sdumont11: cannot connect to local mpd (/tmp/mpd2.console_alfredo.jaramillo); possible causes: > 1. no mpd is running on this host > 2. an mpd is running but was started without a "console" (-n option) /scratch/simulreserv/softwares/petsc-3.13.0/src/snes/tutorials Possible problem with ex19 running with hypre, diffs above = 1,9c1,3 < lid velocity = 0.0625, prandtl # = 1., grashof # = 1. < 0 SNES Function norm 0.239155 < 0 KSP Residual norm 0.239155 < 1 KSP Residual norm < 1.e-11 < 1 SNES Function norm 6.81968e-05 < 0 KSP Residual norm 6.81968e-05 < 1 KSP Residual norm < 1.e-11 < 2 SNES Function norm < 1.e-11 < Number of SNES iterations = 2 --- > mpiexec_sdumont11: cannot connect to local mpd (/tmp/mpd2.console_alfredo.jaramillo); possible causes: > 1. no mpd is running on this host > 2. an mpd is running but was started without a "console" (-n option) /scratch/simulreserv/softwares/petsc-3.13.0/src/snes/tutorials Possible problem with ex19 running with mumps, diffs above = Possible error running Fortran example src/snes/tutorials/ex5f with 1 MPI process See http://www.mcs.anl.gov/petsc/documentation/faq.html mpiexec_sdumont11: cannot connect to local mpd (/tmp/mpd2.console_alfredo.jaramillo); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option) Completed test examples === I entered in src/snes/tutorials/ and executed "make ex5f". The binary exf5 was created On Thu, May 21, 2020 at 6:37 PM Satish Balay wrote: > Do you get this error when building PETSc examples [C and/or fortran] - > when you build them with the corresponding petsc makefile? > > Can you send the log of the example compiles? > > Satish > > --- > > [the attachment got deleted - don't know by who..] > > DENIAL OF SERVICE ALERT > > A denial of service protection limit was exceeded. The file has been > removed. > Context: 'configure.log.7z' > Reason: The data size limit was exceeded > Limit: 10 MB > Ticket Number : 0c9c-5ec6-f30f-0001 > > > For further information, contact your system administrator. > Copyright 1999-2014 McAfee, Inc. > All Rights Reserved. > http://www.mcafee.com > > > > On Thu, 21 May 2020, Alfredo Jaramillo wrote: > > > dear PETSc team, > > > > I have compiled PETSc with a 2016 version of the intel compilers. The > > installation went well, but when I tried to compile my code the following > > error appears in the final step of compilation (linking with ld) > > > > ../build/linux_icc/obj_linux_icc_opt/main.o: In function `main': > > main.c:(.text+0x0): multiple definition of `main' > > > /opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/compiler/lib/intel64/libifcore_pic.a(for_main.o):for_main.c:(.text+0x0): > > first defined here > > > /opt/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/compiler/lib/intel64/libifcore_pic.a(for_main.o): > > In function `main': > > for_main.c:(.text+0x3e): undefined reference to `MAIN__' > > > > I searched for this and I found that the option "-nofor_main" should be > > added when compiling with ifort, but our code is written only in C an > C++. > > The FORTRAN compiler is used when PETSc compiles MUMPS. So I dont know if > > this would work for this case. > > > > The configure.log file and the log of the compilation giving the error > are > > attached to this message. These logs were obtained in a cluster, I'm > > getting the same error on my personal computer with a 2020 version of the > > Intel Parallel Studio. > > > > thank you for any help on this > > Alfredo > > > >
Re: [petsc-users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: deactivating x11 in PETSc 3.13.0 with pastix/hwloc
it worked fine! the previous output: [alfredo.jaramillo@sdumont14 petsc]$ ldd arch-linux2-c-debug-actual/lib/libhwloc.so linux-vdso.so.1 => (0x7ffde6f49000) libm.so.6 => /usr/lib64/libm.so.6 (0x7fb8be4ed000) libXNVCtrl.so.0 => /usr/lib64/libXNVCtrl.so.0 (0x7fb8be2e8000) libXext.so.6 => /usr/lib64/libXext.so.6 (0x7fb8be0d6000) libX11.so.6 => /usr/lib64/libX11.so.6 (0x7fb8bdd98000) libc.so.6 => /usr/lib64/libc.so.6 (0x7fb8bd9cb000) /lib64/ld-linux-x86-64.so.2 (0x7fb8bea4f000) libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x7fb8bd7a3000) libdl.so.2 => /usr/lib64/libdl.so.2 (0x7fb8bd59f000) libXau.so.6 => /usr/lib64/libXau.so.6 (0x7fb8bd39b000) the updated output: [alfredo.jaramillo@sdumont14 petsc]$ ldd arch-linux2-c-debug-previous/lib/libhwloc.so linux-vdso.so.1 => (0x7fff135c7000) libm.so.6 => /usr/lib64/libm.so.6 (0x7f5ecd5f3000) libc.so.6 => /usr/lib64/libc.so.6 (0x7f5ecd226000) /lib64/ld-linux-x86-64.so.2 (0x7f5ecdb54000 I added the option --download-hwloc-configure-arguments="--disable-opencl --disable-cuda --disable-nvml --disable-gl" to my installation and pastix is working fine, thank you very much regards On Mon, Apr 6, 2020 at 6:39 PM Satish Balay wrote: > I pushed another change to branch balay/fix-hwloc-x-dependency/maint > > Can you check again? > > BTW: The following options don't make sense when using --with-mpi=0 > > CPPFLAGS=-I/scratch/app/openmpi/4.0_gnu/include > -I/scratch/app/mpc/1.0.3/include -I/scratch/app/isl/0.18/include > -I/scratch/app/gcc/6.5/include LDFLAGS=-L/scratch/app/openmpi/4.0_gnu/lib > -L/scratch/app/mpc/1.0.3/lib -L/scratch/app/isl/0.18/lib > -L/scratch/app/gcc/6.5/lib64 -L/scratch/app/gcc/6.5/lib > > [And likely these options are missing appropriate quotes..] > > Satish > > On Mon, 6 Apr 2020, Alfredo Jaramillo wrote: > > > ops... you can find the attachments here > > > https://www.dropbox.com/sh/n8scz7wioe01t6p/AAC3P3iU1jLYsW6m3vs6S5DHa?dl=0 > > > > I got > > > > $ ldd minimal/lib/libhwloc.so > > linux-vdso.so.1 => (0x7fffcc3ae000) > > libm.so.6 => /usr/lib64/libm.so.6 (0x7ff0b90a2000) > > libXNVCtrl.so.0 => /usr/lib64/libXNVCtrl.so.0 (0x7ff0b8e9d000) > > libXext.so.6 => /usr/lib64/libXext.so.6 (0x7ff0b8c8b000) > > libX11.so.6 => /usr/lib64/libX11.so.6 (0x7ff0b894d000) > > libc.so.6 => /usr/lib64/libc.so.6 (0x7ff0b858) > > /lib64/ld-linux-x86-64.so.2 (0x7ff0b9604000) > > libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x7ff0b8358000) > > libdl.so.2 => /usr/lib64/libdl.so.2 (0x7ff0b8154000) > > libXau.so.6 => /usr/lib64/libXau.so.6 (0x7ff0b7f5) > > > > On Mon, Apr 6, 2020 at 6:02 PM Satish Balay wrote: > > > > > The attachments didn't come through.. > > > > > > >>>> > > > DENIAL OF SERVICE ALERT > > > > > > A denial of service protection limit was exceeded. The file has been > > > removed. > > > Context: 'hwloc-x11=no.tar.gz\hwloc-x11=no.tar' > > > Reason: The data size limit was exceeded > > > Limit: 10 MB > > > Ticket Number : 0c20-5e8b-97c0-000f > > > <<<<<< > > > > > > Can you retry the test below - i.e './configure --with-mpi=0 > > > --download-hwloc --with-x=0' - and see if this works? > > > > > > What do you get for ldd [as shown below]? > > > > > > Satish > > > > > > On Mon, 6 Apr 2020, Alfredo Jaramillo wrote: > > > > > > > I've tried two installations in that branch, a "full installation" > with > > > > every package I need, and a minimal one. I attach the results here, > these > > > > libraries are still being linked > > > > > > > > On Mon, Apr 6, 2020 at 4:41 PM Satish Balay > wrote: > > > > > > > > > Hm - its working for me on CentOS7 [I see you are using > RHEL7/CentOS7] > > > > > > > > > > > > > > > Can you try the branch balay/fix-hwloc-x-dependency/maint - and see > > > what > > > > > you get? > > > > > > > > > > [balay@petsc-c7 petsc:d409bfc]$ ./configure --with-mpi=0 > > > --download-hwloc > > > > > --with-x=0 > > > > > > > > > > [balay@petsc-c7 petsc:d409bfc]$ ldd > > > arch-linux2-c-debug/lib/libhwloc.so > > > > > > > > > > > > > > > linux-vdso.so.1 => (0x7ffe88bae000) > > > > > libm.so.6 => /lib64/libm.so.6 (0x7ff75
Re: [petsc-users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: deactivating x11 in PETSc 3.13.0 with pastix/hwloc
ops... you can find the attachments here https://www.dropbox.com/sh/n8scz7wioe01t6p/AAC3P3iU1jLYsW6m3vs6S5DHa?dl=0 I got $ ldd minimal/lib/libhwloc.so linux-vdso.so.1 => (0x7fffcc3ae000) libm.so.6 => /usr/lib64/libm.so.6 (0x7ff0b90a2000) libXNVCtrl.so.0 => /usr/lib64/libXNVCtrl.so.0 (0x7ff0b8e9d000) libXext.so.6 => /usr/lib64/libXext.so.6 (0x7ff0b8c8b000) libX11.so.6 => /usr/lib64/libX11.so.6 (0x7ff0b894d000) libc.so.6 => /usr/lib64/libc.so.6 (0x7ff0b858) /lib64/ld-linux-x86-64.so.2 (0x7ff0b9604000) libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x7ff0b8358000) libdl.so.2 => /usr/lib64/libdl.so.2 (0x7ff0b8154000) libXau.so.6 => /usr/lib64/libXau.so.6 (0x7ff0b7f5) On Mon, Apr 6, 2020 at 6:02 PM Satish Balay wrote: > The attachments didn't come through.. > > >>>> > DENIAL OF SERVICE ALERT > > A denial of service protection limit was exceeded. The file has been > removed. > Context: 'hwloc-x11=no.tar.gz\hwloc-x11=no.tar' > Reason: The data size limit was exceeded > Limit: 10 MB > Ticket Number : 0c20-5e8b-97c0-000f > <<<<<< > > Can you retry the test below - i.e './configure --with-mpi=0 > --download-hwloc --with-x=0' - and see if this works? > > What do you get for ldd [as shown below]? > > Satish > > On Mon, 6 Apr 2020, Alfredo Jaramillo wrote: > > > I've tried two installations in that branch, a "full installation" with > > every package I need, and a minimal one. I attach the results here, these > > libraries are still being linked > > > > On Mon, Apr 6, 2020 at 4:41 PM Satish Balay wrote: > > > > > Hm - its working for me on CentOS7 [I see you are using RHEL7/CentOS7] > > > > > > > > > Can you try the branch balay/fix-hwloc-x-dependency/maint - and see > what > > > you get? > > > > > > [balay@petsc-c7 petsc:d409bfc]$ ./configure --with-mpi=0 > --download-hwloc > > > --with-x=0 > > > > > > [balay@petsc-c7 petsc:d409bfc]$ ldd > arch-linux2-c-debug/lib/libhwloc.so > > > > > > > > > linux-vdso.so.1 => (0x7ffe88bae000) > > > libm.so.6 => /lib64/libm.so.6 (0x7ff75c7b9000) > > > libc.so.6 => /lib64/libc.so.6 (0x7ff75c3eb000) > > > /lib64/ld-linux-x86-64.so.2 (0x7ff75cd1a000) > > > [balay@petsc-c7 petsc:d409bfc]$ ./configure --with-mpi=0 > --download-hwloc > > > --with-x=1 > > > > > > [balay@petsc-c7 petsc:d409bfc]$ ldd > arch-linux2-c-debug/lib/libhwloc.so > > > > > > > > > linux-vdso.so.1 => (0x7ffe129cc000) > > > libm.so.6 => /lib64/libm.so.6 (0x7f89920fa000) > > > libX11.so.6 => /lib64/libX11.so.6 (0x7f8991dbc000) > > > libc.so.6 => /lib64/libc.so.6 (0x7f89919ee000) > > > /lib64/ld-linux-x86-64.so.2 (0x7f899265b000) > > > libxcb.so.1 => /lib64/libxcb.so.1 (0x7f89917c6000) > > > libdl.so.2 => /lib64/libdl.so.2 (0x7f89915c2000) > > > libXau.so.6 => /lib64/libXau.so.6 (0x7f89913be000) > > > [balay@petsc-c7 petsc:d409bfc]$ > > > > > > Satish > > > > > > On Mon, 6 Apr 2020, Alfredo Jaramillo wrote: > > > > > > > No, it doesn't work... I also tried --with-x=disabled. I gave a look > to > > > > ./configure --help in the hwloc directory and to the configure file > > > itself, > > > > but Im not finding the right option. > > > > > > > > On Mon, Apr 6, 2020 at 3:53 PM Satish Balay > wrote: > > > > > > > > > Looks like the option is --with-x=no > > > > > > > > > > Can you give this a try? Branch balay/fix-hwloc-x-dependency/maint > has > > > > > this update now > > > > > > > > > > Satish > > > > > > > > > > On Mon, 6 Apr 2020, Alfredo Jaramillo wrote: > > > > > > > > > > > Hello Satish, Im sorry but I think I tested this workaround with > the > > > > > wrong > > > > > > installation, my bad. > > > > > > In fact the libXNVC, X11 libraries are still being linked even > with > > > the > > > > > > --download-hwloc-configure-arguments=--without-x option > > > > > > > > > > > > Im attaching the config.log file located in > > > > > > $PETSC_DIR/$PETSC_ARCH/externalpackages/hwloc-2.1.0 > > &g
Re: [petsc-users] deactivating x11 in PETSc 3.13.0 with pastix/hwloc
No, it doesn't work... I also tried --with-x=disabled. I gave a look to ./configure --help in the hwloc directory and to the configure file itself, but Im not finding the right option. On Mon, Apr 6, 2020 at 3:53 PM Satish Balay wrote: > Looks like the option is --with-x=no > > Can you give this a try? Branch balay/fix-hwloc-x-dependency/maint has > this update now > > Satish > > On Mon, 6 Apr 2020, Alfredo Jaramillo wrote: > > > Hello Satish, Im sorry but I think I tested this workaround with the > wrong > > installation, my bad. > > In fact the libXNVC, X11 libraries are still being linked even with the > > --download-hwloc-configure-arguments=--without-x option > > > > Im attaching the config.log file located in > > $PETSC_DIR/$PETSC_ARCH/externalpackages/hwloc-2.1.0 > > you can see that the option --without-x is there (line 7) but by means of > > $ ldd foo > > I see that the links are still there > > > > On Mon, Apr 6, 2020 at 2:57 PM Satish Balay wrote: > > > > > Great! > > > > > > Wrt pastix dependency on hwloc - > > > config/BuildSystem/config/packages/PaStiX.py has the following comment: > > > > > > # PaStiX.py does not absolutely require hwloc, but it performs > better > > > with it and can fail (in ways not easily tested) without it > > > # > > > > https://gforge.inria.fr/forum/forum.php?thread_id=32824_id=599_id=186 > > > # https://solverstack.gitlabpages.inria.fr/pastix/Bindings.html > > > > > > I have a fix in branch balay/fix-hwloc-x-dependency/maint [that does > not > > > need the extra --download-hwloc-configure-arguments=--without-x > option]. > > > Can you give this a try? > > > > > > Satish > > > > > > On Mon, 6 Apr 2020, Alfredo Jaramillo wrote: > > > > > > > hello Satish, > > > > adding > > > > --download-hwloc-configure-arguments=--without-x > > > > worked perfectly > > > > > > > > thank you! > > > > > > > > On Mon, Apr 6, 2020 at 2:12 PM Satish Balay > wrote: > > > > > > > > > you can try: > > > > > > > > > > --download-pastix --download-hwloc > > > > > --download-hwloc-configure-arguments=--without-x > > > > > > > > > > We should fix this to automatically use --with-x=0/1 > > > > > > > > > > Satish > > > > > > > > > > On Mon, 6 Apr 2020, Alfredo Jaramillo wrote: > > > > > > > > > > > hello everyone, > > > > > > > > > > > > I have a fresh installation of the 3.13.0 version with pastix. > Like > > > with > > > > > > previous versions, I'm using the options > > > > > > > > > > > > --with-x11=0 --with-x=0 --with-windows-graphics=0 > > > > > > > > > > > > to disable X11 > > > > > > > > > > > > however, when compiling my program foo and doing > > > > > > > > > > > > $ ldd foo > > > > > > > > > > > > between the linked libraries there appear: > > > > > > libXNVCtrl.so.0 and libX11.so.6 > > > > > > > > > > > > the first one related to NVIDIA. I observed that this does not > happen > > > > > when > > > > > > installing PETSc without hwloc. In this new version, PETSc > requires > > > to > > > > > > install hwloc when trying to install pastix. In previous > versions of > > > > > PETSc > > > > > > (eg 3.11.2) that wasn't necessary. > > > > > > > > > > > > I'm working in a cluster where I have no access to these > X11-related > > > > > > libraries and that's why I need them not be linked. Is it there > some > > > way > > > > > to > > > > > > disable X11 when installing hwloc? maybe enforcing some > configuration > > > > > > variables when installing it through petsc or installing it > > > > > independently? > > > > > > > > > > > > thanks a lot! > > > > > > > > > > > > Below the configuration command of the two installations I've > tried > > > with > > > > > > the 3.13.0 version. > > > > > > > > > > > > === WITH PASTIX === > > > > > > > > > > > > ./configure --with-make-np=20 > > > > > > --with-petsc-arch=x64go-3.13-openmpi-4.0.1-pastix-64 > > > --with-debugging=0 > > > > > > --doCleanup=0 \ > > > > > > --with-mpi=1 \ > > > > > > --with-valgrind=1 > > > --with-valgrind-dir=$PATH_TO_VALGRIND/valgrind-3.15.0 \ > > > > > > --download-scalapack \ > > > > > > --download-openblas \ > > > > > > --download-mumps \ > > > > > > --download-superlu_dist \ > > > > > > --download-metis \ > > > > > > --download-parmetis \ > > > > > > --download-ptscotch \ > > > > > > --download-hypre \ > > > > > > > > > > > > *--download-pastix \--download-hwloc \* > > > > > > --with-64-bit-indices=1 \ > > > > > > LDFLAGS=$LDFLAGS CPPFLAGS=$CPPFLAGS \ > > > > > > --with-cxx-dialect=C++11 \ > > > > > > --with-x11=0 --with-x=0 --with-windows-graphics=0 \ > > > > > > COPTFLAGS="-O3 -march=native -mtune=native" \ > > > > > > CXXOPTFLAGS="-O3 -march=native -mtune=native" \ > > > > > > FOPTFLAGS="-O3 -march=native -mtune=native" > > > > > > > > > > > > === WITHOUT PASTIX === > > > > > > > > > > > > the same as above but the options "--download-pastix > > > --download-hwloc" > > > > > > > > > > > > == > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [petsc-users] deactivating x11 in PETSc 3.13.0 with pastix/hwloc
I understand, Matthew, thank you both! On Mon, Apr 6, 2020 at 2:29 PM Matthew Knepley wrote: > Okay, what it must be is that hwloc dynamically links to X11, so its never > given explicitly anywhere by us, but hwloc sucks it in. > Satish is right about the fix. > > Thanks, > > Matt > > On Mon, Apr 6, 2020 at 1:18 PM Alfredo Jaramillo < > ajaramillopa...@gmail.com> wrote: > >> attaching the file >> thanks! >> >> On Mon, Apr 6, 2020 at 2:12 PM Matthew Knepley wrote: >> >>> Hmm, there is no sign of libX11 in that log. Can you send me >>> >>> $PETSC_DIR/$PETSC_ARCH/lib/petsc/conf/petscvariables >>> >>> Thanks, >>> >>> Matt >>> >>> On Mon, Apr 6, 2020 at 1:02 PM Alfredo Jaramillo < >>> ajaramillopa...@gmail.com> wrote: >>> >>>> Sure, here it is. >>>> thx >>>> >>>> On Mon, Apr 6, 2020 at 1:59 PM Matthew Knepley >>>> wrote: >>>> >>>>> On Mon, Apr 6, 2020 at 12:55 PM Alfredo Jaramillo < >>>>> ajaramillopa...@gmail.com> wrote: >>>>> >>>>>> hello everyone, >>>>>> >>>>>> I have a fresh installation of the 3.13.0 version with pastix. Like >>>>>> with previous versions, I'm using the options >>>>>> >>>>>> --with-x11=0 --with-x=0 --with-windows-graphics=0 >>>>>> >>>>>> to disable X11 >>>>>> >>>>>> however, when compiling my program foo and doing >>>>>> >>>>>> $ ldd foo >>>>>> >>>>>> between the linked libraries there appear: >>>>>> libXNVCtrl.so.0 and libX11.so.6 >>>>>> >>>>>> the first one related to NVIDIA. I observed that this does not happen >>>>>> when installing PETSc without hwloc. In this new version, PETSc requires >>>>>> to >>>>>> install hwloc when trying to install pastix. In previous versions of >>>>>> PETSc >>>>>> (eg 3.11.2) that wasn't necessary. >>>>>> >>>>>> I'm working in a cluster where I have no access to these X11-related >>>>>> libraries and that's why I need them not be linked. Is it there some way >>>>>> to >>>>>> disable X11 when installing hwloc? maybe enforcing some configuration >>>>>> variables when installing it through petsc or installing it >>>>>> independently? >>>>>> >>>>> >>>>> Can you send configure.log? >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> >>>>>> thanks a lot! >>>>>> >>>>>> Below the configuration command of the two installations I've tried >>>>>> with the 3.13.0 version. >>>>>> >>>>>> === WITH PASTIX === >>>>>> >>>>>> ./configure --with-make-np=20 >>>>>> --with-petsc-arch=x64go-3.13-openmpi-4.0.1-pastix-64 --with-debugging=0 >>>>>> --doCleanup=0 \ >>>>>> --with-mpi=1 \ >>>>>> --with-valgrind=1 >>>>>> --with-valgrind-dir=$PATH_TO_VALGRIND/valgrind-3.15.0 \ >>>>>> --download-scalapack \ >>>>>> --download-openblas \ >>>>>> --download-mumps \ >>>>>> --download-superlu_dist \ >>>>>> --download-metis \ >>>>>> --download-parmetis \ >>>>>> --download-ptscotch \ >>>>>> --download-hypre \ >>>>>> >>>>>> *--download-pastix \--download-hwloc \* >>>>>> --with-64-bit-indices=1 \ >>>>>> LDFLAGS=$LDFLAGS CPPFLAGS=$CPPFLAGS \ >>>>>> --with-cxx-dialect=C++11 \ >>>>>> --with-x11=0 --with-x=0 --with-windows-graphics=0 \ >>>>>> COPTFLAGS="-O3 -march=native -mtune=native" \ >>>>>> CXXOPTFLAGS="-O3 -march=native -mtune=native" \ >>>>>> FOPTFLAGS="-O3 -march=native -mtune=native" >>>>>> >>>>>> === WITHOUT PASTIX === >>>>>> >>>>>> the same as above but the options "--download-pastix --download-hwloc" >>>>>> >>>>>> == >>>>>> >>>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> https://www.cse.buffalo.edu/~knepley/ >>>>> <http://www.cse.buffalo.edu/~knepley/> >>>>> >>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> <http://www.cse.buffalo.edu/~knepley/> >>> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> >
Re: [petsc-users] deactivating x11 in PETSc 3.13.0 with pastix/hwloc
hello Satish, adding --download-hwloc-configure-arguments=--without-x worked perfectly thank you! On Mon, Apr 6, 2020 at 2:12 PM Satish Balay wrote: > you can try: > > --download-pastix --download-hwloc > --download-hwloc-configure-arguments=--without-x > > We should fix this to automatically use --with-x=0/1 > > Satish > > On Mon, 6 Apr 2020, Alfredo Jaramillo wrote: > > > hello everyone, > > > > I have a fresh installation of the 3.13.0 version with pastix. Like with > > previous versions, I'm using the options > > > > --with-x11=0 --with-x=0 --with-windows-graphics=0 > > > > to disable X11 > > > > however, when compiling my program foo and doing > > > > $ ldd foo > > > > between the linked libraries there appear: > > libXNVCtrl.so.0 and libX11.so.6 > > > > the first one related to NVIDIA. I observed that this does not happen > when > > installing PETSc without hwloc. In this new version, PETSc requires to > > install hwloc when trying to install pastix. In previous versions of > PETSc > > (eg 3.11.2) that wasn't necessary. > > > > I'm working in a cluster where I have no access to these X11-related > > libraries and that's why I need them not be linked. Is it there some way > to > > disable X11 when installing hwloc? maybe enforcing some configuration > > variables when installing it through petsc or installing it > independently? > > > > thanks a lot! > > > > Below the configuration command of the two installations I've tried with > > the 3.13.0 version. > > > > === WITH PASTIX === > > > > ./configure --with-make-np=20 > > --with-petsc-arch=x64go-3.13-openmpi-4.0.1-pastix-64 --with-debugging=0 > > --doCleanup=0 \ > > --with-mpi=1 \ > > --with-valgrind=1 --with-valgrind-dir=$PATH_TO_VALGRIND/valgrind-3.15.0 \ > > --download-scalapack \ > > --download-openblas \ > > --download-mumps \ > > --download-superlu_dist \ > > --download-metis \ > > --download-parmetis \ > > --download-ptscotch \ > > --download-hypre \ > > > > *--download-pastix \--download-hwloc \* > > --with-64-bit-indices=1 \ > > LDFLAGS=$LDFLAGS CPPFLAGS=$CPPFLAGS \ > > --with-cxx-dialect=C++11 \ > > --with-x11=0 --with-x=0 --with-windows-graphics=0 \ > > COPTFLAGS="-O3 -march=native -mtune=native" \ > > CXXOPTFLAGS="-O3 -march=native -mtune=native" \ > > FOPTFLAGS="-O3 -march=native -mtune=native" > > > > === WITHOUT PASTIX === > > > > the same as above but the options "--download-pastix --download-hwloc" > > > > == > > > >
Re: [petsc-users] deactivating x11 in PETSc 3.13.0 with pastix/hwloc
attaching the file thanks! On Mon, Apr 6, 2020 at 2:12 PM Matthew Knepley wrote: > Hmm, there is no sign of libX11 in that log. Can you send me > > $PETSC_DIR/$PETSC_ARCH/lib/petsc/conf/petscvariables > > Thanks, > > Matt > > On Mon, Apr 6, 2020 at 1:02 PM Alfredo Jaramillo < > ajaramillopa...@gmail.com> wrote: > >> Sure, here it is. >> thx >> >> On Mon, Apr 6, 2020 at 1:59 PM Matthew Knepley wrote: >> >>> On Mon, Apr 6, 2020 at 12:55 PM Alfredo Jaramillo < >>> ajaramillopa...@gmail.com> wrote: >>> >>>> hello everyone, >>>> >>>> I have a fresh installation of the 3.13.0 version with pastix. Like >>>> with previous versions, I'm using the options >>>> >>>> --with-x11=0 --with-x=0 --with-windows-graphics=0 >>>> >>>> to disable X11 >>>> >>>> however, when compiling my program foo and doing >>>> >>>> $ ldd foo >>>> >>>> between the linked libraries there appear: >>>> libXNVCtrl.so.0 and libX11.so.6 >>>> >>>> the first one related to NVIDIA. I observed that this does not happen >>>> when installing PETSc without hwloc. In this new version, PETSc requires to >>>> install hwloc when trying to install pastix. In previous versions of PETSc >>>> (eg 3.11.2) that wasn't necessary. >>>> >>>> I'm working in a cluster where I have no access to these X11-related >>>> libraries and that's why I need them not be linked. Is it there some way to >>>> disable X11 when installing hwloc? maybe enforcing some configuration >>>> variables when installing it through petsc or installing it independently? >>>> >>> >>> Can you send configure.log? >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> thanks a lot! >>>> >>>> Below the configuration command of the two installations I've tried >>>> with the 3.13.0 version. >>>> >>>> === WITH PASTIX === >>>> >>>> ./configure --with-make-np=20 >>>> --with-petsc-arch=x64go-3.13-openmpi-4.0.1-pastix-64 --with-debugging=0 >>>> --doCleanup=0 \ >>>> --with-mpi=1 \ >>>> --with-valgrind=1 --with-valgrind-dir=$PATH_TO_VALGRIND/valgrind-3.15.0 >>>> \ >>>> --download-scalapack \ >>>> --download-openblas \ >>>> --download-mumps \ >>>> --download-superlu_dist \ >>>> --download-metis \ >>>> --download-parmetis \ >>>> --download-ptscotch \ >>>> --download-hypre \ >>>> >>>> *--download-pastix \--download-hwloc \* >>>> --with-64-bit-indices=1 \ >>>> LDFLAGS=$LDFLAGS CPPFLAGS=$CPPFLAGS \ >>>> --with-cxx-dialect=C++11 \ >>>> --with-x11=0 --with-x=0 --with-windows-graphics=0 \ >>>> COPTFLAGS="-O3 -march=native -mtune=native" \ >>>> CXXOPTFLAGS="-O3 -march=native -mtune=native" \ >>>> FOPTFLAGS="-O3 -march=native -mtune=native" >>>> >>>> === WITHOUT PASTIX === >>>> >>>> the same as above but the options "--download-pastix --download-hwloc" >>>> >>>> == >>>> >>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> <http://www.cse.buffalo.edu/~knepley/> >>> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> > petscvariables Description: Binary data
[petsc-users] deactivating x11 in PETSc 3.13.0 with pastix/hwloc
hello everyone, I have a fresh installation of the 3.13.0 version with pastix. Like with previous versions, I'm using the options --with-x11=0 --with-x=0 --with-windows-graphics=0 to disable X11 however, when compiling my program foo and doing $ ldd foo between the linked libraries there appear: libXNVCtrl.so.0 and libX11.so.6 the first one related to NVIDIA. I observed that this does not happen when installing PETSc without hwloc. In this new version, PETSc requires to install hwloc when trying to install pastix. In previous versions of PETSc (eg 3.11.2) that wasn't necessary. I'm working in a cluster where I have no access to these X11-related libraries and that's why I need them not be linked. Is it there some way to disable X11 when installing hwloc? maybe enforcing some configuration variables when installing it through petsc or installing it independently? thanks a lot! Below the configuration command of the two installations I've tried with the 3.13.0 version. === WITH PASTIX === ./configure --with-make-np=20 --with-petsc-arch=x64go-3.13-openmpi-4.0.1-pastix-64 --with-debugging=0 --doCleanup=0 \ --with-mpi=1 \ --with-valgrind=1 --with-valgrind-dir=$PATH_TO_VALGRIND/valgrind-3.15.0 \ --download-scalapack \ --download-openblas \ --download-mumps \ --download-superlu_dist \ --download-metis \ --download-parmetis \ --download-ptscotch \ --download-hypre \ *--download-pastix \--download-hwloc \* --with-64-bit-indices=1 \ LDFLAGS=$LDFLAGS CPPFLAGS=$CPPFLAGS \ --with-cxx-dialect=C++11 \ --with-x11=0 --with-x=0 --with-windows-graphics=0 \ COPTFLAGS="-O3 -march=native -mtune=native" \ CXXOPTFLAGS="-O3 -march=native -mtune=native" \ FOPTFLAGS="-O3 -march=native -mtune=native" === WITHOUT PASTIX === the same as above but the options "--download-pastix --download-hwloc" ==
