Hi Cody,
Yes, I'm able to compile and run a simple MPI program which prints rank and
size of MPI_COMM_WORLD. I used the same environment as my LibMesh
environment:
-bash-4.1$ icc -v
icc version 15.0.3 (gcc version 4.4.7 compatibility)
-bash-4.1$ mpiicc -v
mpiicc for the Intel(R) MPI Library 5.0 Update 3 for Linux*
Copyright(C) 2003-2015, Intel Corporation. All rights reserved.
icc version 15.0.3 (gcc version 4.4.7 compatibility)
Also, the code I pasted in my previous email gives me this output:
-bash-4.1$ ./petsc_vector_test
MPI_num_procs = 1, LibMesh_num_procs = 1, MPI_rank = 0, LibMesh_rank = 0, a
= 0.000000, b = 59.000000
*** Warning, AutoPtr is deprecated and will be removed in a future library
version! ./include/libmesh/auto_ptr.h, line 271, compiled Mar 9 2016 at
11:01:34 ***
-bash-4.1$ mpiexec -n 2 ./petsc_vector_test
MPI_num_procs = 2, LibMesh_num_procs = 1, MPI_rank = 1, LibMesh_rank = 0, a
= 0.000000, b = 59.000000
MPI_num_procs = 2, LibMesh_num_procs = 1, MPI_rank = 0, LibMesh_rank = 0, a
= 0.000000, b = 59.000000
*** Warning, AutoPtr is deprecated and will be removed in a future library
version! ./include/libmesh/auto_ptr.h, line 271, compiled Mar 9 2016 at
11:01:34 ***
Thanks,
Harshad
On Wed, Mar 9, 2016 at 4:15 PM, Cody Permann <codyperm...@gmail.com> wrote:
> Can you successfully compile and run a simple MPI hello world program and
> get the right rank size and messages out? If you your environment is not
> sane (different at runtime than it was during compile) you might have
> problems with MPI in general.
>
> Cody
>
> On Wed, Mar 9, 2016 at 2:10 PM Harshad Sahasrabudhe <hsaha...@purdue.edu>
> wrote:
>
>> Hi John,
>>
>> I tried compiling my simple test code (pasted) with -DNDEBUG flag. I don't
>> get the error due to ghosted vector. The problem is
>> libmesh_init.comm().size() is always 1 irrespective of how many processes
>> I
>> run it with. I also tried initializing MPI before libmesh and passing the
>> communicator, but the same happens.
>>
>> I am completely lost on where I'm going wrong with the LibMesh build! Any
>> help is appreciated.
>>
>> Thanks,
>> Harshad
>>
>> petsc_vector_test.cpp:
>>
>> #include "libmesh.h"
>> #include "petsc_vector.h"
>>
>> using namespace libMesh;
>>
>> int main(int argc, char* argv[])
>> {
>> MPI_Init(&argc, &argv);
>> int mpi_rank, mpi_size;
>> MPI_Comm_rank(MPI_COMM_WORLD, &mpi_rank);
>> MPI_Comm_size(MPI_COMM_WORLD, &mpi_size);
>> LibMeshInit libmesh_init(argc, argv, MPI_COMM_WORLD);
>> Parallel::Communicator comm;
>> libmesh_init.comm().duplicate(comm);
>> int num_procs = comm.size();
>> int rank = comm.rank();
>>
>> unsigned int N = 60;
>> unsigned int n = 60/num_procs;
>> std::vector<unsigned int> send_list(2);
>>
>> if(num_procs != 1)
>> {
>> if(rank != 0)
>> send_list[0] = rank*n-1;
>> else
>> send_list[0] = n+2;
>>
>> if(rank != num_procs - 1)
>> send_list[1] = (rank+1)*n+1;
>> else
>> send_list[1] = rank*n-2;
>> }
>> else
>> {
>> send_list[0] = 0;
>> send_list[1] = N-1;
>> }
>>
>>
>> PetscVector<double> parallel(comm, N, n, PARALLEL);
>>
>> for(unsigned int i=0; i<n; i++)
>> parallel.set(rank*n+i, rank*n+i);
>> parallel.close();
>>
>> PetscVector<double> *ghosted = NULL;
>>
>> if(num_procs != 1)
>> ghosted = new PetscVector<double>(comm, N, n+2, send_list, GHOSTED);
>> else
>> ghosted = new PetscVector<double>(comm, N, n, send_list, GHOSTED);
>>
>> parallel.localize(*ghosted, send_list);
>>
>> double a = (*ghosted)(send_list[0]);
>> double b = (*ghosted)(send_list[1]);
>>
>> printf("MPI_num_procs = %d, LibMesh_num_procs = %d, MPI_rank = %d,
>> LibMesh_rank = %d, a = %f, b = %f\n", mpi_size, num_procs, mpi_rank, rank,
>> a, b);
>>
>> delete ghosted;
>> }
>>
>> On Tue, Mar 8, 2016 at 7:45 PM, Harshad Sahasrabudhe <hsaha...@purdue.edu
>> >
>> wrote:
>>
>> > Forgot to mention, this probably indicates that something is very wrong
>> >> with your build, i.e. you are linking against a out-of-date debug
>> version
>> >> of the library by accident, or something else equally weird. I would
>> >> definitely try to fix this issue before worrying about other issues.
>> >
>> >
>> > It seems NDEBUG is not defined in libmesh_config.h for me, which is very
>> > weird. I'm not sure if that's the place at which it should be defined,
>> but
>> > looking at include/utils/restore_warnings.h, since it's used right after
>> > #include "libmesh_config.h" I'm guessing it should be there. I get
>> > libmesh_opt as the library, but NDEBUG still isn't defined! Here is the
>> > configure command I'm using:
>> >
>> > ./configure PETSC_DIR=$(PETSC_REAL_BUILD) MPIHOME=$(MPIHOME)
>> > PETSC_ARCH=$(PETSC_REAL_ARCH) \
>> > F77="$(MPIF77)" CC="$(MPICC)" GCC="$(GCC)" CXX="$(MPICXX)" --enable-vtk
>> > --with-vtk-include=$(VTKINC_PATH) \
>> > --with-vtk-lib=$(VTKLIB_PATH) --disable-tetgen --disable-nemesis
>> > --with-methods="opt" --with-metis=PETSc --enable-default-comm-world \
>> > --enable-amr --enable-shared=yes --disable-cxx11
>> > --enable-legacy-include-paths --disable-strict-lgpl --disable-examples
>> > --enable-slepc;
>> >
>> > I have attached the resultant config.log and libmesh_config.h. Any ideas
>> > what could be causing this issue?
>> >
>> > Thanks,
>> > Harshad
>> >
>> > On Tue, Mar 8, 2016 at 4:14 PM, Harshad Sahasrabudhe <
>> hsaha...@purdue.edu>
>> > wrote:
>> >
>> >> Hi John,
>> >>>
>> >>>
>> >>> Forgot to mention, this probably indicates that something is very
>> wrong
>> >>> with your build, i.e. you are linking against a out-of-date debug
>> version
>> >>> of the library by accident, or something else equally weird. I would
>> >>> definitely try to fix this issue before worrying about other issues.
>> >>
>> >>
>> >> It definitely looks like I'm linking with a debug version too. I need
>> to
>> >> find out where that's happening. Thanks for your insights!
>> >>
>> >> Harshad
>> >>
>> >>>
>> >>
>> >
>>
>> ------------------------------------------------------------------------------
>> Transform Data into Opportunity.
>> Accelerate data analysis in your applications with
>> Intel Data Analytics Acceleration Library.
>> Click to learn more.
>> http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
>> _______________________________________________
>> Libmesh-users mailing list
>> Libmesh-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/libmesh-users
>>
>
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
_______________________________________________
Libmesh-users mailing list
Libmesh-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libmesh-users
