Can you successfully compile and run a simple MPI hello world program and
get the right rank size and messages out? If you your environment is not
sane (different at runtime than it was during compile) you might have
problems with MPI in general.
Cody
On Wed, Mar 9, 2016 at 2:10 PM Harshad Sahasrabudhe <hsaha...@purdue.edu>
wrote:
> Hi John,
>
> I tried compiling my simple test code (pasted) with -DNDEBUG flag. I don't
> get the error due to ghosted vector. The problem is
> libmesh_init.comm().size() is always 1 irrespective of how many processes I
> run it with. I also tried initializing MPI before libmesh and passing the
> communicator, but the same happens.
>
> I am completely lost on where I'm going wrong with the LibMesh build! Any
> help is appreciated.
>
> Thanks,
> Harshad
>
> petsc_vector_test.cpp:
>
> #include "libmesh.h"
> #include "petsc_vector.h"
>
> using namespace libMesh;
>
> int main(int argc, char* argv[])
> {
> MPI_Init(&argc, &argv);
> int mpi_rank, mpi_size;
> MPI_Comm_rank(MPI_COMM_WORLD, &mpi_rank);
> MPI_Comm_size(MPI_COMM_WORLD, &mpi_size);
> LibMeshInit libmesh_init(argc, argv, MPI_COMM_WORLD);
> Parallel::Communicator comm;
> libmesh_init.comm().duplicate(comm);
> int num_procs = comm.size();
> int rank = comm.rank();
>
> unsigned int N = 60;
> unsigned int n = 60/num_procs;
> std::vector<unsigned int> send_list(2);
>
> if(num_procs != 1)
> {
> if(rank != 0)
> send_list[0] = rank*n-1;
> else
> send_list[0] = n+2;
>
> if(rank != num_procs - 1)
> send_list[1] = (rank+1)*n+1;
> else
> send_list[1] = rank*n-2;
> }
> else
> {
> send_list[0] = 0;
> send_list[1] = N-1;
> }
>
>
> PetscVector<double> parallel(comm, N, n, PARALLEL);
>
> for(unsigned int i=0; i<n; i++)
> parallel.set(rank*n+i, rank*n+i);
> parallel.close();
>
> PetscVector<double> *ghosted = NULL;
>
> if(num_procs != 1)
> ghosted = new PetscVector<double>(comm, N, n+2, send_list, GHOSTED);
> else
> ghosted = new PetscVector<double>(comm, N, n, send_list, GHOSTED);
>
> parallel.localize(*ghosted, send_list);
>
> double a = (*ghosted)(send_list[0]);
> double b = (*ghosted)(send_list[1]);
>
> printf("MPI_num_procs = %d, LibMesh_num_procs = %d, MPI_rank = %d,
> LibMesh_rank = %d, a = %f, b = %f\n", mpi_size, num_procs, mpi_rank, rank,
> a, b);
>
> delete ghosted;
> }
>
> On Tue, Mar 8, 2016 at 7:45 PM, Harshad Sahasrabudhe <hsaha...@purdue.edu>
> wrote:
>
> > Forgot to mention, this probably indicates that something is very wrong
> >> with your build, i.e. you are linking against a out-of-date debug
> version
> >> of the library by accident, or something else equally weird. I would
> >> definitely try to fix this issue before worrying about other issues.
> >
> >
> > It seems NDEBUG is not defined in libmesh_config.h for me, which is very
> > weird. I'm not sure if that's the place at which it should be defined,
> but
> > looking at include/utils/restore_warnings.h, since it's used right after
> > #include "libmesh_config.h" I'm guessing it should be there. I get
> > libmesh_opt as the library, but NDEBUG still isn't defined! Here is the
> > configure command I'm using:
> >
> > ./configure PETSC_DIR=$(PETSC_REAL_BUILD) MPIHOME=$(MPIHOME)
> > PETSC_ARCH=$(PETSC_REAL_ARCH) \
> > F77="$(MPIF77)" CC="$(MPICC)" GCC="$(GCC)" CXX="$(MPICXX)" --enable-vtk
> > --with-vtk-include=$(VTKINC_PATH) \
> > --with-vtk-lib=$(VTKLIB_PATH) --disable-tetgen --disable-nemesis
> > --with-methods="opt" --with-metis=PETSc --enable-default-comm-world \
> > --enable-amr --enable-shared=yes --disable-cxx11
> > --enable-legacy-include-paths --disable-strict-lgpl --disable-examples
> > --enable-slepc;
> >
> > I have attached the resultant config.log and libmesh_config.h. Any ideas
> > what could be causing this issue?
> >
> > Thanks,
> > Harshad
> >
> > On Tue, Mar 8, 2016 at 4:14 PM, Harshad Sahasrabudhe <
> hsaha...@purdue.edu>
> > wrote:
> >
> >> Hi John,
> >>>
> >>>
> >>> Forgot to mention, this probably indicates that something is very wrong
> >>> with your build, i.e. you are linking against a out-of-date debug
> version
> >>> of the library by accident, or something else equally weird. I would
> >>> definitely try to fix this issue before worrying about other issues.
> >>
> >>
> >> It definitely looks like I'm linking with a debug version too. I need to
> >> find out where that's happening. Thanks for your insights!
> >>
> >> Harshad
> >>
> >>>
> >>
> >
>
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
> _______________________________________________
> Libmesh-users mailing list
> Libmesh-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/libmesh-users
>
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
_______________________________________________
Libmesh-users mailing list
Libmesh-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libmesh-users
