On Tue, Mar 29, 2022 at 5:25 PM Satish Balay via petsc-dev <
petsc-dev@mcs.anl.gov> wrote:
> I'm not sure why we have PetscSFCount - and not always use MPI_Count.
>
> Maybe this would work?
>
> Perhaps Junchao can clarify
>
I used MPIU_Ineighbor_alltoallv() to wrap MPI_Ineighbor_alltoallv() or
MPI_Neighbor_alltoallv_c(), depending on whether PETSc uses 64-bit indices
and the MPI supports large count.
Such that the input count array arguments are of type int* or MPI_Count*. I
defined PetscSFCount for convenience so that I can use that to declare
variables working for both cases.
I have thought in 64-bit, I could pass PetscInt * (long long *) to
MPI_Count* (long *), but apparently from Fande's report, it is wrong.
Fande's fix is right. I will create an MR with that.
>
> Satish
>
> ---
>
> diff --git a/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c
> b/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c
> index 5dc2e8c0b2..10f42fc302 100644
> --- a/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c
> +++ b/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c
> @@ -1,12 +1,7 @@
> #include <../src/vec/is/sf/impls/basic/sfpack.h>
> #include <../src/vec/is/sf/impls/basic/sfbasic.h>
>
> -/* A convenience temporary type */
> -#if defined(PETSC_HAVE_MPI_LARGE_COUNT) &&
> defined(PETSC_USE_64BIT_INDICES)
> - typedef PetscInt PetscSFCount;
> -#else
> - typedef PetscMPIInt PetscSFCount;
> -#endif
> +typedef MPI_Count PetscSFCount;
>
> typedef struct {
> SFBASICHEADER;
>
>
> On Tue, 29 Mar 2022, Fande Kong wrote:
>
> > OK, this works for me.
> >
> > (moose) kongf@FN428781 petsc1 % git diff
> >
> > *diff --git a/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c
> > b/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c*
> >
> > *index 5dc2e8c0b2..c2cc72dfa9 100644*
> >
> > *--- a/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c*
> >
> > *+++ b/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c*
> >
> > @@ -3,7 +3,7 @@
> >
> >
> >
> > /* A convenience temporary type */
> >
> > #if defined(PETSC_HAVE_MPI_LARGE_COUNT) &&
> defined(PETSC_USE_64BIT_INDICES)
> >
> > - typedef PetscInt PetscSFCount;
> >
> > + typedef MPI_Count PetscSFCount;
> >
> > #else
> >
> > typedef PetscMPIInt PetscSFCount;
> >
> > #endif
> >
> > On Tue, Mar 29, 2022 at 3:49 PM Fande Kong <fdkong...@gmail.com> wrote:
> >
> > > OK, I attached the configure log here so that we have move information.
> > >
> > > I feel like we should do
> > >
> > > typedef MPI_Count PetscSFCount
> > >
> > > Do we have the target of 64-bit-indices with C++ in CI? I was
> > > surprised that I am the only guy who saw this issue
> > >
> > > Thanks,
> > >
> > > Fande
> > >
> > > On Tue, Mar 29, 2022 at 2:50 PM Satish Balay <ba...@mcs.anl.gov>
> wrote:
> > >
> > >> What MPI is this? How to reproduce?
> > >>
> > >> Perhaps its best if you can send the relevant logs.
> > >>
> > >> The likely trigger code in sfneighbor.c:
> > >>
> > >> >>>>
> > >> /* A convenience temporary type */
> > >> #if defined(PETSC_HAVE_MPI_LARGE_COUNT) &&
> > >> defined(PETSC_USE_64BIT_INDICES)
> > >> typedef PetscInt PetscSFCount;
> > >> #else
> > >> typedef PetscMPIInt PetscSFCount;
> > >> #endif
> > >>
> > >> This change is at https://gitlab.com/petsc/petsc/-/commit/c87b50c4628
> > >>
> > >> Hm - if MPI supported LARGE_COUNT - perhaps it also provides a type
> that
> > >> should go with it which we could use - instead of PetscInt?
> > >>
> > >>
> > >> Perhaps it should be: "typedef log PetscSFCount;"
> > >>
> > >> Satish
> > >>
> > >>
> > >> On Tue, 29 Mar 2022, Fande Kong wrote:
> > >>
> > >> > It seems correct according to
> > >> >
> > >> > #define PETSC_SIZEOF_LONG 8
> > >> >
> > >> > #define PETSC_SIZEOF_LONG_LONG 8
> > >> >
> > >> >
> > >> > Can not convert from "non-constant" to "constant"?
> > >> >
> > >> > Fande
> > >> >
> > >> > On Tue, Mar 29, 2022 at 2:22 PM Fande Kong <fdkong...@gmail.com>
> wrote:
> > >> >
> > >> > > Hi All,
> > >> > >
> > >> > > When building PETSc with 64 bit indices, it seems that
> PetscSFCount is
> > >> > > 64-bit integer while MPI_Count is still 32 bit.
> > >> > >
> > >> > > typedef long MPI_Count;
> > >> > >
> > >> > > typedef PetscInt PetscSFCount;
> > >> > >
> > >> > >
> > >> > > I had the following errors. Do I have a bad MPI?
> > >> > >
> > >> > > Thanks,
> > >> > >
> > >> > > Fande
> > >> > >
> > >> > >
> > >> > >
> > >>
> Users/kongf/projects/moose6/petsc1/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c:171:18:
> > >> > > error: no matching function for call to
> 'MPI_Ineighbor_alltoallv_c'
> > >> > >
> > >> > >
> > >>
> PetscCallMPI(MPIU_Ineighbor_alltoallv(rootbuf,dat->rootcounts,dat->rootdispls,unit,leafbuf,dat->leafcounts,dat->leafdispls,unit,distcomm,req));
> > >> > >
> > >> > >
> > >>
> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >> > >
> > >>
> /Users/kongf/projects/moose6/petsc1/include/petsc/private/mpiutils.h:97:79:
> > >> > > note: expanded from macro 'MPIU_Ineighbor_alltoallv'
> > >> > > #define MPIU_Ineighbor_alltoallv(a,b,c,d,e,f,g,h,i,j)
> > >> > > MPI_Ineighbor_alltoallv_c(a,b,c,d,e,f,g,h,i,j)
> > >> > >
> > >> > > ^~~~~~~~~~~~~~~~~~~~~~~~~
> > >> > > /Users/kongf/projects/moose6/petsc1/include/petscerror.h:407:32:
> note:
> > >> > > expanded from macro 'PetscCallMPI'
> > >> > > PetscMPIInt _7_errorcode = __VA_ARGS__;
> > >> > > \
> > >> > > ^~~~~~~~~~~
> > >> > > /Users/kongf/mambaforge3/envs/moose/include/mpi_proto.h:945:5:
> note:
> > >> > > candidate function not viable: no known conversion from
> 'PetscSFCount
> > >> *'
> > >> > > (aka 'long long *') to 'const MPI_Count *' (aka 'const long *')
> for
> > >> 2nd
> > >> > > argument
> > >> > > int MPI_Ineighbor_alltoallv_c(const void *sendbuf, const MPI_Count
> > >> > > sendcounts[],
> > >> > > ^
> > >> > >
> > >>
> /Users/kongf/projects/moose6/petsc1/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c:195:18:
> > >> > > error: no matching function for call to
> 'MPI_Ineighbor_alltoallv_c'
> > >> > >
> > >> > >
> > >>
> PetscCallMPI(MPIU_Ineighbor_alltoallv(leafbuf,dat->leafcounts,dat->leafdispls,unit,rootbuf,dat->rootcounts,dat->rootdispls,unit,distcomm,req));
> > >> > >
> > >> > >
> > >>
> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >> > >
> > >>
> /Users/kongf/projects/moose6/petsc1/include/petsc/private/mpiutils.h:97:79:
> > >> > > note: expanded from macro 'MPIU_Ineighbor_alltoallv'
> > >> > > #define MPIU_Ineighbor_alltoallv(a,b,c,d,e,f,g,h,i,j)
> > >> > > MPI_Ineighbor_alltoallv_c(a,b,c,d,e,f,g,h,i,j)
> > >> > >
> > >> > > ^~~~~~~~~~~~~~~~~~~~~~~~~
> > >> > > /Users/kongf/projects/moose6/petsc1/include/petscerror.h:407:32:
> note:
> > >> > > expanded from macro 'PetscCallMPI'
> > >> > > PetscMPIInt _7_errorcode = __VA_ARGS__;
> > >> > > \
> > >> > > ^~~~~~~~~~~
> > >> > > /Users/kongf/mambaforge3/envs/moose/include/mpi_proto.h:945:5:
> note:
> > >> > > candidate function not viable: no known conversion from
> 'PetscSFCount
> > >> *'
> > >> > > (aka 'long long *') to 'const MPI_Count *' (aka 'const long *')
> for
> > >> 2nd
> > >> > > argument
> > >> > > int MPI_Ineighbor_alltoallv_c(const void *sendbuf, const MPI_Count
> > >> > > sendcounts[],
> > >> > > ^
> > >> > >
> > >>
> /Users/kongf/projects/moose6/petsc1/src/vec/is/sf/impls/basic/neighbor/sfneighbor.c:240:18:
> > >> > > error: no matching function for call to 'MPI_Neighbor_alltoallv_c'
> > >> > >
> > >> > >
> > >>
> PetscCallMPI(MPIU_Neighbor_alltoallv(rootbuf,dat->rootcounts,dat->rootdispls,unit,leafbuf,dat->leafcounts,dat->leafdispls,unit,comm));
> > >> > >
> > >> > >
> > >>
> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >> > >
> > >>
> /Users/kongf/projects/moose6/petsc1/include/petsc/private/mpiutils.h:96:79:
> > >> > > note: expanded from macro 'MPIU_Neighbor_alltoallv'
> > >> > > #define MPIU_Neighbor_alltoallv(a,b,c,d,e,f,g,h,i)
> > >> > > MPI_Neighbor_alltoallv_c(a,b,c,d,e,f,g,h,i)
> > >> > >
> > >> > > ^~~~~~~~~~~~~~~~~~~~~~~~
> > >> > > /Users/kongf/projects/moose6/petsc1/include/petscerror.h:407:32:
> note:
> > >> > > expanded from macro 'PetscCallMPI'
> > >> > > PetscMPIInt _7_errorcode = __VA_ARGS__;
> > >> > > \
> > >> > > ^~~~~~~~~~~
> > >> > > /Users/kongf/mambaforge3/envs/moose/include/mpi_proto.h:1001:5:
> note:
> > >> > > candidate function not viable: no known conversion from
> 'PetscSFCount
> > >> *'
> > >> > > (aka 'long long *') to 'const MPI_Count *' (aka 'const long *')
> for
> > >> 2nd
> > >> > > argument
> > >> > > int MPI_Neighbor_alltoallv_c(const void *sendbuf, const MPI_Count
> > >> > > sendcounts[],
> > >> > > ^
> > >> > >
> > >> > >
> > >> > >
> > >> >
> > >>
> > >>
> >
>
>
