Re: [petsc-users] Why PetscDestroy global collective semantics?
Junchao Zhang writes: > On Fri, Oct 22, 2021 at 9:13 PM Barry Smith wrote: > >> >> One technical reason is that PetscHeaderDestroy_Private() may call >> PetscCommDestroy() which may call MPI_Comm_free() which is defined by the >> standard to be collective. Though PETSc tries to limit its use of new MPI >> communicators (for example generally many objects shared the same >> communicator) if we did not free those we no longer need when destroying >> objects we could run out. >> > PetscCommDestroy() might call MPI_Comm_free() , but it is very unlikely. > Petsc uses reference counting on communicators, so in PetscCommDestroy(), > it likely just decreases the count. In other words, PetscCommDestroy() is > cheap and in effect not collective. Unless it's the last reference to a given communicator, which is a risky/difficult thing for a user to guarantee and the consequences are potentially dire (deadlock being way worse than a crash) when the user's intent is to relax ordering for destruction. Alberto, what is the use case in which deterministic destruction is problematic? If you relax it for individual objects, is there a place you can be collective to collect any stale communicators?
Re: [petsc-users] Why PetscDestroy global collective semantics?
On Fri, Oct 22, 2021 at 9:13 PM Barry Smith wrote: > > One technical reason is that PetscHeaderDestroy_Private() may call > PetscCommDestroy() which may call MPI_Comm_free() which is defined by the > standard to be collective. Though PETSc tries to limit its use of new MPI > communicators (for example generally many objects shared the same > communicator) if we did not free those we no longer need when destroying > objects we could run out. > PetscCommDestroy() might call MPI_Comm_free() , but it is very unlikely. Petsc uses reference counting on communicators, so in PetscCommDestroy(), it likely just decreases the count. In other words, PetscCommDestroy() is cheap and in effect not collective. > > I cannot off-hand think of another specific technical reason they must > be collective besides this good housekeeping. > > In what use case can you not call them collectively? > > Barry > > > > On Oct 22, 2021, at 9:21 PM, Alberto F. Martín > wrote: > > > > Dear PETSc users, > > > > What is the main reason underlying PetscDestroy subroutines having > global collective semantics? Is this actually true for all PETSc objects? > Can this be relaxed/deactivated by, e.g., compilation macros/configuration > options? > > > > Thanks in advance! > > > > Best regards, > > > > Alberto. > > > >
Re: [petsc-users] Why PetscDestroy global collective semantics?
One technical reason is that PetscHeaderDestroy_Private() may call PetscCommDestroy() which may call MPI_Comm_free() which is defined by the standard to be collective. Though PETSc tries to limit its use of new MPI communicators (for example generally many objects shared the same communicator) if we did not free those we no longer need when destroying objects we could run out. I cannot off-hand think of another specific technical reason they must be collective besides this good housekeeping. In what use case can you not call them collectively? Barry > On Oct 22, 2021, at 9:21 PM, Alberto F. Martín wrote: > > Dear PETSc users, > > What is the main reason underlying PetscDestroy subroutines having global > collective semantics? Is this actually true for all PETSc objects? Can this > be relaxed/deactivated by, e.g., compilation macros/configuration options? > > Thanks in advance! > > Best regards, > > Alberto. >
[petsc-users] Why PetscDestroy global collective semantics?
Dear PETSc users, What is the main reason underlying PetscDestroy subroutines having global collective semantics? Is this actually true for all PETSc objects? Can this be relaxed/deactivated by, e.g., compilation macros/configuration options? Thanks in advance! Best regards, Alberto.
