there is certainly frustration with changes. And then there could be real issues. If similar major changes land in sept release [at the last minite] in any critical packages [that others packages don't quickly add it to their own sept release ] - that might break things in a way that xsdk release could not be delivered.
[and usually triggers hard to resolve discussion of who should address this failure - that changed package - or dependent packages. And even if patches are available - they might not get in - due to workflow isues and such]. Satish On Sun, 3 Apr 2022, Barry Smith wrote: > > We have not updated Sundials because they developed an entirely new code > with new APIs, it is essentially a new package with tons of new > functionality. Had they been incrementally changing things over the years we > would have actually kept up with it; so this is not a good example of how > small API changes keep us from upgrading, not at all. It is just an example > of how it takes a lot of work to wrap a new large package like Sundials 3 > from scratch and someone must make a big effort to do it. Note: I think > Sundials was right to do a rewrite, their classic design was preventing them > from making dramatic additions to the old code by doing a complete rewrite > they could accomplish so much more. > > MOAB has not been updated presumably because there are no or very > unaggressive users. > > Side note: I understand the frustration and grumbling that takes place when > one has to deal with change, especially when from a perspective as an > outer-sider to a project the change may seem unnecessary, that frustration > and grumbling is normal, I do it all the time. But it should not dictate > policy. > > > > > > > On Apr 3, 2022, at 12:58 PM, Satish Balay <ba...@mcs.anl.gov> wrote: > > > > On Sun, 3 Apr 2022, Barry Smith wrote: > > > >> > >> > >>> On Apr 3, 2022, at 12:24 PM, Satish Balay <ba...@mcs.anl.gov> wrote: > >>> > >>>> If we had this attitude with the external packages PETSc uses we would > >>>> have to stop using most of the packages/*.py. > >>> > >>> Sure one can take extreme view on both sides. [no change, vs won't > >>> hesitate to change] - having a manageable (minimal) change is harder to > >>> do. > >>> > >>> I would point out that most externalpackages don't change much and we > >>> benefit from it - hence we are able to support so may. Some packages had > >>> major changes - and we haven't upgraded to their new versions. > >> > >> What packages are these? We should have a tool that runs through all the > >> packages/xxx.py and determines the date of release of the version we are > >> using and if there are any newer versions available. We could run this > >> tool automatically a month before each PETSc release sending its output to > >> petsc-dev to see what we should be updating. > > > > sundials, moab [,trilinos - ml was only recently updated] - that I can > > think off right now. > > > > Satish > > > >> > >> Note also that some packages we don't update to, not because of API > >> changes but because the new releases are broken in some way, this is life > >> in the HPC world. > >> > >> > >> > >> > >>> > >>> [i.e with the current state one can use them only if they completely buy > >>> into petsc ecosystem i.e use old version - but not any larger one - as in > >>> use newer features from them] > >>> > >>> We did update to newer interfaces in some packages. > >>> > >>> But these problems remain - and have to be dealt with - and sometimes the > >>> complexity increases based on the dependency tree. > >>> > >>> [and also results in folk using and requiring help with older petsc > >>> versions] > >>> > >>> Satish > >>> > >>> > >>> On Sun, 3 Apr 2022, Barry Smith wrote: > >>> > >>>> > >>>> I would say it is not reasonable for the package developers in the xsdk > >>>> ecosystem to expect that they can just continue to use another HPC > >>>> package for multiple years without doing some minimal amount of work to > >>>> keep up with the other packages' new releases. If we had this attitude > >>>> with the external packages PETSc uses we would have to stop using most > >>>> of the packages/*.py. Yes, it is a constant race to keep up the versions > >>>> in packages/*.py and requires some effort but if you want to play in > >>>> this game that is a race you have to remain in. And it goes way beyond > >>>> HPC, to say you do software development but don't need to manage > >>>> constant change in everything is an oxymoron. There was never a golden > >>>> age of computing where things didn't change rapidly, pretending there > >>>> was or can be is not productive. Of course, we want to minimize public > >>>> change, but having a goal of no public change is not a realistic or even > >>>> desirable goal. > >>>> > >>>>> Just noticed - CHKERRQ() got removed from fortran interface - breaking > >>>>> pflotran > >>>> > >>>> This was just a oversight, easily fixed. > >>>> > >>>>> On Apr 3, 2022, at 11:13 AM, Satish Balay <ba...@mcs.anl.gov> wrote: > >>>>> > >>>>> > >>>>> Note this is not just 'users should update their code' issue. > >>>>> - all packages (that use petsc) would need to do this update > >>>>> - and this update doesn't always happen - so pakages will stay at old > >>>>> release - some might not > >>>>> - so now we cant build PETSc with both these packages together. > >>>>> > >>>>> this type of change causes major issues in xsdk ecosystem (depends on > >>>>> how many direct/indirect dependencies are on the given package) > >>>>> > >>>>> Just noticed - CHKERRQ() got removed from fortran interface - breaking > >>>>> pflotran > >>>>> > >>>>> https://gitlab.com/xsdk-project/spack-xsdk/-/jobs/2285145624 > >>>>> > >>>>> [also CHKERRABORT]. Perhaps they can be added back in. > >>>>> > >>>>> $ git diff release-3.16..release include/petsc/finclude/petscsys.h > >>>>> diff --git a/include/petsc/finclude/petscsys.h > >>>>> b/include/petsc/finclude/petscsys.h > >>>>> <snip> > >>>>> #define SETERRABORT(c,ierr,s) call PetscError(c,ierr,0,s); call > >>>>> MPI_Abort(c,ierr) > >>>>> -#define CHKERRQ(ierr) if (ierr .ne. 0) then;call > >>>>> PetscErrorF(ierr);return;endif > >>>>> +#define PetscCall(ierr) if (ierr .ne. 0) then;call > >>>>> PetscErrorF(ierr);return;endif > >>>>> #define CHKERRA(ierr) if (ierr .ne. 0) then;call PetscErrorF(ierr);call > >>>>> MPIU_Abort(PETSC_COMM_SELF,ierr);endif > >>>>> -#define CHKERRABORT(c,ierr) if (ierr .ne. 0) then;call > >>>>> PetscErrorF(ierr);call MPI_Abort(c,ierr);endif > >>>>> +#define PetscCallAbort(c,ierr) if (ierr .ne. 0) then;call > >>>>> PetscErrorF(ierr);call MPI_Abort(c,ierr);endif > >>>>> #define CHKMEMQ call chkmemfortran(__LINE__,__FILE__,ierr) > >>>>> > >>>>> Satish > >>>>> > >>>>> On Sun, 3 Apr 2022, Barry Smith wrote: > >>>>> > >>>>>> > >>>>>> To use the latest version of PETSc, each user needs to remove the > >>>>>> error checks on these calls. The resulting code will work with > >>>>>> previous versions of PETSc as well as the current version of PETSc. > >>>>>> PETSc has never promised complete backward compatibility in the sense > >>>>>> of promising that one can use new PETSc releases without any changes > >>>>>> to their code; the documentation has always stated new releases will > >>>>>> contain changes in the API. We began using depreciate a few years ago > >>>>>> to limit the number of changes that needed to be made immediately for > >>>>>> each release but depreciate is not suitable for all changes and so > >>>>>> users do need to make some changes for each new release. > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On Apr 3, 2022, at 7:23 AM, Lisandro Dalcin <dalc...@gmail.com> wrote: > >>>>>>> > >>>>>>> The recent PetscUse/TryMethod changes are backward incompatible. > >>>>>>> Third-party codes cannot compile without modification. Our users > >>>>>>> deserve better. > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Lisandro Dalcin > >>>>>>> ============ > >>>>>>> Senior Research Scientist > >>>>>>> Extreme Computing Research Center (ECRC) > >>>>>>> King Abdullah University of Science and Technology (KAUST) > >>>>>>> http://ecrc.kaust.edu.sa/ <http://ecrc.kaust.edu.sa/> > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > > >
