On Tue, 16 Mar 2021, Barry Smith wrote: > > Jacob, > > You very well may be right, splitting may not be needed or relevant. I > only advocated it since for Mat it did fix a VM gfortran memory issue. How > closely the IBM compiler and gfortran issues are I do not know. > > My MR Mat split is read to to go into main. > > https://gitlab.com/petsc/petsc/-/merge_requests/3715 > <https://gitlab.com/petsc/petsc/-/merge_requests/3715> The MPI one is still > needing some babysitting, hopefully it can pass the CI and get into main soon. > > Both should be in main before starting to muck with changing the use stuff > for the IBM compiler. > > Satish, does your Vec file split pass the CI? If so I think we should also > get it into main before messing with the use for the IBM compiler.
Barry - if the switch from 'use' to 'import' is sufficient [for VM builds] - should we still split the sources? Right now it changes only the auto-fortran stubs, fixing custom stubs should give further improvements. If spliting sources is useful - I'll look into pushing the Vec split into this MR. BTW: Matt - can you test balay/reorg-f90-for-xlf branch against 'gfortran VM'? Satish > > > Who has access to the wonky IBM compiler? Is there any way to have general > access so it can be added to the CI and we can maintain code that works with > it? Or does someone need to fix things one at a time, first Vec, then Mat, > then .... > > > Barry > > > > On Mar 16, 2021, at 2:14 PM, Satish Balay via petsc-dev > > <[email protected]> wrote: > > > > On Tue, 16 Mar 2021, Matthew Knepley wrote: > > > >> On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch <[email protected]> > >> wrote: > >> > >>> To me the best solution is to first split the files when possible (it is > >>> often not or just too painful) and then adding imports if needed. > >>> > >>> > >>> I’m not sure I agree. Bear in mind my familiarity with fortran is very > >>> limited, but to me the “use” statement is similar to #include <header> in > >>> C > >>> or using namespace xxx in cpp. > >>> > >>> Our fortran interface is not optimal code atm, it’s as if we put a > >>> "#include <petscvec.h>” before every Vec function in the C source > >>> (assuming include guards are not used). Sure you can fix this by splitting > >>> the vec src files in two, but at the end of the day that’s a bandaid > >>> solution. I think I can get the import stuff to work reasonably well (a > >>> lot > >>> of the infrastructure to capture types is already present in the current > >>> generatefortranstubs). > >>> > >> > >> If you can get this to work, we should definitely do it. Was there a > >> question about getting it to work? > > > > Likely these additional changes should be done starting with Barry's > > split-up of sources. > > > > Satish > > > >> > >> Thanks, > >> > >> Matt > >> > >> > >>> Best regards, > >>> > >>> Jacob Faibussowitsch > >>> (Jacob Fai - booss - oh - vitch) > >>> Cell: (312) 694-3391 > >>> > >>> On Mar 16, 2021, at 12:56, Barry Smith <[email protected]> wrote: > >>> > >>> > >>> Jacob, > >>> > >>> I split the mat definitions in the MR > >>> https://gitlab.com/petsc/petsc/-/merge_requests/3696 and this reduced > >>> memory requirements enough to get builds through on some VM that failed > >>> previously ran out of memory (with gfortran). > >>> > >>> The petsc*mod.F90 files are all handwritten so it is ok to manually > >>> change them with imports if that helps the IBM compiler. To me the best > >>> solution is to first split the files when possible (it is often not or > >>> just > >>> too painful) and then adding imports if needed. > >>> > >>> Note also the MR > >>> https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs which may help > >>> a great deal with the IBM compiler or not. > >>> > >>> Thanks > >>> > >>> Barry > >>> > >>> > >>> > >>> On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch <[email protected]> > >>> wrote: > >>> > >>> Ok something I have gotten to work, but only doing things by hand in > >>> petscvecmod.F90: > >>> > >>> diff --git a/src/vec/f90-mod/petscvecmod.F90 > >>> b/src/vec/f90-mod/petscvecmod.F90 > >>> index 0c447156b9..ef3e2e2e55 100644 > >>> --- a/src/vec/f90-mod/petscvecmod.F90 > >>> +++ b/src/vec/f90-mod/petscvecmod.F90 > >>> module petscisdef > >>> use petscisdefdummy > >>> interface operator(.ne.) > >>> function isnotequal(A,B) > >>> - use petscisdefdummy > >>> + import tIS > >>> logical isnotequal > >>> > >>> type(tIS), intent(in) :: A,B > >>> end function isnotequal > >>> > >>> This works for everything wrapped in a module which contains an interface > >>> block. For dangling functions the following works: > >>> > >>> function isnotequal(A,B) > >>> - use petscisdefdummy > >>> + use petscisdef, only: tIs > >>> logical isnotequal > >>> type(tIS), intent(in) :: A,B > >>> isnotequal = (A%v .ne. B%v) > >>> end function isnotequal > >>> > >>> Do our fortran stub generators have any notion of types? Specifically > >>> types that originate from petsc? > >>> > >>> Best regards, > >>> > >>> Jacob Faibussowitsch > >>> (Jacob Fai - booss - oh - vitch) > >>> Cell: (312) 694-3391 > >>> > >>> On Mar 16, 2021, at 11:11, Satish Balay <[email protected]> wrote: > >>> > >>> On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote: > >>> > >>> My [partial] change is in branch balay/reorg-f90-for-xlf > >>> > >>> > >>> Satish is this branch pushed? I’d like to send it to the ibm folks to see > >>> if it works for them as well. > >>> > >>> > >>> Sorry - pushed now. From what I remember - it didn't work. > >>> > >>> Satish > >>> > >>> > >>> They also added this extra follow up: > >>> > >>> The change we did in the source files is to replace all the "use pet*" > >>> statements in all the Interface blocks with IMPORT statement. > >>> > >>> The nature of this workaround is to reduce the number of symbols that the > >>> compiler have to create, which exceeded the limitation and caused ICE. > >>> > >>> Every USE statement in an interface block opens up the module symbol file > >>> and reads all the symbols from it and creates an entity for each symbol in > >>> compiler. This is unnecessary when the module already has the same USE > >>> statement in the module scope. Instead, users can use IMPORT statement to > >>> make the module symbols accessible inside interface face blocks. > >>> > >>> With the change, the compiler would only read the module symbol file once > >>> and create one set of symbols where the old code reads the module symbol > >>> files as many times as the number of the USE statements in Interface > >>> blocks > >>> and create that many sets of duplicate symbols. Replacing those USE > >>> statements with IMPORT statements also shortens the compile time > >>> significantly. > >>> > >>> Best regards, > >>> > >>> Jacob Faibussowitsch > >>> (Jacob Fai - booss - oh - vitch) > >>> Cell: (312) 694-3391 > >>> > >>> On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev < > >>> [email protected]> wrote: > >>> > >>> The only change I can get working (i.e avoid compile errors) is to split > >>> petscvecmod.F90 into 2 source files - but I don't know if this will help > >>> with xlf.. > >>> > >>> My [partial] change is in branch balay/reorg-f90-for-xlf > >>> > >>> Satish > >>> > >>> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote: > >>> > >>> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote: > >>> > >>> Sure - once any change works locally [for gcc and xlf] > >>> > >>> When I try - I get a bunch of errors.. [yet to digest them.] > >>> > >>> > >>> Can you please give the following source code workaround a try? > >>> Since there is already "use petscvecdefdummy" at the module scope, one > >>> workaround might be to remove the unnecessary "use petscvecdefdummy" in > >>> vecnotequal and vecequals > >>> and all similar procedures. > >>> > >>> For example, the test case has: > >>> module petscvecdef > >>> use petscvecdefdummy > >>> ... > >>> function vecnotequal(A,B) > >>> use petscvecdefdummy > >>> logical vecnotequal > >>> type(tVec), intent(in) :: A,B > >>> vecnotequal = (A%v .ne. B%v) > >>> end function > >>> > >>> > >>> > >>> Ok - try this suggestion: > >>> > >>> diff --git a/src/vec/f90-mod/petscvecmod.F90 > >>> b/src/vec/f90-mod/petscvecmod.F90 > >>> index 0c447156b9..81968c7ca1 100644 > >>> --- a/src/vec/f90-mod/petscvecmod.F90 > >>> +++ b/src/vec/f90-mod/petscvecmod.F90 > >>> @@ -77,7 +77,6 @@ > >>> use petscvecdefdummy > >>> interface operator(.ne.) > >>> function vecnotequal(A,B) > >>> - use petscvecdefdummy > >>> logical vecnotequal > >>> type(tVec), intent(in) :: A,B > >>> end function > >>> > >>> > >>> > >>> FC arch-linux-c-debug/obj/vec/f90-mod/petscvecmod.o > >>> /home/balay/petsc/src/vec/f90-mod/petscvecmod.F90:81:22: > >>> > >>> 81 | type(tVec), intent(in) :: A,B > >>> | 1 > >>> Error: Derived type ‘tvec’ at (1) is being used before it is defined > >>> > >>> /home/balay/petsc/include/../src/vec/f90-mod/ftn-auto-interfaces/petscis.h90:2:10: > >>> > >>> 2 | use petscvecdef > >>> | 1 > >>> Fatal Error: Cannot open module file ‘petscvecdef.mod’ for reading at (1): > >>> No such file or directory > >>> <<<<<< > >>> > >>> Satish > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >> > >> > >
