Thanks Barry, Your branch works very well. Thanks for your help!!!
Could you merge it to upstream? Fande On Wed, Mar 10, 2021 at 6:30 PM Barry Smith <[email protected]> wrote: > > Fande, > > Before send the files I requested in my last email could you try with > the branch *barry/2021-03-10/handle-pie-flag-conda/release *and send its > configure.log if it fails. > > Thanks > > Barry > > > On Mar 10, 2021, at 5:59 PM, Fande Kong <[email protected]> wrote: > > Do not know what the fix should look like, but this works for me > > > @staticmethod > @@ -1194,7 +1194,6 @@ class Configure(config.base.Configure): > output.find('unrecognized command line option') >= 0 or > output.find('unrecognized option') >= 0 or output.find('unrecognised > option') >= 0 or > output.find('not recognized') >= 0 or output.find('not recognised') > >= 0 or > output.find('unknown option') >= 0 or output.find('unknown flag') >= > 0 or output.find('Unknown switch') >= 0 or > - output.find('ignoring option') >= 0 or output.find('ignored') >= 0 or > output.find('argument unused') >= 0 or output.find('not supported') > >= 0 or > # When checking for the existence of 'attribute' > output.find('is unsupported and will be skipped') >= 0 or > > > > Thanks, > > Fande > > On Wed, Mar 10, 2021 at 4:21 PM Fande Kong <[email protected]> wrote: > >> >> >> On Wed, Mar 10, 2021 at 1:36 PM Satish Balay <[email protected]> wrote: >> >>> Can you use a different MPI for this conda install? >>> >> >> We control how to build MPI. If I take "-pie" options out of LDFLAGS, >> conda can not compile mpich. >> >> >> >> >>> >>> Alternative: >>> >>> ./configure CC=x86_64-apple-darwin13.4.0-clang COPTFLAGS="-march=core2 >>> -mtune=haswell" CPPFLAGS=-I/Users/kongf/miniconda3/envs/testpetsc/include >>> LDFLAGS="-Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs >>> -Wl,-commons,use_dylibs" >>> LIBS="-Wl,-rpath,/Users/kongf/miniconda3/envs/testpetsc/lib -lmpi -lpmpi" >>> >> >> MPI can not generate an executable because we took out "-pie". >> >> Thanks, >> >> Fande >> >> >>> >>> etc.. [don't know if you really need LDFLAGS options] >>> >>> Satish >>> >>> On Wed, 10 Mar 2021, Fande Kong wrote: >>> >>> > I guess it was encoded in mpicc >>> > >>> > petsc % mpicc -show >>> > x86_64-apple-darwin13.4.0-clang -march=core2 -mtune=haswell -Wl,-pie >>> > -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs >>> > -Wl,-rpath,/Users/kongf/miniconda3/envs/testpetsc/lib >>> > -L/Users/kongf/miniconda3/envs/testpetsc/lib -Wl,-commons,use_dylibs >>> > -I/Users/kongf/miniconda3/envs/testpetsc/include >>> > -L/Users/kongf/miniconda3/envs/testpetsc/lib -lmpi -lpmpi >>> > >>> > >>> > Thanks, >>> > >>> > Fande >>> > >>> > On Wed, Mar 10, 2021 at 12:51 PM Satish Balay <[email protected]> >>> wrote: >>> > >>> > > > LDFLAGS_LD=-pie -headerpad_max_install_names -dead_strip_dylibs >>> -rpath >>> > > /Users/kongf/miniconda3/envs/testpetsc/lib >>> > > -L/Users/kongf/miniconda3/envs/testpetsc/lib >>> > > >>> > > Does conda compiler pick up '-pie' from this env variable? If so - >>> perhaps >>> > > its easier to just modify it? >>> > > >>> > > Or is it encoded in mpicc wrapper? [mpicc -show] >>> > > >>> > > Satish >>> > > >>> > > On Wed, 10 Mar 2021, Fande Kong wrote: >>> > > >>> > > > Thanks Barry, >>> > > > >>> > > > Got the same result, but "-pie" was not filtered out somehow. >>> > > > >>> > > > I did changes like this: >>> > > > >>> > > > kongf@x86_64-apple-darwin13 petsc % git diff >>> > > > diff --git a/config/BuildSystem/config/framework.py >>> > > > b/config/BuildSystem/config/framework.py >>> > > > index beefe82956..c31fbeb95e 100644 >>> > > > --- a/config/BuildSystem/config/framework.py >>> > > > +++ b/config/BuildSystem/config/framework.py >>> > > > @@ -504,6 +504,8 @@ class Framework(config.base.Configure, >>> > > > script.LanguageProcessor): >>> > > > lines = [s for s in lines if s.find('Load a valid targeting >>> module or >>> > > > set CRAY_CPU_TARGET') < 0] >>> > > > # pgi dumps filename on stderr - but returns 0 errorcode' >>> > > > lines = [s for s in lines if lines != 'conftest.c:'] >>> > > > + # in case -pie is always being passed to linker >>> > > > + lines = [s for s in lines if s.find('-pie being ignored. It is >>> only >>> > > > used when linking a main executable') < 0] >>> > > > if lines: output = reduce(lambda s, t: s+t, lines, '\n') >>> > > > else: output = '' >>> > > > log.write("Linker stderr after filtering:\n"+output+":\n") >>> > > > >>> > > > The log was attached again. >>> > > > >>> > > > Thanks, >>> > > > >>> > > > Fande >>> > > > >>> > > > >>> > > > On Wed, Mar 10, 2021 at 12:05 PM Barry Smith <[email protected]> >>> wrote: >>> > > > >>> > > > > Fande, >>> > > > > >>> > > > > Please add in config/BuildSystem/config/framework.py line 528 >>> two >>> > > new >>> > > > > lines >>> > > > > >>> > > > > # pgi dumps filename on stderr - but returns 0 errorcode' >>> > > > > lines = [s for s in lines if lines != 'conftest.c:'] >>> > > > > # in case -pie is always being passed to linker >>> > > > > lines = [s for s in lines if s.find('-pie being ignored. >>> It is >>> > > only >>> > > > > used when linking a main executable') < 0] >>> > > > > >>> > > > > Barry >>> > > > > >>> > > > > You have (another of Conda's "take over the world my way" >>> approach) >>> > > > > >>> > > > > LDFLAGS_LD=-pie -headerpad_max_install_names >>> -dead_strip_dylibs >>> > > -rpath >>> > > > > /Users/kongf/miniconda3/envs/testpetsc/lib >>> > > > > -L/Users/kongf/miniconda3/envs/testpetsc/lib >>> > > > > >>> > > > > Executing: mpicc -o >>> > > > > >>> > > >>> /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest >>> > > > > -dynamiclib -single_module >>> > > > > >>> > > >>> /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.o >>> > > > > Possible ERROR while running linker: >>> > > > > stderr: >>> > > > > ld: warning: -pie being ignored. It is only used when linking a >>> main >>> > > > > executable >>> > > > > Rejecting C linker flag -dynamiclib -single_module >>> due to >>> > > > > >>> > > > > ld: warning: -pie being ignored. It is only used when linking a >>> main >>> > > > > executable >>> > > > > >>> > > > > This is the correct link command for the Mac but it is being >>> rejected >>> > > due >>> > > > > to the warning message. >>> > > > > >>> > > > > >>> > > > > On Mar 10, 2021, at 10:11 AM, Fande Kong <[email protected]> >>> wrote: >>> > > > > >>> > > > > Thanks, Barry, >>> > > > > >>> > > > > It seems PETSc works fine with manually built compilers. We are >>> pretty >>> > > > > much sure that the issue is related to conda. Conda might >>> introduce >>> > > extra >>> > > > > flags. >>> > > > > >>> > > > > We still need to make it work with conda because we deliver our >>> package >>> > > > > via conda for users. >>> > > > > >>> > > > > >>> > > > > I unset all flags from conda, and got slightly different results >>> this >>> > > > > time. The log was attached. Anyone could explain the >>> motivation that >>> > > we >>> > > > > try to build executable without a main function? >>> > > > > >>> > > > > Thanks, >>> > > > > >>> > > > > Fande >>> > > > > >>> > > > > Executing: mpicc -c -o >>> > > > > >>> > > >>> /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.o >>> > > > > >>> > > >>> -I/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers >>> > > > > -fPIC >>> > > > > >>> > > >>> /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.c >>> > > > > >>> > > > > Successful compile: >>> > > > > Source: >>> > > > > #include "confdefs.h" >>> > > > > #include "conffix.h" >>> > > > > #include <stdio.h> >>> > > > > int (*fprintf_ptr)(FILE*,const char*,...) = fprintf; >>> > > > > void foo(void){ >>> > > > > fprintf_ptr(stdout,"hello"); >>> > > > > return; >>> > > > > } >>> > > > > void bar(void){foo();} >>> > > > > Running Executable WITHOUT threads to time it out >>> > > > > Executing: mpicc -o >>> > > > > >>> > > >>> /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/libconftest.so >>> > > > > -dynamic -fPIC >>> > > > > >>> > > >>> /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.o >>> > > > > >>> > > > > Possible ERROR while running linker: exit code 1 >>> > > > > stderr: >>> > > > > Undefined symbols for architecture x86_64: >>> > > > > "_main", referenced from: >>> > > > > implicit entry/start for main executable >>> > > > > ld: symbol(s) not found for architecture x86_64 >>> > > > > clang-11: error: linker command failed with exit code 1 (use -v >>> to see >>> > > > > invocation) >>> > > > > Rejected C compiler flag -fPIC because it was not >>> compatible >>> > > > > with shared linker mpicc using flags ['-dynamic'] >>> > > > > >>> > > > > >>> > > > > On Mon, Mar 8, 2021 at 7:28 PM Barry Smith <[email protected]> >>> wrote: >>> > > > > >>> > > > >> >>> > > > >> Fande, >>> > > > >> >>> > > > >> I see you are using CONDA, this can cause issues since it >>> sticks >>> > > all >>> > > > >> kinds of things into the environment. PETSc tries to remove >>> some of >>> > > them >>> > > > >> but perhaps not enough. If you run printenv you will see all >>> the mess >>> > > it is >>> > > > >> dumping in. >>> > > > >> >>> > > > >> Can you trying the same build without CONDA environment? >>> > > > >> >>> > > > >> Barry >>> > > > >> >>> > > > >> >>> > > > >> On Mar 8, 2021, at 7:31 PM, Matthew Knepley <[email protected]> >>> > > wrote: >>> > > > >> >>> > > > >> On Mon, Mar 8, 2021 at 8:23 PM Fande Kong <[email protected]> >>> > > wrote: >>> > > > >> >>> > > > >>> Thanks Matthew, >>> > > > >>> >>> > > > >>> Hmm, we still have the same issue after shutting off all >>> unknown >>> > > flags. >>> > > > >>> >>> > > > >> >>> > > > >> Oh, I was misinterpreting the error message: >>> > > > >> >>> > > > >> ld: can't link with a main executable file >>> > > > >> >>> > > >>> '/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/libconftest.dylib' >>> > > > >> >>> > > > >> So clang did not _actually_ make a shared library, it made an >>> > > executable. >>> > > > >> Did clang-11 change the options it uses to build a shared >>> library? >>> > > > >> >>> > > > >> Satish, do we test with clang-11? >>> > > > >> >>> > > > >> Thanks, >>> > > > >> >>> > > > >> Matt >>> > > > >> >>> > > > >> Thanks, >>> > > > >>> >>> > > > >>> Fande >>> > > > >>> >>> > > > >>> On Mon, Mar 8, 2021 at 6:07 PM Matthew Knepley < >>> [email protected]> >>> > > > >>> wrote: >>> > > > >>> >>> > > > >>>> On Mon, Mar 8, 2021 at 7:55 PM Fande Kong < >>> [email protected]> >>> > > wrote: >>> > > > >>>> >>> > > > >>>>> Hi All, >>> > > > >>>>> >>> > > > >>>>> mpicc rejected "-fPIC". Anyone has a clue how to work around >>> this >>> > > > >>>>> issue? >>> > > > >>>>> >>> > > > >>>> >>> > > > >>>> The failure is at the last step >>> > > > >>>> >>> > > > >>>> Executing: mpicc -o >>> > > > >>>> >>> > > >>> /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/conftest >>> > > > >>>> -fPIC >>> > > > >>>> >>> > > >>> /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/conftest.o >>> > > > >>>> >>> > > >>> -L/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers >>> > > > >>>> -lconftest >>> > > > >>>> Possible ERROR while running linker: exit code 1 >>> > > > >>>> stderr: >>> > > > >>>> ld: can't link with a main executable file >>> > > > >>>> >>> > > >>> '/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/libconftest.dylib' >>> > > > >>>> for architecture x86_64 >>> > > > >>>> clang-11: error: linker command failed with exit code 1 (use >>> -v to >>> > > see >>> > > > >>>> invocation) >>> > > > >>>> >>> > > > >>>> but you have some flags stuck in which may or may not affect >>> this. I >>> > > > >>>> would try shutting them off: >>> > > > >>>> >>> > > > >>>> LDFLAGS_LD=-pie -headerpad_max_install_names >>> -dead_strip_dylibs >>> > > -rpath >>> > > > >>>> /Users/kongf/miniconda3/envs/moose/lib >>> > > > >>>> -L/Users/kongf/miniconda3/envs/moose/lib >>> > > > >>>> >>> > > > >>>> I cannot tell exactly why clang is failing because it does not >>> > > report a >>> > > > >>>> specific error. >>> > > > >>>> >>> > > > >>>> Thanks, >>> > > > >>>> >>> > > > >>>> Matt >>> > > > >>>> >>> > > > >>>> The log was attached. >>> > > > >>>>> >>> > > > >>>>> Thanks so much, >>> > > > >>>>> >>> > > > >>>>> Fande >>> > > > >>>>> >>> > > > >>>> >>> > > > >>>> >>> > > > >>>> -- >>> > > > >>>> What most experimenters take for granted before they begin >>> their >>> > > > >>>> experiments is infinitely more interesting than any results to >>> > > which their >>> > > > >>>> experiments lead. >>> > > > >>>> -- Norbert Wiener >>> > > > >>>> >>> > > > >>>> https://www.cse.buffalo.edu/~knepley/ >>> > > > >>>> <http://www.cse.buffalo.edu/~knepley/> >>> > > > >>>> >>> > > > >>> >>> > > > >> >>> > > > >> -- >>> > > > >> What most experimenters take for granted before they begin their >>> > > > >> experiments is infinitely more interesting than any results to >>> which >>> > > their >>> > > > >> experiments lead. >>> > > > >> -- Norbert Wiener >>> > > > >> >>> > > > >> https://www.cse.buffalo.edu/~knepley/ >>> > > > >> <http://www.cse.buffalo.edu/~knepley/> >>> > > > >> >>> > > > >> >>> > > > >> <configure.log> >>> > > > > >>> > > > > >>> > > > > >>> > > > >>> > > >>> > > >>> > >>> >>> >
