Re: [Autoconf] help configure project for HPC - non-standard ways to run the executable
On 2017-09-21 03:40, suzuki toshiya wrote: Dear Anton, I'm afraid there is no standard mechanism to control the execution of cross compiled binary. Yet I've not detected which autoconf macro defines it, but I can find many attempts trying to execute "./conftest" directly, in generated configure script - no variable to insert the intermediate executor. # my experience with HPC related field is very legacy, # but I think it was very popular situation that the # accessible UNIX frontend provides the cross compilers # and cannot execute the compiled binary, and the system # executing the compiled binary do not have Unix like # frontend. autotools supports the situation that the cross compiler makes a binary executable but the system cannot execute it directly. (e.g. ./configure --host=m68k-coff --build=i386-pc-linux-gnu) if you do not expect any results from the execution of the compiled binary, the nearest way might be using autotools as if you work for cross build. However, in my impression, current autotools are optimized for GNU-based cross development toolchain which has well-structured prefix for each tools (cc, as, ar, ld, etc). The proprietary toolchain for HPC systems might be something what the original designers of autotool expected. I agree that there is and IMO there will be no standard way to control the execution of cross compiled binaries. In our project, we therefore explicitly switch to autoconf's cross-compile mode when we are on a (HPC) system that is known to be a cross-compile system (Blue Gene, Fujitsu, Cray). I.e., we don't 'run' conftests during configure, we just compile and link. Also, we don't specify --host and --build for these systems but provide the native (MPI) compilers to CC, CXX, F77. FC. This is how to switch to cross-compile mode: # in configure.ac cross_compiling=yes In cross-compile mode the macros AC_CHECK_FILE AC_CHECK_FILES AC_RUN_IFELSE will die. For the latter one you can define an 'action-if-cross-compiling' though. What kind of the automation you want to realize by autotool? Regards, mpsuzuki On 9/21/2017 12:43 AM, Anton Shterenlikht wrote: Hi, I'm new to this list. I develop a Fortran 2008 coarrays project targeting HPC systems. I'd like to simplify my build process with autotools. My Fortran experience is limited. But I seem to remember that automatic dependency generation and Fortran modules don't play well together. This is an automake issue though. Christian The immediate problem is that the executable is run in different ways on different platforms. For example, OpenCoarrays compiler is "caf", but the executable is run with "cafrun -np .. ./a.out". This is similar to "mpif90" and "mpirun" for MPI programs. On Cray I compile with "ftn" and run with "aprun" (it's also cross-compiled, but I'm not there yet). So I get errors like: configure:2850: caf -o conftestconftest.f >&5 configure:2854: $? = 0 configure:2861: ./conftest [mpie...@20474701626e.anet.bris.ac.uk] match_arg (utils/args/args.c:159): unrecognized argument pmi_ args [mpie...@20474701626e.anet.bris.ac.uk] HYDU_parse_array (utils/args/args.c:174): argument matching r eturned error [mpie...@20474701626e.anet.bris.ac.uk] parse_args (ui/mpich/utils.c:1596): error parsing input array [mpie...@20474701626e.anet.bris.ac.uk] HYD_uii_mpx_get_parameters (ui/mpich/utils.c:1648): unable to parse user arguments [mpie...@20474701626e.anet.bris.ac.uk] main (ui/mpich/mpiexec.c:153): error parsing parameters because the invocation should have been "cafrun -np .. ./conftest" for this compiler. Is there a standard way of implementing this in configure.ac? I looked at ax_prog_fc_mpi: https://www.gnu.org/software/autoconf-archive/ax_prog_fc_mpi.html which looks similar but not exactly what I need. Thanks Anton ___ Autoconf mailing list Autoconf@gnu.org https://lists.gnu.org/mailman/listinfo/autoconf ___ Autoconf mailing list Autoconf@gnu.org https://lists.gnu.org/mailman/listinfo/autoconf smime.p7s Description: S/MIME Cryptographic Signature ___ Autoconf mailing list Autoconf@gnu.org https://lists.gnu.org/mailman/listinfo/autoconf
Re: [Autoconf] help configure project for HPC - non-standard ways to run the executable
Dear Anton, On 9/21/2017 4:51 PM, Anton Shterenlikht wrote: At present I have a collection of Makefiles, one for each platform (HPC system, compiler, libraries, etc.). This works ok, but sometimes the users are not sure which libraries are available or which they should use, i.e. they are not sure which Makefile to use. It's very typical cases that people try autoconf :-) I was wondering if I can automate this choice with autotools. Good! To check the availabilities of the libraries, autoconf is still useful. As far as the compiler use standard syntax (like -I, -L) to specify the directories for header & libraries, configure script does not need the execution of the compiled binary. It would be possible for configure script to check the availabilities of the libraries, and override sometimes (e.g. ignore the libraries even if it was found). You mention cross-compiling. However, my examples with mpif90 and mpirun, or caf and cafrun, are native, not cross-compiling. The executables are run on the same platform, it's just the invocation of the executable is not simply ./a.out, but "mpirun -np ./a.out", or "cafrun -np ./a.out". Ahh... sorry for my overlooking on the errors you pasted in your first post. I can see that the program starts but aborts immediately. Maybe "cafrun" prepares the appropriate environment and commandline options, even if the original source is very small bit like "hello world", they are needed. I think, Christian's idea to insert "cross_compiling=yes" to your skeleton could make the first issue passed. Anyway, it sounds that autotools will give me more headache than help. Maybe I made you misunderstood, I'm quite sorry. Regards, mpsuzuki ___ Autoconf mailing list Autoconf@gnu.org https://lists.gnu.org/mailman/listinfo/autoconf
Re: [Autoconf] help configure project for HPC - non-standard ways to run the executable
> From mpsuz...@hiroshima-u.ac.jp Thu Sep 21 01:39:51 2017 > > What kind of the automation you want to realize by autotool? At present I have a collection of Makefiles, one for each platform (HPC system, compiler, libraries, etc.). This works ok, but sometimes the users are not sure which libraries are available or which they should use, i.e. they are not sure which Makefile to use. I was wondering if I can automate this choice with autotools. You mention cross-compiling. However, my examples with mpif90 and mpirun, or caf and cafrun, are native, not cross-compiling. The executables are run on the same platform, it's just the invocation of the executable is not simply ./a.out, but "mpirun -np ./a.out", or "cafrun -np ./a.out". Anyway, it sounds that autotools will give me more headache than help. Thanks! Anton ___ Autoconf mailing list Autoconf@gnu.org https://lists.gnu.org/mailman/listinfo/autoconf
Re: [Autoconf] help configure project for HPC - non-standard ways to run the executable
Oops, sorry for my poor English. > The proprietary > toolchain for HPC systems might be something what the original > designers of autotool expected. I meant "might be something *different from* what the original designers of autotool expected". Regards, mpsuzuki suzuki toshiya wrote: > Dear Anton, > > I'm afraid there is no standard mechanism to control the > execution of cross compiled binary. Yet I've not detected > which autoconf macro defines it, but I can find many > attempts trying to execute "./conftest" directly, in > generated configure script - no variable to insert the > intermediate executor. > > # my experience with HPC related field is very legacy, > # but I think it was very popular situation that the > # accessible UNIX frontend provides the cross compilers > # and cannot execute the compiled binary, and the system > # executing the compiled binary do not have Unix like > # frontend. > > autotools supports the situation that the cross compiler makes > a binary executable but the system cannot execute it directly. > (e.g. ./configure --host=m68k-coff --build=i386-pc-linux-gnu) > > if you do not expect any results from the execution of the > compiled binary, the nearest way might be using autotools as > if you work for cross build. > > However, in my impression, current autotools are optimized for > GNU-based cross development toolchain which has well-structured > prefix for each tools (cc, as, ar, ld, etc). The proprietary > toolchain for HPC systems might be something what the original > designers of autotool expected. > > What kind of the automation you want to realize by autotool? > > Regards, > mpsuzuki > > On 9/21/2017 12:43 AM, Anton Shterenlikht wrote: >> Hi, I'm new to this list. >> >> I develop a Fortran 2008 coarrays project >> targeting HPC systems. >> I'd like to simplify my build process with autotools. >> >> The immediate problem is that the executable is >> run in different ways on different platforms. >> >> For example, OpenCoarrays compiler is "caf", but >> the executable is run with "cafrun -np .. ./a.out". >> This is similar to "mpif90" and "mpirun" for MPI >> programs. On Cray I compile with "ftn" and run with >> "aprun" (it's also cross-compiled, but I'm not there yet). >> >> So I get errors like: >> >> configure:2850: caf -o conftestconftest.f >&5 >> configure:2854: $? = 0 >> configure:2861: ./conftest >> [mpie...@20474701626e.anet.bris.ac.uk] match_arg (utils/args/args.c:159): >> unrecognized argument pmi_ >> args >> [mpie...@20474701626e.anet.bris.ac.uk] HYDU_parse_array >> (utils/args/args.c:174): argument matching r >> eturned error >> [mpie...@20474701626e.anet.bris.ac.uk] parse_args (ui/mpich/utils.c:1596): >> error parsing input array >> [mpie...@20474701626e.anet.bris.ac.uk] HYD_uii_mpx_get_parameters >> (ui/mpich/utils.c:1648): unable to >> parse user arguments >> [mpie...@20474701626e.anet.bris.ac.uk] main (ui/mpich/mpiexec.c:153): error >> parsing parameters >> >> because the invocation should have been >> "cafrun -np .. ./conftest" >> for this compiler. >> >> Is there a standard way of implementing this >> in configure.ac? >> >> I looked at ax_prog_fc_mpi: >> >> https://www.gnu.org/software/autoconf-archive/ax_prog_fc_mpi.html >> >> which looks similar but not exactly what I need. >> >> Thanks >> >> Anton >> >> ___ >> Autoconf mailing list >> Autoconf@gnu.org >> https://lists.gnu.org/mailman/listinfo/autoconf >> > > > ___ > Autoconf mailing list > Autoconf@gnu.org > https://lists.gnu.org/mailman/listinfo/autoconf > ___ Autoconf mailing list Autoconf@gnu.org https://lists.gnu.org/mailman/listinfo/autoconf
Re: [Autoconf] help configure project for HPC - non-standard ways to run the executable
Dear Anton, I'm afraid there is no standard mechanism to control the execution of cross compiled binary. Yet I've not detected which autoconf macro defines it, but I can find many attempts trying to execute "./conftest" directly, in generated configure script - no variable to insert the intermediate executor. # my experience with HPC related field is very legacy, # but I think it was very popular situation that the # accessible UNIX frontend provides the cross compilers # and cannot execute the compiled binary, and the system # executing the compiled binary do not have Unix like # frontend. autotools supports the situation that the cross compiler makes a binary executable but the system cannot execute it directly. (e.g. ./configure --host=m68k-coff --build=i386-pc-linux-gnu) if you do not expect any results from the execution of the compiled binary, the nearest way might be using autotools as if you work for cross build. However, in my impression, current autotools are optimized for GNU-based cross development toolchain which has well-structured prefix for each tools (cc, as, ar, ld, etc). The proprietary toolchain for HPC systems might be something what the original designers of autotool expected. What kind of the automation you want to realize by autotool? Regards, mpsuzuki On 9/21/2017 12:43 AM, Anton Shterenlikht wrote: > Hi, I'm new to this list. > > I develop a Fortran 2008 coarrays project > targeting HPC systems. > I'd like to simplify my build process with autotools. > > The immediate problem is that the executable is > run in different ways on different platforms. > > For example, OpenCoarrays compiler is "caf", but > the executable is run with "cafrun -np .. ./a.out". > This is similar to "mpif90" and "mpirun" for MPI > programs. On Cray I compile with "ftn" and run with > "aprun" (it's also cross-compiled, but I'm not there yet). > > So I get errors like: > > configure:2850: caf -o conftestconftest.f >&5 > configure:2854: $? = 0 > configure:2861: ./conftest > [mpie...@20474701626e.anet.bris.ac.uk] match_arg (utils/args/args.c:159): > unrecognized argument pmi_ > args > [mpie...@20474701626e.anet.bris.ac.uk] HYDU_parse_array > (utils/args/args.c:174): argument matching r > eturned error > [mpie...@20474701626e.anet.bris.ac.uk] parse_args (ui/mpich/utils.c:1596): > error parsing input array > [mpie...@20474701626e.anet.bris.ac.uk] HYD_uii_mpx_get_parameters > (ui/mpich/utils.c:1648): unable to > parse user arguments > [mpie...@20474701626e.anet.bris.ac.uk] main (ui/mpich/mpiexec.c:153): error > parsing parameters > > because the invocation should have been > "cafrun -np .. ./conftest" > for this compiler. > > Is there a standard way of implementing this > in configure.ac? > > I looked at ax_prog_fc_mpi: > > https://www.gnu.org/software/autoconf-archive/ax_prog_fc_mpi.html > > which looks similar but not exactly what I need. > > Thanks > > Anton > > ___ > Autoconf mailing list > Autoconf@gnu.org > https://lists.gnu.org/mailman/listinfo/autoconf > ___ Autoconf mailing list Autoconf@gnu.org https://lists.gnu.org/mailman/listinfo/autoconf
