Re: [Patch] Edit contrib/ files to download gfortran prerequisites

[Jerry DeLisle]

On 10/21/2017 11:17 AM, Bernhard Reutner-Fischer wrote:
> On 21 October 2017 at 02:26, Damian Rouson  
> wrote:
>>
>> Hi Richard,
>>
>> Attached is a revised patch that makes the downloading of Fortran 
>> prerequisites optional via a new --no-fortran flag that can be passed to 
>> contrib/download_prerequisites as requested in your reply below.
>>
>> As Jerry mentioned in his response, he has been working on edits to the 
>> top-level build machinery, but we need additional guidance to complete his 
>> work.  Given that there were no responses to his request for guidance and 
>> it’s not clear when that work will complete, I’m hoping this minor change 
>> can be approved independently so that this patch doesn’t suffer bit rot in 
>> the interim.
>>
>> Ok for trunk?
> 
> + die "Invoking wget and curl in the 'download_prerequisites' script failed."
> 
> I suggest "Neither wget nor curl found, cannot download tarballs."
> 
> As an open-mpi user i question why this hardcodes MPICH, fwiw.
> 
> thanks,
> 

Hi Bernhard,

Well, open-mpi does not yet support failed images so we are focusing on mpich.

Maybe what we should do is default to mpich if not otherwise specified and
provide a -mpi=mpich or -mpi=open-mpi, or similar. Then later we can add any
other implementations that show up.

Cheers,

Jerry

Re: [Patch] Edit contrib/ files to download gfortran prerequisites

[Richard Biener]

On Tue, Oct 24, 2017 at 12:02 AM, Damian Rouson
 wrote:
>
>
> On October 23, 2017 at 3:54:22 AM, Richard Biener
> (richard.guent...@gmail.com) wrote:
>
> On Sat, Oct 21, 2017 at 2:26 AM, Damian Rouson
>  wrote:
>>
>> Hi Richard,
>>
>> Attached is a revised patch that makes the downloading of Fortran
>> prerequisites optional via a new --no-fortran flag that can be passed to
>> contrib/download_prerequisites as requested in your reply below.
>>
>> As Jerry mentioned in his response, he has been working on edits to the
>> top-level build machinery, but we need additional guidance to complete his
>> work. Given that there were no responses to his request for guidance and
>> it’s not clear when that work will complete, I’m hoping this minor change
>> can be approved independently so that this patch doesn’t suffer bit rot in
>> the interim.
>
> But the change doesn't make sense without the build actually picking up
> things.
>
> Each prerequisite tar ball contains build scripts so the change is useful
> even without the ultimate integration into the GCC build system.  Our first
> step was to get the tar balls onto the GCC ftp server.  Our next step was to
> set up for the files to be downloaded automatically with this patch.  For
> now, users can use the build scripts in the prerequisite tar balls.  Our
> final step will be to finish the integration into the GCC build system.
> Jerry has requested guidance on this mailing list, but I don’t think there
> have been any replies to his request.  In the interim, I’m stuck updating
> this patch indefinitely.  I’ve already updated it several times because the
> files it patches are changing.

What I am completely missing is documentation on the Coarrays feature.
doc/install.texi doesn't list any install requirements like for example
where GCC will look for and for what libraries (and headers?) and what
the constraints on the libraries are (like mpich vs. openmpi).

Simply downloading a tarball from download_prerequesites doesn't
sound useful.  People can simply follow URLs listed in install.texi
to get at those.

> Most importantly, the features these prerequisites enable are part of the
> Fortran standard so they are necessary for full access to the language —
> namely access to any Fortran 2008 or Fortran 2015 parallel features.  I hope
> I’ve made a case for the benefit of the patch. Is there a way in which the
> patch is harmful?

It will unconditionally download stuff and the user doesn't know he needs
to do sth "manual" to enable -fcoarrays=lib(?)?  From what I gather
from the scattered information on the wiki there's nothing one needs to
strictly do at GCC _build_ time.  Just when one wants to use
-fcoarrays=lib you need to manually add -lXYZ (why's that not done
with GCCs driver specs processing?) and have that library available
somewhere in the search path.

It looks then this is a similar level of integration as -mveclibabi={acml,svml}
where we don't download or build/install a vector math library either.

So - can you - as a first step - add something about Fortran Coarray
requirements to doc/install.texi?  Just put it in an item after 'isl Library'.
I presume the MPI library is a requirement of OpenCoarrays and  not
of GFortran?  If so people should refer to OpenCoarrays documentation
to figure out that requirement (and decide themselves what MPI library
to use?)

Thanks,
Richard.

>
> Damian

Re: [Patch] Edit contrib/ files to download gfortran prerequisites

[Damian Rouson]

 


On October 21, 2017 at 11:17:49 AM, Bernhard Reutner-Fischer 
(rep.dot@gmail.com(mailto:rep.dot@gmail.com)) wrote:

>  
> + die "Invoking wget and curl in the 'download_prerequisites' script failed."
>  
> I suggest "Neither wget nor curl found, cannot download tarballs.”

Thanks for the suggestion.  I have now replaced the previous message with your 
suggested one.

>  
> As an open-mpi user i question why this hardcodes MPICH, fwiw.

The primary reason is because Fortran 2015 has features that support the 
writing 
of fault-tolerant parallel algorithms. GCC 7 supports these features using 
OpenCoarrays,
which in turn uses features that are in MPICH 3.2.  I don’t think there is 
anything
comparable in an OpenMPI release yet, although I’m aware that OpenMPI 
developers are
working on it.  

Nonetheless, I would be glad to add a “—-with-mpi” flag that would enable users 
to specify
the MPI of their choice. We could also allow for non-MPI options.  For example, 
OpenSHEM
has been tested as an alternative recently.  Either way, I’m thinking I’ll hold 
off on 
investing any more time in this patch for now.  It seems unlikely to be 
approved for the
trunk until other work completes and it’s uncertain when the other work will 
complete.

Damian

Re: [Patch] Edit contrib/ files to download gfortran prerequisites

[Damian Rouson]

On October 23, 2017 at 3:54:22 AM, Richard Biener (richard.guent...@gmail.com) 
wrote:

On Sat, Oct 21, 2017 at 2:26 AM, Damian Rouson 
 wrote: 
> 
> Hi Richard, 
> 
> Attached is a revised patch that makes the downloading of Fortran 
> prerequisites optional via a new --no-fortran flag that can be passed to 
> contrib/download_prerequisites as requested in your reply below. 
> 
> As Jerry mentioned in his response, he has been working on edits to the 
> top-level build machinery, but we need additional guidance to complete his 
> work. Given that there were no responses to his request for guidance and it’s 
> not clear when that work will complete, I’m hoping this minor change can be 
> approved independently so that this patch doesn’t suffer bit rot in the 
> interim. 

But the change doesn't make sense without the build actually picking up things. 
Each prerequisite tar ball contains build scripts so the change is useful even 
without the ultimate integration into the GCC build system.  Our first step was 
to get the tar balls onto the GCC ftp server.  Our next step was to set up for 
the files to be downloaded automatically with this patch.  For now, users can 
use the build scripts in the prerequisite tar balls.  Our final step will be to 
finish the integration into the GCC build system.   Jerry has requested 
guidance on this mailing list, but I don’t think there have been any replies to 
his request.  In the interim, I’m stuck updating this patch indefinitely.  I’ve 
already updated it several times because the files it patches are changing.  

Most importantly, the features these prerequisites enable are part of the 
Fortran standard so they are necessary for full access to the language — namely 
access to any Fortran 2008 or Fortran 2015 parallel features.  I hope I’ve made 
a case for the benefit of the patch. Is there a way in which the patch is 
harmful?

Damian

Re: [Patch] Edit contrib/ files to download gfortran prerequisites

[Richard Biener]

On Sat, Oct 21, 2017 at 2:26 AM, Damian Rouson
 wrote:
>
> Hi Richard,
>
> Attached is a revised patch that makes the downloading of Fortran 
> prerequisites optional via a new --no-fortran flag that can be passed to 
> contrib/download_prerequisites as requested in your reply below.
>
> As Jerry mentioned in his response, he has been working on edits to the 
> top-level build machinery, but we need additional guidance to complete his 
> work.  Given that there were no responses to his request for guidance and 
> it’s not clear when that work will complete, I’m hoping this minor change can 
> be approved independently so that this patch doesn’t suffer bit rot in the 
> interim.

But the change doesn't make sense without the build actually picking up things.

> Ok for trunk?

No.

Thanks,
Richard.

> Damian
>
>
>
>
> On September 21, 2017 at 12:40:49 AM, Richard Biener 
> (richard.guent...@gmail.com(mailto:richard.guent...@gmail.com)) wrote:
>
>> On Wed, Sep 20, 2017 at 10:35 PM, Damian Rouson
>> wrote:
>> > Attached is a patch that adds the downloading of gfortran prerequisites 
>> > OpenCoarrays and MPICH in the contrib/download_prerequisites script. The 
>> > patch also provides a useful error message when neither wget or curl are 
>> > available on the target platform. I tested this patch with several choices 
>> > for the command-line options on macOS (including --md5 and --sha512) and 
>> > Ubuntu Linux (including --sha512). A suggested ChangeLog entry is
>> >
>> > * contrib/download_prerequisites: Download OpenCoarrays and MPICH.
>> > * contrib/prerequisites.sha5: Add sha512 message digests for OpenCoarrays 
>> > and MPICH.
>> > * contrib/prerequisites.md5: Add md5 message digests for OpenCoarrays and 
>> > MPICH.
>> >
>> >
>> > OK for trunk? If so, I’ll ask Jerry to commit this. I don’t have commit 
>> > rights.
>>
>> Can you make this optional similar to graphite/isl? Also I see no support in
>> the toplevel build machinery to build/install the libs as part of GCC
>> so how does
>> that work in the end?
>>
>> Thanks,
>> Richard.
>>
>> > Damian

Re: [Patch] Edit contrib/ files to download gfortran prerequisites

[Bernhard Reutner-Fischer]

On 21 October 2017 at 02:26, Damian Rouson  wrote:
>
> Hi Richard,
>
> Attached is a revised patch that makes the downloading of Fortran 
> prerequisites optional via a new --no-fortran flag that can be passed to 
> contrib/download_prerequisites as requested in your reply below.
>
> As Jerry mentioned in his response, he has been working on edits to the 
> top-level build machinery, but we need additional guidance to complete his 
> work.  Given that there were no responses to his request for guidance and 
> it’s not clear when that work will complete, I’m hoping this minor change can 
> be approved independently so that this patch doesn’t suffer bit rot in the 
> interim.
>
> Ok for trunk?

+ die "Invoking wget and curl in the 'download_prerequisites' script failed."

I suggest "Neither wget nor curl found, cannot download tarballs."

As an open-mpi user i question why this hardcodes MPICH, fwiw.

thanks,

Re: [Patch] Edit contrib/ files to download gfortran prerequisites

[Damian Rouson]

 
Hi Richard,

Attached is a revised patch that makes the downloading of Fortran prerequisites 
optional via a new --no-fortran flag that can be passed to 
contrib/download_prerequisites as requested in your reply below. 

As Jerry mentioned in his response, he has been working on edits to the 
top-level build machinery, but we need additional guidance to complete his 
work.  Given that there were no responses to his request for guidance and it’s 
not clear when that work will complete, I’m hoping this minor change can be 
approved independently so that this patch doesn’t suffer bit rot in the interim.

Ok for trunk?

Damian




On September 21, 2017 at 12:40:49 AM, Richard Biener 
(richard.guent...@gmail.com(mailto:richard.guent...@gmail.com)) wrote:

> On Wed, Sep 20, 2017 at 10:35 PM, Damian Rouson
> wrote:
> > Attached is a patch that adds the downloading of gfortran prerequisites 
> > OpenCoarrays and MPICH in the contrib/download_prerequisites script. The 
> > patch also provides a useful error message when neither wget or curl are 
> > available on the target platform. I tested this patch with several choices 
> > for the command-line options on macOS (including --md5 and --sha512) and 
> > Ubuntu Linux (including --sha512). A suggested ChangeLog entry is
> >
> > * contrib/download_prerequisites: Download OpenCoarrays and MPICH.
> > * contrib/prerequisites.sha5: Add sha512 message digests for OpenCoarrays 
> > and MPICH.
> > * contrib/prerequisites.md5: Add md5 message digests for OpenCoarrays and 
> > MPICH.
> >
> >
> > OK for trunk? If so, I’ll ask Jerry to commit this. I don’t have commit 
> > rights.
>  
> Can you make this optional similar to graphite/isl? Also I see no support in
> the toplevel build machinery to build/install the libs as part of GCC
> so how does
> that work in the end?
>  
> Thanks,
> Richard.
>  
> > Damian


downlaod-prereqs.diff
Description: Binary data

Re: [Patch] Edit contrib/ files to download gfortran prerequisites

[Jerry DeLisle]

On Thu, 21 Sep 2017 09:40:49Richard Biener wrote:
> On Wed, Sep 20, 2017 at 10:35 PM, Damian Rouson
>  wrote:
> >> Attached is a patch that adds the downloading of gfortran prerequisites
> OpenCoarrays and MPICH in the contrib/download_prerequisites script.  The
> patch also provides a useful error message when neither wget or curl are
> available on the target platform.  I tested this patch with several choices
> for the command-line options on macOS (including --md5 and --sha512) and
> Ubuntu Linux (including --sha512).A suggested ChangeLog entry is
>>
>> * contrib/download_prerequisites: Download OpenCoarrays and MPICH.
>> * contrib/prerequisites.sha5: Add sha512 message digests for
> OpenCoarrays and MPICH.
>> * contrib/prerequisites.md5: Add md5 message digests for OpenCoarrays
> and MPICH.
>
>
> OK for trunk?  If so, I’ll ask Jerry to commit this.  I don’t have commit
> rights.

> Can you make this optional similar to graphite/isl?  Also I see no support in
> the toplevel build machinery to build/install the libs as part of GCC
> so how does that work in the end?

In the end gcc needs to know what to do and just do it just like mpfr and
libquadmath.

I started looking at Makefile.def and I am guessing this is where the changes
need to be made to enable gcc machinery to actually build the two packages.  I
am not knowledgeable enough to figure this out and I recently got a new job so
now am very time constrained as far as reverse engineering what is going on.

I did manage to add Gnu Build System autotools features to OpenCoarrays so we
could be closer to integrating this in. (./configure && make vs cmake)

We definitely need some guidance from someone regarding the changes required for
the top level gcc items.

Also we are getting some libtool related build failures with the OpenCoarrays on
Mac and some other platforms where we know gcc builds fine, so I suspect we need
to build a libtool script or something to ensure that it works everywhere. It
works fine on Fedora which is what I use (naturally).

So if anyone would step up to the plate and advise on the toplevel machinery it
would be greatly appreciated.

Regards,

Jerry

Re: [Patch] Edit contrib/ files to download gfortran prerequisites

[Richard Biener]

On Wed, Sep 20, 2017 at 10:35 PM, Damian Rouson
 wrote:
> Attached is a patch that adds the downloading of gfortran prerequisites 
> OpenCoarrays and MPICH in the contrib/download_prerequisites script.  The 
> patch also provides a useful error message when neither wget or curl are 
> available on the target platform.  I tested this patch with several choices 
> for the command-line options on macOS (including --md5 and --sha512) and 
> Ubuntu Linux (including --sha512).A suggested ChangeLog entry is
>
> * contrib/download_prerequisites: Download OpenCoarrays and MPICH.
> * contrib/prerequisites.sha5: Add sha512 message digests for 
> OpenCoarrays and MPICH.
> * contrib/prerequisites.md5: Add md5 message digests for OpenCoarrays 
> and MPICH.
>
>
> OK for trunk?  If so, I’ll ask Jerry to commit this.  I don’t have commit 
> rights.

Can you make this optional similar to graphite/isl?  Also I see no support in
the toplevel build machinery to build/install the libs as part of GCC
so how does
that work in the end?

Thanks,
Richard.

> Damian

[Patch] Edit contrib/ files to download gfortran prerequisites

[Damian Rouson]

Attached is a patch that adds the downloading of gfortran prerequisites 
OpenCoarrays and MPICH in the contrib/download_prerequisites script.  The patch 
also provides a useful error message when neither wget or curl are available on 
the target platform.  I tested this patch with several choices for the 
command-line options on macOS (including --md5 and --sha512) and Ubuntu Linux 
(including --sha512).    A suggested ChangeLog entry is

        * contrib/download_prerequisites: Download OpenCoarrays and MPICH.
        * contrib/prerequisites.sha5: Add sha512 message digests for 
OpenCoarrays and MPICH.
        * contrib/prerequisites.md5: Add md5 message digests for OpenCoarrays 
and MPICH.


OK for trunk?  If so, I’ll ask Jerry to commit this.  I don’t have commit 
rights.

Damian

download-opencoarrays-mpich.diff
Description: Binary data