Re: [Mpi-forum] Giving up on C11 _Generic

[N.M. Maclaren via mpi-forum]

On Aug 17 2019, Jed Brown via mpi-forum wrote:

I don't want to go over old ground, but I feel these statements need a
bit of balancing.


They might even build
using -Wconversion (with a C11 compiler) as a diagnostic to check that
they have used MPI_Count everywhere that it is needed.  But the fallback
will just rely on silent (default -Wno-conversion) conversion to int,
thereby preserving today's behavior.


They are unlikely to use -Wconversion, because of the number of false
positives it generates; I fairly often use it, but I have been coding for
and teaching extreme portability for many decades.  I also teach the use of
-Wconversion but most people who try it on production codes give up on it.
And, unfortunately, the reasons are inherent in the C language.

The big problem with failure is NOT the direct MPI programmers on their
development systems, but when a working program is ported to a new system,
often by someone else entirely.  The danger there is that they are not
expecting a subtle, quiet error that will show up only in very big runs,
NOT most test runs, and may well not realise they are getting wrong answers.
That's seriously bad news and causes a LOT of false information to be
published.

Another is, as I said, C++.  Far more codes are developed in C++ than C
nowadays, and I discovered a new issue there when working on something
else.  C11 generics cause havoc with C++'s type-matching rules, so including
 and using namespace std (both VERY common) causes any use of the
built-in C++ abs function (at least on double) to fail.  This may be a bug
in the compiler, but I don't think so.  My guess is that most vendors will
alleviate that by not supporting C11 generics in C++'s C interface.


Regards,
Nick Maclaren.

___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jed Brown via mpi-forum]

Joseph Schuchart via mpi-forum  writes:

>> I'm not taking a position on C11 _Generic in the standard, but it would
>> significantly reduce the configure complexity for apps to upgrade to
>> MPI_Count without dropping support for previous standards.
>
> I don't see how C11 _Generic will make life easier. if an application 
> expects to transfer >2G elements it will eventually transition to 
> MPI_Count. With C11 _Generic (as originally proposed), checks are 
> required to ensure that both the compiler and the MPI implementation 
> support it to avoid silent corruption. 

Most apps today do not have code to detect when message sizes exceed int
and work around the limit or fail gracefully.  Instead, they just crash
unpredictably when those limits are exceeded.  I don't see that changing
for the majority of applications, but I do see them switching types to
MPI_Count so that they can get correct behavior when C11 generics are
present and the MPI implementation supports it.  They might even build
using -Wconversion (with a C11 compiler) as a diagnostic to check that
they have used MPI_Count everywhere that it is needed.  But the fallback
will just rely on silent (default -Wno-conversion) conversion to int,
thereby preserving today's behavior.

It isn't robust, but it is *pragmatic* and will work when the
environment allows.  I see the question as whether the Forum wishes to
implicitly endorse this short-cut by including the C11 generics or leave
that to apps to learn about and include an internal header with the
_Generic macros.

> Without _Generic, the compiler has no role in this and compilation
> will fail if the MPI implementation does not offer the MPI_*_x
> calls. The error is thus obvious. Thus, the application has to have
> configure checks and wrapper calls to properly support large
> transfers. And if they are not needed then there is no reason to use
> the MPI_*_x calls in the first place.
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[N.M. Maclaren via mpi-forum]

On Aug 15 2019, Joseph Schuchart via mpi-forum wrote:

On 8/13/19 5:54 AM, Jed Brown via mpi-forum wrote:


Few packages will accept a hard dependency on MPI-4 for at least 10
years.  MS-MPI still doesn't fully support MPI-2.1, for example, and
PETSc only recently began requiring MPI-2.0.


That is a sad truth: I just had a colleague asking me about a >6y old 
Open MPI release...


It's not just MPI!  But it's not as sad as all that.  Interfaces that
handle upwards compatibility well, like Fortran and MPI, continue to
accept old code.  And, if people don't have problems with their MPI,
there is simply no need for most of them to upgrade - especially if they
are not changing that part of their code.

Please continue to preserve this tradition!


Regards,
Nick Maclaren.

___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Joseph Schuchart via mpi-forum]

Jed,

On 8/13/19 5:54 AM, Jed Brown via mpi-forum wrote:

"Jeff Squyres \(jsquyres\) via mpi-forum"  
writes:


Let me ask a simple question: how will users to write portable MPI programs in 
C with large count values?

Answer: they will explicitly call MPI_Send_x(), and not rely on C11 _Generic.


Few packages will accept a hard dependency on MPI-4 for at least 10
years.  MS-MPI still doesn't fully support MPI-2.1, for example, and
PETSc only recently began requiring MPI-2.0.


That is a sad truth: I just had a colleague asking me about a >6y old 
Open MPI release...


>

I'm not taking a position on C11 _Generic in the standard, but it would
significantly reduce the configure complexity for apps to upgrade to
MPI_Count without dropping support for previous standards.


I don't see how C11 _Generic will make life easier. if an application 
expects to transfer >2G elements it will eventually transition to 
MPI_Count. With C11 _Generic (as originally proposed), checks are 
required to ensure that both the compiler and the MPI implementation 
support it to avoid silent corruption. Without _Generic, the compiler 
has no role in this and compilation will fail if the MPI implementation 
does not offer the MPI_*_x calls. The error is thus obvious. Thus, the 
application has to have configure checks and wrapper calls to properly 
support large transfers. And if they are not needed then there is no 
reason to use the MPI_*_x calls in the first place.


The point Jeff is making below is totally valid.

Joseph




Which then raises the question: what's the point of using C11
_Generic?  Its main functionality can lead to [potentially] silent
run-time errors, and/or require additional error checking in every
single code path by the implementation.  That just seems like bad
design, IMNSHO.  That's why the WG decided to bring this to the Forum
list (especially given the compressed timeframe for MPI-4).


___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum


___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jed Brown via mpi-forum]

"Jeff Squyres \(jsquyres\) via mpi-forum"  
writes:

> Let me ask a simple question: how will users to write portable MPI programs 
> in C with large count values?
>
> Answer: they will explicitly call MPI_Send_x(), and not rely on C11 _Generic.

Few packages will accept a hard dependency on MPI-4 for at least 10
years.  MS-MPI still doesn't fully support MPI-2.1, for example, and
PETSc only recently began requiring MPI-2.0.

Instead, each package that wishes to upgrade to MPI_Count args will
write configure tests (autoconf, etc.) to detect availability of
MPI_Send_x (as individual functions, not as MPI_VERSION == 4) and define
macros/wrappers OurMPI_Send() that forwards to MPI_Send_x (when
available) or MPI_Send (otherwise).  When the implementation doesn't
provide MPI_Send_x, they'll either have a smart wrapper that errors at
run-time on truncation or (likely most apps) will silently fail with an
FAQ that suggests using a compliant MPI-4 implementation.

I'm not taking a position on C11 _Generic in the standard, but it would
significantly reduce the configure complexity for apps to upgrade to
MPI_Count without dropping support for previous standards.

> Which then raises the question: what's the point of using C11
> _Generic?  Its main functionality can lead to [potentially] silent
> run-time errors, and/or require additional error checking in every
> single code path by the implementation.  That just seems like bad
> design, IMNSHO.  That's why the WG decided to bring this to the Forum
> list (especially given the compressed timeframe for MPI-4).

___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jeff Squyres (jsquyres) via mpi-forum]

Some points:

1. C11 _Generic support in an MPI implementation has to be optional, at least 
for now, because not all compilers support C11 _Generic (just like the F08 
bindings are still optional, and just like aspects of the C++ bindings were 
optional back in the '90s when C++ compilers still hadn't implemented all of 
the required features).

2. Given that an MPI implementation and/or compiler may or may not support 
MPI_Count-enabled C11 _Generic-style polymorphism for functions like MPI_Send, 
and given the defined behavior of the C/C++ language in terms of casting a 
large int down into a small int, it is quite possible that a valid MPI-4 
enabled application (i.e., using MPI_Count params with >2^32 values for 
functions like MPI_Send) with a valid MPI implementation can get truncated at 
run time.   **That is bad** (Dan's email exemplified this point)

--> Jeff Hammond has pointed out that there are some compiler flags (outside of 
-Wall) that can emit such warnings.
--> Jim Dinan has pointed out that implementations can add additional sizeof 
checks via macros. 

Fair points.  However, my response to both is: this is just making it hard for 
the average user to utilize the functionality correctly.  Sure, we can say 
"it's the users' fault", but the fact is that the language polymorphism 
mechanism itself is pretty weak and downright confusing (as a Forum, we've been 
discussing this for ***months*** -- how will average users/programmers to sort 
this stuff out?).

Let me ask a simple question: how will users to write portable MPI programs in 
C with large count values?

Answer: they will explicitly call MPI_Send_x(), and not rely on C11 _Generic.

Which then raises the question: what's the point of using C11 _Generic?  Its 
main functionality can lead to [potentially] silent run-time errors, and/or 
require additional error checking in every single code path by the 
implementation.  That just seems like bad design, IMNSHO.  That's why the WG 
decided to bring this to the Forum list (especially given the compressed 
timeframe for MPI-4).



> On Aug 8, 2019, at 7:37 AM, Jim Dinan via mpi-forum 
>  wrote:
> 
> In fairness, unsafe conversions are a feature of the C language, I assume 
> since very early on.  Assignment gets a pass to do horribly unsafe 
> conversions.  Unfortunately, when calling a function (with a prototype), the 
> argument expressions are converted as if by assignment, which is the 
> underlying cause for the truncation sadness.
> 
>  ~Jim.
> 
> On Thu, Aug 8, 2019 at 9:45 AM Jeff Hammond via mpi-forum 
>  wrote:
> That you use code that does unsafe conversions really has nothing to do with 
> the business of the MPI Forum. 
> 
> Again, if you think C89 is the best C, then use and teach that. No one here 
> is trying to make you use C11.
> 
> Jeff
> 
> > On Aug 8, 2019, at 5:56 AM, N.M. Maclaren via mpi-forum 
> >  wrote:
> > 
> >> On Aug 7 2019, Jeff Hammond via mpi-forum wrote:
> >> 
> >> "silently truncated at run time" is trivially addressed with -Wconversion
> >> or -Wshorten-64-to-32.  The example program below is addressed by this.
> > 
> > Unfortunately, no.  While I have no trouble using such options on MY
> > code, I have frequently found them unusable on imported packages, because
> > of the flood of non-errors they generate.  For example, the following
> > code is both common and reasonable, and includes narrowing conversions:
> > 
> >   int i;
> >   i = sizeof(double);
> >   float x;
> >   x = 1.0;
> ___
> mpi-forum mailing list
> mpi-forum@lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum
> ___
> mpi-forum mailing list
> mpi-forum@lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum


-- 
Jeff Squyres
jsquy...@cisco.com

___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[N.M. Maclaren via mpi-forum]

On Aug 8 2019, Jeff Hammond wrote:

That you use code that does unsafe conversions really has nothing to do 
with the business of the MPI Forum.


Again, if you think C89 is the best C, then use and teach that. No one 
here is trying to make you use C11.


Please don't be infantile.  I am thankfully retired, but spent 43 years
supporting scientific computing, including as our organisation's C expert,
and 15 years on SC22WG14.  As I have said (twice now, if I recall) "This
is not about MY code."

Jim Dinan has pointed out that unsafe conversions are a feature of the
C language, and I can confirm that they have been basic to the language
since K&R C and still are, in both C and C++, and are relied on in the
vast majority of the many dozens of large C codes I have ported, worked
on and advised people with.

He actually understated the problem, as assignment is not the only factor
that makes this such a foul issue.


Regards,
Nick Maclaren.

___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jim Dinan via mpi-forum]

In fairness, unsafe conversions are a feature of the C language, I assume
since very early on.  Assignment gets a pass to do horribly unsafe
conversions.  Unfortunately, when calling a function (with a prototype),
the argument expressions are converted as if by assignment, which is the
underlying cause for the truncation sadness.

 ~Jim.

On Thu, Aug 8, 2019 at 9:45 AM Jeff Hammond via mpi-forum <
mpi-forum@lists.mpi-forum.org> wrote:

> That you use code that does unsafe conversions really has nothing to do
> with the business of the MPI Forum.
>
> Again, if you think C89 is the best C, then use and teach that. No one
> here is trying to make you use C11.
>
> Jeff
>
> > On Aug 8, 2019, at 5:56 AM, N.M. Maclaren via mpi-forum <
> mpi-forum@lists.mpi-forum.org> wrote:
> >
> >> On Aug 7 2019, Jeff Hammond via mpi-forum wrote:
> >>
> >> "silently truncated at run time" is trivially addressed with
> -Wconversion
> >> or -Wshorten-64-to-32.  The example program below is addressed by this.
> >
> > Unfortunately, no.  While I have no trouble using such options on MY
> > code, I have frequently found them unusable on imported packages, because
> > of the flood of non-errors they generate.  For example, the following
> > code is both common and reasonable, and includes narrowing conversions:
> >
> >   int i;
> >   i = sizeof(double);
> >   float x;
> >   x = 1.0;
> ___
> mpi-forum mailing list
> mpi-forum@lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum
>
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jim Dinan via mpi-forum]

The polymorphic C front-end (i.e. macros) can be bypassed by talking a
pointer to the underlying function [1].  This is the desired behavior for
backward compatibility.  I agree that this could be surprising to users and
it must be clearly documented.

In the case where the user is not bypassing the C polymorphic front-end and
the compiler does not support _Generic, we can either detect or prevent
silent truncation using one of the approaches that I posted.

 ~Jim.

[1] The C preprocessor only invokes a function-like macro if the macro name
is followed by (.  So, &MPI_Send takes the pointer of the function even
though there is a function-like macro with the same name.

On Thu, Aug 8, 2019 at 6:51 AM HOLMES Daniel  wrote:

> Hi Jim,
>
> Are we sure the function pointer “worked" in your example? It silently
> truncated the large value, because the function pointer bypasses the
> redirection macro.
>
> Cheers,
> Dan.
> —
> Dr Daniel Holmes PhD
> Architect (HPC Research)
> d.hol...@epcc.ed.ac.uk
> Phone: +44 (0) 131 651 3465
> Mobile: +44 (0) 7940 524 088
> Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh,
> EH8 9BT
> —
> The University of Edinburgh is a charitable body, registered in Scotland,
> with registration number SC005336.
> —
>
> On 7 Aug 2019, at 22:00, Jim Dinan via mpi-forum <
> mpi-forum@lists.mpi-forum.org> wrote:
>
> Even simpler than this, we could just forward all calls to the MPI_Count
> interface (see below).  The int count argument should type convert to
> MPI_Count without issue.  Note that it still needs to be a function-like
> macro so that function pointers work.
>
> Don't give up yet!  :D
>
>  ~Jim.
>
> #include 
>
>
> typedef int MPI_Datatype;
> typedef int MPI_Comm;
>
> int MPI_Send(const void* buf, int count, MPI_Datatype datatype, int dest,
>  int tag, MPI_Comm comm)
> {
> printf("MPI_Send(count = %d)\n", count);
> return 0;
> }
>
> int MPI_Send_x(const void* buf, long long count, MPI_Datatype datatype,
> int dest,
>int tag, MPI_Comm comm)
> {
> printf("MPI_Send_x(count = %lld)\n", count);
> return 0;
> }
>
> #define MPI_Send(buf, count, datatype, dest, tag, comm) MPI_Send_x(buf,
> count, datatype, dest, tag, comm)
>
> int main(int argc, char *argv[]) {
> /* 8589934592LL == 2^33 */
> long long i = 8589934592LL + 11;
> int ret;
> int (*snd_ptr)(const void*, int, MPI_Datatype, int, int, MPI_Comm) =
> &MPI_Send;
> ret = MPI_Send(NULL, i, 0, 0, 0, 0);
> ret = MPI_Send(NULL, 5, 0, 0, 0, 0);
> ret = (*snd_ptr)(NULL, i, 0, 0, 0, 0);
> ret = (*snd_ptr)(NULL, 5, 0, 0, 0, 0);
> return 0;
>
> }
>
> MPI_Send_x(count = 8589934603)
> MPI_Send_x(count = 5)
> MPI_Send(count = 11)
>
> MPI_Send(count = 5)
> ___
> mpi-forum mailing list
> mpi-forum@lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum
>
>
>
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jeff Hammond via mpi-forum]

That you use code that does unsafe conversions really has nothing to do with 
the business of the MPI Forum. 

Again, if you think C89 is the best C, then use and teach that. No one here is 
trying to make you use C11.

Jeff

> On Aug 8, 2019, at 5:56 AM, N.M. Maclaren via mpi-forum 
>  wrote:
> 
>> On Aug 7 2019, Jeff Hammond via mpi-forum wrote:
>> 
>> "silently truncated at run time" is trivially addressed with -Wconversion
>> or -Wshorten-64-to-32.  The example program below is addressed by this.
> 
> Unfortunately, no.  While I have no trouble using such options on MY
> code, I have frequently found them unusable on imported packages, because
> of the flood of non-errors they generate.  For example, the following
> code is both common and reasonable, and includes narrowing conversions:
> 
>   int i;
>   i = sizeof(double);
>   float x;
>   x = 1.0;
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[N.M. Maclaren via mpi-forum]

On Aug 7 2019, Jeff Squyres (jsquyres) via mpi-forum wrote:


After much debate, the BigCount WG has decided that C11 _Generic just 
isn't worth it. That's no reason to penalize Fortran, though.


That is certainly the simplest solution.  There may be others, but
see below.

At a wild guess, this is probably trappable by implementations (at some
effort, and needing deep knowledge of the standards and issues) for MPI
being called directly from conforming C compilers, but not from C++ ones
using their C interface.  And that is a very serious gotcha.


On Aug 7 2019, Anthony Skjellum via mpi-forum wrote:


Conclusion: You can't use non-compliant MPI-4 with C11 compilers that are
non-compliant with C11, except maybe with a special flag???
Is there more that I am missing?


It's a LOT more complicated than that, as I tried to explain earlier.

Even when I was most active in this area, it would have taken me many
hours, probably days, of investigation (including on systems to which I
had no access), studying at least the C99, C11, C++ and MPI standards in
great detail, and working out flow-charts, to be fairly sure of the
situation.  This area is a gibbering nightmare, and is a major reason
the UK voted "no" to C99.


On Aug 7 2019, Jeff Hammond via mpi-forum wrote:


"silently truncated at run time" is trivially addressed with -Wconversion
or -Wshorten-64-to-32.  The example program below is addressed by this.


Unfortunately, no.  While I have no trouble using such options on MY
code, I have frequently found them unusable on imported packages, because
of the flood of non-errors they generate.  For example, the following
code is both common and reasonable, and includes narrowing conversions:

   int i;
   i = sizeof(double);
   float x;
   x = 1.0;


On Aug 8 2019, Gropp, William D via mpi-forum wrote:

That -Wall is a lie is also true for gcc. Yes, this is a extremely poor 
design. If you want to provide -Wmostcommon then do that (and maybe 
abbreviate to -Wmost).


Agreed, but it's a digression here!  I believe that the gcc people agree,
too, but changing it now is just too evil.


Regards,
Nick Maclaren.

___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Gropp, William D via mpi-forum]

That -Wall is a lie is also true for gcc. Yes, this is a extremely poor design. 
If you want to provide -Wmostcommon then do that (and maybe abbreviate to 
-Wmost).

Bill

William Gropp
Director and Chief Scientist, NCSA
Thomas M. Siebel Chair in Computer Science
University of Illinois Urbana-Champaign






On Aug 8, 2019, at 6:42 AM, HOLMES Daniel via mpi-forum 
mailto:mpi-forum@lists.mpi-forum.org>> wrote:

After a brief search that I never knew was necessary, I find that “all” is 
smaller than “everything” in clang world.
http://clang.llvm.org/docs/UsersManual.html#enabling-all-diagnostics

So, my question “why isn’t -Wconversion implied by -Wall?” might have an 
astonishing answer.

___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[HOLMES Daniel via mpi-forum]

Hi JeffH,

After a brief search that I never knew was necessary, I find that “all” is 
smaller than “everything” in clang world.
http://clang.llvm.org/docs/UsersManual.html#enabling-all-diagnostics

So, my question “why isn’t -Wconversion implied by -Wall?” might have an 
astonishing answer.

The resulting workflow is then:
1) notice the symptoms of a bug in the output/effect of the code
2) re-compile with -Weverything -Wno-c++98-compat -Wno-c++-compat
3) re-compile with -Weverything -Wno-c++98-compat -Wno-c++-compat 
-Wno-
4) re-compile with -Weverything -Wno-c++98-compat -Wno-c++-compat 
-Wno- -Wno-
5) address all remaining warnings by changing code
Experience and repeated exposure would permit combining steps 2-4.

This is documented, once you know to go look for it. However, it was not 
obvious to me that looking beyond “-Wall” was needed.

Whilst I agree wholeheartedly that “incorrect programs are incorrect”, I think 
designers of interfaces/standards have some responsibility to select a design 
that makes it easy to discover incorrect usage over one that makes it hard. 
Ideally, a design should be selected that makes it hard to form incorrect usage 
rather than one that makes misuse easy.

Cheers,
Dan.
—
Dr Daniel Holmes PhD
Architect (HPC Research)
d.hol...@epcc.ed.ac.uk
Phone: +44 (0) 131 651 3465
Mobile: +44 (0) 7940 524 088
Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT
—
The University of Edinburgh is a charitable body, registered in Scotland, with 
registration number SC005336.
—

On 7 Aug 2019, at 21:14, Jeff Hammond via mpi-forum 
mailto:mpi-forum@lists.mpi-forum.org>> wrote:

I don't care that much about C11 _Generic, which is why I have always focused 
on a C99 solution to the large-count problem, but I disagree with your reasons 
for abandoning it.

"silently truncated at run time" is trivially addressed with -Wconversion or 
-Wshorten-64-to-32.  The example program below is addressed by this.

$ clang -Wshorten-64-to-32 -c truncation.c
truncation.c:10:9: warning: implicit conversion loses integer precision: 'long 
long' to 'int' [-Wshorten-64-to-32]
foo(i);
~~~ ^
1 warning generated.

$ gcc-9 -Wconversion -c truncation.c
truncation.c: In function 'main':
truncation.c:10:9: warning: conversion from 'long long int' to 'int' may change 
value [-Wconversion]
   10 | foo(i);
  | ^

$ icc -Wconversion -c truncation.c
truncation.c(10): warning #1682: implicit conversion of a 64-bit integral type 
to a smaller integral type (potential portability problem)
  foo(i);
  ^

In any case, incorrect programs are incorrect.  It is a 
quality-of-implementation issue whether C compilers and MPI libraries detect 
incorrect usage.  We have never designed MPI around people who can't follow 
directions and we should not start now.

Jeff

On Wed, Aug 7, 2019 at 6:59 AM Jeff Squyres (jsquyres) via mpi-forum 
mailto:mpi-forum@lists.mpi-forum.org>> wrote:
SHORT VERSION
=

Due to the possibility of silently introducing errors into user applications, 
the BigCount WG no longer thinks that C11 _Generic is a good idea.  We are 
therefore dropping that from our proposal.  The new proposal will therefore 
essentially just be the addition of a bunch of MPI_Count-enabled "_x" functions 
in C, combined with the addition of a bunch of polymorphic MPI_Count-enabled 
interfaces in Fortran.

MORE DETAIL
===

Joseph Schuchart raised a very important point in a recent mailing thread: the 
following C/C++ code does not raise a compiler warning:

-
#include 

static void foo(int j) {
printf("foo(j) = %d\n", j);
}

int main(int argc, char *argv[]) {
/* 8589934592LL == 2^33 */
long long i = 8589934592LL + 11;
foo(i);
return 0;
}
-

If you compile and run this program on a commodity x86-64 platform, a) you 
won't get a warning from the compiler, and b) you'll see "11" printed out.  I 
tried with gcc 9 and clang 8 -- both with the C and C++ compilers.  I even 
tried with "-Wall -pedantic".  No warnings.

This is because casting from a larger int type to a smaller int type is 
perfectly valid C/C++.

Because of this, there is a possibility that we could be silently introducing 
errors into user applications.  Consider:

1. An application upgrades its "count" parameters to type MPI_Count for all 
calls to MPI_Send.
   --> Recall that "MPI_Count" already exists in MPI-3.1, and is likely of type 
(long long) on commodity x86-64 platforms
2. The application then uses values in that "count" parameter that are greater 
than 2^32.

If the user's MPI implementation and compiler both support C11 _Generic, 
everything is great.

But if either the MPI implementation or the compiler do not support C11 
_Generic, ***the "count" value will be silently truncated at run time***.

This seems like a very bad idea, from a design standpoint.

We have therefore come full circle: we are back to adding a bunch o

Re: [Mpi-forum] Giving up on C11 _Generic

[HOLMES Daniel via mpi-forum]

Hi Jim,

Are we sure the function pointer “worked" in your example? It silently 
truncated the large value, because the function pointer bypasses the 
redirection macro.

Cheers,
Dan.
—
Dr Daniel Holmes PhD
Architect (HPC Research)
d.hol...@epcc.ed.ac.uk
Phone: +44 (0) 131 651 3465
Mobile: +44 (0) 7940 524 088
Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT
—
The University of Edinburgh is a charitable body, registered in Scotland, with 
registration number SC005336.
—

On 7 Aug 2019, at 22:00, Jim Dinan via mpi-forum 
mailto:mpi-forum@lists.mpi-forum.org>> wrote:

Even simpler than this, we could just forward all calls to the MPI_Count 
interface (see below).  The int count argument should type convert to MPI_Count 
without issue.  Note that it still needs to be a function-like macro so that 
function pointers work.

Don't give up yet!  :D

 ~Jim.

#include 



typedef int MPI_Datatype;
typedef int MPI_Comm;

int MPI_Send(const void* buf, int count, MPI_Datatype datatype, int dest,
 int tag, MPI_Comm comm)
{
printf("MPI_Send(count = %d)\n", count);
return 0;
}

int MPI_Send_x(const void* buf, long long count, MPI_Datatype datatype, int 
dest,
   int tag, MPI_Comm comm)
{
printf("MPI_Send_x(count = %lld)\n", count);
return 0;
}

#define MPI_Send(buf, count, datatype, dest, tag, comm) MPI_Send_x(buf, count, 
datatype, dest, tag, comm)

int main(int argc, char *argv[]) {
/* 8589934592LL == 2^33 */
long long i = 8589934592LL + 11;
int ret;
int (*snd_ptr)(const void*, int, MPI_Datatype, int, int, MPI_Comm) = 
&MPI_Send;
ret = MPI_Send(NULL, i, 0, 0, 0, 0);
ret = MPI_Send(NULL, 5, 0, 0, 0, 0);
ret = (*snd_ptr)(NULL, i, 0, 0, 0, 0);
ret = (*snd_ptr)(NULL, 5, 0, 0, 0, 0);
return 0;

}

MPI_Send_x(count = 8589934603)
MPI_Send_x(count = 5)
MPI_Send(count = 11)

MPI_Send(count = 5)
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jim Dinan via mpi-forum]

That case should not have the implicit type conversion issue that Joseph
raised and should be fine without the extra macro when _Generic is
unsupported.

On Wed, Aug 7, 2019, 5:43 PM Jeff Hammond  wrote:

> You can't do that forwarding for vectors of counts.
>
> On Wed, Aug 7, 2019 at 2:10 PM Jim Dinan via mpi-forum <
> mpi-forum@lists.mpi-forum.org> wrote:
>
>> Even simpler than this, we could just forward all calls to the MPI_Count
>> interface (see below).  The int count argument should type convert to
>> MPI_Count without issue.  Note that it still needs to be a function-like
>> macro so that function pointers work.
>>
>> Don't give up yet!  :D
>>
>>  ~Jim.
>>
>> #include 
>>
>>
>>
>> typedef int MPI_Datatype;
>>
>> typedef int MPI_Comm;
>>
>>
>> int MPI_Send(const void* buf, int count, MPI_Datatype datatype, int dest,
>>
>>  int tag, MPI_Comm comm)
>>
>> {
>>
>> printf("MPI_Send(count = %d)\n", count);
>>
>> return 0;
>>
>> }
>>
>>
>> int MPI_Send_x(const void* buf, long long count, MPI_Datatype datatype,
>> int dest,
>>
>>int tag, MPI_Comm comm)
>>
>> {
>>
>> printf("MPI_Send_x(count = %lld)\n", count);
>>
>> return 0;
>>
>> }
>>
>>
>> #define MPI_Send(buf, count, datatype, dest, tag, comm) MPI_Send_x(buf,
>> count, datatype, dest, tag, comm)
>>
>>
>> int main(int argc, char *argv[]) {
>>
>> /* 8589934592LL == 2^33 */
>>
>> long long i = 8589934592LL + 11;
>>
>> int ret;
>>
>> int (*snd_ptr)(const void*, int, MPI_Datatype, int, int, MPI_Comm) =
>> &MPI_Send;
>>
>> ret = MPI_Send(NULL, i, 0, 0, 0, 0);
>>
>> ret = MPI_Send(NULL, 5, 0, 0, 0, 0);
>>
>> ret = (*snd_ptr)(NULL, i, 0, 0, 0, 0);
>>
>> ret = (*snd_ptr)(NULL, 5, 0, 0, 0, 0);
>>
>> return 0;
>>
>> }
>>
>>
>> MPI_Send_x(count = 8589934603)
>>
>> MPI_Send_x(count = 5)
>>
>> MPI_Send(count = 11)
>>
>> MPI_Send(count = 5)
>> ___
>> mpi-forum mailing list
>> mpi-forum@lists.mpi-forum.org
>> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum
>>
>
>
> --
> Jeff Hammond
> jeff.scie...@gmail.com
> http://jeffhammond.github.io/
>
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jeff Hammond via mpi-forum]

You can't do that forwarding for vectors of counts.

On Wed, Aug 7, 2019 at 2:10 PM Jim Dinan via mpi-forum <
mpi-forum@lists.mpi-forum.org> wrote:

> Even simpler than this, we could just forward all calls to the MPI_Count
> interface (see below).  The int count argument should type convert to
> MPI_Count without issue.  Note that it still needs to be a function-like
> macro so that function pointers work.
>
> Don't give up yet!  :D
>
>  ~Jim.
>
> #include 
>
>
>
> typedef int MPI_Datatype;
>
> typedef int MPI_Comm;
>
>
> int MPI_Send(const void* buf, int count, MPI_Datatype datatype, int dest,
>
>  int tag, MPI_Comm comm)
>
> {
>
> printf("MPI_Send(count = %d)\n", count);
>
> return 0;
>
> }
>
>
> int MPI_Send_x(const void* buf, long long count, MPI_Datatype datatype,
> int dest,
>
>int tag, MPI_Comm comm)
>
> {
>
> printf("MPI_Send_x(count = %lld)\n", count);
>
> return 0;
>
> }
>
>
> #define MPI_Send(buf, count, datatype, dest, tag, comm) MPI_Send_x(buf,
> count, datatype, dest, tag, comm)
>
>
> int main(int argc, char *argv[]) {
>
> /* 8589934592LL == 2^33 */
>
> long long i = 8589934592LL + 11;
>
> int ret;
>
> int (*snd_ptr)(const void*, int, MPI_Datatype, int, int, MPI_Comm) =
> &MPI_Send;
>
> ret = MPI_Send(NULL, i, 0, 0, 0, 0);
>
> ret = MPI_Send(NULL, 5, 0, 0, 0, 0);
>
> ret = (*snd_ptr)(NULL, i, 0, 0, 0, 0);
>
> ret = (*snd_ptr)(NULL, 5, 0, 0, 0, 0);
>
> return 0;
>
> }
>
>
> MPI_Send_x(count = 8589934603)
>
> MPI_Send_x(count = 5)
>
> MPI_Send(count = 11)
>
> MPI_Send(count = 5)
> ___
> mpi-forum mailing list
> mpi-forum@lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum
>


-- 
Jeff Hammond
jeff.scie...@gmail.com
http://jeffhammond.github.io/
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jim Dinan via mpi-forum]

Even simpler than this, we could just forward all calls to the MPI_Count
interface (see below).  The int count argument should type convert to
MPI_Count without issue.  Note that it still needs to be a function-like
macro so that function pointers work.

Don't give up yet!  :D

 ~Jim.

#include 



typedef int MPI_Datatype;

typedef int MPI_Comm;


int MPI_Send(const void* buf, int count, MPI_Datatype datatype, int dest,

 int tag, MPI_Comm comm)

{

printf("MPI_Send(count = %d)\n", count);

return 0;

}


int MPI_Send_x(const void* buf, long long count, MPI_Datatype datatype, int
dest,

   int tag, MPI_Comm comm)

{

printf("MPI_Send_x(count = %lld)\n", count);

return 0;

}


#define MPI_Send(buf, count, datatype, dest, tag, comm) MPI_Send_x(buf,
count, datatype, dest, tag, comm)


int main(int argc, char *argv[]) {

/* 8589934592LL == 2^33 */

long long i = 8589934592LL + 11;

int ret;

int (*snd_ptr)(const void*, int, MPI_Datatype, int, int, MPI_Comm) =
&MPI_Send;

ret = MPI_Send(NULL, i, 0, 0, 0, 0);

ret = MPI_Send(NULL, 5, 0, 0, 0, 0);

ret = (*snd_ptr)(NULL, i, 0, 0, 0, 0);

ret = (*snd_ptr)(NULL, 5, 0, 0, 0, 0);

return 0;

}


MPI_Send_x(count = 8589934603)

MPI_Send_x(count = 5)

MPI_Send(count = 11)

MPI_Send(count = 5)
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jim Dinan via mpi-forum]

This example is a bit more representative of how we would use this to
implement the MPI bindings (renamed "bar" macro to "foo" and shifted down
to enable the name aliasing):

#include 



static void foo(int j) {

printf("foo(j) = %d\n", j);

}


#define foo(j) foo(sizeof(j) > sizeof(int) ? -1 : j)


int main(int argc, char *argv[]) {

/* 8589934592LL == 2^33 */

long long i = 8589934592LL + 11;

foo(i);

return 0;

}

foo(j) = -1

 ~Jim.

On Wed, Aug 7, 2019 at 4:16 PM Jim Dinan  wrote:

> Jeff,
>
> You can wrap the function in a macro that checks for truncation:
>
> #include 
>
>
>
> #define bar(j) foo(sizeof(j) > sizeof(int) ? -1 : j)
>
>
> static void foo(int j) {
>
> printf("foo(j) = %d\n", j);
>
> }
>
>
> int main(int argc, char *argv[]) {
>
> /* 8589934592LL == 2^33 */
>
> long long i = 8589934592LL + 11;
>
> foo(i);
>
> bar(i);
>
> return 0;
>
> }
>
> foo(j) = 11
>
> foo(j) = -1
>
>  ~Jim.
>
> On Wed, Aug 7, 2019 at 9:59 AM Jeff Squyres (jsquyres) via mpi-forum <
> mpi-forum@lists.mpi-forum.org> wrote:
>
>> SHORT VERSION
>> =
>>
>> Due to the possibility of silently introducing errors into user
>> applications, the BigCount WG no longer thinks that C11 _Generic is a good
>> idea.  We are therefore dropping that from our proposal.  The new proposal
>> will therefore essentially just be the addition of a bunch of
>> MPI_Count-enabled "_x" functions in C, combined with the addition of a
>> bunch of polymorphic MPI_Count-enabled interfaces in Fortran.
>>
>> MORE DETAIL
>> ===
>>
>> Joseph Schuchart raised a very important point in a recent mailing
>> thread: the following C/C++ code does not raise a compiler warning:
>>
>> -
>> #include 
>>
>> static void foo(int j) {
>> printf("foo(j) = %d\n", j);
>> }
>>
>> int main(int argc, char *argv[]) {
>> /* 8589934592LL == 2^33 */
>> long long i = 8589934592LL + 11;
>> foo(i);
>> return 0;
>> }
>> -
>>
>> If you compile and run this program on a commodity x86-64 platform, a)
>> you won't get a warning from the compiler, and b) you'll see "11" printed
>> out.  I tried with gcc 9 and clang 8 -- both with the C and C++ compilers.
>> I even tried with "-Wall -pedantic".  No warnings.
>>
>> This is because casting from a larger int type to a smaller int type is
>> perfectly valid C/C++.
>>
>> Because of this, there is a possibility that we could be silently
>> introducing errors into user applications.  Consider:
>>
>> 1. An application upgrades its "count" parameters to type MPI_Count for
>> all calls to MPI_Send.
>>--> Recall that "MPI_Count" already exists in MPI-3.1, and is likely
>> of type (long long) on commodity x86-64 platforms
>> 2. The application then uses values in that "count" parameter that are
>> greater than 2^32.
>>
>> If the user's MPI implementation and compiler both support C11 _Generic,
>> everything is great.
>>
>> But if either the MPI implementation or the compiler do not support C11
>> _Generic, ***the "count" value will be silently truncated at run time***.
>>
>> This seems like a very bad idea, from a design standpoint.
>>
>> We have therefore come full circle: we are back to adding a bunch of "_x"
>> functions for C, and there will be no polymorphism (in C).  Sorry, folks.
>>
>> Note that Fortran does not have similar problems:
>>
>> 1. Fortran compilers have supported polymorphism for 20+ years
>> 2. Fortran does not automatically cast between INTEGER values of
>> different sizes
>>
>> After much debate, the BigCount WG has decided that C11 _Generic just
>> isn't worth it.  That's no reason to penalize Fortran, though.
>>
>> --
>> Jeff Squyres
>> jsquy...@cisco.com
>>
>> ___
>> mpi-forum mailing list
>> mpi-forum@lists.mpi-forum.org
>> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum
>>
>
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jim Dinan via mpi-forum]

Jeff,

You can wrap the function in a macro that checks for truncation:

#include 



#define bar(j) foo(sizeof(j) > sizeof(int) ? -1 : j)


static void foo(int j) {

printf("foo(j) = %d\n", j);

}


int main(int argc, char *argv[]) {

/* 8589934592LL == 2^33 */

long long i = 8589934592LL + 11;

foo(i);

bar(i);

return 0;

}

foo(j) = 11

foo(j) = -1

 ~Jim.

On Wed, Aug 7, 2019 at 9:59 AM Jeff Squyres (jsquyres) via mpi-forum <
mpi-forum@lists.mpi-forum.org> wrote:

> SHORT VERSION
> =
>
> Due to the possibility of silently introducing errors into user
> applications, the BigCount WG no longer thinks that C11 _Generic is a good
> idea.  We are therefore dropping that from our proposal.  The new proposal
> will therefore essentially just be the addition of a bunch of
> MPI_Count-enabled "_x" functions in C, combined with the addition of a
> bunch of polymorphic MPI_Count-enabled interfaces in Fortran.
>
> MORE DETAIL
> ===
>
> Joseph Schuchart raised a very important point in a recent mailing thread:
> the following C/C++ code does not raise a compiler warning:
>
> -
> #include 
>
> static void foo(int j) {
> printf("foo(j) = %d\n", j);
> }
>
> int main(int argc, char *argv[]) {
> /* 8589934592LL == 2^33 */
> long long i = 8589934592LL + 11;
> foo(i);
> return 0;
> }
> -
>
> If you compile and run this program on a commodity x86-64 platform, a) you
> won't get a warning from the compiler, and b) you'll see "11" printed out.
> I tried with gcc 9 and clang 8 -- both with the C and C++ compilers.  I
> even tried with "-Wall -pedantic".  No warnings.
>
> This is because casting from a larger int type to a smaller int type is
> perfectly valid C/C++.
>
> Because of this, there is a possibility that we could be silently
> introducing errors into user applications.  Consider:
>
> 1. An application upgrades its "count" parameters to type MPI_Count for
> all calls to MPI_Send.
>--> Recall that "MPI_Count" already exists in MPI-3.1, and is likely of
> type (long long) on commodity x86-64 platforms
> 2. The application then uses values in that "count" parameter that are
> greater than 2^32.
>
> If the user's MPI implementation and compiler both support C11 _Generic,
> everything is great.
>
> But if either the MPI implementation or the compiler do not support C11
> _Generic, ***the "count" value will be silently truncated at run time***.
>
> This seems like a very bad idea, from a design standpoint.
>
> We have therefore come full circle: we are back to adding a bunch of "_x"
> functions for C, and there will be no polymorphism (in C).  Sorry, folks.
>
> Note that Fortran does not have similar problems:
>
> 1. Fortran compilers have supported polymorphism for 20+ years
> 2. Fortran does not automatically cast between INTEGER values of different
> sizes
>
> After much debate, the BigCount WG has decided that C11 _Generic just
> isn't worth it.  That's no reason to penalize Fortran, though.
>
> --
> Jeff Squyres
> jsquy...@cisco.com
>
> ___
> mpi-forum mailing list
> mpi-forum@lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum
>
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Jeff Hammond via mpi-forum]

I don't care that much about C11 _Generic, which is why I have always
focused on a C99 solution to the large-count problem, but I disagree with
your reasons for abandoning it.

"silently truncated at run time" is trivially addressed with -Wconversion
or -Wshorten-64-to-32.  The example program below is addressed by this.

$ clang -Wshorten-64-to-32 -c truncation.c
truncation.c:10:9: warning: implicit conversion loses integer precision:
'long long' to 'int' [-Wshorten-64-to-32]
foo(i);
~~~ ^
1 warning generated.

$ gcc-9 -Wconversion -c truncation.c
truncation.c: In function 'main':
truncation.c:10:9: warning: conversion from 'long long int' to 'int' may
change value [-Wconversion]
   10 | foo(i);
  | ^

$ icc -Wconversion -c truncation.c
truncation.c(10): warning #1682: implicit conversion of a 64-bit integral
type to a smaller integral type (potential portability problem)
  foo(i);
  ^

In any case, incorrect programs are incorrect.  It is a
quality-of-implementation issue whether C compilers and MPI libraries
detect incorrect usage.  We have never designed MPI around people who can't
follow directions and we should not start now.

Jeff

On Wed, Aug 7, 2019 at 6:59 AM Jeff Squyres (jsquyres) via mpi-forum <
mpi-forum@lists.mpi-forum.org> wrote:

> SHORT VERSION
> =
>
> Due to the possibility of silently introducing errors into user
> applications, the BigCount WG no longer thinks that C11 _Generic is a good
> idea.  We are therefore dropping that from our proposal.  The new proposal
> will therefore essentially just be the addition of a bunch of
> MPI_Count-enabled "_x" functions in C, combined with the addition of a
> bunch of polymorphic MPI_Count-enabled interfaces in Fortran.
>
> MORE DETAIL
> ===
>
> Joseph Schuchart raised a very important point in a recent mailing thread:
> the following C/C++ code does not raise a compiler warning:
>
> -
> #include 
>
> static void foo(int j) {
> printf("foo(j) = %d\n", j);
> }
>
> int main(int argc, char *argv[]) {
> /* 8589934592LL == 2^33 */
> long long i = 8589934592LL + 11;
> foo(i);
> return 0;
> }
> -
>
> If you compile and run this program on a commodity x86-64 platform, a) you
> won't get a warning from the compiler, and b) you'll see "11" printed out.
> I tried with gcc 9 and clang 8 -- both with the C and C++ compilers.  I
> even tried with "-Wall -pedantic".  No warnings.
>
> This is because casting from a larger int type to a smaller int type is
> perfectly valid C/C++.
>
> Because of this, there is a possibility that we could be silently
> introducing errors into user applications.  Consider:
>
> 1. An application upgrades its "count" parameters to type MPI_Count for
> all calls to MPI_Send.
>--> Recall that "MPI_Count" already exists in MPI-3.1, and is likely of
> type (long long) on commodity x86-64 platforms
> 2. The application then uses values in that "count" parameter that are
> greater than 2^32.
>
> If the user's MPI implementation and compiler both support C11 _Generic,
> everything is great.
>
> But if either the MPI implementation or the compiler do not support C11
> _Generic, ***the "count" value will be silently truncated at run time***.
>
> This seems like a very bad idea, from a design standpoint.
>
> We have therefore come full circle: we are back to adding a bunch of "_x"
> functions for C, and there will be no polymorphism (in C).  Sorry, folks.
>
> Note that Fortran does not have similar problems:
>
> 1. Fortran compilers have supported polymorphism for 20+ years
> 2. Fortran does not automatically cast between INTEGER values of different
> sizes
>
> After much debate, the BigCount WG has decided that C11 _Generic just
> isn't worth it.  That's no reason to penalize Fortran, though.
>
> --
> Jeff Squyres
> jsquy...@cisco.com
>
> ___
> mpi-forum mailing list
> mpi-forum@lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum
>


-- 
Jeff Hammond
jeff.scie...@gmail.com
http://jeffhammond.github.io/
___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

Re: [Mpi-forum] Giving up on C11 _Generic

[Anthony Skjellum via mpi-forum]

Jeff, I don't think we all agree 100% to abandon C11 _Generic, and let me
feedback to you what you write below for further argumentation;
I am a bit surprised this is in front of the whole forum just one day after
WG discussions, when more WG discussions are still in order weeks before
the finalization of our proposals for Zurich.

This statement alone

"But if either the MPI implementation or the compiler do not support C11
_Generic, ***the "count" value will be silently truncated at run time***."

is not a sufficient reason for abandoning C11 _Generic.
1) MPI Implementations have to either support MPI-4 or not; it would not be
optional.  Therefore the MPI implementation is incomplete.
Therefore this is not a reason to abandon C11.

2) If the compiler environment doesn't support C11 _Generic, then the build
process for MPI would have to warn the user or that kind of platform would
be impossible to use with MPI-4.   Seems harsh for non-compliant compilers.

These are not sufficient conditions alone for abandoning C11 _Generic.

Conclusion: You can't use non-compliant MPI-4 with C11 compilers that are
non-compliant with C11, except maybe with a special flag???
Is there more that I am missing?

Do you have other reasons that I've missed or you don't cite below besides
a) non-compliant MPI implementation, b) non-compliant C11 compiler for
abandoning this feature?  These are both reasons that an MPI program would
not do what it should.  But you also write

"If the user's MPI implementation and compiler both support C11 _Generic,
everything is great."

I realize that I was not on the full meetings recently, but this major
change of heart, although shared by others on the phone yesterday, is not
unanimous (and not everyone spoke up).Maybe I am missing deeper
arguments, because you yourself say that correct implementation with a
correct compiler produces valid outcomes.

Notice how this kind of problem with Long vs. int exists in the C interface
currently; whenever you have a long long int argument, and you pass this to
a current C API that takes an int (for a count), this truncation occurs.
Do you agree?

Help me to understand better why we should overthrow this approach.

Regards,
Tony


On Wed, Aug 7, 2019 at 9:59 AM Jeff Squyres (jsquyres) via mpi-forum <
mpi-forum@lists.mpi-forum.org> wrote:

> SHORT VERSION
> =
>
> Due to the possibility of silently introducing errors into user
> applications, the BigCount WG no longer thinks that C11 _Generic is a good
> idea.  We are therefore dropping that from our proposal.  The new proposal
> will therefore essentially just be the addition of a bunch of
> MPI_Count-enabled "_x" functions in C, combined with the addition of a
> bunch of polymorphic MPI_Count-enabled interfaces in Fortran.
>
> MORE DETAIL
> ===
>
> Joseph Schuchart raised a very important point in a recent mailing thread:
> the following C/C++ code does not raise a compiler warning:
>
> -
> #include 
>
> static void foo(int j) {
> printf("foo(j) = %d\n", j);
> }
>
> int main(int argc, char *argv[]) {
> /* 8589934592LL == 2^33 */
> long long i = 8589934592LL + 11;
> foo(i);
> return 0;
> }
> -
>
> If you compile and run this program on a commodity x86-64 platform, a) you
> won't get a warning from the compiler, and b) you'll see "11" printed out.
> I tried with gcc 9 and clang 8 -- both with the C and C++ compilers.  I
> even tried with "-Wall -pedantic".  No warnings.
>
> This is because casting from a larger int type to a smaller int type is
> perfectly valid C/C++.
>
> Because of this, there is a possibility that we could be silently
> introducing errors into user applications.  Consider:
>
> 1. An application upgrades its "count" parameters to type MPI_Count for
> all calls to MPI_Send.
>--> Recall that "MPI_Count" already exists in MPI-3.1, and is likely of
> type (long long) on commodity x86-64 platforms
> 2. The application then uses values in that "count" parameter that are
> greater than 2^32.
>
> If the user's MPI implementation and compiler both support C11 _Generic,
> everything is great.
>
> But if either the MPI implementation or the compiler do not support C11
> _Generic, ***the "count" value will be silently truncated at run time***.
>
> This seems like a very bad idea, from a design standpoint.
>
> We have therefore come full circle: we are back to adding a bunch of "_x"
> functions for C, and there will be no polymorphism (in C).  Sorry, folks.
>
> Note that Fortran does not have similar problems:
>
> 1. Fortran compilers have supported polymorphism for 20+ years
> 2. Fortran does not automatically cast between INTEGER values of different
> sizes
>
> After much debate, the BigCount WG has decided that C11 _Generic just
> isn't worth it.  That's no reason to penalize Fortran, though.
>
> --
> Jeff Squyres
> jsquy...@cisco.com
>
> ___
> mpi-forum mailing list
> mpi-forum@lists.mpi-forum.org

[Mpi-forum] Giving up on C11 _Generic

[Jeff Squyres (jsquyres) via mpi-forum]

SHORT VERSION
=

Due to the possibility of silently introducing errors into user applications, 
the BigCount WG no longer thinks that C11 _Generic is a good idea.  We are 
therefore dropping that from our proposal.  The new proposal will therefore 
essentially just be the addition of a bunch of MPI_Count-enabled "_x" functions 
in C, combined with the addition of a bunch of polymorphic MPI_Count-enabled 
interfaces in Fortran.

MORE DETAIL
===

Joseph Schuchart raised a very important point in a recent mailing thread: the 
following C/C++ code does not raise a compiler warning:

-
#include 

static void foo(int j) {
printf("foo(j) = %d\n", j);
}

int main(int argc, char *argv[]) {
/* 8589934592LL == 2^33 */
long long i = 8589934592LL + 11;
foo(i);
return 0;
}
-

If you compile and run this program on a commodity x86-64 platform, a) you 
won't get a warning from the compiler, and b) you'll see "11" printed out.  I 
tried with gcc 9 and clang 8 -- both with the C and C++ compilers.  I even 
tried with "-Wall -pedantic".  No warnings.

This is because casting from a larger int type to a smaller int type is 
perfectly valid C/C++.

Because of this, there is a possibility that we could be silently introducing 
errors into user applications.  Consider:

1. An application upgrades its "count" parameters to type MPI_Count for all 
calls to MPI_Send.
   --> Recall that "MPI_Count" already exists in MPI-3.1, and is likely of type 
(long long) on commodity x86-64 platforms
2. The application then uses values in that "count" parameter that are greater 
than 2^32.

If the user's MPI implementation and compiler both support C11 _Generic, 
everything is great.

But if either the MPI implementation or the compiler do not support C11 
_Generic, ***the "count" value will be silently truncated at run time***.

This seems like a very bad idea, from a design standpoint.

We have therefore come full circle: we are back to adding a bunch of "_x" 
functions for C, and there will be no polymorphism (in C).  Sorry, folks.

Note that Fortran does not have similar problems:

1. Fortran compilers have supported polymorphism for 20+ years
2. Fortran does not automatically cast between INTEGER values of different sizes

After much debate, the BigCount WG has decided that C11 _Generic just isn't 
worth it.  That's no reason to penalize Fortran, though.

-- 
Jeff Squyres
jsquy...@cisco.com

___
mpi-forum mailing list
mpi-forum@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpi-forum