Re: building GNU gettext on AIX

[Bruno Haible]

I wrote in
:
> > The latest issue is that a few files in gettext ignore --disable-pthreads
> > and creates a dependency on pthread_mutex.
> ...
>   * If no, then the simple solution would be to pass the configure option
>   --enable-threads=isoc
> This should not introduce a link dependency, because the mtx_lock,
> mtx_unlock, and mtx_init functions are in libc in AIX ≥ 7.2. Currently it
> does not work (it still uses pthread_mutex_lock and pthread_mutex_unlock
> despite --enable-threads=isoc). But I could make this work and release
> a gettext 0.22.4 with the fix.

Alas, this approach does not help reducing the dependency towards libpthreads.
On AIX, pthread_mutex_t and mtx_t are the same type. Thus code like this
===
#include 
#include 
#include 
#include 

pthread_mutex_t lock1 = PTHREAD_MUTEX_INITIALIZER;

int main ()
{
  if (mtx_lock () != thrd_success)
abort ();

  if (mtx_unlock () != thrd_success)
abort ();
}
===
compiles and runs fine. But the library dependencies still contain libpthreads.
This is in 32-bit mode:

$ ldd a.out
a.out needs:
 /usr/lib/libc.a(shr.o)
 /usr/lib/libc.a(cthread.o)
 /usr/lib/libc.a(_shr.o)
 /unix
 /usr/lib/libcrypt.a(shr.o)
 /usr/lib/libpthreads.a(shr_xpg5.o)
 /usr/lib/libpthreads.a(_shr_xpg5.o)
 /usr/lib/libpthreads.a(shr_comm.o)

and this in 64-bit mode:

$ ldd a.out 
a.out needs:
 /usr/lib/libc.a(shr_64.o)
 /usr/lib/libc.a(cthread_64.o)
 /usr/lib/libc.a(_shr_64.o)
 /unix
 /usr/lib/libcrypt.a(shr_64.o)
 /usr/lib/libpthreads.a(shr_xpg5_64.o)
 /usr/lib/libpthreads.a(_shr_xpg5_64.o)

Apparently the mtx_* functions are provided by
  /usr/lib/libc.a(cthread_64.o)
and this one depends on
  /usr/lib/libpthreads.a(shr_xpg5_64.o)
  /usr/lib/libpthreads.a(_shr_xpg5_64.o)

So, there can be only three ways to build GCC on AIX:

  - With --disable-nls. No i18n, no libpthreads dependency.
  - With --enable-nls, linked against a libintl created in the build
tree with --disable-shared --disable-threads (requires gettext ≥ 0.22.4).
Has i18n, but no libpthreads dependency.
  - With --enable-nls, linked against a public libintl. Depends on libpthreads.

Bruno

Re: building GNU gettext on AIX

[Arsen Arsenović]

Bruno Haible  writes:

> David Edelsohn and Richard Biener wrote:
>> > >   * GCC can pass --enable-threads=isoc, to avoid the libpthread 
>> > > dependency
>> > > on AIX ≥ 7.2.
>> >
>> > Hmm, would that option work everywhere, though?  Or would we have to
>> > wire up configury to detect which flag to use?  If so, what would it
>> > look like.
>
> --enable-threads=isoc should only be used where needed, i.e. on AIX 7.2
> or higher. Yes, this needs a configure test.
>
>> I'd highly recommend the in-tree gettext to be built with thread
>> support _disabled_,
>> only a static library built and never installed (but linked statically
>> into the host binaries).
>
> As long as this static libintl.a gets never installed (but linked statically
> into the host binaries), that's perfectly fine.
>
> It only needs a fixed gettext.m4, to deal with the bug that David Edelsohn
> noticed on AIX (but that exists on macOS as well). [1]

That should always work for us, then (the only case in which the GCC
build system sets up flags for gettext is when building it in-tree, in
which case we already want a --disable-shared build).

If that library gets installed, then, so far, it is unintentional.  I'll
check that (as well as the docs targets that Eric Gallager mentioned on
IRC) tonight.

> Bruno
>
> [1] https://lists.gnu.org/archive/html/bug-gnulib/2023-11/msg00164.html


-- 
Arsen Arsenović


signature.asc
Description: PGP signature

Re: building GNU gettext on AIX

[Bruno Haible]

David Edelsohn and Richard Biener wrote:
> > >   * GCC can pass --enable-threads=isoc, to avoid the libpthread dependency
> > > on AIX ≥ 7.2.
> >
> > Hmm, would that option work everywhere, though?  Or would we have to
> > wire up configury to detect which flag to use?  If so, what would it
> > look like.

--enable-threads=isoc should only be used where needed, i.e. on AIX 7.2
or higher. Yes, this needs a configure test.

> I'd highly recommend the in-tree gettext to be built with thread
> support _disabled_,
> only a static library built and never installed (but linked statically
> into the host binaries).

As long as this static libintl.a gets never installed (but linked statically
into the host binaries), that's perfectly fine.

It only needs a fixed gettext.m4, to deal with the bug that David Edelsohn
noticed on AIX (but that exists on macOS as well). [1]

Bruno

[1] https://lists.gnu.org/archive/html/bug-gnulib/2023-11/msg00164.html

Re: building GNU gettext on AIX

[Arsen Arsenović]

Richard Biener  writes:

> On Fri, Nov 17, 2023 at 12:13 AM Arsen Arsenović  wrote:
>>
>>
>> Bruno Haible  writes:
>>
>> > Hi David,
>> >
>> >> the default, distributed libintl library will not allow GCC to be built
>> >> with NLS enabled.
>> >
>> > The problem is this configure test from gettext.m4
>> >
>> >   checking for GNU gettext in libintl... no
>> >
>> > It should say
>> >
>> >   checking for GNU gettext in libintl... yes
>> >
>> > I reproduce it with simple hello-world package, outside GCC.
>> >
>> > It tests whether a program that uses gettext() can be linked with
>> >   -lintl -liconv
>> > But now, on AIX, it needs to test whether such a program can be linked with
>> >   -lintl -liconv -lpthread
>> >
>> >> Were you suggesting that --enable-threads=isoc would work now or that it
>> >> would require further changes for a future release?
>> >
>> > It requires a change, effectively to do as if HAVE_PTHREAD_API is undefined
>> > if --enable-threads=isoc was provided.
>> >
>> > I can prepare a new gettext release that has both issues fixed:
>> >   - gettext.m4 that fixes the configure test and sets the variable LIBINTL
>> > to "-Lsome/libdir -lintl -liconv -lpthread",
>> >   - mbrtowc.o and setlocale*.o that use mtx_* locks instead of pthread_*
>> > mutexes when requested.
>> >
>> > But you then need to make up your mind w.r.t. what I wrote in the earlier
>> > mail.
>> >
>> >   * GCC can pass --enable-threads=isoc, to avoid the libpthread dependency
>> > on AIX ≥ 7.2.
>>
>> Hmm, would that option work everywhere, though?  Or would we have to
>> wire up configury to detect which flag to use?  If so, what would it
>> look like.
>
> I'd highly recommend the in-tree gettext to be built with thread
> support _disabled_,
> only a static library built and never installed (but linked statically
> into the host
> binaries).
>
> So if --disable-threads (or how it is called) does not work you need
> to fix _that_.
> Likewise if --disable-shared doesn't work correctly on all platforms
> you have to fix
> that as well.

That seems like the best choice to me too (and I'm testing a patch that
does that currently).

> Richard.
>
>>
>> >   * Or GCC can (continue to?) use the variable LIBINTL. This will work on
>>
>> If you mean the one generated by gettext.m4/uninstalled-config.sh, it is
>> utilized today:
>>
>> LIBS = @LIBS@ libcommon.a $(CPPLIB) $(LIBINTL) $(LIBICONV) $(LIBBACKTRACE) \
>> $(LIBIBERTY) $(LIBDECNUMBER) $(HOST_LIBS)
>>
>> (from gcc/Makefile.in)
>>
>>
>> > AIX 7.1 as well but the programs will then be linked against 
>> > libpthread.
>> > One additional library.
>> > $ ldd gcc
>> > /opt/freeware/bin/gcc needs:
>> >  /usr/lib/libc.a(shr.o)
>> >  /opt/freeware/lib/libiconv.a(libiconv.so.2)
>> >  /usr/lib/libc.a(_shr.o)
>> >  /unix
>> >  /usr/lib/libcrypt.a(shr.o)
>> >  /opt/freeware/lib/libgcc_s.a(shr.o)
>> > libpthread.a will be added to this list.
>> >
>> > ibm-clang links against libpthread.a as well:
>> > $ ldd /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig
>> > /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig needs:
>> >  /usr/lib/libpthreads.a(shr_xpg5_64.o)
>> >  /usr/opt/zlibNX/lib/libz.a(libz.so.1)
>> >  /usr/lib/libcurses.a(shr42_64.o)
>> >  /usr/lib/libiconv.a(shr4_64.o)
>> >  /usr/lib/libc++.a(shr_64.o)
>> >  /usr/lib/libc++abi.a(libc++abi.so.1)
>> >  /usr/lib/libc.a(shr_64.o)
>> >  /usr/lib/libpthreads.a(_shr_xpg5_64.o)
>> >  /usr/lib/libc++.a(libc++.so.1)
>> >  /usr/lib/libunwind.a(libunwind.so.1)
>> >  /usr/lib/libc.a(_shr_64.o)
>> >  /unix
>> >  /usr/lib/libcrypt.a(shr_64.o)
>>
>> David, I'll leave that decision up to you.
>>
>> > Bruno
>>
>>
>> --
>> Arsen Arsenović


-- 
Arsen Arsenović


signature.asc
Description: PGP signature

Re: building GNU gettext on AIX

[Richard Biener]

On Fri, Nov 17, 2023 at 12:13 AM Arsen Arsenović  wrote:
>
>
> Bruno Haible  writes:
>
> > Hi David,
> >
> >> the default, distributed libintl library will not allow GCC to be built
> >> with NLS enabled.
> >
> > The problem is this configure test from gettext.m4
> >
> >   checking for GNU gettext in libintl... no
> >
> > It should say
> >
> >   checking for GNU gettext in libintl... yes
> >
> > I reproduce it with simple hello-world package, outside GCC.
> >
> > It tests whether a program that uses gettext() can be linked with
> >   -lintl -liconv
> > But now, on AIX, it needs to test whether such a program can be linked with
> >   -lintl -liconv -lpthread
> >
> >> Were you suggesting that --enable-threads=isoc would work now or that it
> >> would require further changes for a future release?
> >
> > It requires a change, effectively to do as if HAVE_PTHREAD_API is undefined
> > if --enable-threads=isoc was provided.
> >
> > I can prepare a new gettext release that has both issues fixed:
> >   - gettext.m4 that fixes the configure test and sets the variable LIBINTL
> > to "-Lsome/libdir -lintl -liconv -lpthread",
> >   - mbrtowc.o and setlocale*.o that use mtx_* locks instead of pthread_*
> > mutexes when requested.
> >
> > But you then need to make up your mind w.r.t. what I wrote in the earlier
> > mail.
> >
> >   * GCC can pass --enable-threads=isoc, to avoid the libpthread dependency
> > on AIX ≥ 7.2.
>
> Hmm, would that option work everywhere, though?  Or would we have to
> wire up configury to detect which flag to use?  If so, what would it
> look like.

I'd highly recommend the in-tree gettext to be built with thread
support _disabled_,
only a static library built and never installed (but linked statically
into the host
binaries).

So if --disable-threads (or how it is called) does not work you need
to fix _that_.
Likewise if --disable-shared doesn't work correctly on all platforms
you have to fix
that as well.

Richard.

>
> >   * Or GCC can (continue to?) use the variable LIBINTL. This will work on
>
> If you mean the one generated by gettext.m4/uninstalled-config.sh, it is
> utilized today:
>
> LIBS = @LIBS@ libcommon.a $(CPPLIB) $(LIBINTL) $(LIBICONV) $(LIBBACKTRACE) \
> $(LIBIBERTY) $(LIBDECNUMBER) $(HOST_LIBS)
>
> (from gcc/Makefile.in)
>
>
> > AIX 7.1 as well but the programs will then be linked against libpthread.
> > One additional library.
> > $ ldd gcc
> > /opt/freeware/bin/gcc needs:
> >  /usr/lib/libc.a(shr.o)
> >  /opt/freeware/lib/libiconv.a(libiconv.so.2)
> >  /usr/lib/libc.a(_shr.o)
> >  /unix
> >  /usr/lib/libcrypt.a(shr.o)
> >  /opt/freeware/lib/libgcc_s.a(shr.o)
> > libpthread.a will be added to this list.
> >
> > ibm-clang links against libpthread.a as well:
> > $ ldd /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig
> > /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig needs:
> >  /usr/lib/libpthreads.a(shr_xpg5_64.o)
> >  /usr/opt/zlibNX/lib/libz.a(libz.so.1)
> >  /usr/lib/libcurses.a(shr42_64.o)
> >  /usr/lib/libiconv.a(shr4_64.o)
> >  /usr/lib/libc++.a(shr_64.o)
> >  /usr/lib/libc++abi.a(libc++abi.so.1)
> >  /usr/lib/libc.a(shr_64.o)
> >  /usr/lib/libpthreads.a(_shr_xpg5_64.o)
> >  /usr/lib/libc++.a(libc++.so.1)
> >  /usr/lib/libunwind.a(libunwind.so.1)
> >  /usr/lib/libc.a(_shr_64.o)
> >  /unix
> >  /usr/lib/libcrypt.a(shr_64.o)
>
> David, I'll leave that decision up to you.
>
> > Bruno
>
>
> --
> Arsen Arsenović

Re: building GNU gettext on AIX

[David Edelsohn]

On Thu, Nov 16, 2023 at 7:07 PM Bruno Haible  wrote:

> David Edelsohn wrote:
> > > ibm-clang links against libpthread.a as well:
> > > $ ldd /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig
> > > /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig needs:
> > >  /usr/lib/libpthreads.a(shr_xpg5_64.o)
> > >  /usr/opt/zlibNX/lib/libz.a(libz.so.1)
> > >  /usr/lib/libcurses.a(shr42_64.o)
> > >  /usr/lib/libiconv.a(shr4_64.o)
> > >  /usr/lib/libc++.a(shr_64.o)
> > >  /usr/lib/libc++abi.a(libc++abi.so.1)
> > >  /usr/lib/libc.a(shr_64.o)
> > >  /usr/lib/libpthreads.a(_shr_xpg5_64.o)
> > >  /usr/lib/libc++.a(libc++.so.1)
> > >  /usr/lib/libunwind.a(libunwind.so.1)
> > >  /usr/lib/libc.a(_shr_64.o)
> > >  /unix
> > >  /usr/lib/libcrypt.a(shr_64.o)
> > >
> >
> > I have asked the IBM Clang team why ibm-clang depends on libpthreads.
>
> The reason is that
>   - For a library, it is a normal expectation nowadays that it is
> multithread-safe.
>   - Making a library multithread-safe (without major hacks) means to do
> locking or to call pthread_once / call_once in some places.
>   - The ISO C 11 threading functions in libc have some drawbacks compared
> to the pthread functions. [1] So most developer prefer to rely on the
> POSIX threads API.
>   - Since AIX does not have the POSIX mutex functions in libc and does not
> support weak symbols like in ELF, this means a dependency to
> pthread_mutex_lock or pthread_once.
>   - Accordingly, in the list of libraries above, 3 libraries need pthread*
> symbols:
>
> $ nm -X 64 /usr/lib/libc++abi.a | grep ' U ' | grep pthread_mutex
> pthread_mutex_lock   U   -
> pthread_mutex_unlock U   -
> $ nm -X 64 /usr/lib/libc++.a | grep ' U ' | grep pthread_mutex
> pthread_mutex_destroy U   -
> pthread_mutex_init   U   -
> pthread_mutex_lock   U   -
> pthread_mutex_trylock U   -
> pthread_mutex_unlock U   -
> pthread_mutexattr_destroy U   -
> pthread_mutexattr_init U   -
> pthread_mutexattr_settype U   -
> $ nm -X 64 /usr/opt/zlibNX/lib/libz.a | grep ' U ' | grep pthread_mutex
> pthread_mutex_destroy U   -
> pthread_mutex_init   U   -
> pthread_mutex_lock   U   -
> pthread_mutex_unlock U   -
>

There are ibm_clang and ibm_clang_r (previous xlc and xlc_r) to compile
with and without thread safe.   If IBM Clang team
chose to only provide a thread safe version of libc++, okay, but that
doesn't seem like a fundamental requirement.
zlibNX is another can of worms.

David

Re: building GNU gettext on AIX

[Bruno Haible]

David Edelsohn wrote:
> > ibm-clang links against libpthread.a as well:
> > $ ldd /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig
> > /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig needs:
> >  /usr/lib/libpthreads.a(shr_xpg5_64.o)
> >  /usr/opt/zlibNX/lib/libz.a(libz.so.1)
> >  /usr/lib/libcurses.a(shr42_64.o)
> >  /usr/lib/libiconv.a(shr4_64.o)
> >  /usr/lib/libc++.a(shr_64.o)
> >  /usr/lib/libc++abi.a(libc++abi.so.1)
> >  /usr/lib/libc.a(shr_64.o)
> >  /usr/lib/libpthreads.a(_shr_xpg5_64.o)
> >  /usr/lib/libc++.a(libc++.so.1)
> >  /usr/lib/libunwind.a(libunwind.so.1)
> >  /usr/lib/libc.a(_shr_64.o)
> >  /unix
> >  /usr/lib/libcrypt.a(shr_64.o)
> >
> 
> I have asked the IBM Clang team why ibm-clang depends on libpthreads.

The reason is that
  - For a library, it is a normal expectation nowadays that it is
multithread-safe.
  - Making a library multithread-safe (without major hacks) means to do
locking or to call pthread_once / call_once in some places.
  - The ISO C 11 threading functions in libc have some drawbacks compared
to the pthread functions. [1] So most developer prefer to rely on the
POSIX threads API.
  - Since AIX does not have the POSIX mutex functions in libc and does not
support weak symbols like in ELF, this means a dependency to
pthread_mutex_lock or pthread_once.
  - Accordingly, in the list of libraries above, 3 libraries need pthread*
symbols:

$ nm -X 64 /usr/lib/libc++abi.a | grep ' U ' | grep pthread_mutex
pthread_mutex_lock   U   -
pthread_mutex_unlock U   -
$ nm -X 64 /usr/lib/libc++.a | grep ' U ' | grep pthread_mutex
pthread_mutex_destroy U   -
pthread_mutex_init   U   -
pthread_mutex_lock   U   -
pthread_mutex_trylock U   -
pthread_mutex_unlock U   -
pthread_mutexattr_destroy U   -
pthread_mutexattr_init U   -
pthread_mutexattr_settype U   -
$ nm -X 64 /usr/opt/zlibNX/lib/libz.a | grep ' U ' | grep pthread_mutex
pthread_mutex_destroy U   -
pthread_mutex_init   U   -
pthread_mutex_lock   U   -
pthread_mutex_unlock U   -

Bruno

[1] Lock initialization is clumsy. The return value of a thread is only an
'int', not a pointer. Etc.

Re: building GNU gettext on AIX

[David Edelsohn]

On Thu, Nov 16, 2023 at 5:47 PM Bruno Haible  wrote:

> Hi David,
>
> > the default, distributed libintl library will not allow GCC to be built
> > with NLS enabled.
>
> The problem is this configure test from gettext.m4
>
>   checking for GNU gettext in libintl... no
>
> It should say
>
>   checking for GNU gettext in libintl... yes
>
> I reproduce it with simple hello-world package, outside GCC.
>
> It tests whether a program that uses gettext() can be linked with
>   -lintl -liconv
> But now, on AIX, it needs to test whether such a program can be linked with
>   -lintl -liconv -lpthread
>
> > Were you suggesting that --enable-threads=isoc would work now or that it
> > would require further changes for a future release?
>
> It requires a change, effectively to do as if HAVE_PTHREAD_API is undefined
> if --enable-threads=isoc was provided.
>
> I can prepare a new gettext release that has both issues fixed:
>   - gettext.m4 that fixes the configure test and sets the variable LIBINTL
> to "-Lsome/libdir -lintl -liconv -lpthread",
>   - mbrtowc.o and setlocale*.o that use mtx_* locks instead of pthread_*
> mutexes when requested.
>
> But you then need to make up your mind w.r.t. what I wrote in the earlier
> mail.
>
>   * GCC can pass --enable-threads=isoc, to avoid the libpthread dependency
> on AIX ≥ 7.2.
>

I have reached out to the AIX Open Source Tools team for their
perspective.  Normally GCC and other
FOSS packages have not based their support for OS versions on official
vendor support lifecycles,
within reason.  In fact, many users have appreciated longer duration
support.


>
>   * Or GCC can (continue to?) use the variable LIBINTL. This will work on
> AIX 7.1 as well but the programs will then be linked against
> libpthread.
> One additional library.
> $ ldd gcc
> /opt/freeware/bin/gcc needs:
>  /usr/lib/libc.a(shr.o)
>  /opt/freeware/lib/libiconv.a(libiconv.so.2)
>  /usr/lib/libc.a(_shr.o)
>  /unix
>  /usr/lib/libcrypt.a(shr.o)
>  /opt/freeware/lib/libgcc_s.a(shr.o)
> libpthread.a will be added to this list.
>

My builds of GCC only rely on AIX libc.  All other libraries are statically
linked.


>
> ibm-clang links against libpthread.a as well:
> $ ldd /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig
> /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig needs:
>  /usr/lib/libpthreads.a(shr_xpg5_64.o)
>  /usr/opt/zlibNX/lib/libz.a(libz.so.1)
>  /usr/lib/libcurses.a(shr42_64.o)
>  /usr/lib/libiconv.a(shr4_64.o)
>  /usr/lib/libc++.a(shr_64.o)
>  /usr/lib/libc++abi.a(libc++abi.so.1)
>  /usr/lib/libc.a(shr_64.o)
>  /usr/lib/libpthreads.a(_shr_xpg5_64.o)
>  /usr/lib/libc++.a(libc++.so.1)
>  /usr/lib/libunwind.a(libunwind.so.1)
>  /usr/lib/libc.a(_shr_64.o)
>  /unix
>  /usr/lib/libcrypt.a(shr_64.o)
>

I have asked the IBM Clang team why ibm-clang depends on libpthreads.

One option is to add -lpthreads to the link line, even if the tools are not
built
thread-safe.  Only the intl support would invoke the extraneous overhead.
The downside is that other pthreads dependencies could sneak in.

Thanks, David


>
> Bruno
>
>
>
>

Re: building GNU gettext on AIX

[Arsen Arsenović]

Bruno Haible  writes:

> Hi David,
>
>> the default, distributed libintl library will not allow GCC to be built
>> with NLS enabled.
>
> The problem is this configure test from gettext.m4
>
>   checking for GNU gettext in libintl... no
>
> It should say
>
>   checking for GNU gettext in libintl... yes
>
> I reproduce it with simple hello-world package, outside GCC.
>
> It tests whether a program that uses gettext() can be linked with
>   -lintl -liconv
> But now, on AIX, it needs to test whether such a program can be linked with
>   -lintl -liconv -lpthread
>
>> Were you suggesting that --enable-threads=isoc would work now or that it
>> would require further changes for a future release?
>
> It requires a change, effectively to do as if HAVE_PTHREAD_API is undefined
> if --enable-threads=isoc was provided.
>
> I can prepare a new gettext release that has both issues fixed:
>   - gettext.m4 that fixes the configure test and sets the variable LIBINTL
> to "-Lsome/libdir -lintl -liconv -lpthread",
>   - mbrtowc.o and setlocale*.o that use mtx_* locks instead of pthread_*
> mutexes when requested.
>
> But you then need to make up your mind w.r.t. what I wrote in the earlier
> mail.
>
>   * GCC can pass --enable-threads=isoc, to avoid the libpthread dependency
> on AIX ≥ 7.2.

Hmm, would that option work everywhere, though?  Or would we have to
wire up configury to detect which flag to use?  If so, what would it
look like.

>   * Or GCC can (continue to?) use the variable LIBINTL. This will work on

If you mean the one generated by gettext.m4/uninstalled-config.sh, it is
utilized today:

LIBS = @LIBS@ libcommon.a $(CPPLIB) $(LIBINTL) $(LIBICONV) $(LIBBACKTRACE) \
$(LIBIBERTY) $(LIBDECNUMBER) $(HOST_LIBS)

(from gcc/Makefile.in)


> AIX 7.1 as well but the programs will then be linked against libpthread.
> One additional library.
> $ ldd gcc
> /opt/freeware/bin/gcc needs:
>  /usr/lib/libc.a(shr.o)
>  /opt/freeware/lib/libiconv.a(libiconv.so.2)
>  /usr/lib/libc.a(_shr.o)
>  /unix
>  /usr/lib/libcrypt.a(shr.o)
>  /opt/freeware/lib/libgcc_s.a(shr.o)
> libpthread.a will be added to this list.
>
> ibm-clang links against libpthread.a as well:
> $ ldd /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig
> /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig needs:
>  /usr/lib/libpthreads.a(shr_xpg5_64.o)
>  /usr/opt/zlibNX/lib/libz.a(libz.so.1)
>  /usr/lib/libcurses.a(shr42_64.o)
>  /usr/lib/libiconv.a(shr4_64.o)
>  /usr/lib/libc++.a(shr_64.o)
>  /usr/lib/libc++abi.a(libc++abi.so.1)
>  /usr/lib/libc.a(shr_64.o)
>  /usr/lib/libpthreads.a(_shr_xpg5_64.o)
>  /usr/lib/libc++.a(libc++.so.1)
>  /usr/lib/libunwind.a(libunwind.so.1)
>  /usr/lib/libc.a(_shr_64.o)
>  /unix
>  /usr/lib/libcrypt.a(shr_64.o)

David, I'll leave that decision up to you.

> Bruno


-- 
Arsen Arsenović


signature.asc
Description: PGP signature

Re: building GNU gettext on AIX

[Bruno Haible]

Hi David,

> the default, distributed libintl library will not allow GCC to be built
> with NLS enabled.

The problem is this configure test from gettext.m4

  checking for GNU gettext in libintl... no

It should say

  checking for GNU gettext in libintl... yes

I reproduce it with simple hello-world package, outside GCC.

It tests whether a program that uses gettext() can be linked with
  -lintl -liconv
But now, on AIX, it needs to test whether such a program can be linked with
  -lintl -liconv -lpthread

> Were you suggesting that --enable-threads=isoc would work now or that it
> would require further changes for a future release?

It requires a change, effectively to do as if HAVE_PTHREAD_API is undefined
if --enable-threads=isoc was provided.

I can prepare a new gettext release that has both issues fixed:
  - gettext.m4 that fixes the configure test and sets the variable LIBINTL
to "-Lsome/libdir -lintl -liconv -lpthread",
  - mbrtowc.o and setlocale*.o that use mtx_* locks instead of pthread_*
mutexes when requested.

But you then need to make up your mind w.r.t. what I wrote in the earlier
mail.

  * GCC can pass --enable-threads=isoc, to avoid the libpthread dependency
on AIX ≥ 7.2.

  * Or GCC can (continue to?) use the variable LIBINTL. This will work on
AIX 7.1 as well but the programs will then be linked against libpthread.
One additional library.
$ ldd gcc
/opt/freeware/bin/gcc needs:
 /usr/lib/libc.a(shr.o)
 /opt/freeware/lib/libiconv.a(libiconv.so.2)
 /usr/lib/libc.a(_shr.o)
 /unix
 /usr/lib/libcrypt.a(shr.o)
 /opt/freeware/lib/libgcc_s.a(shr.o)
libpthread.a will be added to this list.

ibm-clang links against libpthread.a as well:
$ ldd /opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig
/opt/IBM/openxlC/17.1.1/bin/.ibm-clang.orig needs:
 /usr/lib/libpthreads.a(shr_xpg5_64.o)
 /usr/opt/zlibNX/lib/libz.a(libz.so.1)
 /usr/lib/libcurses.a(shr42_64.o)
 /usr/lib/libiconv.a(shr4_64.o)
 /usr/lib/libc++.a(shr_64.o)
 /usr/lib/libc++abi.a(libc++abi.so.1)
 /usr/lib/libc.a(shr_64.o)
 /usr/lib/libpthreads.a(_shr_xpg5_64.o)
 /usr/lib/libc++.a(libc++.so.1)
 /usr/lib/libunwind.a(libunwind.so.1)
 /usr/lib/libc.a(_shr_64.o)
 /unix
 /usr/lib/libcrypt.a(shr_64.o)

Bruno

Re: building GNU gettext on AIX

[David Edelsohn]

On Thu, Nov 16, 2023 at 1:52 PM Bruno Haible  wrote:

> David Edelsohn wrote:
> > I manually commented out HAVE_PTHREAD_API from config.h and produced a
> > libintl.a without references to pthreads.
>
> Good finding!
>
> Commenting out HAVE_PTHREAD_API from config.h is also what makes the
> option --enable-threads=isoc work as expected on AIX 7.3.
>

I reconfigured and built gettext with --enable-threads=isoc .  libintl.a
still contains references to pthread_mutex and friends:

$ nm -BCpg libintl.a  | grep pthread

 - U __n_pthreads

 - U .pthread_mutex_lock

 - U .pthread_mutex_unlock

 - U .pthread_mutex_lock

 - U .pthread_mutex_unlock
 - U __n_pthreads

from files mbrtowc, setlocale_null, and vasnwprintf.

I tested on an AIX 7.2.5 system and confirmed that libc does provide the
mtx_ symbols:

$ nm -BCpg libc.a | grep mtx_

 0 T .mtx_timedlock

   160 T .mtx_unlock

   256 T .mtx_trylock

   416 T .mtx_lock

   512 T .mtx_init

   736 T .mtx_destroy

80 D mtx_timedlock

92 D mtx_unlock

   104 D mtx_trylock

   116 D mtx_lock

   128 D mtx_init

   140 D mtx_destroy


Were you suggesting that --enable-threads=isoc would work now or that it
would require further changes for a future release?


At the moment, configuring gettext with --disable-threads and manually
modifying config.h is the only method that produces

libintl.a without references to pthreads allowing GCC to build on AIX with
NLS enabled.


Thanks, David

Re: building GNU gettext on AIX

[Arsen Arsenović]

Bruno Haible  writes:

> Arsen Arsenović wrote:
>> >   * If yes, then the question is how distributors will in general package
>> > libintl on AIX. If it's installed in public locations (such as in
>> > /opt/freeware/{lib,lib64}/libintl.a on gcc119.fsffrance.org), then we
>> > have a problem: It may cause undefined behaviour in multithreaded
>> > packages that use GNU libintl.
>> > If you can guarantee that it will be installed in GCC-private 
>> > directories
>> > (and outside the path where GCC looks for libraries to link with!) then
>> > it would be OK to install such a non-thread-safe libintl.
>> > But if you cannot guarantee that, we are in trouble.
>> 
>> The in-tree configuration already passes --disable-shared, so I imagine
>> passing --disable-threads would be OK too, for the case that it is
>> utilized.  (relevant for the latter case: GCC-private build of libintl)
>
> Yeah, but this affects only those people who use the in-tree build of
> the libraries.
>
> The problem for distributors remains the same: They have a strong tendency
> of building libraries indepently, with --enable-shared (so that they can
> easily apply fixes without rebuilding the world). These distributors on AIX
> would notice that the GCC configuration attempts to link with "-lintl"
> but not with "-lintl -pthread" and thus the configuration detects that NLS
> is not usable.
>
> Arsen: Where in the GCC tree is this part of the GCC configuration? Is it
> in some configure.ac owned by GCC, or does it come from gettext.m4 ?

See Makefile.def.  It specifies a host-gettext module that has the extra
flags set.

If the in-tree configuration is used, then the uninstalled-config.sh
gettext generates is used.  See config/gettext-sister.m4.

gettext.m4 is unaltered, but it is essentially only used when the
gettext in-tree source is not present (because, otherwise,
gettext-runtime generates uninstalled-config.sh even if it builds
nothing)

Hope that answers it.

> Bruno


-- 
Arsen Arsenović


signature.asc
Description: PGP signature

Re: building GNU gettext on AIX

[Bruno Haible]

Arsen Arsenović wrote:
> >   * If yes, then the question is how distributors will in general package
> > libintl on AIX. If it's installed in public locations (such as in
> > /opt/freeware/{lib,lib64}/libintl.a on gcc119.fsffrance.org), then we
> > have a problem: It may cause undefined behaviour in multithreaded
> > packages that use GNU libintl.
> > If you can guarantee that it will be installed in GCC-private 
> > directories
> > (and outside the path where GCC looks for libraries to link with!) then
> > it would be OK to install such a non-thread-safe libintl.
> > But if you cannot guarantee that, we are in trouble.
> 
> The in-tree configuration already passes --disable-shared, so I imagine
> passing --disable-threads would be OK too, for the case that it is
> utilized.  (relevant for the latter case: GCC-private build of libintl)

Yeah, but this affects only those people who use the in-tree build of
the libraries.

The problem for distributors remains the same: They have a strong tendency
of building libraries indepently, with --enable-shared (so that they can
easily apply fixes without rebuilding the world). These distributors on AIX
would notice that the GCC configuration attempts to link with "-lintl"
but not with "-lintl -pthread" and thus the configuration detects that NLS
is not usable.

Arsen: Where in the GCC tree is this part of the GCC configuration? Is it
in some configure.ac owned by GCC, or does it come from gettext.m4 ?

Bruno

Re: building GNU gettext on AIX

[Bruno Haible]

David Edelsohn wrote:
> I manually commented out HAVE_PTHREAD_API from config.h and produced a
> libintl.a without references to pthreads.

Good finding!

Commenting out HAVE_PTHREAD_API from config.h is also what makes the
option --enable-threads=isoc work as expected on AIX 7.3.

Bruno

Re: building GNU gettext on AIX

[Arsen Arsenović]

Bruno Haible  writes:

> David Edelsohn wrote:
>> > It is great that gettext and libintl can be built thread-safe, but GCC
>> > (cc1, gcov, etc.) are not pthreads applications and are not built with
>> > pthreads.  Because libintl defaults to pthreads enabled, NLS cannot
>> > function in GCC on AIX by default.  
>> ...
>> The latest issue is that a few files in gettext ignore --disable-pthreads
>> and creates a dependency on pthread_mutex.
>
> GNU gettext does not have an option '--disable-pthreads'. Instead, it has
> options
>
>   --enable-threads={isoc|posix|isoc+posix|windows}
>   specify multithreading API
>
>   --disable-threads   build without multithread safety
>
>> The issue appears to be that intl/gnulib-lib/{mbrtowc.c,setlocale_null.c}
>> include pthread.h based on HAVE_PTHREAD_API, which is defined as 1 in
>> intl/config.h build directory
>
> Yup, I confirm that the dependency comes from these two object files.
>
> Will the next GCC release support AIX 7.1.x ? Recall that AIX 7.1 went
> end-of-life on 2023-04-30 [1]
>
>   * If no, then the simple solution would be to pass the configure option
>   --enable-threads=isoc
> This should not introduce a link dependency, because the mtx_lock,
> mtx_unlock, and mtx_init functions are in libc in AIX ≥ 7.2. Currently it
> does not work (it still uses pthread_mutex_lock and pthread_mutex_unlock
> despite --enable-threads=isoc). But I could make this work and release
> a gettext 0.22.4 with the fix.
>
>   * If yes, then the question is how distributors will in general package
> libintl on AIX. If it's installed in public locations (such as in
> /opt/freeware/{lib,lib64}/libintl.a on gcc119.fsffrance.org), then we
> have a problem: It may cause undefined behaviour in multithreaded
> packages that use GNU libintl.
> If you can guarantee that it will be installed in GCC-private directories
> (and outside the path where GCC looks for libraries to link with!) then
> it would be OK to install such a non-thread-safe libintl.
> But if you cannot guarantee that, we are in trouble.

The in-tree configuration already passes --disable-shared, so I imagine
passing --disable-threads would be OK too, for the case that it is
utilized.  (relevant for the latter case: GCC-private build of libintl)

> How do other library vendors handle this issue on AIX? Do they ship two
> libraries, one MT-safe and one not? Under different names? Or in different
> library search paths?
>
> Bruno
>
> [1] https://www.ibm.com/support/pages/aix-support-lifecycle-information


-- 
Arsen Arsenović


signature.asc
Description: PGP signature

Re: building GNU gettext on AIX

[Bruno Haible]

David Edelsohn wrote:
> > It is great that gettext and libintl can be built thread-safe, but GCC
> > (cc1, gcov, etc.) are not pthreads applications and are not built with
> > pthreads.  Because libintl defaults to pthreads enabled, NLS cannot
> > function in GCC on AIX by default.  
> ...
> The latest issue is that a few files in gettext ignore --disable-pthreads
> and creates a dependency on pthread_mutex.

GNU gettext does not have an option '--disable-pthreads'. Instead, it has
options

  --enable-threads={isoc|posix|isoc+posix|windows}
  specify multithreading API

  --disable-threads   build without multithread safety

> The issue appears to be that intl/gnulib-lib/{mbrtowc.c,setlocale_null.c}
> include pthread.h based on HAVE_PTHREAD_API, which is defined as 1 in
> intl/config.h build directory

Yup, I confirm that the dependency comes from these two object files.

Will the next GCC release support AIX 7.1.x ? Recall that AIX 7.1 went
end-of-life on 2023-04-30 [1]

  * If no, then the simple solution would be to pass the configure option
  --enable-threads=isoc
This should not introduce a link dependency, because the mtx_lock,
mtx_unlock, and mtx_init functions are in libc in AIX ≥ 7.2. Currently it
does not work (it still uses pthread_mutex_lock and pthread_mutex_unlock
despite --enable-threads=isoc). But I could make this work and release
a gettext 0.22.4 with the fix.

  * If yes, then the question is how distributors will in general package
libintl on AIX. If it's installed in public locations (such as in
/opt/freeware/{lib,lib64}/libintl.a on gcc119.fsffrance.org), then we
have a problem: It may cause undefined behaviour in multithreaded
packages that use GNU libintl.
If you can guarantee that it will be installed in GCC-private directories
(and outside the path where GCC looks for libraries to link with!) then
it would be OK to install such a non-thread-safe libintl.
But if you cannot guarantee that, we are in trouble.

How do other library vendors handle this issue on AIX? Do they ship two
libraries, one MT-safe and one not? Under different names? Or in different
library search paths?

Bruno

[1] https://www.ibm.com/support/pages/aix-support-lifecycle-information

Re: building GNU gettext on AIX

[David Edelsohn]

I manually commented out HAVE_PTHREAD_API from config.h and produced a
libintl.a without references to pthreads.  Configuring GCC with that custom
libintl.a enables NLS.  I now am building GCC with NLS and we will see how
well it functions.

gettext depends on pthreads by default and the versions distributed.

Thanks, David


On Thu, Nov 16, 2023 at 1:01 PM David Edelsohn  wrote:

> Bruno,
>
> The issue appears to be that intl/gnulib-lib/{mbrtowc.c,setlocale_null.c}
> include pthread.h based on HAVE_PTHREAD_API, which is defined as 1 in
> intl/config.h build directory despite requesting --disable-pthreads.
>
> Thanks, David
>
> On Thu, Nov 16, 2023 at 11:35 AM David Edelsohn  wrote:
>
>> I configured gettext with --disable-pthreads and libintl.a still contains
>> references to pthread_mutex_lock and pthread_mutex_unlock, which causes NLS
>> configure to fail on AIX.
>>
>> How can this be corrected?
>>
>> Thanks, David
>>
>> libintl.a[libgnu_la-mbrtowc.o]:
>>
>>  - U __lc_charmap
>>
>>  - U errno
>>
>>  - U .locale_encoding_classification
>>
>>  - U .gl_get_mbtowc_lock
>>
>>  - U .pthread_mutex_lock
>>
>>  - U .mbtowc
>>
>>  - U .pthread_mutex_unlock
>>
>>  - U .abort
>>
>>  0 T ._libintl_mbrtowc
>>
>>   1952 D _libintl_mbrtowc
>>
>> libintl.a[libgnu_la-setlocale_null.o]:
>>
>>  - U .gl_get_setlocale_null_lock
>>
>>  - U .pthread_mutex_lock
>>
>>  - U .setlocale
>>
>>  - U .strlen
>>
>>  - U .memcpy
>>
>>  - U .pthread_mutex_unlock
>>
>>  - U .abort
>>
>>  - U .strcpy
>>
>>336 T ._libintl_setlocale_null_r
>>
>>400 T ._libintl_setlocale_null
>>
>>812 D _libintl_setlocale_null_r
>>
>>824 D _libintl_setlocale_null
>>
>> On Thu, Nov 16, 2023 at 11:00 AM David Edelsohn 
>> wrote:
>>
>>> Bruno,
>>>
>>> I have been able to tweak the environment and build gettext and
>>> libintl.  With the updated libintl and environment, GCC reliably does not
>>> use NLS.
>>>
>>> The issue is that libintl utilizes pthreads.  AIX does not provide no-op
>>> pthread stubs in libc.  pthreads is an explicit multilib on AIX.
>>>
>>> It is great that gettext and libintl can be built thread-safe, but GCC
>>> (cc1, gcov, etc.) are not pthreads applications and are not built with
>>> pthreads.  Because libintl defaults to pthreads enabled, NLS cannot
>>> function in GCC on AIX by default.  The GCC included gettext was built in
>>> the default for GCC libraries, which was not pthreads enabled.
>>>
>>> I can rebuild libintl with --disable-pthreads and I will see if that
>>> works, but the default, distributed libintl library will not allow GCC to
>>> be built with NLS enabled.  And, no, GCC on AIX should not be forced to
>>> build with pthreads.
>>>
>>> This is a regression in NLS support in GCC.
>>>
>>> Thanks, David
>>>
>>>
>>> On Wed, Nov 15, 2023 at 5:39 PM Bruno Haible  wrote:
>>>
 David Edelsohn wrote:
 > I am using my own install of GCC for a reason.

 I have built GNU gettext 0.22.3 in various configurations on the AIX 7.1
 and 7.3 machines in the compilefarm, and haven't encountered issues with
 'max_align_t' nor with 'getpeername'. So, from my point of view, GNU
 gettext
 works fine on AIX with gcc and xlc (but not ibm-clang, which I haven't
 tested).

 You will surely understand that I cannot test a release against a
 compiler
 that exists only on your hard disk.

 The hint I gave you, based on the partial logs that you provided, is to
 look at the configure test for intmax_t first.

 Bruno

Re: building GNU gettext on AIX

[David Edelsohn]

Bruno,

The issue appears to be that intl/gnulib-lib/{mbrtowc.c,setlocale_null.c}
include pthread.h based on HAVE_PTHREAD_API, which is defined as 1 in
intl/config.h build directory despite requesting --disable-pthreads.

Thanks, David

On Thu, Nov 16, 2023 at 11:35 AM David Edelsohn  wrote:

> I configured gettext with --disable-pthreads and libintl.a still contains
> references to pthread_mutex_lock and pthread_mutex_unlock, which causes NLS
> configure to fail on AIX.
>
> How can this be corrected?
>
> Thanks, David
>
> libintl.a[libgnu_la-mbrtowc.o]:
>
>  - U __lc_charmap
>
>  - U errno
>
>  - U .locale_encoding_classification
>
>  - U .gl_get_mbtowc_lock
>
>  - U .pthread_mutex_lock
>
>  - U .mbtowc
>
>  - U .pthread_mutex_unlock
>
>  - U .abort
>
>  0 T ._libintl_mbrtowc
>
>   1952 D _libintl_mbrtowc
>
> libintl.a[libgnu_la-setlocale_null.o]:
>
>  - U .gl_get_setlocale_null_lock
>
>  - U .pthread_mutex_lock
>
>  - U .setlocale
>
>  - U .strlen
>
>  - U .memcpy
>
>  - U .pthread_mutex_unlock
>
>  - U .abort
>
>  - U .strcpy
>
>336 T ._libintl_setlocale_null_r
>
>400 T ._libintl_setlocale_null
>
>812 D _libintl_setlocale_null_r
>
>824 D _libintl_setlocale_null
>
> On Thu, Nov 16, 2023 at 11:00 AM David Edelsohn  wrote:
>
>> Bruno,
>>
>> I have been able to tweak the environment and build gettext and libintl.
>> With the updated libintl and environment, GCC reliably does not use NLS.
>>
>> The issue is that libintl utilizes pthreads.  AIX does not provide no-op
>> pthread stubs in libc.  pthreads is an explicit multilib on AIX.
>>
>> It is great that gettext and libintl can be built thread-safe, but GCC
>> (cc1, gcov, etc.) are not pthreads applications and are not built with
>> pthreads.  Because libintl defaults to pthreads enabled, NLS cannot
>> function in GCC on AIX by default.  The GCC included gettext was built in
>> the default for GCC libraries, which was not pthreads enabled.
>>
>> I can rebuild libintl with --disable-pthreads and I will see if that
>> works, but the default, distributed libintl library will not allow GCC to
>> be built with NLS enabled.  And, no, GCC on AIX should not be forced to
>> build with pthreads.
>>
>> This is a regression in NLS support in GCC.
>>
>> Thanks, David
>>
>>
>> On Wed, Nov 15, 2023 at 5:39 PM Bruno Haible  wrote:
>>
>>> David Edelsohn wrote:
>>> > I am using my own install of GCC for a reason.
>>>
>>> I have built GNU gettext 0.22.3 in various configurations on the AIX 7.1
>>> and 7.3 machines in the compilefarm, and haven't encountered issues with
>>> 'max_align_t' nor with 'getpeername'. So, from my point of view, GNU
>>> gettext
>>> works fine on AIX with gcc and xlc (but not ibm-clang, which I haven't
>>> tested).
>>>
>>> You will surely understand that I cannot test a release against a
>>> compiler
>>> that exists only on your hard disk.
>>>
>>> The hint I gave you, based on the partial logs that you provided, is to
>>> look at the configure test for intmax_t first.
>>>
>>> Bruno
>>>
>>>
>>>
>>>

Re: building GNU gettext on AIX

[David Edelsohn]

On Thu, Nov 16, 2023 at 11:58 AM Richard Biener 
wrote:

>
>
> Am 16.11.2023 um 17:00 schrieb David Edelsohn :
>
> 
> Bruno,
>
> I have been able to tweak the environment and build gettext and libintl.
> With the updated libintl and environment, GCC reliably does not use NLS.
>
> The issue is that libintl utilizes pthreads.  AIX does not provide no-op
> pthread stubs in libc.  pthreads is an explicit multilib on AIX.
>
> It is great that gettext and libintl can be built thread-safe, but GCC
> (cc1, gcov, etc.) are not pthreads applications and are not built with
> pthreads.  Because libintl defaults to pthreads enabled, NLS cannot
> function in GCC on AIX by default.  The GCC included gettext was built in
> the default for GCC libraries, which was not pthreads enabled.
>
> I can rebuild libintl with --disable-pthreads and I will see if that
> works, but the default, distributed libintl library will not allow GCC to
> be built with NLS enabled.  And, no, GCC on AIX should not be forced to
> build with pthreads.
>
> This is a regression in NLS support in GCC.
>
>
> If that’s for the in-tree libintl we can arrange configure to pass down
> —disable-pthreads like we adjust configure args for gmp and friends as well.
>

The latest issue is that a few files in gettext ignore --disable-pthreads
and creates a dependency on pthread_mutex.

David



>
> Richard
>
> Thanks, David
>
>
> On Wed, Nov 15, 2023 at 5:39 PM Bruno Haible  wrote:
>
>> David Edelsohn wrote:
>> > I am using my own install of GCC for a reason.
>>
>> I have built GNU gettext 0.22.3 in various configurations on the AIX 7.1
>> and 7.3 machines in the compilefarm, and haven't encountered issues with
>> 'max_align_t' nor with 'getpeername'. So, from my point of view, GNU
>> gettext
>> works fine on AIX with gcc and xlc (but not ibm-clang, which I haven't
>> tested).
>>
>> You will surely understand that I cannot test a release against a compiler
>> that exists only on your hard disk.
>>
>> The hint I gave you, based on the partial logs that you provided, is to
>> look at the configure test for intmax_t first.
>>
>> Bruno
>>
>>
>>
>>

Re: building GNU gettext on AIX

[Richard Biener]

Am 16.11.2023 um 17:00 schrieb David Edelsohn :Bruno,I have been able to tweak the environment and build gettext and libintl.  With the updated libintl and environment, GCC reliably does not use NLS.The issue is that libintl utilizes pthreads.  AIX does not provide no-op pthread stubs in libc.  pthreads is an explicit multilib on AIX.It is great that gettext and libintl can be built thread-safe, but GCC (cc1, gcov, etc.) are not pthreads applications and are not built with pthreads.  Because libintl defaults to pthreads enabled, NLS cannot function in GCC on AIX by default.  The GCC included gettext was built in the default for GCC libraries, which was not pthreads enabled.I can rebuild libintl with --disable-pthreads and I will see if that works, but the default, distributed libintl library will not allow GCC to be built with NLS enabled.  And, no, GCC on AIX should not be forced to build with pthreads.This is a regression in NLS support in GCC.If that’s for the in-tree libintl we can arrange configure to pass down —disable-pthreads like we adjust configure args for gmp and friends as well.Richard Thanks, DavidOn Wed, Nov 15, 2023 at 5:39 PM Bruno Haible  wrote:David Edelsohn wrote:
> I am using my own install of GCC for a reason.

I have built GNU gettext 0.22.3 in various configurations on the AIX 7.1
and 7.3 machines in the compilefarm, and haven't encountered issues with
'max_align_t' nor with 'getpeername'. So, from my point of view, GNU gettext
works fine on AIX with gcc and xlc (but not ibm-clang, which I haven't
tested).

You will surely understand that I cannot test a release against a compiler
that exists only on your hard disk.

The hint I gave you, based on the partial logs that you provided, is to
look at the configure test for intmax_t first.

Bruno

Re: building GNU gettext on AIX

[David Edelsohn]

I configured gettext with --disable-pthreads and libintl.a still contains
references to pthread_mutex_lock and pthread_mutex_unlock, which causes NLS
configure to fail on AIX.

How can this be corrected?

Thanks, David

libintl.a[libgnu_la-mbrtowc.o]:

 - U __lc_charmap

 - U errno

 - U .locale_encoding_classification

 - U .gl_get_mbtowc_lock

 - U .pthread_mutex_lock

 - U .mbtowc

 - U .pthread_mutex_unlock

 - U .abort

 0 T ._libintl_mbrtowc

  1952 D _libintl_mbrtowc

libintl.a[libgnu_la-setlocale_null.o]:

 - U .gl_get_setlocale_null_lock

 - U .pthread_mutex_lock

 - U .setlocale

 - U .strlen

 - U .memcpy

 - U .pthread_mutex_unlock

 - U .abort

 - U .strcpy

   336 T ._libintl_setlocale_null_r

   400 T ._libintl_setlocale_null

   812 D _libintl_setlocale_null_r

   824 D _libintl_setlocale_null

On Thu, Nov 16, 2023 at 11:00 AM David Edelsohn  wrote:

> Bruno,
>
> I have been able to tweak the environment and build gettext and libintl.
> With the updated libintl and environment, GCC reliably does not use NLS.
>
> The issue is that libintl utilizes pthreads.  AIX does not provide no-op
> pthread stubs in libc.  pthreads is an explicit multilib on AIX.
>
> It is great that gettext and libintl can be built thread-safe, but GCC
> (cc1, gcov, etc.) are not pthreads applications and are not built with
> pthreads.  Because libintl defaults to pthreads enabled, NLS cannot
> function in GCC on AIX by default.  The GCC included gettext was built in
> the default for GCC libraries, which was not pthreads enabled.
>
> I can rebuild libintl with --disable-pthreads and I will see if that
> works, but the default, distributed libintl library will not allow GCC to
> be built with NLS enabled.  And, no, GCC on AIX should not be forced to
> build with pthreads.
>
> This is a regression in NLS support in GCC.
>
> Thanks, David
>
>
> On Wed, Nov 15, 2023 at 5:39 PM Bruno Haible  wrote:
>
>> David Edelsohn wrote:
>> > I am using my own install of GCC for a reason.
>>
>> I have built GNU gettext 0.22.3 in various configurations on the AIX 7.1
>> and 7.3 machines in the compilefarm, and haven't encountered issues with
>> 'max_align_t' nor with 'getpeername'. So, from my point of view, GNU
>> gettext
>> works fine on AIX with gcc and xlc (but not ibm-clang, which I haven't
>> tested).
>>
>> You will surely understand that I cannot test a release against a compiler
>> that exists only on your hard disk.
>>
>> The hint I gave you, based on the partial logs that you provided, is to
>> look at the configure test for intmax_t first.
>>
>> Bruno
>>
>>
>>
>>

Re: building GNU gettext on AIX

[David Edelsohn]

Bruno,

I have been able to tweak the environment and build gettext and libintl.
With the updated libintl and environment, GCC reliably does not use NLS.

The issue is that libintl utilizes pthreads.  AIX does not provide no-op
pthread stubs in libc.  pthreads is an explicit multilib on AIX.

It is great that gettext and libintl can be built thread-safe, but GCC
(cc1, gcov, etc.) are not pthreads applications and are not built with
pthreads.  Because libintl defaults to pthreads enabled, NLS cannot
function in GCC on AIX by default.  The GCC included gettext was built in
the default for GCC libraries, which was not pthreads enabled.

I can rebuild libintl with --disable-pthreads and I will see if that works,
but the default, distributed libintl library will not allow GCC to be built
with NLS enabled.  And, no, GCC on AIX should not be forced to build with
pthreads.

This is a regression in NLS support in GCC.

Thanks, David


On Wed, Nov 15, 2023 at 5:39 PM Bruno Haible  wrote:

> David Edelsohn wrote:
> > I am using my own install of GCC for a reason.
>
> I have built GNU gettext 0.22.3 in various configurations on the AIX 7.1
> and 7.3 machines in the compilefarm, and haven't encountered issues with
> 'max_align_t' nor with 'getpeername'. So, from my point of view, GNU
> gettext
> works fine on AIX with gcc and xlc (but not ibm-clang, which I haven't
> tested).
>
> You will surely understand that I cannot test a release against a compiler
> that exists only on your hard disk.
>
> The hint I gave you, based on the partial logs that you provided, is to
> look at the configure test for intmax_t first.
>
> Bruno
>
>
>
>

Re: building GNU gettext on AIX

[Bruno Haible]

David Edelsohn wrote:
> I am using my own install of GCC for a reason.

I have built GNU gettext 0.22.3 in various configurations on the AIX 7.1
and 7.3 machines in the compilefarm, and haven't encountered issues with
'max_align_t' nor with 'getpeername'. So, from my point of view, GNU gettext
works fine on AIX with gcc and xlc (but not ibm-clang, which I haven't
tested).

You will surely understand that I cannot test a release against a compiler
that exists only on your hard disk.

The hint I gave you, based on the partial logs that you provided, is to
look at the configure test for intmax_t first.

Bruno

Re: building GNU gettext on AIX

[David Edelsohn]

On Wed, Nov 15, 2023 at 4:22 PM Bruno Haible  wrote:

> David Edelsohn wrote:
> > When I try to configure gettext-0.22.3, I receive the following error:
> >
> > checking for socklen_t equivalent... configure: error: Cannot find a type
> > to use in place of socklen_t
> >
> > configure: error:
> > /nasfarm/edelsohn/src/gettext-0.22.3/libtextstyle/configure failed for
> > libtextstyle
> >
> >
> > configure:43943: /nasfarm/edelsohn/install/GCC12/bin/gcc -c -g -O2
> > -D_THREAD_SAFE
> > conftest.c >&5
> >
> > conftest.c:112:18: error: two or more data types in declaration
> specifiers
> >
> >   112 | #define intmax_t long long
> >
> >   |  ^~~~
> >
> > conftest.c:112:23: error: two or more data types in declaration
> specifiers
> >
> >   112 | #define intmax_t long long
> >
> >   |   ^~~~
> >
> > In file included from conftest.c:212:
> >
> > conftest.c:214:24: error: conflicting types for 'ngetpeername'; have
> > 'int(int,  void *, long unsigned int *)'
> >
> >   214 |int getpeername (int, void *, unsigned long
> int
> > *);
> >
> >   |^~~
> >
> >
> /nasfarm/edelsohn/install/GCC12/lib/gcc/powerpc-ibm-aix7.2.5.0/12.1.1/include-fixed/sys/socket.h:647:9:
> > note: previous declaration of 'ngetpeername' with type 'int(int,  struct
> > sockaddr * restrict,  socklen_t * restrict)' {aka 'int(int,  struct
> > sockaddr * restrict,  long unsigned int * restrict)'}
> >
> >   647 | int getpeername(int, struct sockaddr *__restrict__, socklen_t
> > *__restrict__);
> >
> >   | ^~~
> >
> >
> > configure and config.h seems to get itself confused about types.
>
> There seem to be two problems, both related to the include files of
> your compiler:
>
>   - The configure test "checking for intmax_t..." must have found the
> answer "no". But on a modern system,  should be defining
> intmax_t already.
>
>   - This configure test that tries to find the getpeername declaration,
> but cannot find it (maybe because of the first problem?):
>
>
> 
>  for arg2 in "struct sockaddr" void; do
>for t in int size_t "unsigned int" "long int" "unsigned long
> int"; do
>  cat confdefs.h - <<_ACEOF >conftest.$ac_ext
> /* end confdefs.h.  */
> #include 
>#include 
>
>int getpeername (int, $arg2 *, $t *);
> int
> main (void)
> {
> $t len;
>   getpeername (0, 0, );
>   ;
>   return 0;
> }
> _ACEOF
> if ac_fn_c_try_compile "$LINENO"
> then :
>   gl_cv_socklen_t_equiv="$t"
> fi
>
> 
>
> I would concentrate on the first problem. If you don't get it fixed, then
> I'd
> suggest to try 'gcc' from the AIX Toolbox [1] or 'xlc' (as an IBM product)
> instead of 'gcc' (that looks like you built it yourself).
>
> Bruno
>
> [1]
> https://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/SPECS/gcc12-12.3.0-1.spec


Bruno,

I am using my own install of GCC for a reason.  The build of GCC works for
everything else, including bootstrap of GCC, GDB, GMP, etc.  The only
problem is gettext.

Thanks, David

Re: building GNU gettext on AIX

[Bruno Haible]

David Edelsohn wrote:
> When I try to configure gettext-0.22.3, I receive the following error:
> 
> checking for socklen_t equivalent... configure: error: Cannot find a type
> to use in place of socklen_t
> 
> configure: error:
> /nasfarm/edelsohn/src/gettext-0.22.3/libtextstyle/configure failed for
> libtextstyle
> 
> 
> configure:43943: /nasfarm/edelsohn/install/GCC12/bin/gcc -c -g -O2
> -D_THREAD_SAFE
> conftest.c >&5
> 
> conftest.c:112:18: error: two or more data types in declaration specifiers
> 
>   112 | #define intmax_t long long
> 
>   |  ^~~~
> 
> conftest.c:112:23: error: two or more data types in declaration specifiers
> 
>   112 | #define intmax_t long long
> 
>   |   ^~~~
> 
> In file included from conftest.c:212:
> 
> conftest.c:214:24: error: conflicting types for 'ngetpeername'; have
> 'int(int,  void *, long unsigned int *)'
> 
>   214 |int getpeername (int, void *, unsigned long int
> *);
> 
>   |^~~
> 
> /nasfarm/edelsohn/install/GCC12/lib/gcc/powerpc-ibm-aix7.2.5.0/12.1.1/include-fixed/sys/socket.h:647:9:
> note: previous declaration of 'ngetpeername' with type 'int(int,  struct
> sockaddr * restrict,  socklen_t * restrict)' {aka 'int(int,  struct
> sockaddr * restrict,  long unsigned int * restrict)'}
> 
>   647 | int getpeername(int, struct sockaddr *__restrict__, socklen_t
> *__restrict__);
> 
>   | ^~~
> 
> 
> configure and config.h seems to get itself confused about types.

There seem to be two problems, both related to the include files of
your compiler:

  - The configure test "checking for intmax_t..." must have found the
answer "no". But on a modern system,  should be defining
intmax_t already.

  - This configure test that tries to find the getpeername declaration,
but cannot find it (maybe because of the first problem?):


 for arg2 in "struct sockaddr" void; do
   for t in int size_t "unsigned int" "long int" "unsigned long int"; do
 cat confdefs.h - <<_ACEOF >conftest.$ac_ext
/* end confdefs.h.  */
#include 
   #include 

   int getpeername (int, $arg2 *, $t *);
int
main (void)
{
$t len;
  getpeername (0, 0, );
  ;
  return 0;
}
_ACEOF
if ac_fn_c_try_compile "$LINENO"
then :
  gl_cv_socklen_t_equiv="$t"
fi


I would concentrate on the first problem. If you don't get it fixed, then I'd
suggest to try 'gcc' from the AIX Toolbox [1] or 'xlc' (as an IBM product)
instead of 'gcc' (that looks like you built it yourself).

Bruno

[1] 
https://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/SPECS/gcc12-12.3.0-1.spec

Re: building GNU gettext on AIX

[David Edelsohn]

When I try to configure gettext-0.22.3, I receive the following error:

checking for socklen_t equivalent... configure: error: Cannot find a type
to use in place of socklen_t

configure: error:
/nasfarm/edelsohn/src/gettext-0.22.3/libtextstyle/configure failed for
libtextstyle


configure:43943: /nasfarm/edelsohn/install/GCC12/bin/gcc -c -g -O2
-D_THREAD_SAFE
conftest.c >&5

conftest.c:112:18: error: two or more data types in declaration specifiers

  112 | #define intmax_t long long

  |  ^~~~

conftest.c:112:23: error: two or more data types in declaration specifiers

  112 | #define intmax_t long long

  |   ^~~~

In file included from conftest.c:212:

conftest.c:214:24: error: conflicting types for 'ngetpeername'; have
'int(int,  void *, long unsigned int *)'

  214 |int getpeername (int, void *, unsigned long int
*);

  |^~~

/nasfarm/edelsohn/install/GCC12/lib/gcc/powerpc-ibm-aix7.2.5.0/12.1.1/include-fixed/sys/socket.h:647:9:
note: previous declaration of 'ngetpeername' with type 'int(int,  struct
sockaddr * restrict,  socklen_t * restrict)' {aka 'int(int,  struct
sockaddr * restrict,  long unsigned int * restrict)'}

  647 | int getpeername(int, struct sockaddr *__restrict__, socklen_t
*__restrict__);

  | ^~~


configure and config.h seems to get itself confused about types.


David



On Wed, Nov 15, 2023 at 7:29 AM Bruno Haible  wrote:

> [CCing bug-gettext]
>
> David Edelsohn wrote in
> :
> > The current gettext-0.22.3 fails to build for me on AIX.
>
> Here are some hints to get a successful build of GNU gettext on AIX:
>
> 1. Set the recommended environment variables before running configure:
>https://gitlab.com/ghwiki/gnow-how/-/wikis/Platforms/Configuration
>
>Namely:
>* for a 32-bit build with gcc:
>  CC=gcc
>  CXX=g++
>  CPPFLAGS="-I$PREFIX/include"
>  LDFLAGS="-L$PREFIX/lib"
>  unset AR NM
>* for a 32-bit build with xlc:
>  CC="xlc -qthreaded -qtls"
>  CXX="xlC -qthreaded -qtls"
>  CPPFLAGS="-I$PREFIX/include"
>  LDFLAGS="-L$PREFIX/lib"
>  unset AR NM
>* for a 64-bit build with gcc:
>  CC="gcc -maix64"
>  CXX="g++ -maix64"
>  CPPFLAGS="-I$PREFIX/include"
>  LDFLAGS="-L$PREFIX/lib"
>  AR="ar -X 64"; NM="nm -X 64 -B"
>* for a 64-bit build with xlc:
>  CC="xlc -q64 -qthreaded -qtls"
>  CXX="xlC -q64 -qthreaded -qtls"
>  CPPFLAGS="-I$PREFIX/include"
>  LDFLAGS="-L$PREFIX/lib"
>  AR="ar -X 64"; NM="nm -X 64 -B"
>
>where $PREFIX is the value that you pass to the --prefix configure
> option.
>
>Rationale: you can run into all sorts of problems if you choose compiler
>options at random and haven't experience with compiler options on that
>platform.
>
> 2. Don't use ibm-clang.
>
>Rationale: It's broken.
>
> 3. Don't use -Wall with gcc 10.3.
>
>Rationale: If you specify -Wall, gettext's configure adds -fanalyzer,
> which
>has excessive memory requirements in gcc 10.x. In particular, on AIX, it
>makes cc1 crash while compiling regex.c after it has consumed 1 GiB of
> RAM.
>
> 4. Avoid using a --prefix that contains earlier installations of the same
>package.
>
>Rationale: Because the AIX linker hardcodes directory names in shared
>libraries, GNU libtool has a peculiar configuration on AIX. It ends up
>mixing the in-build-tree libraries with the libraries in the install
>locations, leading to all sorts of errors.
>
>If you really need to use a --prefix that contains an earlier
>installation of the same package:
>  - Either use --disable-shared and remove libgettextlib.a and
>libgettextsrc.a from $PREFIX/lib before starting the build.
>  - Or use a mix of "make -k", "make -k install" and ad-hoc workarounds
>that cannot be described in a general way.
>
> Bruno
>
>
>
>

Re: building GNU gettext on AIX

[Bruno Haible]

[CCing bug-gettext]

David Edelsohn wrote in
:
> The current gettext-0.22.3 fails to build for me on AIX.

Here are some hints to get a successful build of GNU gettext on AIX:

1. Set the recommended environment variables before running configure:
   https://gitlab.com/ghwiki/gnow-how/-/wikis/Platforms/Configuration

   Namely:
   * for a 32-bit build with gcc:
 CC=gcc
 CXX=g++
 CPPFLAGS="-I$PREFIX/include"
 LDFLAGS="-L$PREFIX/lib"
 unset AR NM
   * for a 32-bit build with xlc:
 CC="xlc -qthreaded -qtls"
 CXX="xlC -qthreaded -qtls"
 CPPFLAGS="-I$PREFIX/include"
 LDFLAGS="-L$PREFIX/lib"
 unset AR NM
   * for a 64-bit build with gcc:
 CC="gcc -maix64"
 CXX="g++ -maix64"
 CPPFLAGS="-I$PREFIX/include"
 LDFLAGS="-L$PREFIX/lib"
 AR="ar -X 64"; NM="nm -X 64 -B"
   * for a 64-bit build with xlc:
 CC="xlc -q64 -qthreaded -qtls"
 CXX="xlC -q64 -qthreaded -qtls"
 CPPFLAGS="-I$PREFIX/include"
 LDFLAGS="-L$PREFIX/lib"
 AR="ar -X 64"; NM="nm -X 64 -B"

   where $PREFIX is the value that you pass to the --prefix configure option.

   Rationale: you can run into all sorts of problems if you choose compiler
   options at random and haven't experience with compiler options on that
   platform.

2. Don't use ibm-clang.

   Rationale: It's broken.

3. Don't use -Wall with gcc 10.3.

   Rationale: If you specify -Wall, gettext's configure adds -fanalyzer, which
   has excessive memory requirements in gcc 10.x. In particular, on AIX, it
   makes cc1 crash while compiling regex.c after it has consumed 1 GiB of RAM.

4. Avoid using a --prefix that contains earlier installations of the same
   package.

   Rationale: Because the AIX linker hardcodes directory names in shared
   libraries, GNU libtool has a peculiar configuration on AIX. It ends up
   mixing the in-build-tree libraries with the libraries in the install
   locations, leading to all sorts of errors.

   If you really need to use a --prefix that contains an earlier
   installation of the same package:
 - Either use --disable-shared and remove libgettextlib.a and
   libgettextsrc.a from $PREFIX/lib before starting the build.
 - Or use a mix of "make -k", "make -k install" and ad-hoc workarounds
   that cannot be described in a general way.

Bruno