Re: hard_locale in a single-threaded app

[Bruno Haible via Gnulib discussion list]

Paul Eggert wrote:
> > In detail, we have dependency chains
> >   exclude -> regex -> lock -> once -> pthread-once
> >   nstrftime -> localename-unsafe-limited -> getlocalename_l-simple -> lock 
> > -> once -> pthread-once
> >   regex -> lock -> once -> pthread-once
> > 
> > Here, in fact, the dependency chain
> >   localename-unsafe-limited -> getlocalename_l-simple -> lock
> > can be reduced
> 
> Thanks for doing that. I hope that diffutils's existing use of 
> GNULIB_EXCLUDE_SINGLE_THREAD and GNULIB_REGEX_SINGLE_THREAD are enough 
> to break the other chains.

These flags don't break dependency chains at the module level. They
optimize the code, despite of the dependencies.

> > * Omitting any multithreading code. Users can already force it by 
> > configuring
> >   with --disable-threads. As a package maintainer, you can make this the
> >   default by adding this line to configure.ac:
> > AC_DEFUN([gl_THREADLIB_DEFAULT_NO], [])
> 
> Shouldn't this be documented in doc/multithread.texi? At least to 
> clarify the relationship between it and GNULIB_EXCLUDE_SINGLE_THREAD etc.

I'm OK to document it, once you have made sure that it works as intended :)
Because it has not been sensibly tested so far.

> Also, since it's checked via m4_ifdef, shouldn't it be defined via 
> m4_define([gl_THREADLIB_DEFAULT_NO])?

Since an AC_DEFUN results in an m4-level definition, either AC_DEFUN or
m4_define would likely work. In the documentation, I would prefer AC_DEFUN,
since m4_define is more "exotic" for most configure.ac maintainers.

> It's not simply shrinking the tarball's size. It's making it easier for 
> diffutils developers and builders to see that multithreading need not be 
> worried about

Such a statement is important, yes: The omnipresent use of global and
'static' variables is only possible when multithreading is not an issue.
But isn't the best way to formulate this statement a sentence in the
HACKING file?

> Even if from Gnulib's viewpoint it's 
> simpler to leave the multithreaded source code and tests in, from the 
> consumer's viewpoint it may well be simpler to omit that stuff when the 
> package can't significantly benefit from multithreading.

Maybe the easiest way to bridge the two viewpoints is that, at the end
of running 'bootstrap', you have a command that removes selected files
from the lib/ directory? The bootstrap_post_import_hook and bootstrap_epilogue
functions are good candidates, IMO.

Bruno

Re: hard_locale in a single-threaded app

[Paul Eggert]

On 2025-02-22 10:02, Bruno Haible wrote:

The latest release already contains some multithreading
support already:


That was inadvertent.


When you intersect the resulting list with the gnulib_modules defined in
diffutils/bootstrap.conf, the result is:
  exclude
  nstrftime
  regex
In detail, we have dependency chains
  exclude -> regex -> lock -> once -> pthread-once
  nstrftime -> localename-unsafe-limited -> getlocalename_l-simple -> lock -> once 
-> pthread-once
  regex -> lock -> once -> pthread-once

Here, in fact, the dependency chain
  localename-unsafe-limited -> getlocalename_l-simple -> lock
can be reduced


Thanks for doing that. I hope that diffutils's existing use of 
GNULIB_EXCLUDE_SINGLE_THREAD and GNULIB_REGEX_SINGLE_THREAD are enough 
to break the other chains. (Haven't had time to look into it.)



when you --avoid'ed some multithreading modules, no wonder that some
tests fail.


The idea was to --avoid tests that require multithreading. Formerly 
diffutils could do that by avoiding only lock-tests; it sounds like it 
needs to avoid more now. I plan to look into it.



* Omitting any multithreading code. Users can already force it by configuring
  with --disable-threads. As a package maintainer, you can make this the
  default by adding this line to configure.ac:
AC_DEFUN([gl_THREADLIB_DEFAULT_NO], [])


Shouldn't this be documented in doc/multithread.texi? At least to 
clarify the relationship between it and GNULIB_EXCLUDE_SINGLE_THREAD etc.


Also, since it's checked via m4_ifdef, shouldn't it be defined via 
m4_define([gl_THREADLIB_DEFAULT_NO])?




* As long as you
  can see (with "ldd") that the package's binaries don't make use of libpthread
  and (with "nm | grep ' U '") that the package's binaries don't use
  multithreading functions, there is no reason to forcibly remove source files
  from the tarball.


It's not simply shrinking the tarball's size. It's making it easier for 
diffutils developers and builders to see that multithreading need not be 
worried about, so that 'configure' doesn't check for multithreading, 
tests don't test for it, etc. Even if from Gnulib's viewpoint it's 
simpler to leave the multithreaded source code and tests in, from the 
consumer's viewpoint it may well be simpler to omit that stuff when the 
package can't significantly benefit from multithreading.

Re: hard_locale in a single-threaded app

[Bruno Haible via Gnulib discussion list]

> 2025-02-22  Bruno Haible  
> 
>   * lib/getlocalename_l.c: Most code moved to
>   lib/getlocalename_l-unsafe.c.

After this change, I get a compilation error on OpenBSD (and presumably
also AIX):

cc -DHAVE_CONFIG_H -DNO_XMALLOC -I. -I../../lib -I..  -I. -I../../lib -I.. 
-I../..  -DIN_LIBUNISTRING -DDEPENDS_ON_LIBICONV=1  -I/usr/local/include -Wall  
-g -O2 -c ../../lib/getlocalename_l-unsafe.c
../../lib/getlocalename_l-unsafe.c:122:4: error: "newlocale, duplocale, 
freelocale not being replaced as expected!"
#  error "newlocale, duplocale, freelocale not being replaced as expected!"
   ^

This patch fixes it.


2025-02-24  Bruno Haible  

getlocalename_l-unsafe: Fix compilation error on OpenBSD, AIX.
* modules/getlocalename_l-unsafe (configure.ac): Add module indicator.
* m4/locale_h.m4 (gl_LOCALE_H_REQUIRE_DEFAULTS): Initialize
GNULIB_GETLOCALENAME_L_UNSAFE.
* modules/locale-h (Makefile.am): Substitute
GNULIB_GETLOCALENAME_L_UNSAFE.
* lib/locale.in.h (newlocale, duplocale, freelocale): Test the module
indicator of 'getlocalename_l-unsafe', not the module indicator of
'getlocalename_l'.

diff --git a/lib/locale.in.h b/lib/locale.in.h
index 2b10a6b95d..5eead0a360 100644
--- a/lib/locale.in.h
+++ b/lib/locale.in.h
@@ -300,7 +300,7 @@ _GL_WARN_ON_USE (setlocale, "setlocale works differently on 
native Windows - "
 # include "setlocale_null.h"
 #endif
 
-#if @GNULIB_NEWLOCALE@ || (@GNULIB_GETLOCALENAME_L@ && 
@LOCALENAME_ENHANCE_LOCALE_FUNCS@ && @HAVE_NEWLOCALE@)
+#if @GNULIB_NEWLOCALE@ || (@GNULIB_GETLOCALENAME_L_UNSAFE@ && 
@LOCALENAME_ENHANCE_LOCALE_FUNCS@ && @HAVE_NEWLOCALE@)
 # if @REPLACE_NEWLOCALE@
 #  if !(defined __cplusplus && defined GNULIB_NAMESPACE)
 #   undef newlocale
@@ -331,7 +331,7 @@ _GL_WARN_ON_USE (newlocale, "newlocale is not portable");
 # endif
 #endif
 
-#if @GNULIB_DUPLOCALE@ || (@GNULIB_GETLOCALENAME_L@ && 
@LOCALENAME_ENHANCE_LOCALE_FUNCS@ && @HAVE_DUPLOCALE@)
+#if @GNULIB_DUPLOCALE@ || (@GNULIB_GETLOCALENAME_L_UNSAFE@ && 
@LOCALENAME_ENHANCE_LOCALE_FUNCS@ && @HAVE_DUPLOCALE@)
 # if @REPLACE_DUPLOCALE@
 #  if !(defined __cplusplus && defined GNULIB_NAMESPACE)
 #   undef duplocale
@@ -357,7 +357,7 @@ _GL_WARN_ON_USE (duplocale, "duplocale is unportable and 
buggy on some glibc sys
 # endif
 #endif
 
-#if @GNULIB_FREELOCALE@ || (@GNULIB_GETLOCALENAME_L@ && 
@LOCALENAME_ENHANCE_LOCALE_FUNCS@ && @HAVE_FREELOCALE@)
+#if @GNULIB_FREELOCALE@ || (@GNULIB_GETLOCALENAME_L_UNSAFE@ && 
@LOCALENAME_ENHANCE_LOCALE_FUNCS@ && @HAVE_FREELOCALE@)
 # if @REPLACE_FREELOCALE@
 #  if !(defined __cplusplus && defined GNULIB_NAMESPACE)
 #   undef freelocale
diff --git a/m4/locale_h.m4 b/m4/locale_h.m4
index 0b6b6f182d..058dc7924b 100644
--- a/m4/locale_h.m4
+++ b/m4/locale_h.m4
@@ -1,5 +1,5 @@
 # locale_h.m4
-# serial 35
+# serial 36
 dnl Copyright (C) 2007, 2009-2025 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -180,6 +180,7 @@ AC_DEFUN([gl_LOCALE_H_REQUIRE_DEFAULTS]
 gl_MODULE_INDICATOR_INIT_VARIABLE([GNULIB_DUPLOCALE])
 gl_MODULE_INDICATOR_INIT_VARIABLE([GNULIB_FREELOCALE])
 gl_MODULE_INDICATOR_INIT_VARIABLE([GNULIB_GETLOCALENAME_L])
+gl_MODULE_INDICATOR_INIT_VARIABLE([GNULIB_GETLOCALENAME_L_UNSAFE])
 gl_MODULE_INDICATOR_INIT_VARIABLE([GNULIB_LOCALENAME_UNSAFE])
   ])
   m4_require(GL_MODULE_INDICATOR_PREFIX[_LOCALE_H_MODULE_INDICATOR_DEFAULTS])
diff --git a/modules/getlocalename_l-unsafe b/modules/getlocalename_l-unsafe
index 5a37d46b85..b3873d7b02 100644
--- a/modules/getlocalename_l-unsafe
+++ b/modules/getlocalename_l-unsafe
@@ -24,6 +24,7 @@ free-posix
 configure.ac:
 gl_FUNC_GETLOCALENAME_L_UNSAFE
 gl_PREREQ_GETLOCALENAME_L_UNSAFE
+gl_LOCALE_MODULE_INDICATOR([getlocalename_l-unsafe])
 
 Makefile.am:
 if !GL_COND_OBJ_GETLOCALENAME_L_UNSAFE_LIMITED
diff --git a/modules/locale-h b/modules/locale-h
index 42eb9b5fbc..00910fde2a 100644
--- a/modules/locale-h
+++ b/modules/locale-h
@@ -42,6 +42,7 @@ locale.h: locale.in.h $(top_builddir)/config.status 
$(CXXDEFS_H) $(ARG_NONNULL_H
  -e 's/@''GNULIB_DUPLOCALE''@/$(GNULIB_DUPLOCALE)/g' \
  -e 's/@''GNULIB_FREELOCALE''@/$(GNULIB_FREELOCALE)/g' \
  -e 's/@''GNULIB_GETLOCALENAME_L''@/$(GNULIB_GETLOCALENAME_L)/g' \
+ -e 
's/@''GNULIB_GETLOCALENAME_L_UNSAFE''@/$(GNULIB_GETLOCALENAME_L_UNSAFE)/g' \
  -e 
's/@''GNULIB_LOCALENAME_UNSAFE''@/$(GNULIB_LOCALENAME_UNSAFE)/g' \
  -e 's|@''HAVE_NEWLOCALE''@|$(HAVE_NEWLOCALE)|g' \
  -e 's|@''HAVE_DUPLOCALE''@|$(HAVE_DUPLOCALE)|g' \

Re: hard_locale in a single-threaded app

[Bruno Haible via Gnulib discussion list]

Paul Eggert wrote:
> I tried to update GNU diffutils to the latest Gnulib, and found that 
> this dragged in a bunch of stuff about multithreading that diffutils 
> doesn't need.

That's nothing new. The latest release already contains some multithreading
support already:

$ ls -1 diffutils-3.11/m4/*thread*
diffutils-3.11/m4/intl-thread-locale.m4
diffutils-3.11/m4/pthread-cond.m4
diffutils-3.11/m4/pthread_h.m4
diffutils-3.11/m4/pthread-mutex.m4
diffutils-3.11/m4/pthread_mutex_timedlock.m4
diffutils-3.11/m4/pthread-once.m4
diffutils-3.11/m4/pthread-rwlock.m4
diffutils-3.11/m4/pthread_rwlock_rdlock.m4
diffutils-3.11/m4/pthread_sigmask.m4
diffutils-3.11/m4/pthread-spin.m4
diffutils-3.11/m4/pthread-thread.m4
diffutils-3.11/m4/threadlib.m4
diffutils-3.11/m4/thread.m4
$ ls -1 diffutils-3.11/lib/*thread*
diffutils-3.11/lib/pthread.in.h
diffutils-3.11/lib/pthread-once.c

diffutils-3.11/lib/glthread:
...

> The first function I found was hard_locale. What's the 
> recommended way to use hard_locale in a single-threaded app, without 
> pulling in the thread library part of Gnulib?

'hard-locale' relies on 'setlocale-null'. There are 3 aspects:

* Runtime speed optimization. The documentation
  

  lists optimizations for several modules. There is no
  SETLOCALE_NULL_SINGLE_THREAD or SETLOCALE_NULL_SINGLE_FLAG that would
  have an effect. We could introduce one if relevant.
  But since on glibc systems, setlocale-null does not do any locking anyway
  (because SETLOCALE_NULL_ALL_MTSAFE and SETLOCALE_NULL_ONE_MTSAFE are 1
  on glibc systems), possible optimizations would be relevant only for
  other platforms.
  Use of the 'thread-optim' module in setlocale-null is not needed, for the
  same reason.

* Omitting any multithreading code. Users can already force it by configuring
  with --disable-threads. As a package maintainer, you can make this the
  default by adding this line to configure.ac:
AC_DEFUN([gl_THREADLIB_DEFAULT_NO], [])
  See m4/threadlib.m4 for how it works.

* Other than that, added files in the tarball are harmless. As long as you
  can see (with "ldd") that the package's binaries don't make use of libpthread
  and (with "nm | grep ' U '") that the package's binaries don't use
  multithreading functions, there is no reason to forcibly remove source files
  from the tarball.
  In other words, maintaining two versions of each module, one multithread-safe
  and one with reduced dependencies for single-threaded packages, is not worth
  the effort.

Still, anyone can investigate where dependencies come from. For example,
diffutils-3.11 contains lib/pthread-once.c, whereas diffutils-3.10 doesn't.
Where does it come from?
  $ ./gnulib-tool --find lib/pthread-once.c 
  pthread-once
Which are the modules with an indirect dependency on it?
  $ ./gnulib-tool --extract-recursive-dependents pthread-once | grep -v tests
When you intersect the resulting list with the gnulib_modules defined in
diffutils/bootstrap.conf, the result is:
  exclude
  nstrftime
  regex
In detail, we have dependency chains
  exclude -> regex -> lock -> once -> pthread-once
  nstrftime -> localename-unsafe-limited -> getlocalename_l-simple -> lock -> 
once -> pthread-once
  regex -> lock -> once -> pthread-once

Here, in fact, the dependency chain
  localename-unsafe-limited -> getlocalename_l-simple -> lock
can be reduced, by introducing a function getlocalename_l_unsafe
and modules 'getlocalename_l-unsafe' and 'getlocalename_l-unsafe-limited'.
Done through the patches below.

> Also, a couple of the multithread-based Gnulib tests failed in the 
> updated diffutils, but only there - they don't fail standalone. I assume 
> this is because diffutils is excluding some modules (based on what it 
> needed to do with old Gnulib to avoid the threading stuff), so I doubt 
> whether this rabbit hole is worth going down.

Yes, when you --avoid'ed some multithreading modules, no wonder that some
tests fail.


2025-02-22  Bruno Haible  

localename-unsafe-limited: Use getlocalename_l-unsafe-limited.
* modules/localename-unsafe-limited (Depends-on): Add
getlocalename_l-unsafe-limited. Remove getlocalename_l-unsafe.

2025-02-22  Bruno Haible  

getlocalename_l-unsafe-limited: New module.
* modules/getlocalename_l-unsafe-limited: New file.
* modules/getlocalename_l-unsafe (Depends-on): Add
getlocalename_l-unsafe-limited.
(Makefile.am): Don't compile getlocalename_l-unsafe.c if already
compiled as part of module 'getlocalename_l-unsafe-limited'.

2025-02-22  Bruno Haible  

localename-unsafe: Use getlocalename_l-unsafe.
* lib/localename-unsafe.c: Include getlocalename_l-unsafe.h.
(gl_locale_name_thread_unsafe): Invoke getlocalename_l_unsafe instead of
getlocalename_l.
* modules/localename-unsafe (Depends-on): Add getlocalename_l-unsafe.
Remove getlo