Re: Can we have gnulib-tool.py emit the same copyright header?

[Collin Funk]

On 2/27/24 6:51 PM, Collin Funk wrote:
> [collin@debian gnulib]$ git log --pretty=medium -n 1 | grep '^Date:'

It seems that the 'git command-name --pretty=medium' existed before
git version 1. I've found the commit that introduced the version I was
using that caused breakage [1]. It seems that my git config also
breaks a test case in vc-dwim. I've just changed my config for now,
but assuming that command doesn't have any problems I can submit a fix
there.

[2] https://github.com/git/git/commit/ff56fe1ca7f43087fd84588af87179c2959d0cb3

Collin

Re: Can we have gnulib-tool.py emit the same copyright header?

[Collin Funk]

On 2/27/24 6:26 PM, Bruno Haible wrote:
> git log -n 1 ChangeLog | grep ^Date:

Ah, I see what is happening.

[collin@debian gnulib]$ git log -n 1
commit 431d6a7615245e6b32d95b4b27aab5d3af65ad2b (HEAD -> master, origin/master, 
origin/HEAD)
Author: Bruno Haible 
AuthorDate: Wed Feb 28 00:33:49 2024 +0100
Commit: Bruno Haible 
CommitDate: Wed Feb 28 00:33:49 2024 +0100

isnan: Fix compilation error in C++ mode on OpenBSD 7.5-beta.

Reported by Christian Weisgerber  in
.

* lib/math.in.h (GNULIB_NAMESPACE_LACKS_ISNAN): Define on all platforms
with clang ≥ 14.

So the grep ^Date fails. I don't remember changing this but in my git
config it is caused by this [1]:

[format]
pretty = fuller

The regular gnulib-tool gives me this:

[collin@debian gnulib]$ gnulib-tool --version
gnulib-tool (GNU gnulib 2024-02-28 00:00:00) 0.1.7153-431d6
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Bruno Haible, Paul Eggert, and Simon Josefsson.

It seems that some people might have their git log settings so that
there isn't even a date or name:

[collin@debian gnulib]$ git log --pretty=oneline
431d6a7615245e6b32d95b4b27aab5d3af65ad2b (HEAD -> master, origin/master, 
origin/HEAD) isnan: Fix compilation error in C++ mode on OpenBSD 7.5-beta.
38a9bb6e0ae3f5c71da8eb5db5ea7aa41b14f3e4 gnulib-tool.py: Fix configure.ac 
output.
e1f51c8e0431fe2598479daf76690e1d4d7c1a88 gnulib-tool.py: Follow gnulib-tool 
changes, part 31.
bb09847b7d87d1c99a29f889372bc4771eb9d2bd gnulib-tool.py: Small fix of part 27.

I wonder which version this feature was added... I'm not sure how git
will act with invalid arguments but ideally we could run something
like:

[collin@debian gnulib]$ git log --pretty=medium -n 1 | grep '^Date:'
Date:   Wed Feb 28 00:33:49 2024 +0100

I think medium looks like the default, but I'd have to double check.

[1] https://git-scm.com/docs/pretty-formats

Collin

Re: Can we have gnulib-tool.py emit the same copyright header?

[Bruno Haible]

Collin Funk wrote:
> I tried to run --version and got this:
> 
> [collin@debian gnulib]$ gnulib-tool.py --version
> Traceback (most recent call last):
>   File "/home/collin/.local/src/gnulib/pygnulib/main.py", line 1197, in 
> 
> main()
>   File "/home/collin/.local/src/gnulib/pygnulib/main.py", line 442, in main
> % (info.package(), info.date(), version, info.copyright(),
>^^^
>   File "/home/collin/.local/src/gnulib/pygnulib/GLInfo.py", line 109, in date
> result = pattern.findall(result)[0]
>  ~~~^^^
> IndexError: list index out of range

Huh? For me, it works fine:

$ ./gnulib-tool.py --version
gnulib-tool (GNU gnulib 2024-02-27 23:33:49) 0.1.7153-431d6
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Bruno Haible, Paul Eggert, Simon Josefsson, and Dmitry Selyutin.
$ python3 --version
Python 3.10.12
$ git --version
git version 2.34.1
$ git log -n 1 ChangeLog | grep ^Date:
Date:   Wed Feb 28 00:33:49 2024 +0100

Bruno

Re: Can we have gnulib-tool.py emit the same copyright header?

[Collin Funk]

On 2/27/24 6:07 PM, Bruno Haible wrote:
> When I wrote this piece of code (2022-08-05), it was for the --version
> output. I had apparently overlooked that the method GLInfo.copyright
> also gets used in other contexts than for processing the --version
> option.

Ah I see. I probably should have grep'd for that function because I
didn't notice it was used there.

> In *files*, the copyright notice should contain the first and the last year
> of modification; a range - is OK. [1]
> 
> In *--version output*, the copyright notice should contain only the last year.
> [2]

That makes sense, thanks. I tried to run --version and got this:

[collin@debian gnulib]$ gnulib-tool.py --version
Traceback (most recent call last):
  File "/home/collin/.local/src/gnulib/pygnulib/main.py", line 1197, in 
main()
  File "/home/collin/.local/src/gnulib/pygnulib/main.py", line 442, in main
% (info.package(), info.date(), version, info.copyright(),
   ^^^
  File "/home/collin/.local/src/gnulib/pygnulib/GLInfo.py", line 109, in date
result = pattern.findall(result)[0]
 ~~~^^^
IndexError: list index out of range

So, I'll have a look at that fixing that. :)

Collin

Re: Can we have gnulib-tool.py emit the same copyright header?

[Bruno Haible]

Hi Collin,

> When creating a Makefile for example, the diff between gnulib-tool and
> gnulib-tool.py is:
> 
> diff --git a/lib/gnulib.mk.in b/lib/gnulib.mk.in
> index a718c17c0e8..15d15970051 100644
> --- a/lib/gnulib.mk.in
> +++ b/lib/gnulib.mk.in
> @@ -1,5 +1,5 @@
>  ## DO NOT EDIT! GENERATED AUTOMATICALLY!
> -# Copyright (C) 2002-2024 Free Software Foundation, Inc.
> +# Copyright (C) 2024 Free Software Foundation, Inc.
> 
> The gnulib-tool.py one only does a single year instead of the range.
> 
> This diff is copied from the attached patch:
> 
> diff --git a/pygnulib/GLInfo.py b/pygnulib/GLInfo.py
> index 808f11b06f..1759368268 100644
> --- a/pygnulib/GLInfo.py
> +++ b/pygnulib/GLInfo.py
> @@ -80,8 +80,6 @@ class GLInfo(object):
>  '''Return formatted string which contains copyright.
>  The special __copyright__ variable is used (type is str).'''
>  copyright = __copyright__
> -# Per the GNU Coding Standards, show only the last year.
> -copyright = re.compile('^[0-9]*-').sub('', copyright)
>  result = 'Copyright (C) %s' % copyright
>  return result

When I wrote this piece of code (2022-08-05), it was for the --version
output. I had apparently overlooked that the method GLInfo.copyright
also gets used in other contexts than for processing the --version
option.

> The GNU Maintainer guide seems to disagree with this comment (assuming
> that gnulib-tool.py is not a separate package) [1]. Am I missing
> something or can we make this change?
> 
> [1] https://www.gnu.org/prep/maintain/maintain.html#Copyright-Notices

In *files*, the copyright notice should contain the first and the last year
of modification; a range - is OK. [1]

In *--version output*, the copyright notice should contain only the last year.
[2]

Bruno

[1] https://www.gnu.org/prep/maintain/html_node/Copyright-Notices.html
[2] https://www.gnu.org/prep/standards/html_node/_002d_002dversion.html

Can we have gnulib-tool.py emit the same copyright header?

[Collin Funk]

When creating a Makefile for example, the diff between gnulib-tool and
gnulib-tool.py is:

diff --git a/lib/gnulib.mk.in b/lib/gnulib.mk.in
index a718c17c0e8..15d15970051 100644
--- a/lib/gnulib.mk.in
+++ b/lib/gnulib.mk.in
@@ -1,5 +1,5 @@
 ## DO NOT EDIT! GENERATED AUTOMATICALLY!
-# Copyright (C) 2002-2024 Free Software Foundation, Inc.
+# Copyright (C) 2024 Free Software Foundation, Inc.

The gnulib-tool.py one only does a single year instead of the range.

This diff is copied from the attached patch:

diff --git a/pygnulib/GLInfo.py b/pygnulib/GLInfo.py
index 808f11b06f..1759368268 100644
--- a/pygnulib/GLInfo.py
+++ b/pygnulib/GLInfo.py
@@ -80,8 +80,6 @@ class GLInfo(object):
 '''Return formatted string which contains copyright.
 The special __copyright__ variable is used (type is str).'''
 copyright = __copyright__
-# Per the GNU Coding Standards, show only the last year.
-copyright = re.compile('^[0-9]*-').sub('', copyright)
 result = 'Copyright (C) %s' % copyright
 return result

The GNU Maintainer guide seems to disagree with this comment (assuming
that gnulib-tool.py is not a separate package) [1]. Am I missing
something or can we make this change?

[1] https://www.gnu.org/prep/maintain/maintain.html#Copyright-Notices

Thanks,
CollinFrom 202d152d8a0a6fa3875c44ff5cd38bdd4b0d485d Mon Sep 17 00:00:00 2001
From: Collin Funk 
Date: Tue, 27 Feb 2024 17:30:18 -0800
Subject: [PATCH] gnulib-tool.py: Emit copyright notices as a year range.

* pygnulib/GLInfo.py (GLInfo.copyright): Fix copyright string. Remove
comment to reflect change.
---
 ChangeLog  | 6 ++
 pygnulib/GLInfo.py | 2 --
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index af3af3707e..84f5bcee88 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2024-02-27  Collin Funk  
+
+	gnulib-tool.py: Emit copyright notices as a year range.
+	* pygnulib/GLInfo.py (GLInfo.copyright): Fix copyright string. Remove
+	comment to reflect change.
+
 2024-02-27  Bruno Haible  
 
 	isnan: Fix compilation error in C++ mode on OpenBSD 7.5-beta.
diff --git a/pygnulib/GLInfo.py b/pygnulib/GLInfo.py
index 808f11b06f..1759368268 100644
--- a/pygnulib/GLInfo.py
+++ b/pygnulib/GLInfo.py
@@ -80,8 +80,6 @@ class GLInfo(object):
 '''Return formatted string which contains copyright.
 The special __copyright__ variable is used (type is str).'''
 copyright = __copyright__
-# Per the GNU Coding Standards, show only the last year.
-copyright = re.compile('^[0-9]*-').sub('', copyright)
 result = 'Copyright (C) %s' % copyright
 return result
 
-- 
2.39.2

Re: [PATCH] gnulib-tool.py: Fix configure.ac output.

[Collin Funk]

On 2/27/24 3:39 PM, Bruno Haible wrote:
> $ pycodestyle *.py
> ...
> GLEmiter.py:535:26: W605 invalid escape sequence '\.'
> GLEmiter.py:535:35: W605 invalid escape sequence '\.'
> ...

I discovered it that way too. I like using Eglot from Emacs though. If
you are using Emacs 29+ the `eglot' command should be built-in. In
that case in the following line:

  sed_drop_objext='s/\.o$//;s/\.obj$//'

The '\.o$//;s/\.obj$//' part will be crossed out. Not sure if that is
your perfected way of doing things but I find it useful so I figured
I'd mention it. :)

Collin

Re: [PATCH] gnulib-tool.py: Fix configure.ac output.

[Bruno Haible]

Collin Funk wrote:
> > I guess I wasn't very familiar with raw strings when I did this change.
> 
> No worries, it was pretty easy to find and fix. I remember a professor
> in college telling us to always use them for Python regular
> expressions. It seems this is also recommended by the Python people,
> see the '\' character under this page:
> 
> https://docs.python.org/3/library/re.html#regular-expression-syntax

Indeed, this doc contains the explanation for the 'pycodestyle'
warning that I was seeing. With a plain string instead of a raw string,
I get:

$ pycodestyle *.py
...
GLEmiter.py:535:26: W605 invalid escape sequence '\.'
GLEmiter.py:535:35: W605 invalid escape sequence '\.'
...

which indicates that the Python parser would have eaten up the backslash
before the regular expression engine could see it.

Bruno

Re: Fwd: gnulib: isnan: Fix compilation error in C++ mode

[Bruno Haible]

Hi,

Christian Weisgerber wrote:
> isnan: Fix compilation error in C++ mode on macOS 14.4 / Xcode 15.2
> https://lists.gnu.org/archive/html/bug-gnulib/2024-02/msg00121.html
> 
> I just came across the same problem on OpenBSD 7.5-beta (clang 16.0.6).
> I guess the defined() check needs to be extended unless a more generic
> solution can be found.
> 
> Index: gettext-tools/gnulib-lib/math.in.h
> --- gettext-tools/gnulib-lib/math.in.h.orig
> +++ gettext-tools/gnulib-lib/math.in.h
> @@ -2668,7 +2668,7 @@ _GL_MATH_CXX_REAL_FLOATING_DECL_1 (isnan)
>  _GL_MATH_CXX_REAL_FLOATING_DECL_2 (isnan, rpl_isnan, bool)
>  #define isnan rpl_isnan
>  #define GNULIB_NAMESPACE_LACKS_ISNAN 1
> -#   elif (((defined __APPLE__ && defined __MACH__) || defined __FreeBSD__) 
> && __clang_major__ >= 14)
> +#   elif (((defined __APPLE__ && defined __MACH__) || defined __FreeBSD__ || 
> defined __OpenBSD__) && __clang_major__ >= 14)
>/* Neither of the two possible _GL_MATH_CXX_REAL_FLOATING_DECL_2 
> invocations
>   works.  Inline functions are already present in 
> /usr/include/c++/v1/math.h,
>   which comes from LLVM.  */

Thanks for the report and patch. It appears that the issue comes from
the clang++ header files and that these header files are incorporated
in the OS (macOS, FreeBSD, OpenBSD so far). It is likely that this will
also hold for other OSes. I'm therefore committing a slightly more
general patch:


2024-02-27  Bruno Haible  

isnan: Fix compilation error in C++ mode on OpenBSD 7.5-beta.
Reported by Christian Weisgerber  in
.
* lib/math.in.h (GNULIB_NAMESPACE_LACKS_ISNAN): Define on all platforms
with clang ≥ 14.

diff --git a/lib/math.in.h b/lib/math.in.h
index 85b7e6ff95..72a61d7044 100644
--- a/lib/math.in.h
+++ b/lib/math.in.h
@@ -2668,7 +2668,7 @@ _GL_MATH_CXX_REAL_FLOATING_DECL_1 (isnan)
 _GL_MATH_CXX_REAL_FLOATING_DECL_2 (isnan, rpl_isnan, bool)
 #define isnan rpl_isnan
 #define GNULIB_NAMESPACE_LACKS_ISNAN 1
-#   elif (((defined __APPLE__ && defined __MACH__) || defined __FreeBSD__) && 
__clang_major__ >= 14)
+#   elif __clang_major__ >= 14
   /* Neither of the two possible _GL_MATH_CXX_REAL_FLOATING_DECL_2 invocations
  works.  Inline functions are already present in 
/usr/include/c++/v1/math.h,
  which comes from LLVM.  */

Re: [PATCH] gnulib-tool.py: Fix configure.ac output.

[Collin Funk]

Hi Bruno,

On 2/27/24 2:39 PM, Bruno Haible wrote:
> I guess I wasn't very familiar with raw strings when I did this change.

No worries, it was pretty easy to find and fix. I remember a professor
in college telling us to always use them for Python regular
expressions. It seems this is also recommended by the Python people,
see the '\' character under this page:

https://docs.python.org/3/library/re.html#regular-expression-syntax

Collin

Fwd: gnulib: isnan: Fix compilation error in C++ mode

[Bruno Haible]

[CCing bug-gnulib]

--  Forwarded Message  --

Subject: gnulib: isnan: Fix compilation error in C++ mode
Date: Dienstag, 27. Februar 2024, 23:08:47 CET
From: Christian Weisgerber 
To: Bruno Haible 

Hi,

regarding

isnan: Fix compilation error in C++ mode on macOS 14.4 / Xcode 15.2
https://lists.gnu.org/archive/html/bug-gnulib/2024-02/msg00121.html

I just came across the same problem on OpenBSD 7.5-beta (clang 16.0.6).
I guess the defined() check needs to be extended unless a more generic
solution can be found.

Index: gettext-tools/gnulib-lib/math.in.h
--- gettext-tools/gnulib-lib/math.in.h.orig
+++ gettext-tools/gnulib-lib/math.in.h
@@ -2668,7 +2668,7 @@ _GL_MATH_CXX_REAL_FLOATING_DECL_1 (isnan)
 _GL_MATH_CXX_REAL_FLOATING_DECL_2 (isnan, rpl_isnan, bool)
 #define isnan rpl_isnan
 #define GNULIB_NAMESPACE_LACKS_ISNAN 1
-#   elif (((defined __APPLE__ && defined __MACH__) || defined __FreeBSD__) && 
__clang_major__ >= 14)
+#   elif (((defined __APPLE__ && defined __MACH__) || defined __FreeBSD__ || 
defined __OpenBSD__) && __clang_major__ >= 14)
   /* Neither of the two possible _GL_MATH_CXX_REAL_FLOATING_DECL_2 invocations
  works.  Inline functions are already present in 
/usr/include/c++/v1/math.h,
  which comes from LLVM.  */
-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

-

Re: [PATCH] gnulib-tool.py: Fix configure.ac output.

[Bruno Haible]

Collin Funk wrote:
> When running the following command,
> 
> gnulib-tool.py --create-testdir --dir test-dir-python dummy
> 
> creates a configure.ac like so:
> 
> gl_source_base='gllib'
> 
> \
>   m4_ifval(gl_LIBSOURCES_LIST, [
> m4_syscmd([test ! -d ]m4_defn([gl_LIBSOURCES_DIR])[ ||
> ...
> AC_SUBST([gl_LIBOBJS], [$gl_libobjs])
> AC_SUBST([gl_LTLIBOBJS], [$gl_ltlibobjs])
>   ])\n  LIBGNU_LIBDEPS="$gl_libdeps"
> ...

Indeed, that's not right.

Thanks for the patch; applied.

> https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=f00ef30ee0a68bf91cfd081afc784cb8ffd00656
> 
> I assume that this string literal was changed to a raw string [1] in
> order to avoid having to escape a sed statement. Since it is already
> wrapped in triple quotes we can just add a newline in the code

I guess I wasn't very familiar with raw strings when I did this change.

Bruno

[PATCH] gnulib-tool.py: Fix configure.ac output.

[Collin Funk]

Here is a patch fixing an issue I've noticed in the output of
configure.ac (and configure) created by gnulib.py.

When running the following command,

gnulib-tool.py --create-testdir --dir test-dir-python dummy

creates a configure.ac like so:

gl_source_base='gllib'

\
  m4_ifval(gl_LIBSOURCES_LIST, [
m4_syscmd([test ! -d ]m4_defn([gl_LIBSOURCES_DIR])[ ||
...
AC_SUBST([gl_LIBOBJS], [$gl_libobjs])
AC_SUBST([gl_LTLIBOBJS], [$gl_ltlibobjs])
  ])\n  LIBGNU_LIBDEPS="$gl_libdeps"
...

This causes the 'configure' script to try to execute 'n' twice like
so:

checking for Minix Amsterdam compiler... no
checking for ar... ar
checking for ranlib... ranlib
./configure: line 5004: n: command not found

This seems to be caused by this commit trying to clean up the code:

https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=f00ef30ee0a68bf91cfd081afc784cb8ffd00656

I assume that this string literal was changed to a raw string [1] in
order to avoid having to escape a sed statement. Since it is already
wrapped in triple quotes we can just add a newline in the code and
have the output change instead of using '\n'.

I've also changed the """foo""" quoting to '''foo''' quoting in that
statement. I assume that is also the preferred coding style, feel free
to change it if I misinterpreted. I will have to modify this file a
few times while handling the TODO, so I will fix the other double
quote offenders then.

[1] https://docs.python.org/3/library/re.html#raw-string-notation

CollinFrom a9932bf82e3b5e1526e801a55f743a0c8aeb2331 Mon Sep 17 00:00:00 2001
From: Collin Funk 
Date: Tue, 27 Feb 2024 14:15:10 -0800
Subject: [PATCH] gnulib-tool.py: Fix configure.ac output.

* pygnulib/GLEmiter.py (GLEmiter.initmacro_end): Fix backslash escapes
in raw string.
---
 ChangeLog| 6 ++
 pygnulib/GLEmiter.py | 5 +++--
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index a71c5a771c..7d65e27151 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2024-02-27  Collin Funk  
+
+	gnulib-tool.py: Fix configure.ac output.
+	* pygnulib/GLEmiter.py (GLEmiter.initmacro_end): Fix backslash escapes
+	in raw string.
+
 2024-02-27  Collin Funk  
 
 	gnulib-tool.py: Follow gnulib-tool changes, part 31.
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index 56e8156761..d8cc085f59 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -508,7 +508,7 @@ USE_MSGCTXT = no\n"""
 # arguments. The check is performed only when autoconf is run from the
 # directory where the configure.ac resides; if it is run from a different
 # directory, the check is skipped.
-emit += r"""\
+emit += r'''
   m4_ifval(%V1%_LIBSOURCES_LIST, [
 m4_syscmd([test ! -d ]m4_defn([%V1%_LIBSOURCES_DIR])[ ||
   for gl_file in ]%V1%_LIBSOURCES_LIST[ ; do
@@ -540,7 +540,8 @@ USE_MSGCTXT = no\n"""
 fi
 AC_SUBST([%V1%_LIBOBJS], [$%V1%_libobjs])
 AC_SUBST([%V1%_LTLIBOBJS], [$%V1%_ltlibobjs])
-  ])\n"""
+  ])
+'''
 emit = emit.replace('%V1%', macro_prefix_arg)
 return emit
 
-- 
2.39.2

Re: sort dynamic linking overhead

[Bruno Haible]

Pádraig Brady wrote:
> > Does this work for all the various names of libcrypto in various distros?
> > 
> > Debian 12   libcrypto.so.3
> > Ubuntu 22.04libcrypto.so.1.1 libcrypto.so.3
> > Slackware 15libcrypto.so.1.1
> > openSUSE 15.5   libcrypto.so.1.1
> > CentOS Stream 9 libcrypto.so.3
> > Guix 1.4libcrypto.so.1.1
> > Alpine 3.19 libcrypto.so.3
> > FreeBSD 14.0libcrypto.so.38
> > NetBSD 9.3  libcrypto.so.14
> > OpenBSD 7.4 libcrypto.so.52.0
> > 
> 
> I only tested with libcrypto.so.3, but it should match all of the above.
> It matches libcrypto.so.[.0-9]*

Here are my testing results (with the LIB_DL fix):

* On some machines, I had to install the packages with the 
  header files first:

  - Debian 12:
# apt install libssl-dev
  - openSUSE 15.5:
YaST software > install libopenssl-3-devel
  - Slackware 15:
Download and unpack the openssl-1.1.1w binary packages
  - Alpine Linux 3.19:
# apk add openssl openssl-dev

* On some platforms, the configure test gave

 checking whether openssl is GPL compatible... no

  due to the  header files being absent:

- Guix 1.4
- macOS 12.5
- Cygwin 2.9.0

* On some platforms, the configure test gave

 checking whether openssl is GPL compatible... no

  because the openssl version is not >= 3.

- Slackware 15 (has libcrypto.so.1.1)
- NetBSD 9.3 (has libcrypto.so.14)
- OpenBSD 7.4 (has libcrypto.so.52.0)
- Solaris 11.4 (has libcrypto.so.1.0.0)

* On these platforms, the configure test gave

 checking whether openssl is GPL compatible... yes

  and LIBCRYPTO_SONAME got defined.

- Debian 12
- Ubuntu 22.04
- openSUSE 15.5
- CentOS Stream 9
- Alpine Linux 3.19
- FreeBSD 14.0
- Android

  The value of LIBCRYPTO_SONAME is
"libcrypto.so.30" on Android,
"libcrypto.so.3" on the other platforms.

Bruno

Re: sort dynamic linking overhead

[Bruno Haible]

Paul Eggert wrote:
> Thanks for the patch. I was hoping that we didn't need to worry about 
> older platforms needing -ldl. Oh well.

Distros with glibc < 2.34 include:
  Debian 11
  CentOS 8 stream
  Fedora 34
  openSUSE 15.5
  Slackware 15
These are not obsolete, so far.

> The patch causes 'configure' to search for dlopen even when there's no 
> crypto library. 'configure' could instead use AC_SEARCH_LIBS only if the 
> AC_LINK_IFELSE fails (or simply put AC_LINK_ELSE in an 'for LIB_DL in "" 
> -ldl' loop). But perhaps it's better to leave things be, in case we ever 
> need dlopen for something else.

If we were to start optimizing configure.ac scripts like this, they would
quickly become hard to maintain, because here and there we would be
accessing a shell variable that has not been initialized (because the
initialization was in an 'if' branch). (There's a reason why AC_REQUIRE
has been invented...)

The configure tests which nag me the most are those which take one second
of time or more: the fcntl test, the sleep test, the strstr test, and a few
others. If someone is starting to optimize, here would be the starting point :)

> Also, if I understand things correctly, with this patch it's 
> theoretically possible to pass -ldl to gcc even when 'sort' doesn't do 
> dynamic linking, and we could complicate configure.ac further to omit 
> -ldl in that case. But I doubt whether it's worth worrying about this.

Correct, but this applies only to glibc and Android systems, and on these
systems configure finds

  checking for C compiler flag to ignore unused libraries... -Wl,--as-needed

So, if $(LIB_DL) is -ldl and this linker option gets used for 'sort', but
'sort' does not use dlopen(), the linker will eliminate it.

Bruno

Re: sort dynamic linking overhead

[Paul Eggert]

Thanks for the patch. I was hoping that we didn't need to worry about 
older platforms needing -ldl. Oh well.


The patch causes 'configure' to search for dlopen even when there's no 
crypto library. 'configure' could instead use AC_SEARCH_LIBS only if the 
AC_LINK_IFELSE fails (or simply put AC_LINK_ELSE in an 'for LIB_DL in "" 
-ldl' loop). But perhaps it's better to leave things be, in case we ever 
need dlopen for something else.


Also, if I understand things correctly, with this patch it's 
theoretically possible to pass -ldl to gcc even when 'sort' doesn't do 
dynamic linking, and we could complicate configure.ac further to omit 
-ldl in that case. But I doubt whether it's worth worrying about this.

Re: sort dynamic linking overhead

[Pádraig Brady]

On 27/02/2024 11:15, Bruno Haible wrote:

Paul Eggert wrote:

I installed the
attached patch into coreutils. With it, 'sort -R' continues to use MD5
but on GNUish platforms 'sort' links libcrypto dynamically only if -R is
used (Bruno's suggestion). This doesn't significantly affect 'sort -R'
performance, and reduces the startup overhead of plain 'sort'


The patch has no effect on openSUSE 15.5: DLOPEN_LIBCRYPTO is not defined
in config.h.

config.log shows a link error of the test program:
   undefined reference to symbol 'dlopen@@GLIBC_2.2.5'
Both the test program and 'sort' need to link with -ldl.

This is generally true on the following platforms:
   - glibc < 2.34,
   - Android (with some compilers, not with Termux (*)).

(*) In the Android Termux environment, the compiler is configured to pass
the options '-ldl -lc', rather than just '-lc', to the linker. Thus,
we don't need to pass '-ldl' to the compiler. But the 'sort' program will
be linked with -ldl. In other Android environments, things may be different,
though.

The proposed attached patch fixes the problem: It defines a variable LIB_DL
that contains '-ldl' where needed or '' if not needed, and uses it with
the test program and with 'sort'.

You might think that this patch is no win, because it trades one link
dependency for another link dependency? But that's not what it does:
'ldd' shows that without the patch, 'sort' loads the libraries

   linux-vdso.so.1
   libcrypto.so.3
   libpthread.so.0
   libc.so.6
   libz.so.1
   libdl.so.2
   /lib64/ld-linux-x86-64.so.2

and with the patch, 'sort' loads the libraries

   linux-vdso.so.1
   libdl.so.2
   libpthread.so.0
   libc.so.6
   /lib64/ld-linux-x86-64.so.2

— that is, libdl.so.2 is getting loaded anyway.


Applied.

thanks,
Pádraig

Re: sort dynamic linking overhead

[Bruno Haible]

Paul Eggert wrote:
> I installed the 
> attached patch into coreutils. With it, 'sort -R' continues to use MD5 
> but on GNUish platforms 'sort' links libcrypto dynamically only if -R is 
> used (Bruno's suggestion). This doesn't significantly affect 'sort -R' 
> performance, and reduces the startup overhead of plain 'sort'

The patch has no effect on openSUSE 15.5: DLOPEN_LIBCRYPTO is not defined
in config.h.

config.log shows a link error of the test program:
  undefined reference to symbol 'dlopen@@GLIBC_2.2.5'
Both the test program and 'sort' need to link with -ldl.

This is generally true on the following platforms:
  - glibc < 2.34,
  - Android (with some compilers, not with Termux (*)).

(*) In the Android Termux environment, the compiler is configured to pass
the options '-ldl -lc', rather than just '-lc', to the linker. Thus,
we don't need to pass '-ldl' to the compiler. But the 'sort' program will
be linked with -ldl. In other Android environments, things may be different,
though.

The proposed attached patch fixes the problem: It defines a variable LIB_DL
that contains '-ldl' where needed or '' if not needed, and uses it with
the test program and with 'sort'.

You might think that this patch is no win, because it trades one link
dependency for another link dependency? But that's not what it does:
'ldd' shows that without the patch, 'sort' loads the libraries

  linux-vdso.so.1
  libcrypto.so.3
  libpthread.so.0
  libc.so.6
  libz.so.1
  libdl.so.2
  /lib64/ld-linux-x86-64.so.2

and with the patch, 'sort' loads the libraries

  linux-vdso.so.1
  libdl.so.2
  libpthread.so.0
  libc.so.6
  /lib64/ld-linux-x86-64.so.2

— that is, libdl.so.2 is getting loaded anyway.

>From c30a0e55c95e0ae7062ee2ececf85cd0dbfe49fb Mon Sep 17 00:00:00 2001
From: Bruno Haible 
Date: Tue, 27 Feb 2024 12:12:59 +0100
Subject: [PATCH] sort: Make the startup time optimization effective on glibc <
 2.34

* configure.ac: Test where to find the dlopen function. Set LIB_DL.
Use it in the DLOPEN_LIBCRYPTO test.
* src/local.mk (src_sort_LDADD): Add $(LIB_DL).
---
 configure.ac | 11 ++-
 src/local.mk |  2 +-
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/configure.ac b/configure.ac
index fe8408a06..248e30ca2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -351,6 +351,15 @@ if test $utils_cv_localtime_cache = yes; then
   AC_DEFINE([LOCALTIME_CACHE], [1], [FIXME])
 fi
 
+# Find the library for dynamic loading of shared libraries.
+AC_SEARCH_LIBS([dlopen], [dl])
+AS_CASE([$ac_cv_search_dlopen],
+  [no | 'none required'],
+[LIB_DL=],
+  [*],
+[LIB_DL="$ac_cv_search_dlopen"])
+AC_SUBST([LIB_DL])
+
 # Should 'sort' link libcrypto dynamically?
 AS_CASE([$LIB_CRYPTO],
   [-lcrypto],
@@ -360,7 +369,7 @@ AS_CASE([$LIB_CRYPTO],
[utils_cv_dlopen_libcrypto],
[utils_cv_dlopen_libcrypto=no
 saved_LIBS=$LIBS
-LIBS="$LIBS $LIB_CRYPTO"
+LIBS="$LIBS $LIB_DL $LIB_CRYPTO"
 AC_LINK_IFELSE(
   [AC_LANG_PROGRAM(
  [[#include 
diff --git a/src/local.mk b/src/local.mk
index 7bc5ba5bc..96ee941ca 100644
--- a/src/local.mk
+++ b/src/local.mk
@@ -304,7 +304,7 @@ src_printf_LDADD += $(LIBICONV)
 
 # for libcrypto hash routines
 src_md5sum_LDADD += $(LIB_CRYPTO)
-src_sort_LDADD += $(LIB_CRYPTO)
+src_sort_LDADD += $(LIB_DL) $(LIB_CRYPTO)
 src_sha1sum_LDADD += $(LIB_CRYPTO)
 src_sha224sum_LDADD += $(LIB_CRYPTO)
 src_sha256sum_LDADD += $(LIB_CRYPTO)
-- 
2.34.1

Re: gnulib-tool.py: Follow gnulib-tool changes, part 31

[Bruno Haible]

Collin Funk wrote:
> And another patch to stop gnulib-tool.py from messing with copyright
> headers. This patch is just doing the same as this commit did for
> gnulib-tool:
> 
> https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=f3553bdefc9202e81996c73686e3ca53cd827417

Thanks! The gnu_make tweak is unrelated; so, I put it in a separate commit
(and moved it a bit: in GLConfig, the 'gnu_make' property is typically
handled between 'lgpl' and 'makefile_name').

> I assume that this commit somehow escaped being added to
> gnulibtool.py.TODO so I've made a reference to it anyways.

Yes. Somehow my original commit included only part of the needed
pygnulib/* modifications; I missed to add it to the TODO file.

Bruno

Re: gnulib-tool.py: Follow gnulib-tool changes, part 29, 30.

[Bruno Haible]

Collin Funk wrote:
> These two TODO entries were pretty simple. One of the ChangeLog
> entries listed to a test case Bruno wrote that worked for testing both
> of them.
> 
> [collin@debian gnulib]$ gnulib-tool.py --create-testdir --dir=testdir-all 
> --single-configure
> module parse-datetime2 depends on a module with an incompatible license: 
> gettime
> module parse-datetime2 depends on a module with an incompatible license: 
> nstrftime
> module parse-datetime2 depends on a module with an incompatible license: 
> parse-datetime
> module parse-datetime2 depends on a module with an incompatible license: 
> time_rz
> module parse-datetime2 depends on a module with an incompatible license: 
> timespec
> module parse-datetime2 depends on a module with an incompatible license: tzset
> module ucs4-utf16 depends on a module with an incompatible license: 
> unistr/u16-uctomb
> module utf16-ucs4 depends on a module with an incompatible license: 
> unistr/u16-mbtouc
> module utf16-ucs4-unsafe depends on a module with an incompatible license: 
> unistr/u16-mbtouc-unsafe
> 
> The first patch fixes the parse-datetime2 warnings in the same way
> that gnulib-tool does (ready for the release of parse-datetime3). The
> second fixes the libunistring ones. Let me know if I missed anything.

Thanks! Applied both patches.

Bruno

gnulib-tool.py: Follow gnulib-tool changes, part 31

[Collin Funk]

And another patch to stop gnulib-tool.py from messing with copyright
headers. This patch is just doing the same as this commit did for
gnulib-tool:

https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=f3553bdefc9202e81996c73686e3ca53cd827417

I assume that this commit somehow escaped being added to
gnulibtool.py.TODO so I've made a reference to it anyways.

CollinFrom 983e8e95e90b108a2e126bb2a34d3ee0ad3dd721 Mon Sep 17 00:00:00 2001
From: Collin Funk 
Date: Tue, 27 Feb 2024 02:01:59 -0800
Subject: [PATCH 3/3] gnulib-tool.py: Follow gnulib-tool changes, part 31.

Follow gnulib-tool change
2021-06-04  Bruno Haible  
gnulib-tool: Stop doing license notice replacements.

* pygnulib/GLConfig.py (GLConfig.default): Remove 'copyrights' key. Add
'gnu_make' key which returns False.
(checkCopyrights): Remove function.
(setSymbolic, resetSymbolic): Remove 'copyrights' key.
* pygnulib/GLImport.py (GLImport.prepare): Remove copyrights variable.
Don't modify license notice on source files.
(GLImport.execute): Remove copyrights variable.
---
 ChangeLog| 14 ++
 pygnulib/GLConfig.py | 10 +-
 pygnulib/GLImport.py | 46 
 3 files changed, 15 insertions(+), 55 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index fdf08ad934..ed517802e8 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,17 @@
+2024-02-27  Collin Funk  
+
+	gnulib-tool.py: Follow gnulib-tool changes, part 31.
+	Follow gnulib-tool change
+	2021-06-04  Bruno Haible  
+	gnulib-tool: Stop doing license notice replacements.
+	* pygnulib/GLConfig.py (GLConfig.default): Remove 'copyrights' key. Add
+	'gnu_make' key which returns False.
+	(checkCopyrights): Remove function.
+	(setSymbolic, resetSymbolic): Remove 'copyrights' key.
+	* pygnulib/GLImport.py (GLImport.prepare): Remove copyrights variable.
+	Don't modify license notice on source files.
+	(GLImport.execute): Remove copyrights variable.
+
 2024-02-26  Collin Funk  
 
 	gnulib-tool.py: Follow gnulib-tool changes, part 30.
diff --git a/pygnulib/GLConfig.py b/pygnulib/GLConfig.py
index bdebf243cc..7c26a4873f 100644
--- a/pygnulib/GLConfig.py
+++ b/pygnulib/GLConfig.py
@@ -274,12 +274,10 @@ class GLConfig(object):
 return 2.59
 elif key == 'verbosity':
 return 0
-elif key == 'copyrights':
-return True
 elif key in ['modules', 'avoids', 'tests', 'incl_test_categories', 'excl_test_categories']:
 return list()
 elif key in ['libtool', 'lgpl', 'conddeps', 'symbolic', 'lsymbolic',
- 'libtests', 'dryrun']:
+ 'libtests', 'dryrun', 'gnu_make']:
 return False
 elif key == 'vc_files':
 return None
@@ -960,10 +958,6 @@ class GLConfig(object):
 '''Specify preferred autoconf version. Default value is 2.59.'''
 self.table['ac_version'] = 2.59
 
-def checkCopyrights(self):
-'''Check if copyright notices in files should be replaced.'''
-return self.table['copyrights']
-
 # Define symbolic methods.
 def checkSymbolic(self):
 '''Check if pygnulib will make symbolic links instead of copying files.'''
@@ -973,7 +967,6 @@ class GLConfig(object):
 '''Enable / disable creation of the symbolic links instead of copying files.'''
 if type(value) is bool:
 self.table['symbolic'] = value
-self.table['copyrights'] = not value
 else:  # if type(value) is not bool
 raise TypeError('value must be a bool, not %s'
 % type(value).__name__)
@@ -981,7 +974,6 @@ class GLConfig(object):
 def resetSymbolic(self):
 '''Reset creation of the symbolic links instead of copying files.'''
 self.table['symbolic'] = False
-self.table['copyrights'] = True
 
 # Define lsymbolic methods.
 def checkLSymbolic(self):
diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index 7f332ec760..0f0f463add 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -781,7 +781,6 @@ AC_DEFUN([%s_FILE_LIST], [\n''' % macro_prefix
 docbase = self.config['docbase']
 testsbase = self.config['testsbase']
 lgpl = self.config['lgpl']
-copyrights = self.config['copyrights']
 libname = self.config['libname']
 makefile_name = self.config['makefile_name']
 conddeps = self.config['conddeps']
@@ -885,12 +884,6 @@ AC_DEFUN([%s_FILE_LIST], [\n''' % macro_prefix
 print(notice)
 
 # Determine script to apply to imported library files.
-lgpl2gpl = '''
-  s/GNU Lesser General/GNU General/g
-  s/Lesser General Public License/General Public License/g
-  s/GNU Library General/GNU General/g
-  s/Library General Public License/General Public License/g
-  s/version 2\\(.1\\)\\{0,1\\}\\([ ,]\\)/version 3\\2/g'''
 sed_transform_lib