Pacify -Wmissing-variable-declarations in unit tests.

[Collin Funk]

When building Coreutils I noticed that GCC 14's
-Wmissing-variable-declarations option spams the test output.

Can I apply the attached patch which adds the 'static' specifier to
global variables in unit tests?

Or would it be better to just disable the warning in tests? I'm used
to marking things static even in small single file programs, but maybe
others will forget the convention.

I think a few of the unistr, etc. tests also trigger this warning. I
left them unchanged since IIRC most are auto-generated.

CollinFrom a81727a18045c64b56f2bfd2cebbf61bb072f529 Mon Sep 17 00:00:00 2001
From: Collin Funk 
Date: Sat, 27 Apr 2024 23:37:38 -0700
Subject: [PATCH] Pacify -Wmissing-variable-declarations in unit tests.

* tests/jit/test-cache.c: Add a static specifier to global variables.
* tests/minus-zero.h: Likewise.
* tests/test-acos.c: Likewise.
* tests/test-acosf.c: Likewise.
* tests/test-acosl.c: Likewise.
* tests/test-alignasof.c: Likewise.
* tests/test-alignof.c: Likewise.
* tests/test-alloca-opt.c: Likewise.
* tests/test-argp-version-etc.c: Likewise.
* tests/test-argp.c: Likewise.
* tests/test-asin.c: Likewise.
* tests/test-asinf.c: Likewise.
* tests/test-asinl.c: Likewise.
* tests/test-asyncsafe-spin1.c: Likewise.
* tests/test-atan.c: Likewise.
* tests/test-atan2.c: Likewise.
* tests/test-atan2f.c: Likewise.
* tests/test-atanf.c: Likewise.
* tests/test-atanl.c: Likewise.
* tests/test-cbrt.h: Likewise.
* tests/test-copysign.c: Likewise.
* tests/test-copysignf.c: Likewise.
* tests/test-copysignl.c: Likewise.
* tests/test-cos.c: Likewise.
* tests/test-cosf.c: Likewise.
* tests/test-cosh.c: Likewise.
* tests/test-coshf.c: Likewise.
* tests/test-cosl.c: Likewise.
* tests/test-erf.c: Likewise.
* tests/test-erfc.c: Likewise.
* tests/test-errno.c: Likewise.
* tests/test-exclude.c: Likewise.
* tests/test-exp.h: Likewise.
* tests/test-exp2.h: Likewise.
* tests/test-expm1.h: Likewise.
* tests/test-fabs.h: Likewise.
* tests/test-fcntl-h.c: Likewise.
* tests/test-fenv.c: Likewise.
* tests/test-float.c: Likewise.
* tests/test-fma2.c: Likewise.
* tests/test-fmaf2.c: Likewise.
* tests/test-fmal2.c: Likewise.
* tests/test-fmod.h: Likewise.
* tests/test-fnmatch-h.c: Likewise.
* tests/test-gc-pbkdf2-sha1.c: Likewise.
* tests/test-gc-pbkdf2.c: Likewise.
* tests/test-get-rusage-as.c: Likewise.
* tests/test-get-rusage-data.c: Likewise.
* tests/test-getentropy.c: Likewise.
* tests/test-getrusage.c: Likewise.
* tests/test-glob-h.c: Likewise.
* tests/test-hypot.h: Likewise.
* tests/test-iconv-utf.c: Likewise.
* tests/test-ilogb.h: Likewise.
* tests/test-intprops.c: Likewise.
* tests/test-inttypes.c: Likewise.
* tests/test-isfinite.c: Likewise.
* tests/test-isinf.c: Likewise.
* tests/test-iswblank.c: Likewise.
* tests/test-j0.c: Likewise.
* tests/test-j1.c: Likewise.
* tests/test-jn.c: Likewise.
* tests/test-langinfo.c: Likewise.
* tests/test-lgamma.c: Likewise.
* tests/test-limits-h.c: Likewise.
* tests/test-locale.c: Likewise.
* tests/test-log.h: Likewise.
* tests/test-log10.h: Likewise.
* tests/test-log1p.h: Likewise.
* tests/test-log2.h: Likewise.
* tests/test-logb.h: Likewise.
* tests/test-malloca.c: Likewise.
* tests/test-modf.h: Likewise.
* tests/test-nan-1.c: Likewise.
* tests/test-nan-2.c: Likewise.
* tests/test-netdb.c: Likewise.
* tests/test-nextafter.c: Likewise.
* tests/test-noreturn.c: Likewise.
* tests/test-nullptr.c: Likewise.
* tests/test-pathmax.c: Likewise.
* tests/test-poll-h.c: Likewise.
* tests/test-pow.c: Likewise.
* tests/test-powf.c: Likewise.
* tests/test-pthread.c: Likewise.
* tests/test-remainder.h: Likewise.
* tests/test-sched.c: Likewise.
* tests/test-sethostname1.c: Likewise.
* tests/test-signal-h.c: Likewise.
* tests/test-signbit.c: Likewise.
* tests/test-sigpipe.c: Likewise.
* tests/test-sigsegv-catch-segv1.c: Likewise.
* tests/test-sigsegv-catch-segv2.c: Likewise.
* tests/test-sin.c: Likewise.
* tests/test-sinf.c: Likewise.
* tests/test-sinh.c: Likewise.
* tests/test-sinhf.c: Likewise.
* tests/test-sinl.c: Likewise.
* tests/test-snan-1.c: Likewise.
* tests/test-snan-2.c: Likewise.
* tests/test-spawn.c: Likewise.
* tests/test-sqrt.h: Likewise.
* tests/test-stdbool.c: Likewise.
* tests/test-stddef.c: Likewise.
* tests/test-stdint.c: Likewise.
* tests/test-stdio.c: Likewise.
* tests/test-string-buffer.c: Likewise.
* tests/test-sys_random.c: Likewise.
* tests/test-sys_resource.c: Likewise.
* tests/test-sys_select.c: Likewise.
* tests/test-sys_socket.c: Likewise.
* tests/test-sys_stat.c: Likewise.
* tests/test-sys_time.c: Likewise.
* tests/test-sys_types.c: Likewise.
* tests/test-sys_uio.c: Likewise.
* tests/test-sysexits.c: Likewise.
* tests/test-tan.c: Likewise.
* tests/test-tanf.c: Likewise.
* tests/test-tanh.c: Likewise.
* tests/test-tanhf.c: Likewise.
* tests/test-tanl.c: Likewise.
* tests/test-termios.c: Likewise.
* tests/test-thrd_current.c: Likewise.
* tests/test-thread_self.c: Likewise.
* tests/test-threads.c: Likewise.
* tests/test-time-h.c: Likewise.
* tests/test-uchar.c: Likewise.
* tests/t

Re: Suggestion for cherry-picking to stable branches

[Sam James]

Bruno Haible  writes:

> Sam James wrote:
>> It's metadata in addition to the
>> commit summary (matching based on a title isn't easy, it's way easier if
>> someone says "here's the commits it's based on"; one can give multiple
>> such lines).
>
> It would significantly increase the amount of work for maintaining a stable
> branch. No, thanks.

Sure, no worries. I wasn't aware of your workflow and just wanted to
make the suggestion.

>
>> it makes life far easier if investigating a bug.
>
> The way I check whether a certain bug fix is present is
>   1. to get the fix from master,
>   2. find a particular code line / hunk that is essential for the fix,
>   3. look if this change is present in the branch.
>
> This is more reliable than to look at the git log or ChangeLog, because
> when a patch has not been applied correctly in the branch (due to
> rejects, maybe) there is a mismatch between the advertised list of fixes
> and the actual list of fixes.

Thanks!

best,
sam

Re: gnulib-tool.sh: Match sorting of gnulib-tool.py in output.

[Collin Funk]

Hi Bruno,

On 4/27/24 5:29 PM, Collin Funk wrote:
> I'm thinking that this is due to GLTestDir creating a GLEmiter in
> __init__() like this:
[...]
> The 'libtests' value read from the config in self.emitter is
> incorrect.

This patch seems to support what I was saying here.

diff --git a/pygnulib/GLTestDir.py b/pygnulib/GLTestDir.py
index 002eb30267..11b067e085 100644
--- a/pygnulib/GLTestDir.py
+++ b/pygnulib/GLTestDir.py
@@ -244,8 +244,7 @@ def execute(self) -> None:
 if file.startswith('lib/'):
 libtests = True
 break
-if libtests:
-self.config.setLibtests(True)
+self.emitter.config.setLibtests(libtests)
 
 if single_configure:
 # Add the dummy module to the main module list if needed.

Though, the correct way to fix this would be making instance variables
local when they are only used in one function.

I was having a look at doing that when I submitted this patch:

https://lists.gnu.org/archive/html/bug-gnulib/2024-04/msg00444.html

Since I also noticed some in GLImport. Since we don't want to
introduce any major changes for a bit after your announcement, should
we just leave this for a few days?

Thanks.

Collin

Re: Suggestion for cherry-picking to stable branches

[Bruno Haible]

Sam James wrote:
> It's metadata in addition to the
> commit summary (matching based on a title isn't easy, it's way easier if
> someone says "here's the commits it's based on"; one can give multiple
> such lines).

It would significantly increase the amount of work for maintaining a stable
branch. No, thanks.

> it makes life far easier if investigating a bug.

The way I check whether a certain bug fix is present is
  1. to get the fix from master,
  2. find a particular code line / hunk that is essential for the fix,
  3. look if this change is present in the branch.

This is more reliable than to look at the git log or ChangeLog, because
when a patch has not been applied correctly in the branch (due to
rejects, maybe) there is a mismatch between the advertised list of fixes
and the actual list of fixes.

Bruno

Re: Suggestion for cherry-picking to stable branches

[Sam James]

Bruno Haible  writes:

>> > easy to compare with the original commit and distinguish
>> > branch-only commits from backports.
>
> You can get limited insights by comparing the ChangeLogs of the
> master branch with a stable branch.
>
> But if what you want is a mechanically verifiable assertion of
> any kind, I can tell you that none exists. Backporting patches
> is, ultimately, manual developer work. (This is obvious by the
> fact that developers who have to maintain 6 or 7 backport branches
> spend a *lot* of time on that.) You may trust or may not trust
> this developer work, but there is no mechanical way to prove
> that you can trust it.

I wasn't asking from a perpective of auditing, more that it makes life
far easier if investigating a bug. It's metadata in addition to the
commit summary (matching based on a title isn't easy, it's way easier if
someone says "here's the commits it's based on"; one can give multiple
such lines).

If it's not easy for you to add that metadata with your workflow, that's
fine, of course.

thanks,
sam

Re: Suggestion for cherry-picking to stable branches

[Bruno Haible]

Sam James wrote:
> The expectation isn't that it directly matches the 'cherry-picked from
> ...' commits, but rather that it gives some anchor to compare to /
> expresses intent, rather than having to compare just the commit summary
> (as those aren't unique).

You can get such an "anchor"

(a) by using fuzzy-compare on the 'git log' entries, or
(b) by doing a side-by-side diff of the ChangeLogs, followed by a
"git annotate" of matching ChangeLog entries.

Bruno

Re: Suggestion for cherry-picking to stable branches

[Bruno Haible]

> > easy to compare with the original commit and distinguish
> > branch-only commits from backports.

You can get limited insights by comparing the ChangeLogs of the
master branch with a stable branch.

But if what you want is a mechanically verifiable assertion of
any kind, I can tell you that none exists. Backporting patches
is, ultimately, manual developer work. (This is obvious by the
fact that developers who have to maintain 6 or 7 backport branches
spend a *lot* of time on that.) You may trust or may not trust
this developer work, but there is no mechanical way to prove
that you can trust it.

Bruno

Re: Suggestion for cherry-picking to stable branches

[Sam James]

Bruno Haible  writes:

> Sam James wrote:
>> Could I propose that cherry-picks to the stable-XXX branches are done
>> with 'git cherry-pick -x'? This is often done in projects following a
>> gnulib-like model.
>> 
>> git will append '(cherry picked from commit ...)' to the commit message
>> which makes it easy to compare with the original commit and distinguish
>> branch-only commits from backports.
>
> All commits to the stable branches [1] are backports. There are no commits
> that are created for the branch specifically.
>
> But while doing these backports, I have to make several adjustments:
>   - Combine 2 or more commits from 'master' if the first of these commits
> introduced a regression that was only fixed later on.
>   - Assign new '# serial' numbers to the *.m4 files.
>   - Omit modifications to modules that did not exist when the stable branch
> was forked off.
>   - Omit documentation changes that don't apply.
>   - Fix merge conflicts.
>   - For stable branches from the previous year: Update the (C) year
> of each modified file.
>
> I don't think that a tool like 'git cherry-pick' will allow me to have
> the needed flexibility for this process.

Ah, so, in a way, this is precisely why the line is useful, because
they're *not* direct cherry-picks and it's useful to be able to
correspond them to what the mainline change(s) were.

But if you're not using 'git cherry-pick' already for these, then I can
understand it's a big workflow change and probably not worth it.

The expectation isn't that it directly matches the 'cherry-picked from
...' commits, but rather that it gives some anchor to compare to /
expresses intent, rather than having to compare just the commit summary
(as those aren't unique).

>
> Bruno
>
> [1] https://www.gnu.org/software/gnulib/manual/html_node/Stable-Branches.html

Re: gnulib-tool.sh: Match sorting of gnulib-tool.py in output.

[Collin Funk]

On 4/27/24 4:06 PM, Bruno Haible wrote:
> The remaining problem now is:
> 
> $ rm -rf ../testdir3; GNULIB_TOOL_IMPL=sh+py ./gnulib-tool 
> --create-megatestdir --dir=../testdir3 --single-configure stdio 
> sys_types
> ./gnulib-tool: *** gnulib-tool.py produced different files than 
> gnulib-tool.sh! Compare ../testdir3 and ../testdir3-glpy3133646.
> ./gnulib-tool: *** Stop.

Thanks for figuring out the test case for me.

I'm thinking that this is due to GLTestDir creating a GLEmiter in
__init__() like this:

self.emitter = GLEmiter(self.config)

then in GLTestDir.execute():

# Determine whether a $testsbase/libtests.a is needed.
libtests = False
for module in tests_modules:
files = module.getFiles()
for file in files:
if file.startswith('lib/'):
libtests = True
break
if libtests:
self.config.setLibtests(True)

The emiter has no clue that libtests is changed. So following those
lines when we print the configure.ac:

if single_configure:
# Create $testsbase/Makefile.am.
destfile = joinpath(directory, 'Makefile.am')
witness_macro = '%stests_WITNESS' % macro_prefix
emit = self.emitter.tests_Makefile_am(destfile, tests_modules, 
moduletable,
  self.makefiletable, 
witness_macro, for_test)

The 'libtests' value read from the config in self.emitter is
incorrect.

That's what it seems like at least. Going to eat dinner then work on a
patch. Hopefully my thoughts are correct...

Collin

Re: Suggestion for cherry-picking to stable branches

[Bruno Haible]

Sam James wrote:
> Could I propose that cherry-picks to the stable-XXX branches are done
> with 'git cherry-pick -x'? This is often done in projects following a
> gnulib-like model.
> 
> git will append '(cherry picked from commit ...)' to the commit message
> which makes it easy to compare with the original commit and distinguish
> branch-only commits from backports.

All commits to the stable branches [1] are backports. There are no commits
that are created for the branch specifically.

But while doing these backports, I have to make several adjustments:
  - Combine 2 or more commits from 'master' if the first of these commits
introduced a regression that was only fixed later on.
  - Assign new '# serial' numbers to the *.m4 files.
  - Omit modifications to modules that did not exist when the stable branch
was forked off.
  - Omit documentation changes that don't apply.
  - Fix merge conflicts.
  - For stable branches from the previous year: Update the (C) year
of each modified file.

I don't think that a tool like 'git cherry-pick' will allow me to have
the needed flexibility for this process.

Bruno

[1] https://www.gnu.org/software/gnulib/manual/html_node/Stable-Branches.html

Suggestion for cherry-picking to stable branches

[Sam James]

Hi,

Could I propose that cherry-picks to the stable-XXX branches are done
with 'git cherry-pick -x'? This is often done in projects following a
gnulib-like model.

git will append '(cherry picked from commit ...)' to the commit message
which makes it easy to compare with the original commit and distinguish
branch-only commits from backports.

thanks,
sam


signature.asc
Description: PGP signature

Re: [PATCH 2/2] nullptr: work around GCC 14 nullptr sentinel bug

[Sam James]

Paul Eggert  writes:

> On 2024-04-27 15:17, Sam James wrote:
>> Someone might read this and wrongly think that "GCC 14"
>> is broken.
>> I'd just omit 14 here.
>
> Good point as I think some of these bugs are also in GCC 13.x for some
> value of x.
>
> I installed the attached. It's not quite what you asked for but I hope
> it addresses the issue.
>

WFM, cheers paul!

> [2. text/x-patch; 0001-maint-be-more-precise-and-vague-about-GCC-14.patch]...

Re: [PATCH 2/2] nullptr: work around GCC 14 nullptr sentinel bug

[Paul Eggert]

On 2024-04-27 15:17, Sam James wrote:

Someone might read this and wrongly think that "GCC 14"
is broken.

I'd just omit 14 here.


Good point as I think some of these bugs are also in GCC 13.x for some 
value of x.


I installed the attached. It's not quite what you asked for but I hope 
it addresses the issue.


From b2b3e3b754f865834c8c66f3b2f1d73fecea2216 Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Sat, 27 Apr 2024 16:07:16 -0700
Subject: [PATCH] maint: be more precise and vague about GCC 14
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In commentary, be specific about which particular GCC 14 prerelease is
meant when this matters, and don’t say “GCC 14” otherwise.
Prompted by a remark by Sam James in:
https://lists.gnu.org/r/bug-gnulib/2024-04/msg00484.html
---
 ChangeLog | 4 ++--
 doc/posix-headers/stddef.texi | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 34893ebff1..185daab415 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -31,10 +31,10 @@
 
 2024-04-27  Paul Eggert  
 
-	nullptr: work around GCC 14 nullptr sentinel bug
+	nullptr: work around GCC nullptr sentinel bug
 	* m4/nullptr.m4 (gl_NULLPTR): Work around GCC bug 114780.
 
-	stddef: work around GCC 14 stddef.h bugs
+	stddef: work around GCC stddef.h bugs
 	* lib/stddef.in.h: Do nothing if _@GUARD_PREFIX@_STDDEF_H is
 	defined, as stddef.h has already been included.  This works
 	around GCC bug 114870.
diff --git a/doc/posix-headers/stddef.texi b/doc/posix-headers/stddef.texi
index 0e331481aa..00860bade0 100644
--- a/doc/posix-headers/stddef.texi
+++ b/doc/posix-headers/stddef.texi
@@ -48,7 +48,7 @@ GCC 12, Clang 15, and other pre-2023 C compilers.
 @item
 Some platforms define @code{nullptr_t} even when @code{} is
 not included:
-GCC 14.0.
+GCC 14.0.1 20240411 (Red Hat 14.0.1-0).
 
 @item
 Some platforms provide an @code{offsetof} macro that cannot be used in
-- 
2.40.1

Re: gnulib-tool.sh: Match sorting of gnulib-tool.py in output.

[Bruno Haible]

Hi Collin,

> >  GNULIB_TOOL_IMPL=sh+py ./gnulib-tool --create-megatestdir 
> > --dir=testdir2 --single-configure sys_types stdio
> > 
> > Can you double check on your machine?
> 
> I confirm, it passes for me as well now.

The remaining problem now is:

$ rm -rf ../testdir3; GNULIB_TOOL_IMPL=sh+py ./gnulib-tool --create-megatestdir 
--dir=../testdir3 --single-configure stdio sys_types
./gnulib-tool: *** gnulib-tool.py produced different files than gnulib-tool.sh! 
Compare ../testdir3 and ../testdir3-glpy3133646.
./gnulib-tool: *** Stop.
$ diff -r -q ../testdir3 ../testdir3-glpy3133646
Files ../testdir3/sys_types/gltests/Makefile.am and 
../testdir3-glpy3133646/sys_types/gltests/Makefile.am differ
Files ../testdir3/sys_types/gltests/Makefile.in and 
../testdir3-glpy3133646/sys_types/gltests/Makefile.in differ
$ diff -u ../testdir3/sys_types/gltests/Makefile.am 
../testdir3-glpy3133646/sys_types/gltests/Makefile.am
--- ../testdir3/sys_types/gltests/Makefile.am   2024-04-28 00:59:01.911451722 
+0200
+++ ../testdir3-glpy3133646/sys_types/gltests/Makefile.am   2024-04-28 
00:58:50.951373470 +0200
@@ -32,6 +32,7 @@
 EXTRA_PROGRAMS =
 noinst_HEADERS =
 noinst_LIBRARIES =
+noinst_LIBRARIES += libtests.a
 pkgdata_DATA =
 EXTRA_DIST =
 BUILT_SOURCES =
@@ -52,7 +53,13 @@
   -I.. -I$(srcdir)/.. \
   -I../gllib -I$(srcdir)/../gllib
 
-LDADD = ../gllib/libgnu.a
+LDADD = libtests.a ../gllib/libgnu.a libtests.a ../gllib/libgnu.a libtests.a 
$(LIBTESTS_LIBDEPS)
+
+libtests_a_SOURCES =
+libtests_a_LIBADD = $(gltests_LIBOBJS)
+libtests_a_DEPENDENCIES = $(gltests_LIBOBJS)
+EXTRA_libtests_a_SOURCES =
+AM_LIBTOOLFLAGS = --preserve-dup-deps
 
 TESTS_ENVIRONMENT += EXEEXT='@EXEEXT@' srcdir='$(srcdir)'
 

Re: ac_cv_sys_largefile_opts undocumented?

[dmitrii . pasechnik]

On Sat, Apr 27, 2024 at 11:42:23PM +0200, Bruno Haible wrote:
> dmitrii.pasechnik wrote:
> > By the way, ac_cv_sys_largefile_opts isn't fun to use - as sometimes
> > it's a plain text, and sometimes flags which should be added to CFLAGS.
> > So one has to write things like
> > 
> > AS_CASE([$ac_cv_sys_largefile_opts],
> >   ["none needed"], [],
> >   ["support not detected"], [],
> >   dnl otherwise it is a compiler flag
> >   [LARGEFILECFLAGS="$ac_cv_sys_largefile_opts"]
> >   )
> > 
> > which are unnecessarily complicated.
> 
> All ac_cv_* variables, and in fact all *_cv_* variables, are designed to
> contain the answer to a
>   checking for ...
> output line, in a way that is both machine-parseable and human-readable.
> 
> As a consequence:
>   - Its values look sometimes odd (e.g. 'no (bad O_NOATIME, O_NOFOLLOW)'.
>   - Its values are not stable over time and therefore *cannot be documented*.

From the documentation of AC_SYS_LARGEFILE (which is a wall of text,
talking about many things, including year 2038 support, something that
is quite surprising in the context) it's not possible to figure out what
should be added to CFLAGS, if anything.
So it seems that such a use of ac_cv_sys_largefile_opts as above may be 
described as "off-label".

The AS_CASE() code above also suffers from a problem that in the future
more non-compiler-flag alternatives may be added to ac_cv_sys_largefile_opts
and so it might break.
The only alternative seems to be calling the compiler to check whether 
$ac_cv_sys_largefile_opts
is a compiler option. But that's insane - after all, at the time 
ac_cv_sys_largefile_opts
is set up, it's perfectly known whether or not it's a compiler flag, but
this information appears  to be thrown away.

Dima

> 
> The entities which can often be documented are AC_SUBSTed variables and
> the C/C++ preprocessor macro definitions (AC_DEFINE).
> 
> If there is the need for an Autoconf macro to expose more details, in a
> documented way, it should be done through a new AC_SUBSTed variable or
> a new AC_DEFINE.
> 
> Bruno
> 
> 
> 
> 


signature.asc
Description: PGP signature

Re: ac_cv_sys_largefile_opts undocumented?

[Paul Eggert]

On 2024-04-27 15:39, dmitrii.pasech...@cs.ox.ac.uk wrote:

It's for nauty, a well-known program to deal with graph isomorphisms
etc. I've made a Gentoo patch herehttps://bugs.gentoo.org/921138
and I wanted to upstream it.


Oh my, that is indeed in an undocumented/hacky part of autoconf, one 
that I just changed in the master development sources a few days ago; see:


https://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=b71143738516017f0e0d347a4025301c06c40254

From your patch, it appears that nauty isn't using config.h and instead 
has its own file nauty-h.in that it maintains itself. Perhaps you could 
lessen this sort of problem by having it use config.h instead, at least 
for some things. Failing that, you could instead assume that 
ac_cv_sys_largefile_opts's value is a compiler option if and only if it 
begins with "-"; that's easy to say in a shell case statement and though 
still hacky would be a bit less brittle than what you've got.

Re: gnulib-tool.sh: Match sorting of gnulib-tool.py in output.

[Collin Funk]

On 4/27/24 3:22 PM, Bruno Haible wrote:
> Thanks. There's a nit, though: These lines do not remove duplicates.
> 
> module_set = set(modules)
> modules = [ module
> for module in modules
> if module in module_set ]

Oops... Today isn't my day I guess.

I think the least annoying way to remove duplicates while preserving
order is this trick I mentioned a few months ago:

 print(list(dict.fromkeys([5, 1, 3, 100, 3, 3, 5, 5, 5])))
 [5, 1, 3, 100]

This works since we require Python 3.7 [1]:

Changed in version 3.7: Dictionary order is guaranteed to be insertion
order. This behavior was an implementation detail of CPython from 3.6.

There is also collections.OrderedDict for previous versions though.

> I confirm, it passes for me as well now.
> 
> But I'm confused: I thought part of the problem was that a libtests was being
> generated. Putting the modules in a different order makes the libtests
> disappear?? What is going on?

Likewise. The sorting of gnulib-tool.py was still incorrect, so that
commit was needed minus my silly set mistake.

Sounds like it is time for me to learn how the Python debuggers work
and step through it. :)

[1] https://docs.python.org/3/library/stdtypes.html#mapping-types-dict

Collin

Re: ac_cv_sys_largefile_opts undocumented?

[dmitrii . pasechnik]

On Sat, Apr 27, 2024 at 02:26:24PM -0700, Paul Eggert wrote:
> On 2024-04-27 14:06, dmitrii.pasech...@cs.ox.ac.uk wrote:
> > Thus I got questions whether a patch for a build system I submitted for
> > a project is OK, as it uses an undocumented variable (thus, perhaps,
> > unstable).
> 
> Although it's poorly designed and not documented and not stable, the 
> patch may be good anyway. What's the patch for?

It's for nauty, a well-known program to deal with graph isomorphisms
etc. I've made a Gentoo patch here https://bugs.gentoo.org/921138
and I wanted to upstream it.

Dima



signature.asc
Description: PGP signature

Re: gnulib-tool.sh: Match sorting of gnulib-tool.py in output.

[Bruno Haible]

Hi Collin,

> I've pushed the attached patch

Thanks. There's a nit, though: These lines do not remove duplicates.

module_set = set(modules)
modules = [ module
for module in modules
if module in module_set ]

See:
>>> modules = ['a', 'b', 'a', 'c', 'b']
>>> module_set = set(modules)
>>> modules = [ module  for module in modules  if module in module_set ]
>>> modules
['a', 'b', 'a', 'c', 'b']


> since it allows me to run this and pass:
> 
>  GNULIB_TOOL_IMPL=sh+py ./gnulib-tool --create-megatestdir --dir=testdir2 
> --single-configure sys_types stdio
> 
> Can you double check on your machine?

I confirm, it passes for me as well now.

But I'm confused: I thought part of the problem was that a libtests was being
generated. Putting the modules in a different order makes the libtests
disappear?? What is going on?

Bruno

Re: [PATCH 2/2] nullptr: work around GCC 14 nullptr sentinel bug

[Sam James]

Paul Eggert  writes:

> * m4/nullptr.m4 (gl_NULLPTR): Work around GCC bug 114780.
> ---
>  ChangeLog   |  3 +++
>  doc/gnulib.texi |  4 
>  m4/nullptr.m4   | 18 --
>  3 files changed, 23 insertions(+), 2 deletions(-)
>
> diff --git a/ChangeLog b/ChangeLog
> index b30238f934..e341b62968 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1,5 +1,8 @@
>  2024-04-27  Paul Eggert  
>  
> + nullptr: work around GCC 14 nullptr sentinel bug
> + * m4/nullptr.m4 (gl_NULLPTR): Work around GCC bug 114780.
> +
>   stddef: work around GCC 14 stddef.h bugs
>   * lib/stddef.in.h: Do nothing if _@GUARD_PREFIX@_STDDEF_H is
>   defined, as stddef.h has already been included.  This works
> diff --git a/doc/gnulib.texi b/doc/gnulib.texi
> index ac8c01d1e1..aa0eb57f62 100644
> --- a/doc/gnulib.texi
> +++ b/doc/gnulib.texi
> @@ -978,6 +978,10 @@ Portability problems fixed by Gnulib:
>  Some platforms lack @code{nullptr}:
>  For C: GCC 12, Clang 15, and other pre-2023 C compilers.
>  For C++: pre-2011 C++ compilers.
> +
> +@item
> +Some platforms incorrectly warn when @code{nullptr} is a sentinel argument:
> +GCC 13.2 and 14.0.

This is perhaps a bit confusing, even if technically correct. It'll be
fixed in the final release (GCC 14.1), as Jakub fixed it on 2024-04-23.

GCC X.0 will never be released and refers to a development version (just
like GCC X.Y.1). Someone might read this and wrongly think that "GCC 14"
is broken.

I'd just omit 14 here.

>  @end itemize
>  
>  Portability problems not fixed by Gnulib:
> diff --git a/m4/nullptr.m4 b/m4/nullptr.m4
> index e99495..4f2284296a 100644
> --- a/m4/nullptr.m4
> +++ b/m4/nullptr.m4
> @@ -1,5 +1,5 @@
>  # nullptr.m4
> -# serial 1
> +# serial 2
>  dnl Copyright 2023-2024 Free Software Foundation, Inc.
>  dnl This file is free software; the Free Software Foundation
>  dnl gives unlimited permission to copy and/or distribute it,
> @@ -14,7 +14,21 @@ AC_DEFUN([gl_NULLPTR],
>   AC_CACHE_CHECK([for C nullptr], [gl_cv_c_nullptr],
> [AC_COMPILE_IFELSE(
>[AC_LANG_SOURCE([[int *p = nullptr;]])],
> -  [gl_cv_c_nullptr=yes],
> +  [gl_cv_c_nullptr=yes
> +   # Work around 
> .
> +   gl_saved_CFLAGS=$CFLAGS
> +   CFLAGS="$CFLAGS -Wall -Werror"
> +   AC_COMPILE_IFELSE(
> + [AC_LANG_PROGRAM(
> +[[void f (char const *, ...) __attribute__ ((sentinel));]],
> +[[f ("", nullptr);]])],
> + [],
> + [AC_COMPILE_IFELSE(
> +[AC_LANG_PROGRAM(
> +   [[void f (char const *, ...) __attribute__ 
> ((sentinel));]],
> +   [[f ("", (void *) 0);]])],
> +[gl_cv_c_nullptr='not as a sentinel'])])
> +   CFLAGS=$gl_saved_CFLAGS],
>[gl_cv_c_nullptr=no])])
>gl_c_nullptr=$gl_cv_c_nullptr
>AC_LANG_POP([C])],

Re: gnulib-tool.sh: Match sorting of gnulib-tool.py in output.

[Collin Funk]

On 4/27/24 2:29 PM, Bruno Haible wrote:
> Also, the SUBDIRS variable in a Makefile.am determines the order in which
> the subdirectories are traversed during a build. If a subdirectory has some
> chances to fail the build or the tests, it should be mentioned last, so
> that the other (more reliable) subdirectories can at least be built and
> tested. Thus, sorting SUBDIRS in alphabetical order is generally unwelcome.
> 
> It is gnulib-tool.py which needs to adapt.

Thanks for the explanation. I agree, I overlooked the purpose of
SUBDIRS in the original message.

I've pushed the attached patch since it allows me to run this and
pass:

 GNULIB_TOOL_IMPL=sh+py ./gnulib-tool --create-megatestdir --dir=testdir2 
--single-configure sys_types stdio

Can you double check on your machine?

CollinFrom dba810f623ad02476401faddccfdcaf234db7b5e Mon Sep 17 00:00:00 2001
From: Collin Funk 
Date: Sat, 27 Apr 2024 15:01:24 -0700
Subject: [PATCH] gnulib-tool.py: Preserve module ordering in
 --create-megatestdir.

* pygnulib/GLTestDir.py (GLMegaTestDir.execute): Use a separate set to
remove duplicates from the original list without sorting.
---
 ChangeLog | 6 ++
 pygnulib/GLTestDir.py | 7 ++-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/ChangeLog b/ChangeLog
index 935ddbd1ba..34893ebff1 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2024-04-27  Collin Funk  
+
+	gnulib-tool.py: Preserve module ordering in --create-megatestdir.
+	* pygnulib/GLTestDir.py (GLMegaTestDir.execute): Use a separate set to
+	remove duplicates from the original list without sorting.
+
 2024-04-27  Bruno Haible  
 
 	fcntl-h, stdio, unistd: Ensure off64_t is defined on all platforms.
diff --git a/pygnulib/GLTestDir.py b/pygnulib/GLTestDir.py
index 758a65168e..002eb30267 100644
--- a/pygnulib/GLTestDir.py
+++ b/pygnulib/GLTestDir.py
@@ -876,7 +876,12 @@ def execute(self) -> None:
 modules = self.modulesystem.list()
 modules = [ self.modulesystem.find(m)
 for m in modules ]
-modules = sorted(set(modules))
+# Preserve ordering from the command-line, but remove duplicates.
+# This allows control over the SUBDIRS variable in the top-level Makefile.am.
+module_set = set(modules)
+modules = [ module
+for module in modules
+if module in module_set ]
 
 # First, all modules one by one.
 for module in modules:
-- 
2.44.0

Re: gnulib-tool: Use the Python implementation by default

[Paul Eggert]

On 2024-04-27 11:49, Bruno Haible wrote:

   - In many recent systems, 'python3' exists and 'python' does not.
 This includes Ubuntu 23.10 and 24.04. (No idea why it's different
 on your machine.)


Oh, I see I have the python-is-python3 package installed on my Ubuntu 
platform. I had forgotten that Debian wants users by default to live in 
a compatibility hell where the 'python' command does not work.



I have not seen a single system where 'python' exists, is version 3, and
'python3' does not exist.


I was thinking more of the future, where I expect such systems to exist. 
I suppose we can wait until then.

Re: ac_cv_sys_largefile_opts undocumented?

[Bruno Haible]

dmitrii.pasechnik wrote:
> By the way, ac_cv_sys_largefile_opts isn't fun to use - as sometimes
> it's a plain text, and sometimes flags which should be added to CFLAGS.
> So one has to write things like
> 
> AS_CASE([$ac_cv_sys_largefile_opts],
>   ["none needed"], [],
>   ["support not detected"], [],
>   dnl otherwise it is a compiler flag
>   [LARGEFILECFLAGS="$ac_cv_sys_largefile_opts"]
>   )
> 
> which are unnecessarily complicated.

All ac_cv_* variables, and in fact all *_cv_* variables, are designed to
contain the answer to a
  checking for ...
output line, in a way that is both machine-parseable and human-readable.

As a consequence:
  - Its values look sometimes odd (e.g. 'no (bad O_NOATIME, O_NOFOLLOW)'.
  - Its values are not stable over time and therefore *cannot be documented*.

The entities which can often be documented are AC_SUBSTed variables and
the C/C++ preprocessor macro definitions (AC_DEFINE).

If there is the need for an Autoconf macro to expose more details, in a
documented way, it should be done through a new AC_SUBSTed variable or
a new AC_DEFINE.

Bruno

Re: gnulib-tool.sh: Match sorting of gnulib-tool.py in output.

[Bruno Haible]

Hi Collin,

> diff -ru testdir2/Makefile.am testdir2-glpy85706/Makefile.am
> --- testdir2/Makefile.am  2024-04-27 13:53:51.443692945 -0700
> +++ testdir2-glpy85706/Makefile.am2024-04-27 13:53:33.063660555 -0700
> @@ -2,6 +2,6 @@
>  
>  AUTOMAKE_OPTIONS = 1.14 foreign
>  
> -SUBDIRS = sys_types stdio ALL
> +SUBDIRS = stdio sys_types ALL

Indeed, this part of the failure is a question of sorting.

> The behavior of gnulib-tool.py is to sort the modules and
> gnulib-tool.sh uses them as passed.
> 
> I feel like the best solution here is just to sort the modules in
> gnulib-tool.sh.

I disagree. The route that we have been following over the last two months
— and that you can verify through 'gitk gnulib-tool.sh' — is that we change
gnulib-tool.sh only when it is clearly a bug fix or an improvement.

Adding sorting is neither a bug fix nor an improvement. It is arbitrary.

Also, the SUBDIRS variable in a Makefile.am determines the order in which
the subdirectories are traversed during a build. If a subdirectory has some
chances to fail the build or the tests, it should be mentioned last, so
that the other (more reliable) subdirectories can at least be built and
tested. Thus, sorting SUBDIRS in alphabetical order is generally unwelcome.

It is gnulib-tool.py which needs to adapt.

Bruno

Re: ac_cv_sys_largefile_opts undocumented?

[Paul Eggert]

On 2024-04-27 14:06, dmitrii.pasech...@cs.ox.ac.uk wrote:

Thus I got questions whether a patch for a build system I submitted for
a project is OK, as it uses an undocumented variable (thus, perhaps,
unstable).


Although it's poorly designed and not documented and not stable, the 
patch may be good anyway. What's the patch for?

gnulib-tool.sh: Match sorting of gnulib-tool.py in output.

[Collin Funk]

Hi Bruno,

When looking at your bug report I noticed a few lines of the diff are
just different orderings.

Running:

GNULIB_TOOL_IMPL=sh+py ./gnulib-tool --create-megatestdir --dir=testdir2 
--single-configure sys_types stdio

We see in the diff:

diff -ru testdir2/Makefile.am testdir2-glpy85706/Makefile.am
--- testdir2/Makefile.am2024-04-27 13:53:51.443692945 -0700
+++ testdir2-glpy85706/Makefile.am  2024-04-27 13:53:33.063660555 -0700
@@ -2,6 +2,6 @@
 
 AUTOMAKE_OPTIONS = 1.14 foreign
 
-SUBDIRS = sys_types stdio ALL
+SUBDIRS = stdio sys_types ALL

This can also be seen in configure.ac and the generated configure
script. The behavior of gnulib-tool.py is to sort the modules and
gnulib-tool.sh uses them as passed.

I feel like the best solution here is just to sort the modules in
gnulib-tool.sh. Since gnulib-tool.py turns the strings into a set of
GLModules and then sorts them.

Can I apply the attached patch? Asking since I feel less comfortable
making changes outside of pygnulib/* and I'm not sure if/how to apply
them to stable branches.

CollinFrom c6877b8f0fcd34a96f8874199eafae14b3241d77 Mon Sep 17 00:00:00 2001
From: Collin Funk 
Date: Sat, 27 Apr 2024 14:03:27 -0700
Subject: [PATCH] gnulib-tool.sh: Match sorting of gnulib-tool.py in output.

* gnulib-tool.sh (func_create_megatestdir): Sort the modules so
AC_CONFIG_SUBDIRS in configure.ac and SUBDIRS in Makefile.am are also
sorted.
---
 ChangeLog  | 7 +++
 gnulib-tool.sh | 1 +
 2 files changed, 8 insertions(+)

diff --git a/ChangeLog b/ChangeLog
index 935ddbd1ba..61518a200e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2024-04-27  Collin Funk  
+
+	gnulib-tool.sh: Match sorting of gnulib-tool.py in output.
+	* gnulib-tool.sh (func_create_megatestdir): Sort the modules so
+	AC_CONFIG_SUBDIRS in configure.ac and SUBDIRS in Makefile.am are also
+	sorted.
+
 2024-04-27  Bruno Haible  
 
 	fcntl-h, stdio, unistd: Ensure off64_t is defined on all platforms.
diff --git a/gnulib-tool.sh b/gnulib-tool.sh
index b486e99b1e..13a9bdf535 100755
--- a/gnulib-tool.sh
+++ b/gnulib-tool.sh
@@ -7135,6 +7135,7 @@ func_create_megatestdir ()
   if test -z "$allmodules"; then
 allmodules=`func_all_modules`
   fi
+  allmodules=`for module in $allmodules; do echo $module; done | sort -u`
 
   megasubdirs=
   # First, all modules one by one.
-- 
2.44.0

ac_cv_sys_largefile_opts undocumented?

[dmitrii . pasechnik]

Hello,
I am not 100% sure whether this should go to a gnulib list or to an autoconf 
list,
but anyway, ac_cv_sys_largefile_opts (from a subroutine in
AC_SYS_LARGEFILE) is not documented in either gnulib manual, or in
autoconf manual. (It is documented in the source).

Thus I got questions whether a patch for a build system I submitted for 
a project is OK, as it uses an undocumented variable (thus, perhaps,
unstable).

By the way, ac_cv_sys_largefile_opts isn't fun to use - as sometimes
it's a plain text, and sometimes flags which should be added to CFLAGS.
So one has to write things like

AS_CASE([$ac_cv_sys_largefile_opts],
  ["none needed"], [],
  ["support not detected"], [],
  dnl otherwise it is a compiler flag
  [LARGEFILECFLAGS="$ac_cv_sys_largefile_opts"]
  )

which are unnecessarily complicated.

Please advice, and possibly fix.

Best,
Dima



signature.asc
Description: PGP signature

Re: gnulib-tool.py --create-megatestdir bug

[Collin Funk]

Hi Bruno,

On 4/27/24 8:48 AM, Bruno Haible wrote:
> Could you please look at this one? I created a megatestdir for some
> portability testing, with gnulib-tool.py (of course), and it failed
> on macOS:

Thanks for the report! I can reproduce it.

I have only had a brief look and it looks like gnulib-tool.py is
adding libtests where gnulib-tool.sh isn't.

I'll have a look at fixing that right now.

Collin

Re: gnulib-tool: Use the Python implementation by default

[Collin Funk]

On 4/27/24 11:49 AM, Bruno Haible wrote:
>   - In many older systems, 'python3' exists and 'python' exists as well and is
> a 2.7.x version.
> 
> I have not seen a single system where 'python' exists, is version 3, and
> 'python3' does not exist.

There is this PEP which came about with the Python 2 to Python 3
transition:

https://peps.python.org/pep-0394/

If I am interpreting it correctly 'python3' would be correct.

The only other python implementation that I tried was mypy3 which
works fine. I wanted to try Jython but it only supports version 2.7.

Collin

Re: printf functions without INT_MAX limitation

[Bruno Haible]

Paul Eggert wrote:
> Although I don't use Haiku, my impression is that 
> off_t is 64 bits on Haiku. See, for example, 
> .

Indeed, you're right. Tested on 32-bit Haiku installations from 2017 and 2022.

Bruno

Re: gnulib-tool: Use the Python implementation by default

[Bruno Haible]

Paul Eggert wrote:
> > Are there alternative Python implementations, which could be installed under
> > the name 'python3'?
> 
> On the platforms I currently have terminals open on (Fedora 40, Ubuntu 
> 23.10) it's called 'python'. You can also use 'python3' but I would 
> suggest trying 'python' first if you aren't already doing that. These 
> platforms do not have Python 2, which is typical nowadays.

We would gain nothing by trying 'python' before 'python3'.

  - In many recent systems, 'python3' exists and 'python' does not.
This includes Ubuntu 23.10 and 24.04. (No idea why it's different
on your machine.)

  - In some recent systems, 'python3' exists and 'python' is a symlink to it.
Such as on Fedora 40.

  - In many older systems, 'python3' exists and 'python' exists as well and is
a 2.7.x version.

I have not seen a single system where 'python' exists, is version 3, and
'python3' does not exist.

Bruno

Re: gnulib-tool: Use the Python implementation by default

[Bruno Haible]

Hi Pádraig,

Thanks for the details.

> So this is a general question, unrelated to my particular suggestion.

It _is_ related :-)

> Possible drop in replacements for the python3 command might be ...
> 
> ... pypy, which separates its implementation version from python language 
> support version:
> $ dnf install pypy3
> $ pypy3
> 
>   sys.pypy_version_info
> sys.pypy_version_info(major=7, minor=3, micro=15, releaselevel='final', 
> serial=0)
>   sys.version_info
> sys.version_info(major=3, minor=10, micro=13, releaselevel='final', serial=0)
> 
> ... cinder, which is very close to cpython ...
> $ dnf install docker
> $ sudo systemctl start docker
> $ sudo setfacl --modify user:$USER:rw /var/run/docker.sock
> $ docker run -it --rm ghcr.io/facebookincubator/cinder-runtime:cinder-3.10
> 
> Python 3.10.5+cinder (main, Apr 26 2024, 22:00:58) [GCC 10.3.1 20210422 (Red 
> Hat 10.3.1-1)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import sys
>  >>> sys.version_info
> sys.version_info(major=3, minor=10, micro=5, releaselevel='final', serial=0)
> 
> 
> There are no other real contenders for drop in python 3 replacement.

OK. This means that people might be tempted to create a symlink
python3 -> pypy3
or
python3 -> cinder.

With the values that you have shown, it will be indeed better to use
sys.version_info.

> Testing shows that a subshell does avoid that issue, so the following works:
> 
>if (python3 -c 'import sys; sys.exit(not sys.version_info >= (3,7))') 
> 2>/dev/null; then

Thanks, I'm taking this one.


2024-04-27  Bruno Haible  

gnulib-tool: Simplify the Python version test.
Suggested by Pádraig Brady in
.
* gnulib-tool: Use Python's sys.version_info, not --version.
* gnulib-tool.py: Use the same version test here. Use --version only to
produce a specific error message.

diff --git a/gnulib-tool b/gnulib-tool
index 441958ae7e..56c4473318 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -143,12 +143,7 @@ case "$GNULIB_TOOL_IMPL" in
   '')
 # Use the Python implementation if a suitable Python version is found
 # in $PATH. This is the same Python version test as in gnulib-tool.py.
-if (python3 --version) >/dev/null 2>/dev/null \
-   && case `python3 --version 2>&1` in
-Python\ 3.[0-6] | Python\ 3.[0-6].*) false ;;
-Python\ 3.*) true ;;
-*) false ;;
-  esac; then
+if (python3 -c 'import sys; sys.exit(not sys.version_info >= (3,7))') 
2>/dev/null; then
   exec "$gnulib_dir/gnulib-tool.py" "$@"
 else
   echo "gnulib-tool: warning: python3 not found or too old, using the slow 
shell-based implementation" 1>&2
diff --git a/gnulib-tool.py b/gnulib-tool.py
index 81537c272c..52389dcd78 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -131,17 +131,21 @@
 func_gnulib_dir
 
 # Check the Python version.
-if (python3 --version) >/dev/null 2>/dev/null; then
-  case `python3 --version 2>&1` in
-Python\ 3.[0-6] | Python\ 3.[0-6].*)
-  func_fatal_error "python3 is too old (minimum required version is 3.7); 
try setting GNULIB_TOOL_IMPL=sh" ;;
-Python\ 3.*)
-  ;;
-*)
-  func_fatal_error "python3 version is unsupported" ;;
-  esac
+if (python3 -c 'import sys; sys.exit(not sys.version_info >= (3,7))') 
2>/dev/null; then
+  :
 else
-  func_fatal_error "python3 not found; try setting GNULIB_TOOL_IMPL=sh"
+  if (python3 --version) >/dev/null 2>/dev/null; then
+case `python3 --version 2>&1` in
+  Python\ 3.[0-6] | Python\ 3.[0-6].*)
+func_fatal_error "python3 is too old (minimum required version is 
3.7); try setting GNULIB_TOOL_IMPL=sh" ;;
+  Python\ 3.*)
+;;
+  *)
+func_fatal_error "python3 version is unsupported" ;;
+esac
+  else
+func_fatal_error "python3 not found; try setting GNULIB_TOOL_IMPL=sh"
+  fi
 fi
 
 # Tell Python to store the compiled bytecode outside the gnulib directory.

Re: printf functions without INT_MAX limitation

[Bruno Haible]

Paul Eggert wrote:
> now that you mention off64_t it strikes me that zoff_t would 
> basically be off64_t, and off64_t has had its own problems: its only use 
> in apps is to deal with deficient libraries, and it is a pain in 
> libraries (where its only use is to deal with deficient apps :-). I 
> don't offhand see why zoff_t would do any better than off64_t has done, 
> or why we would need to give a new name to this unloved type.

You've convinced me to use off64_t instead of zoff_t.

My main argument against off64_t was that it is non-standard. But
  - There is no platform where off64_t is different from int64_t.
  - off64_t is defined by glibc, and since Gnulib's job is to offer
the APIs provided by glibc portably, as far as possible, below
are two patches that implement off64_t portably.

With that done, there is indeed no reason any more to invent zoff_t.

Some notes:
- In glibc, off64_t is defined in , , ,
  , .
- In POSIX, off_t is defined in the same header files, and also in
   and .
- off64_t is existent and equivalent to 'long long' on
glibc, musl, FreeBSD, AIX, HP-UX, IRIX, Solaris, mingw, Android.
- off64_t is missing on
macOS, FreeBSD ≤ 10, NetBSD, OpenBSD, MSVC, Cygwin, Haiku, Minix.
- The configure.ac-early section is because otherwise I got an
  autoconf warning

configure.ac:211: warning: AC_COMPILE_IFELSE was called before 
AC_USE_SYSTEM_EXTENSIONS
glm4/off64_t.m4:11: gl_TYPE_OFF64_T is expanded from...
configure.ac:28: gl_INIT is expanded from...
configure.ac:211: the top level
configure.ac:211: warning: AC_CHECK_INCLUDES_DEFAULT was called before 
AC_USE_SYSTEM_EXTENSIONS
glm4/off64_t.m4:11: gl_TYPE_OFF64_T is expanded from...
configure.ac:28: gl_INIT is expanded from...
configure.ac:211: the top level

  I don't know why this has not already occurred earlier; we have
  dozens of modules which require AC_USE_SYSTEM_EXTENSIONS.


2024-04-27  Bruno Haible  

fcntl-h, stdio, unistd: Ensure off64_t is defined on all platforms.
* lib/fcntl.in.h: Update comment regarding off64_t.
* lib/stdio.in.h: Likewise.
* lib/unistd.in.h: Likewise.
* tests/test-fcntl-h.c: Verify that off64_t is defined.
* tests/test-stdio.c: Likewise.
* tests/test-unistd.c: Likewise.
* doc/posix-headers/fcntl.texi: Mention the off64_t workaround.
* doc/posix-headers/stdio.texi: Likewise.
* doc/posix-headers/unistd.texi: Likewise.
* doc/posix-headers/aio.texi: Mention the problem with off64_t.

sys_types: Ensure off64_t is defined on all platforms.
* m4/off64_t.m4: New file.
* m4/sys_types_h.m4 (gl_SYS_TYPES_H): Require gl_TYPE_OFF64_T.
* lib/sys_types.in.h (off64_t): New type.
(GNULIB_defined_off64_t): New macro.
* modules/sys_types (Files): Add m4/off64_t.m4.
(configure.ac-early): Require AC_USE_SYSTEM_EXTENSIONS.
(Makefile.am): Substitute HAVE_OFF64_T.
* tests/test-sys_types.c: Verify that off64_t is defined.
* doc/posix-headers/sys_types.texi: Mention the off64_t workaround.

>From adc372decc54eea2da8f2a7f5587be939aa8b6e3 Mon Sep 17 00:00:00 2001
From: Bruno Haible 
Date: Sat, 27 Apr 2024 19:46:34 +0200
Subject: [PATCH 1/2] sys_types: Ensure off64_t is defined on all platforms.

* m4/off64_t.m4: New file.
* m4/sys_types_h.m4 (gl_SYS_TYPES_H): Require gl_TYPE_OFF64_T.
* lib/sys_types.in.h (off64_t): New type.
(GNULIB_defined_off64_t): New macro.
* modules/sys_types (Files): Add m4/off64_t.m4.
(configure.ac-early): Require AC_USE_SYSTEM_EXTENSIONS.
(Makefile.am): Substitute HAVE_OFF64_T.
* tests/test-sys_types.c: Verify that off64_t is defined.
* doc/posix-headers/sys_types.texi: Mention the off64_t workaround.
---
 ChangeLog| 13 +
 doc/posix-headers/sys_types.texi |  3 +++
 lib/sys_types.in.h   |  9 +
 m4/off64_t.m4| 31 +++
 m4/sys_types_h.m4|  5 -
 modules/sys_types|  5 +
 tests/test-sys_types.c   |  1 +
 7 files changed, 66 insertions(+), 1 deletion(-)
 create mode 100644 m4/off64_t.m4

diff --git a/ChangeLog b/ChangeLog
index e341b62968..8467f57a50 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2024-04-27  Bruno Haible  
+
+	sys_types: Ensure off64_t is defined on all platforms.
+	* m4/off64_t.m4: New file.
+	* m4/sys_types_h.m4 (gl_SYS_TYPES_H): Require gl_TYPE_OFF64_T.
+	* lib/sys_types.in.h (off64_t): New type.
+	(GNULIB_defined_off64_t): New macro.
+	* modules/sys_types (Files): Add m4/off64_t.m4.
+	(configure.ac-early): Require AC_USE_SYSTEM_EXTENSIONS.
+	(Makefile.am): Substitute HAVE_OFF64_T.
+	* tests/test-sys_types.c: Verify that off64_t is defined.
+	* doc/posix-headers/sys_types.texi: Mention the off64_t workaround.
+
 2024-04-27  Paul Eggert  
 
 	nullptr: work around GCC 14 nullptr sentinel bug
diff --git a/doc/posix-headers/sys_t

Re: gnulib-tool: Use the Python implementation by default

[Paul Eggert]

On 2024-04-27 06:32, Bruno Haible wrote:

Are there alternative Python implementations, which could be installed under
the name 'python3'?


On the platforms I currently have terminals open on (Fedora 40, Ubuntu 
23.10) it's called 'python'. You can also use 'python3' but I would 
suggest trying 'python' first if you aren't already doing that. These 
platforms do not have Python 2, which is typical nowadays.


There are other Python implementations - for example, GraalVM supports 
Python 3.8 and the GraalVM folks say they do a lot of compatibility 
checking. If people want (and have the time) to check portability of the 
new Python code in Gnulib, that might be a good alternative to try. 
Among other things, they claim a big performance improvement over 
CPython - I don't know how well that would work for us though.

Re: gnulib-tool: Use the Python implementation by default

[Pádraig Brady]

On 27/04/2024 14:32, Bruno Haible wrote:

Hi Pádraig,


+if (python3 --version) >/dev/null 2>/dev/null \
+   && case `python3 --version 2>&1` in
+Python\ 3.[0-6] | Python\ 3.[0-6].*) false ;;
+Python\ 3.*) true ;;
+*) false ;;
+  esac; then


It may be preferable to state the supported version directly,


I wanted a short warning message. In most cases the absence of python3
will be the problem. It should be rare than anyone still uses an old
version.


and replace the above 6 lines with:

if python3 -c 'import sys; sys.exit(not sys.version_info >= (3,7))' 
2>/dev/null; then


Thanks for the hint. I admit that I did not think at it. But is it
actually better?


Only that's it's shorter, and more directly states the supported version.
No biggie, just a suggestion.



Are there alternative Python implementations, which could be installed under
the name 'python3'?


So this is a general question, unrelated to my particular suggestion.
I think the version check is fine, and should be quite independent from python 
implementation.
Possible drop in replacements for the python3 command might be ...

... pypy, which separates its implementation version from python language 
support version:
$ dnf install pypy3
$ pypy3

 sys.pypy_version_info
sys.pypy_version_info(major=7, minor=3, micro=15, releaselevel='final', 
serial=0)
 sys.version_info
sys.version_info(major=3, minor=10, micro=13, releaselevel='final', serial=0)

... cinder, which is very close to cpython ...
$ dnf install docker
$ sudo systemctl start docker
$ sudo setfacl --modify user:$USER:rw /var/run/docker.sock
$ docker run -it --rm ghcr.io/facebookincubator/cinder-runtime:cinder-3.10

Python 3.10.5+cinder (main, Apr 26 2024, 22:00:58) [GCC 10.3.1 20210422 (Red 
Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=10, micro=5, releaselevel='final', serial=0)


There are no other real contenders for drop in python 3 replacement.
For example codon or mojo are not designed as drop in replacements.
I expect any future drop in replacements would follow the model above.


Note also that your snippet would not replace the 6 lines of code, only the
last 5 lines, because with Solaris 10 /bin/sh your snippet produces the
diagnostic
   python3: not found


Out of interest, I dug out an old dsa ssh key from a backup, to get access to a 
solaris 10 machine.
Testing shows that a subshell does avoid that issue, so the following works:

  if (python3 -c 'import sys; sys.exit(not sys.version_info >= (3,7))') 
2>/dev/null; then

cheers,
Pádraig

Re: GNU gnulib: gnulib-tool has become much faster

[Paul Eggert]

On 2024-04-27 10:11, Tim Rühsen wrote:

Bruno, Dmitry, Collin, thank you so much !

The runtime improvement is amazing :)


Likewise. This is very much appreciated, especially on some of the 
trusty but old and slow hosts that I develop on.

[PATCH 2/2] nullptr: work around GCC 14 nullptr sentinel bug

[Paul Eggert]

* m4/nullptr.m4 (gl_NULLPTR): Work around GCC bug 114780.
---
 ChangeLog   |  3 +++
 doc/gnulib.texi |  4 
 m4/nullptr.m4   | 18 --
 3 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index b30238f934..e341b62968 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,8 @@
 2024-04-27  Paul Eggert  
 
+   nullptr: work around GCC 14 nullptr sentinel bug
+   * m4/nullptr.m4 (gl_NULLPTR): Work around GCC bug 114780.
+
stddef: work around GCC 14 stddef.h bugs
* lib/stddef.in.h: Do nothing if _@GUARD_PREFIX@_STDDEF_H is
defined, as stddef.h has already been included.  This works
diff --git a/doc/gnulib.texi b/doc/gnulib.texi
index ac8c01d1e1..aa0eb57f62 100644
--- a/doc/gnulib.texi
+++ b/doc/gnulib.texi
@@ -978,6 +978,10 @@ Portability problems fixed by Gnulib:
 Some platforms lack @code{nullptr}:
 For C: GCC 12, Clang 15, and other pre-2023 C compilers.
 For C++: pre-2011 C++ compilers.
+
+@item
+Some platforms incorrectly warn when @code{nullptr} is a sentinel argument:
+GCC 13.2 and 14.0.
 @end itemize
 
 Portability problems not fixed by Gnulib:
diff --git a/m4/nullptr.m4 b/m4/nullptr.m4
index e99495..4f2284296a 100644
--- a/m4/nullptr.m4
+++ b/m4/nullptr.m4
@@ -1,5 +1,5 @@
 # nullptr.m4
-# serial 1
+# serial 2
 dnl Copyright 2023-2024 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -14,7 +14,21 @@ AC_DEFUN([gl_NULLPTR],
  AC_CACHE_CHECK([for C nullptr], [gl_cv_c_nullptr],
[AC_COMPILE_IFELSE(
   [AC_LANG_SOURCE([[int *p = nullptr;]])],
-  [gl_cv_c_nullptr=yes],
+  [gl_cv_c_nullptr=yes
+   # Work around .
+   gl_saved_CFLAGS=$CFLAGS
+   CFLAGS="$CFLAGS -Wall -Werror"
+   AC_COMPILE_IFELSE(
+ [AC_LANG_PROGRAM(
+[[void f (char const *, ...) __attribute__ ((sentinel));]],
+[[f ("", nullptr);]])],
+ [],
+ [AC_COMPILE_IFELSE(
+[AC_LANG_PROGRAM(
+   [[void f (char const *, ...) __attribute__ ((sentinel));]],
+   [[f ("", (void *) 0);]])],
+[gl_cv_c_nullptr='not as a sentinel'])])
+   CFLAGS=$gl_saved_CFLAGS],
   [gl_cv_c_nullptr=no])])
   gl_c_nullptr=$gl_cv_c_nullptr
   AC_LANG_POP([C])],
-- 
2.44.0

[PATCH 1/2] stddef: work around GCC 14 stddef.h bugs

[Paul Eggert]

* lib/stddef.in.h: Do nothing if _@GUARD_PREFIX@_STDDEF_H is
defined, as stddef.h has already been included.  This works
around GCC bug 114870.
(_GCC_NULLPTR_T): Define if needed to work around GCC bug 114869.
* m4/stddef_h.m4 (gl_STDDEF_H, gl_STDDEF_H_DEFAULTS):
* modules/stddef (stddef.h):
Detect the two bugs.
---
 ChangeLog | 11 +++
 doc/posix-headers/stddef.texi |  5 +
 lib/stddef.in.h   | 23 +--
 m4/stddef_h.m4| 32 +++-
 modules/stddef|  1 +
 5 files changed, 65 insertions(+), 7 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index dacf6892b1..b30238f934 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2024-04-27  Paul Eggert  
+
+   stddef: work around GCC 14 stddef.h bugs
+   * lib/stddef.in.h: Do nothing if _@GUARD_PREFIX@_STDDEF_H is
+   defined, as stddef.h has already been included.  This works
+   around GCC bug 114870.
+   (_GCC_NULLPTR_T): Define if needed to work around GCC bug 114869.
+   * m4/stddef_h.m4 (gl_STDDEF_H, gl_STDDEF_H_DEFAULTS):
+   * modules/stddef (stddef.h):
+   Detect the two bugs.
+
 2024-04-27  Bruno Haible  
 
bootstrap: Support checking out a recent GNULIB_REVISION.
diff --git a/doc/posix-headers/stddef.texi b/doc/posix-headers/stddef.texi
index 33ad48244c..0e331481aa 100644
--- a/doc/posix-headers/stddef.texi
+++ b/doc/posix-headers/stddef.texi
@@ -45,6 +45,11 @@ Some platforms fail to provide @code{nullptr_t},
 which Gnulib cannot usefully emulate:
 GCC 12, Clang 15, and other pre-2023 C compilers.
 
+@item
+Some platforms define @code{nullptr_t} even when @code{} is
+not included:
+GCC 14.0.
+
 @item
 Some platforms provide an @code{offsetof} macro that cannot be used in
 arbitrary expressions:
diff --git a/lib/stddef.in.h b/lib/stddef.in.h
index fa8998d9b7..fa249259cd 100644
--- a/lib/stddef.in.h
+++ b/lib/stddef.in.h
@@ -27,13 +27,18 @@
 #endif
 @PRAGMA_COLUMNS@
 
-#if defined __need_wchar_t || defined __need_size_t  \
-  || defined __need_ptrdiff_t || defined __need_NULL \
-  || defined __need_wint_t
+#if (!defined _@GUARD_PREFIX@_STDDEF_H \
+ && (defined __need_wchar_t || defined __need_size_t \
+ || defined __need_ptrdiff_t || defined __need_NULL \
+ || defined __need_wint_t))
 /* Special invocation convention inside gcc header files.  In
-   particular, gcc provides a version of  that blindly
-   redefines NULL even when __need_wint_t was defined, even though
-   wint_t is not normally provided by .  Hence, we must
+   particular,  in some ancient versions of GCC blindly
+   redefined NULL when __need_wint_t was defined, even though wint_t
+   is not normally provided by .
+   (FIXME: It's not clear what GCC versions those were - perhaps so
+   ancient that we can stop worrying about this?)
+   Although glibc 2.26 (2017) and later do not use __need_wint_t,
+   for portability to older Glibc + GCC,
remember if special invocation has ever been used to obtain wint_t,
in which case we need to clean up NULL yet again.  */
 
@@ -74,6 +79,12 @@ typedef long max_align_t;
 #   endif
 #  endif
 
+#  if !defined _GCC_NULLPTR_T && !@NULLPTR_T_NEEDS_STDDEF@
+/* Suppress unwanted nullptr_t typedef.  See
+   .  */
+#   define _GCC_NULLPTR_T
+#  endif
+
 /* The include_next requires a split double-inclusion guard.  */
 
 #  @INCLUDE_NEXT@ @NEXT_STDDEF_H@
diff --git a/m4/stddef_h.m4 b/m4/stddef_h.m4
index 84d3bae801..0d4ddf8fcb 100644
--- a/m4/stddef_h.m4
+++ b/m4/stddef_h.m4
@@ -1,5 +1,5 @@
 # stddef_h.m4
-# serial 14
+# serial 15
 dnl Copyright (C) 2009-2024 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -84,6 +84,35 @@ AC_DEFUN_ONCE([gl_STDDEF_H],
 GL_GENERATE_STDDEF_H=true
   fi
 
+  dnl https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114869
+  AC_CACHE_CHECK([whether nullptr_t needs ],
+[gl_cv_nullptr_t_needs_stddef],
+[AC_COMPILE_IFELSE([AC_LANG_DEFINES_PROVIDED[nullptr_t x;]],
+   [gl_cv_nullptr_t_needs_stddef=no],
+   [gl_cv_nullptr_t_needs_stddef=yes])])
+  if test "$gl_cv_nullptr_t_needs_stddef" = no; then
+NULLPTR_T_NEEDS_STDDEF=0
+GL_GENERATE_STDDEF_H=true
+  fi
+
+  AC_CACHE_CHECK([for clean definition of __STDC_VERSION_STDDEF_H__],
+[gl_cv_clean_version_stddef],
+[AC_PREPROC_IFELSE(
+   [AC_LANG_SOURCE(
+  [[/* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114870 */
+#include 
+#undef __STDC_VERSION_STDDEF_H__
+#include 
+#ifdef __STDC_VERSION_STDDEF_H__
+# error " defines __STDC_VERSION_STDDEF_H__"
+#endif
+  ]])],
+[gl_cv_clean_version_stddef=yes],
+[gl_cv_clean_version_stddef=no])])
+  if test "$gl_cv_clean_version_stddef" = no; then
+GL_GENERATE_STDDEF_

Re: Fetch from existing gnulib Git repository if needed

[Bruno Haible]

Hi Markus,

> GNU Octave uses Mercurial as the VCS of its main repository.
> Developers are using the bootstrap script of gnulib to automatically clone 
> its Git repository in a subdirectory of Octave's source tree. The revision 
> that we'd like to use is set in the bootstrap.conf script. Currently, that is
> 
> : ${GNULIB_REVISION=d4ec02b3cc70cddaaa5183cc5a45814e0afb2292}
> 
> 
> This is working perfectly for a fresh clone of Octave's source tree. However, 
> when we update GNULIB_REVISION to a newer revision and a user/developer ran 
> the bootstrap script before, running the bootstrap script again fails with an 
> error like the following:
> 
> ./bootstrap: Bootstrapping from checked-out octave sources...
> fatal: reference is not a tree: d4ec02b3cc70cddaaa5183cc5a45814e0afb2292
> program finished with exit code 128

I see. Indeed there is no 'git fetch' in the code branch that you are
highlighting.

> As a workaround we are applying the attached patch to the 
> bootstrap-funclib.sh script to automatically fetch from the remote gnulib 
> repository if the GNULIB_REVISION isn't found in the local gnulib Git 
> repository.

Thanks for the patch. But note that GNULIB_REVISION can hold either a commit
hash or the name of a branch (such as 'stable-202401'). So, we have 4 cases:
  (a) a commit hash, already present
  (b) a commit hash, not known
  (c) a branch, already present
  (d) a branch, not known

The command 'git cat-file commit $GNULIB_REVISION' returns true in the cases
(a) and (c). So, your patch would trigger a 'git fetch' in the cases (b) and
(d). But in case (d), the 'git fetch' is useless:
'git cat-file commit $GNULIB_REVISION' would still fail afterwards.

One can distinguish the four cases in more detail using the commands
  git rev-list --quiet $GNULIB_REVISION --
which tests for case (a) and
  git show-ref --verify --quiet refs/heads/$GNULIB_REVISION
which tests for case (c). This would allow us to do 'git fetch' only in case
(b).

However, I believe the patch below is simpler and achieves the same goal.


2024-04-27  Bruno Haible  

bootstrap: Support checking out a recent GNULIB_REVISION.
Reported by Markus Mützel  in
.
* top/bootstrap-funclib.sh (prepare_GNULIB_SRCDIR): If the
'git checkout' command fails, fetch the newer commits and then retry it.
* build-aux/bootstrap: Regenerated.

diff --git a/top/bootstrap-funclib.sh b/top/bootstrap-funclib.sh
index 7511d62d9b..620006d320 100644
--- a/top/bootstrap-funclib.sh
+++ b/top/bootstrap-funclib.sh
@@ -1,6 +1,6 @@
 # A library of shell functions for autopull.sh, autogen.sh, and bootstrap.
 
-scriptlibversion=2024-04-13.15; # UTC
+scriptlibversion=2024-04-27.17; # UTC
 
 # Copyright (C) 2003-2024 Free Software Foundation, Inc.
 #
@@ -462,7 +462,17 @@ prepare_GNULIB_SRCDIR ()
   || die "Error: --gnulib-srcdir or \$GNULIB_SRCDIR is specified," \
  "but does not contain gnulib-tool"
 if test -n "$GNULIB_REVISION" && $use_git; then
-  (cd "$GNULIB_SRCDIR" && git checkout "$GNULIB_REVISION") || exit $?
+  # The 'git checkout "$GNULIB_REVISION"' command succeeds if the
+  # GNULIB_REVISION is a commit hash that exists locally, or if it is
+  # branch name that can be fetched from origin. It fails, however,
+  # if the GNULIB_REVISION is a commit hash that only exists in origin.
+  # In this case, we need a 'git fetch' and then retry
+  # 'git checkout "$GNULIB_REVISION"'.
+  (cd "$GNULIB_SRCDIR" \
+   && { git checkout "$GNULIB_REVISION" 2>/dev/null \
+|| { git fetch origin && git checkout "$GNULIB_REVISION"; }
+  }
+  ) || exit $?
 fi
   else
 if ! $use_git; then

Re: GNU gnulib: gnulib-tool has become much faster

[Tim Rühsen]

Bruno, Dmitry, Collin, thank you so much !

The runtime improvement is amazing :)

Regards, Tim

On 4/26/24 12:14, Bruno Haible wrote:

If you are developer on a package that uses GNU gnulib as part of its build
system:

gnulib-tool has been known for being slow for many years. We have listened to
your complaints. We have rewritten gnulib-tool in another programming language
(Python). It is between 8 times and 100 times faster than the previous
implementation.

Both implementations behave identically, that is, produce the same generated
files and the same output. Nothing changes in your way to use Gnulib; it's
only faster.

In order to reap the new speed:

1. Make sure you have Python (version 3.7 or newer) installed on your
machine.

2. Update your gnulib checkout. (For some packages, it comes as a git
submodule named 'gnulib'.) Like this:

   $ git checkout master
   $ git pull

   Set the environment variable GNULIB_SRCDIR, pointing to this checkout.

   If the package is using a git submodule named 'gnulib', it is also advisable
to do

   $ git commit -m 'build: Update gnulib submodule to latest.' gnulib

   (as a preparation for step 4, because the --no-git option does not work as
expected in all variants of 'bootstrap').

3. Clean the built files of your package:

   $ make -k distclean


4. Regenerate the fetched and generated files of your package. Depending on
the package, this may be a command such as

   $ ./bootstrap --no-git --gnulib-srcdir=$GNULIB_SRCDIR

   or

   $ export GNULIB_SRCDIR; ./autopull.sh; ./autogen.sh

   or, if no such script is available:

   $ $GNULIB_SRCDIR/gnulib-tool --update


5. Continue with

   $ ./configure
   $ make

   as usual.

Enjoy! The rewritten gnulib-tool was implemented by Dmitry Selyutin, Collin
Funk, and me.



___
Message sent via Savannah
https://savannah.gnu.org/




OpenPGP_signature.asc
Description: OpenPGP digital signature

gnulib-tool.py --create-megatestdir bug

[Bruno Haible]

Hi Collin,

Could you please look at this one? I created a megatestdir for some
portability testing, with gnulib-tool.py (of course), and it failed
on macOS:

Making all in gltests
##  ##
## --- Gnulib tests --- ##
## You can ignore compiler warnings in this directory.  ##
##  ##
/Library/Developer/CommandLineTools/usr/bin/make  all-recursive
Making all in .
cc -DHAVE_CONFIG_H -I. -I../../../sys_types/gltests -I..  
-DGNULIB_STRICT_CHECKING=1 -DIN_GNULIB_TESTS=1 -I. -I../../../sys_types/gltests 
-I.. -I../../../sys_types/gltests/.. -I../gllib 
-I../../../sys_types/gltests/../gllib -I/Users/haible/include -Wall  -Wno-error 
-g -O2 -MT test-sys_types.o -MD -MP -MF .deps/test-sys_types.Tpo -c -o 
test-sys_types.o ../../../sys_types/gltests/test-sys_types.c
mv -f .deps/test-sys_types.Tpo .deps/test-sys_types.Po
rm -f libtests.a
ar cr libtests.a
ar: no archive members specified
usage:  ar -d [-TLsv] archive file ...
ar -m [-TLsv] archive file ...
ar -m [-abiTLsv] position archive file ...
ar -p [-TLsv] archive [file ...]
ar -q [-cTLsv] archive file ...
ar -r [-cuTLsv] archive file ...
ar -r [-abciuTLsv] position archive file ...
ar -t [-TLsv] archive [file ...]
ar -x [-ouTLsv] archive [file ...]
make[5]: *** [libtests.a] Error 1

The reason appears to be different Makefile.ams generated:

$ rm -rf ../testdir2; GNULIB_TOOL_IMPL=sh+py ./gnulib-tool --create-megatestdir 
--dir=../testdir2 --single-configure sys_types stdio
./gnulib-tool: *** gnulib-tool.py produced different files than gnulib-tool.sh! 
Compare ../testdir2 and ../testdir2-glpy2916071.
./gnulib-tool: *** Stop.

The megatestdir created by the shell implementation builds fine on macOS.

Bruno

Fetch from existing gnulib Git repository if needed

[Markus Mützel]

Dear gnulib developers,

GNU Octave uses Mercurial as the VCS of its main repository.
Developers are using the bootstrap script of gnulib to automatically clone its 
Git repository in a subdirectory of Octave's source tree. The revision that 
we'd like to use is set in the bootstrap.conf script. Currently, that is

: ${GNULIB_REVISION=d4ec02b3cc70cddaaa5183cc5a45814e0afb2292}


This is working perfectly for a fresh clone of Octave's source tree. However, 
when we update GNULIB_REVISION to a newer revision and a user/developer ran the 
bootstrap script before, running the bootstrap script again fails with an error 
like the following:

./bootstrap: Bootstrapping from checked-out octave sources...
fatal: reference is not a tree: d4ec02b3cc70cddaaa5183cc5a45814e0afb2292
program finished with exit code 128


To work around that, a user/developer could manually fetch from the remote 
repository. That is a bit more tedious when it comes to CI installations that 
usually need no manual interaction.

As a workaround we are applying the attached patch to the bootstrap-funclib.sh 
script to automatically fetch from the remote gnulib repository if the 
GNULIB_REVISION isn't found in the local gnulib Git repository.

Would it be possible to make a similar change in gnulib so that updating to a 
newer gnulib revision becomes a bit easier for that configuration?

Markus
Update bootstrap script from upstream gnulib to automatically fetch from 
repository if needed

diff -r c51b07a71421 bootstrap-funclib.sh
--- a/bootstrap-funclib.sh  Fri Apr 26 13:33:37 2024 -0400
+++ b/bootstrap-funclib.sh  Fri Apr 26 20:00:21 2024 +0200
@@ -532,6 +532,10 @@
 # The subdirectory 'gnulib' already exists.
 if test -n "$GNULIB_REVISION"; then
   if test -d "$gnulib_path/.git"; then
+if ! git --git-dir="$gnulib_path"/.git cat-file \
+ commit "$GNULIB_REVISION"; then
+  git --git-dir="$gnulib_path"/.git fetch
+fi
 (cd "$gnulib_path" && git checkout "$GNULIB_REVISION") || exit 1
   else
 die "Error: GNULIB_REVISION is specified in bootstrap.conf," \

Re: Modification of environment variables on Windows

[Markus Mützel]

Hi Bruno,

> > The attached patch avoids those warnings.
>
> Thanks, but it does not do the right thing: *s[1] accesses the first character
> of the string after s. What was meant was to access the second character of
> the string at s; this needs to be written as (*s)[1].

Oof. You are absolutely right. What a difference some parenthesis make.

> I'm committing this patch instead.

Thank you for spotting the error in the proposed patch and pushing a fix so 
quickly.

Markus

Re: Modification of environment variables on Windows

[Bruno Haible]

Hi Markus,

> We recently updated gnulib in GNU Octave to a newer revision 
> (d4ec02b3cc70cddaaa5183cc5a45814e0afb2292). (Kudos on the impressive speedup 
> to the bootstrap process.)
> 
> Since then, we are seeing warnings like the following when building for MinGW:
> 
> ../../libgnu/tzset.c: In function 'rpl_tzset':
> ../../libgnu/tzset.c:68:24: warning: initialization of 'char *' from 
> incompatible pointer type 'char **' [-Wincompatible-pointer-types]
>68 | for (char *s = env; *s != NULL; s++)
>   |^~~
> ../../libgnu/tzset.c:68:32: warning: comparison between pointer and integer
>68 | for (char *s = env; *s != NULL; s++)
>   |^~
> ../../libgnu/tzset.c:72:28: warning: initialization of 'wchar_t *' {aka 
> 'short unsigned int *'} from incompatible pointer type 'wchar_t **' {aka 
> 'short unsigned int **'} [-Wincompatible-pointer-types]
>72 | for (wchar_t *ws = wenv; *ws != NULL; ws++)
>   |^~~~
> ../../libgnu/tzset.c:72:38: warning: comparison between pointer and integer
>72 | for (wchar_t *ws = wenv; *ws != NULL; ws++)
>  ^~
> 
> IIUC, these warnings might be legitimate.

Thanks for the report. What a blunder, indeed!

> The attached patch avoids those warnings.

Thanks, but it does not do the right thing: *s[1] accesses the first character
of the string after s. What was meant was to access the second character of
the string at s; this needs to be written as (*s)[1].

I'm committing this patch instead.


2024-04-27  Bruno Haible  

ctime, localtime, tzset, wcsftime: Fix env access (regr. 2024-02-09).
Reported by Markus Mützel  in
.
* lib/ctime.c (rpl_ctime): Fix logic of environment traversal.
* lib/localtime.c (rpl_localtime): Likewise.
* lib/tzset.c (rpl_tzset): Likewise.
* lib/wcsftime.c (rpl_wcsftime): Likewise.

diff --git a/lib/ctime.c b/lib/ctime.c
index 8c54ef463c..744b153260 100644
--- a/lib/ctime.c
+++ b/lib/ctime.c
@@ -63,13 +63,19 @@ rpl_ctime (const time_t *tp)
   char **env = _environ;
   wchar_t **wenv = _wenviron;
   if (env != NULL)
-for (char *s = env; *s != NULL; s++)
-  if (s[0] == 'T' && s[1] == 'Z' && s[2] == '=')
-s[0] = '$';
+for (char **ep = env; *ep != NULL; ep++)
+  {
+char *s = *ep;
+if (s[0] == 'T' && s[1] == 'Z' && s[2] == '=')
+  s[0] = '$';
+  }
   if (wenv != NULL)
-for (wchar_t *ws = wenv; *ws != NULL; ws++)
-  if (ws[0] == L'T' && ws[1] == L'Z' && ws[2] == L'=')
-ws[0] = L'$';
+for (wchar_t **wep = wenv; *wep != NULL; wep++)
+  {
+wchar_t *ws = *wep;
+if (ws[0] == L'T' && ws[1] == L'Z' && ws[2] == L'=')
+  ws[0] = L'$';
+  }
 }
 #endif
 
diff --git a/lib/localtime.c b/lib/localtime.c
index f0e91ac647..df0278e5c2 100644
--- a/lib/localtime.c
+++ b/lib/localtime.c
@@ -63,13 +63,19 @@ rpl_localtime (const time_t *tp)
   char **env = _environ;
   wchar_t **wenv = _wenviron;
   if (env != NULL)
-for (char *s = env; *s != NULL; s++)
-  if (s[0] == 'T' && s[1] == 'Z' && s[2] == '=')
-s[0] = '$';
+for (char **ep = env; *ep != NULL; ep++)
+  {
+char *s = *ep;
+if (s[0] == 'T' && s[1] == 'Z' && s[2] == '=')
+  s[0] = '$';
+  }
   if (wenv != NULL)
-for (wchar_t *ws = wenv; *ws != NULL; ws++)
-  if (ws[0] == L'T' && ws[1] == L'Z' && ws[2] == L'=')
-ws[0] = L'$';
+for (wchar_t **wep = wenv; *wep != NULL; wep++)
+  {
+wchar_t *ws = *wep;
+if (ws[0] == L'T' && ws[1] == L'Z' && ws[2] == L'=')
+  ws[0] = L'$';
+  }
 }
 #endif
 
diff --git a/lib/tzset.c b/lib/tzset.c
index f307f0c3d1..93dc52e284 100644
--- a/lib/tzset.c
+++ b/lib/tzset.c
@@ -65,13 +65,19 @@ rpl_tzset (void)
   char **env = _environ;
   wchar_t **wenv = _wenviron;
   if (env != NULL)
-for (char *s = env; *s != NULL; s++)
-  if (s[0] == 'T' && s[1] == 'Z' && s[2] == '=')
-s[0] = '$';
+for (char **ep = env; *ep != NULL; ep++)
+  {
+char *s = *ep;
+if (s[0] == 'T' && s[1] == 'Z' && s[2] == '=')
+  s[0] = '$';
+  }
   if (wenv != NULL)
-for (wchar_t *ws = wenv; *ws != NULL; ws++)
-  if (ws[0] == L'T' && ws[1] == L'Z' && ws[2] == L'=')
-ws[0] = L'$';
+for (wchar_t **wep = wenv; *wep != NULL; wep++)
+  {
+wchar_t *ws = *wep;
+if (ws[0] == L'T' && ws[1] == L'Z' && ws[2] == L'=')
+  ws[0] = L'$';
+  }
 }
 
   /* On native Windows, tzset() is

Re: gnulib-tool: Use the Python implementation by default

[Bruno Haible]

Hi Pádraig,

> > +if (python3 --version) >/dev/null 2>/dev/null \
> > +   && case `python3 --version 2>&1` in
> > +Python\ 3.[0-6] | Python\ 3.[0-6].*) false ;;
> > +Python\ 3.*) true ;;
> > +*) false ;;
> > +  esac; then
> 
> It may be preferable to state the supported version directly,

I wanted a short warning message. In most cases the absence of python3
will be the problem. It should be rare than anyone still uses an old
version.

> and replace the above 6 lines with:
> 
>if python3 -c 'import sys; sys.exit(not sys.version_info >= (3,7))' 
> 2>/dev/null; then

Thanks for the hint. I admit that I did not think at it. But is it
actually better?

Are there alternative Python implementations, which could be installed under
the name 'python3'?

Note also that your snippet would not replace the 6 lines of code, only the
last 5 lines, because with Solaris 10 /bin/sh your snippet produces the
diagnostic
  python3: not found

Bruno

Re: gnulib-tool: Use the Python implementation by default

[Pádraig Brady]

On 26/04/2024 10:48, Bruno Haible wrote:

I committed this patch, which activates the Python rewrite of gnulib-tool
for all users who have Python installed, without the need to set any
environment variable. I'll make announcements to info-gnu and planet.gnu.org
soon.



+if (python3 --version) >/dev/null 2>/dev/null \
+   && case `python3 --version 2>&1` in
+Python\ 3.[0-6] | Python\ 3.[0-6].*) false ;;
+Python\ 3.*) true ;;
+*) false ;;
+  esac; then


It may be preferable to state the supported version directly,
and replace the above 6 lines with:

  if python3 -c 'import sys; sys.exit(not sys.version_info >= (3,7))' 
2>/dev/null; then


+  exec "$gnulib_dir/gnulib-tool.py" "$@"
+else
+  echo "gnulib-tool: warning: python3 not found or too old, using the slow shell-based 
implementation" 1>&2
+  exec "$gnulib_dir/gnulib-tool.sh" "$@"
+fi
+;;


I've updated to the latest gnulib now in coreutils,
and the auto python selection works well.

thanks!
Pádraig

Modification of environment variables on Windows

[Markus Mützel]

Dear gnulib developers,

We recently updated gnulib in GNU Octave to a newer revision 
(d4ec02b3cc70cddaaa5183cc5a45814e0afb2292). (Kudos on the impressive speedup to 
the bootstrap process.)

Since then, we are seeing warnings like the following when building for MinGW:

../../libgnu/tzset.c: In function 'rpl_tzset':
../../libgnu/tzset.c:68:24: warning: initialization of 'char *' from 
incompatible pointer type 'char **' [-Wincompatible-pointer-types]
   68 | for (char *s = env; *s != NULL; s++)
  |^~~
../../libgnu/tzset.c:68:32: warning: comparison between pointer and integer
   68 | for (char *s = env; *s != NULL; s++)
  |^~
../../libgnu/tzset.c:72:28: warning: initialization of 'wchar_t *' {aka 'short 
unsigned int *'} from incompatible pointer type 'wchar_t **' {aka 'short 
unsigned int **'} [-Wincompatible-pointer-types]
   72 | for (wchar_t *ws = wenv; *ws != NULL; ws++)
  |^~~~
../../libgnu/tzset.c:72:38: warning: comparison between pointer and integer
   72 | for (wchar_t *ws = wenv; *ws != NULL; ws++)
 ^~

IIUC, these warnings might be legitimate.

The attached patch avoids those warnings.

I'm hoping this is the correct format to propose changes to gnulib.

Best,
Markus Mützel
From 87b83c757f7d249f983410e85d13ba450d57b417 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Markus=20M=C3=BCtzel?= 
Date: Sat, 27 Apr 2024 13:44:29 +0200
Subject: [PATCH] ctime, localtime, tzset, wcsftime: Check content of
 environment.

* lib/ctime.c (rpl_ctime): Index into content of each environment variable.
* lib/localtime.c (rpl_localtime): Likewise.
* lib/tzset.c (rpl_tzset): Likewise.
* lib/wcsftime.c (rpl_wcsftime): Likewise.
---
 lib/ctime.c | 12 ++--
 lib/localtime.c | 12 ++--
 lib/tzset.c | 12 ++--
 lib/wcsftime.c  | 12 ++--
 4 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/lib/ctime.c b/lib/ctime.c
index 8c54ef463c..e30b7997c8 100644
--- a/lib/ctime.c
+++ b/lib/ctime.c
@@ -63,13 +63,13 @@ rpl_ctime (const time_t *tp)
   char **env = _environ;
   wchar_t **wenv = _wenviron;
   if (env != NULL)
-for (char *s = env; *s != NULL; s++)
-  if (s[0] == 'T' && s[1] == 'Z' && s[2] == '=')
-s[0] = '$';
+for (char **s = env; *s != NULL; s++)
+  if (*s[0] == 'T' && *s[1] == 'Z' && *s[2] == '=')
+*s[0] = '$';
   if (wenv != NULL)
-for (wchar_t *ws = wenv; *ws != NULL; ws++)
-  if (ws[0] == L'T' && ws[1] == L'Z' && ws[2] == L'=')
-ws[0] = L'$';
+for (wchar_t **ws = wenv; *ws != NULL; ws++)
+  if (*ws[0] == L'T' && *ws[1] == L'Z' && *ws[2] == L'=')
+*ws[0] = L'$';
 }
 #endif

diff --git a/lib/localtime.c b/lib/localtime.c
index f0e91ac647..1c6bb9856e 100644
--- a/lib/localtime.c
+++ b/lib/localtime.c
@@ -63,13 +63,13 @@ rpl_localtime (const time_t *tp)
   char **env = _environ;
   wchar_t **wenv = _wenviron;
   if (env != NULL)
-for (char *s = env; *s != NULL; s++)
-  if (s[0] == 'T' && s[1] == 'Z' && s[2] == '=')
-s[0] = '$';
+for (char **s = env; *s != NULL; s++)
+  if (*s[0] == 'T' && *s[1] == 'Z' && *s[2] == '=')
+*s[0] = '$';
   if (wenv != NULL)
-for (wchar_t *ws = wenv; *ws != NULL; ws++)
-  if (ws[0] == L'T' && ws[1] == L'Z' && ws[2] == L'=')
-ws[0] = L'$';
+for (wchar_t **ws = wenv; *ws != NULL; ws++)
+  if (*ws[0] == L'T' && *ws[1] == L'Z' && *ws[2] == L'=')
+*ws[0] = L'$';
 }
 #endif

diff --git a/lib/tzset.c b/lib/tzset.c
index f307f0c3d1..c8d48e1afb 100644
--- a/lib/tzset.c
+++ b/lib/tzset.c
@@ -65,13 +65,13 @@ rpl_tzset (void)
   char **env = _environ;
   wchar_t **wenv = _wenviron;
   if (env != NULL)
-for (char *s = env; *s != NULL; s++)
-  if (s[0] == 'T' && s[1] == 'Z' && s[2] == '=')
-s[0] = '$';
+for (char **s = env; *s != NULL; s++)
+  if (*s[0] == 'T' && *s[1] == 'Z' && *s[2] == '=')
+*s[0] = '$';
   if (wenv != NULL)
-for (wchar_t *ws = wenv; *ws != NULL; ws++)
-  if (ws[0] == L'T' && ws[1] == L'Z' && ws[2] == L'=')
-ws[0] = L'$';
+for (wchar_t **ws = wenv; *ws != NULL; ws++)
+  if (*ws[0] == L'T' && *ws[1] == L'Z' && *ws[2] == L'=')
+*ws[0] = L'$';
 }

   /* On native Windows, tzset() is deprecated.  Use _tzset() instead.  See
diff --git a/lib/wcsftime.c b/lib/wcsftime.c
index d8b471ab57..81d1762a92 100644
--- a/lib/wcsftime.c
+++ b/lib/wcsftime.c
@@ -61,13 +61,13 @@ rpl_wcsftime (wchar_t *buf, size_t bufsize, const wchar_t 
*format, const struct
   char **env = _environ;
   wchar_t **wenv = _wenviron;
   if (env != NULL)
-for (char *s = env; *s != NULL; s++)
-  if (s

Re: test-sprintf-posix and test-snprintf-posix test failures

[Bruno Haible]

Hi Collin,

> I would have worked a bit harder and tried to come up with a patch,

At this point of the investigations, it's too early for a patch.
First, one needs to determine whether it's a bug in Gnulib or a
bug in some package we rely on (typically gcc or glibc). In the
latter case, we report the bug and wait for the answer. Depending
on the answer, we may fix the test (if they say that our test is
wrong), or we may put in an officially recommended workaround,
or a workaround of our liking, or disable that part of the unit test.

> but I am not very familiar with the wide character and multibyte
> string functions (wcrtomb is mentioned in the comment).

In order to determine how these functions are supposed to behave,
we often reference
  - ISO C 23 
  - POSIX:2018 

The Linux manpages project and the glibc documentation

are also valuable in this context.

In this case, what matters for

int retval =
  my_sprintf (result, "a%lcz %d", (wint_t) L'\0', 33, 44, 55);

is the specification of sprintf, that is found in ISO C 23. (POSIX:2018
does not yet have the correction mentioned in the comment.)

> > Does the test failure persist when the tests are compiled with "-ggdb"
> > instead of "-g -O2"? (Change CFLAGS in gltests/Makefile.)
> 
> Interesting. Using "-ggdb" in gltests/Makefile fixes both failures.
> I'm assuming that -O2 is the important change here? I know very little
> about GCC's optimizations so feel free to educate me.

Probably -O2 makes the difference, yes (to be tested, though: try "-g O2"
vs "-g", or "-O2" vs "").

Since glibc also has optimizations of string functions, conditionalized by
__OPTIMIZE__, at this point it could be a bug in gcc or in glibc.

> > Does the test program use rpl_sprintf defined by Gnulib, or does it use
> > sprintf from glibc directly? (Use 'nm test-sprintf | grep printf'.) In
> > the first case, does the test failure persist when the Gnulib replacement
> > code is compiled with "-ggdb" instead of "-g -O2"? (Change CFLAGS in
> > gllib/Makefile.)
> 
> Both use them from glibc directly. I feel a bit silly for not checking
> that before...
> 
> $ nm gltests/test-sprintf-posix | grep 'printf'
>  U fprintf@GLIBC_2.2.5
>  U sprintf@GLIBC_2.2.5
> 
> $ nm gltests/test-snprintf-posix | grep 'printf'
>  U fprintf@GLIBC_2.2.5
>  U snprintf@GLIBC_2.2.5

If some glibc-optimized expansions are active, you would also see some
*_chk symbols in the 'nm' output.

So, at this point it looks like a gcc problem. The next step is to prepare
a trimmed-down test case :
  1. Concentrate on the simpler test case. In this case, I would say, it's
 sprintf, since it takes one argument less than snprintf.
  2. Eliminate the -I, -L, -l options specific to gnulib. So that you have
 a program that succeeds without -O2 and fails with -O2. This implies
 inlining include files from gnulib (test-sprintf-posix.h, macros.h, etc.).
  3. Work on minimizing the compilation unit that succeeds without -O2 and
 fails with -O2. Always keep -Wall enabled, to guard against mistakes.
 Remove the test cases that pass, remove unnecessary includes, and so on.
  4. When you can't minimize that further, preprocess it:
 $ gcc OPTIONS -E foo.c > foo-pre.c
 Verify that it still succeeds without -O2 and fails with -O2.
  5. Continue minimizing, with -Wall enabled. Typically the end result will
 be a small file, 10-20 lines of code.

If you got that far and can argue (by the standards) that it is a GCC bug,
open a bug report. The GCC shipped by Fedora is not an upstream release;
nevertheless Paul opens his bug reports about Fedora's gcc directly in the
upstream bug tracker, e.g.
. You can do the same.

Bruno