bug#70214: 'install' fails to copy regular file to autofs/cifs, due to ACL or xattr handling
Hi Pádraig, I wrote: > > 5) The same thing with 'cp -a' succeeds: > > > > $ build-sparc64/src/cp -a /var/tmp/foo3941 $HOME/foo3941; echo $? > > 0 > > $ build-sparc64-no-acl/src/cp -a /var/tmp/foo3941 $HOME/foo3941; echo $? > > 0 You wrote: > The psuedo code that install(1) uses is: > > copy_reg() >if (x->set_mode) /* install */ > set_acl(dest, x->mode /* 600 */) >ctx->acl = acl_from_mode ( /* 600 */) >acl_set_fd (ctx->acl) /* fails EACCES */ >if (! acls_set) > must_chmod = true; >if (must_chmod) > saved_errno = EACCES; > chmod (ctx->mode /* 600 */) > if (save_errno) >return -1; And, for comparison, what is the pseudo-code that 'cp -a' uses? I would guess that there must be a relevant difference between both. Bruno
bug#70214: 'install' fails to copy regular file to autofs/cifs, due to ACL or xattr handling
ot;/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=2998, ...}) = 0
read(3, "# Locale name alias data base.\n#"..., 4096) = 2998
read(3, "", 4096) = 0
close(3)= 0
openat(AT_FDCWD, "/usr/lib/locale/de_DE.UTF-8/LC_IDENTIFICATION",
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/de_DE.utf8/LC_IDENTIFICATION",
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/de_DE/LC_IDENTIFICATION", O_RDONLY|O_CLOEXEC)
= -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/de.UTF-8/LC_IDENTIFICATION",
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/de.utf8/LC_IDENTIFICATION",
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/de/LC_IDENTIFICATION", O_RDONLY|O_CLOEXEC) =
-1 ENOENT (No such file or directory)
geteuid() = 30014
openat(AT_FDCWD, "/media/guest-homedirs/haible/foo3941",
O_RDONLY|O_PATH|O_DIRECTORY) = -1 ENOTDIR (Not a directory)
fstatat64(AT_FDCWD, "/var/tmp/foo3941", {st_mode=S_IFREG|0640, st_size=3, ...},
AT_SYMLINK_NOFOLLOW) = 0
fstatat64(AT_FDCWD, "/media/guest-homedirs/haible/foo3941",
{st_mode=S_IFREG|0600, st_size=3, ...}, 0) = 0
openat(AT_FDCWD, "/var/tmp/foo3941", O_RDONLY|O_NOFOLLOW) = 3
fstat64(3, {st_mode=S_IFREG|0640, st_size=3, ...}) = 0
openat(AT_FDCWD, "/media/guest-homedirs/haible/foo3941", O_WRONLY|O_TRUNC) = 4
ioctl(4, BTRFS_IOC_CLONE or FICLONE, 3) = -1 EXDEV (Invalid cross-device link)
fstat64(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
fadvise64_64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
uname({sysname="Linux", nodename="matoro-sparcdev", ...}) = 0
copy_file_range(3, NULL, 4, NULL, 9223372035781033984, 0) = -1 EXDEV (Invalid
cross-device link)
mmap(NULL, 1064960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xfff8000100038000
read(3, "hi\n", 1048576)= 3
write(4, "hi\n", 3) = 3
read(3, "", 1048576)= 0
utimensat(4, NULL, [{tv_sec=1712251745, tv_nsec=269582795} /*
2024-04-04T13:29:05.269582795-0400 */, {tv_sec=1712251745, tv_nsec=270582848}
/* 2024-04-04T13:29:05.270582848-0400 */], 0) = 0
flistxattr(3, NULL, 0) = 0
flistxattr(3, 0x7feff9860a0, 0) = 0
fchmod(4, 0100640) = 0
flistxattr(3, NULL, 0) = 0
flistxattr(3, 0x7feff9860c0, 0) = 0
close(4)= 0
close(3)= 0
munmap(0xfff8000100038000, 1064960) = 0
_llseek(0, 0, 0x7feff986600, SEEK_CUR) = -1 ESPIPE (Illegal seek)
close(0)= 0
close(1)= 0
close(2)= 0
exit_group(0) = ?
+++ exited with 0 +++
As you can see, it uses 4 flistxattr() calls on the source file descriptor,
apparently detecting that it's a regular file without ACLs, and proceeds to
do a simple fchmod() call on the destination file descriptor.
Probably the same logic is needed in the 'install' program.
Bruno
bug#70104: "FAIL: test-canonicalize" coreutils v9.5 on musl libc
Adept's Lab wrote:
> >> test-canonicalize.c:411: assertion 'strcmp (result1, "//") == 0' failed
Thanks for the report. I reproduce it with gnulib testdirs
$ rm -rf ../testdir1; ./gnulib-tool --create-testdir --dir=../testdir1
--single-configure canonicalize-lgpl
$ rm -rf ../testdir2; ./gnulib-tool --create-testdir --dir=../testdir2
--single-configure canonicalize
when run on Alpine Linux 3.9, 3.14, and 3.19.
History of these failures:
- For many years, one of the tests was failing on Alpine Linux.
- On 2021-01-17, I added a fix, that consists of a realpath() override [1].
- On 2022-07-31, the bug was reported again [2], against the sed-4.8 release
which predates the 2021-01-17 fix [3].
- On 2023-09-04, Paul changed the behaviour of the unit tests on musl libc,
without giving an explanation [4].
- Now, the unit tests fail again, as reported above.
I'm therefore partially reverting Paul's change from 2023-09-04 (attached).
Paul Eggert wrote:
> Does this behavior (of whether / and // are the same directory) depend
> on musl version, or on how musl is configured?
I think you must ask this to yourself:
- What caused you to change the unit tests on 2023-09-04?
- How is the musl version that you used on that date configured?
What I use is Alpine Linux, as I said in versions 3.9, 3.14, 3.19.
- Did you work with gnulib testdirs at that time, or did you use
a package in which some modules are --avoid ed from import?
Bruno
[1] https://lists.gnu.org/archive/html/bug-gnulib/2021-01/msg00215.html
[2] https://lists.gnu.org/archive/html/bug-sed/2022-07/msg3.html
[3] https://ftp.gnu.org/gnu/sed/
[4] https://lists.gnu.org/archive/html/bug-gnulib/2023-09/msg00011.html
From 1defda6356c29c7f731bddb9e9231f594e01d9c9 Mon Sep 17 00:00:00 2001
From: Bruno Haible
Date: Sun, 31 Mar 2024 21:31:34 +0200
Subject: [PATCH] canonicalize[-lgpl] tests: Fix test failure on musl libc.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Reported by Adept's Lab via Pádraig Brady at
<https://lists.gnu.org/archive/html/bug-coreutils/2024-03/msg00086.html>.
* tests/test-canonicalize-lgpl.c (main): Don't special-case "//"
handling for musl libc.
* tests/test-canonicalize.c (main): Likewise.
* modules/canonicalize-lgpl-tests (Files): Remove m4/musl.m4.
(configure.ac): Don't invoke gl_MUSL_LIBC.
---
ChangeLog | 11 +++
modules/canonicalize-lgpl-tests | 2 --
tests/test-canonicalize-lgpl.c | 6 +++---
tests/test-canonicalize.c | 6 +++---
4 files changed, 17 insertions(+), 8 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 2f05098a08..ffae5513ae 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2024-03-31 Bruno Haible
+
+ canonicalize[-lgpl] tests: Fix test failure on musl libc.
+ Reported by Adept's Lab via Pádraig Brady at
+ <https://lists.gnu.org/archive/html/bug-coreutils/2024-03/msg00086.html>.
+ * tests/test-canonicalize-lgpl.c (main): Don't special-case "//"
+ handling for musl libc.
+ * tests/test-canonicalize.c (main): Likewise.
+ * modules/canonicalize-lgpl-tests (Files): Remove m4/musl.m4.
+ (configure.ac): Don't invoke gl_MUSL_LIBC.
+
2024-03-31 Collin Funk
gnulib-tool.py: Fix output of 'po/LINGUAS'.
diff --git a/modules/canonicalize-lgpl-tests b/modules/canonicalize-lgpl-tests
index 409ee7225b..73a0e32abb 100644
--- a/modules/canonicalize-lgpl-tests
+++ b/modules/canonicalize-lgpl-tests
@@ -1,5 +1,4 @@
Files:
-m4/musl.m4
tests/test-canonicalize-lgpl.c
tests/signature.h
tests/null-ptr.h
@@ -11,7 +10,6 @@ same-inode
symlink
configure.ac:
-gl_MUSL_LIBC
Makefile.am:
TESTS += test-canonicalize-lgpl
diff --git a/tests/test-canonicalize-lgpl.c b/tests/test-canonicalize-lgpl.c
index 1b2ad98f51..21211e5eac 100644
--- a/tests/test-canonicalize-lgpl.c
+++ b/tests/test-canonicalize-lgpl.c
@@ -262,9 +262,9 @@ main (void)
ASSERT (stat ("/", ) == 0);
ASSERT (stat ("//", ) == 0);
bool same = psame_inode (, );
-#if defined __MVS__ || defined MUSL_LIBC
-/* On IBM z/OS and musl libc, "/" and "//" both canonicalize to
- themselves, yet they both have st_dev == st_ino == 1. */
+#if defined __MVS__
+/* On IBM z/OS, "/" and "//" both canonicalize to themselves, yet they both
+ have st_dev == st_ino == 1. */
same = false;
#endif
if (same)
diff --git a/tests/test-canonicalize.c b/tests/test-canonicalize.c
index 6763a525c9..5d19285c00 100644
--- a/tests/test-canonicalize.c
+++ b/tests/test-canonicalize.c
@@ -394,9 +394,9 @@ main (void)
ASSERT (stat ("/", ) == 0);
ASSERT (stat ("//", ) == 0);
bool same = psame_inode (, );
-#if defined __MVS__ || defined MUSL_LIBC
-/* On IBM z/OS and musl libc, "/" and "//" both canonicalize to
- themselves, yet they both have st_dev
bug#70070: FAIL: test-localtime_r
Andreas Schwab wrote: > > Yes. And make sure that it has a time zone database installed at all. > > Why? That doesn't make any sense. You can leave the consideration of which test case makes sense or not on my side. What we need from you, as a reporter, is a statement on which platform (distro and version) you saw the test failure. If we don't have this, there is no good way for us to reproduce the issue, and we'll have to close the ticket. Bruno
bug#70070: FAIL: test-localtime_r
Andreas Schwab wrote: > > FAIL: test-localtime_r > > == > > > > test-localtime_r.c:58: assertion 'result->tm_hour == 18' failed > > FAIL test-localtime_r (exit status: 134) > > > > FAIL: test-localtime_r-mt > > = > > > > thread2 disturbed by thread1! > > thread1 disturbed by thread2! > > FAIL test-localtime_r-mt (exit status: 134) The second failure is a consequence of the first one. So, there is no MT-safety problem, "just" that localtime_r produces wrong results. I've seen this happen on OSes with a time zone database that is older than 5 or 10 years. Pádraig Brady wrote: > It may help to detail your platform. Yes. And make sure that it has a time zone database installed at all. Bruno
bug#69418: test failure when no french locale is installed
Testing the current coreutils git master:
On a Debian 12 system, in which I have not installed a French UTF-8 locale,
I see a test failure of tests/misc/join-utf8.
The essential lines from test-suite.log:
+ test set = set
+ LC_ALL=none
../tests/misc/join-utf8.sh: line 24: warning: setlocale: LC_ALL: cannot change
locale (none): No such file or directory
The cause is that on such a system, LOCALE_FR_UTF8, as determined by
gnulib/m4/locale-fr.m4, is 'none', not empty or absent.
The attached patch fixes the failure.
>From 74be78b1efba477016c153e0eacc93a3c661e748 Mon Sep 17 00:00:00 2001
From: Bruno Haible
Date: Mon, 26 Feb 2024 22:33:18 +0100
Subject: [PATCH] join: Avoid test failure on systems which have no French
UTF-8 locale
* tests/misc/join-utf8.sh: Test the value of LOCALE_FR_UTF8 against
'none', not against a missing value.
---
tests/misc/join-utf8.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/misc/join-utf8.sh b/tests/misc/join-utf8.sh
index 2dbe2bb81..9af9e55ce 100755
--- a/tests/misc/join-utf8.sh
+++ b/tests/misc/join-utf8.sh
@@ -19,7 +19,7 @@
. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src
print_ver_ join
-test "${LOCALE_FR_UTF8+set}" = set || skip_ "French UTF-8 locale not available"
+test "$LOCALE_FR_UTF8" != none || skip_ "French UTF-8 locale not available"
LC_ALL=$LOCALE_FR_UTF8
export LC_ALL
--
2.34.1
bug#65992: readutmp should check for sd_booted() and not if /run/utmp exists
Thorsten Kukuk wrote: > Who creates an additional line for "seat0", which no other tools creates > and which it did not create before. Since some display manager writes > wrongly multiple utmp entries To me, that's a feature, not a bug. Systemd has introduced the concept of seats [1], and the concept of tty is not so adequate any more for identifying a session (because some terminal emulators / desktops put a line into /var/run/utmp for each open terminal emulator window [2]). It is thus natural and useful to show the association between user and seat. Bruno [1] https://www.freedesktop.org/wiki/Software/systemd/multiseat/ [2] https://www.thkukuk.de/blog/Y2038_glibc_utmp_64bit/
bug#65992: readutmp should check for sd_booted() and not if /run/utmp exists
[CCing bug-gnulib, because the readutmp code lives in gnulib]
Thorsten Kukuk wrote:
> if there is no /run/utmp file, /usr/bin/who falls back correctly to the
> systemd-logind interface and shows correct data.
>
> But there are applications, which don't use the libc interface for
> reading/writing utmp entries, they have their own implementation. And
> this implementations create the file /run/utmp if this does not exist
> (glibc does not). Which means, /usr/bin/who shows sometimes wrong/incomplete
> data.
>
> procps (especially w) and other packages don't check if /run/utmp exist
> or not, but use sd_booted(). e.g.:
>
> if (sd_booted() > 0) {
> numuser = sd_get_sessions(NULL);
> } else {
> setutent();
> while ((ut = getutent())) {
> if ((ut->ut_type == USER_PROCESS) && (ut->ut_name[0] != '\0'))
> numuser++;
> }
> endutent();
> }
>
> So if logind is running, logind is used, else /run/utmp.
> I think gnulib/coreutils should do the same. It's impossible to find and
> fix all the code writing utmp entries at their own, especially if this
> is 3rd party software and you don't have the source code.
You are actually talking about three different things:
(A) Calling sd_booted in addition to calling sd_get_sessions.
(B) Using the /var/run/utmp file when systemd is not active.
(C) Using the /var/run/utmp file when systemd is active but some
applications use old APIs.
About (A)
=
This is redundant, because when sd_booted() is <= 0, sd_get_sessions returns
NULL. This comes from the implementation, see
https://github.com/systemd/systemd/blob/main/src/libsystemd/sd-daemon/sd-daemon.c#L716
About (B)
=
Is there any distro which has libsystemd installed (so that coreutils could
be built with -lsystemd) but where a different init system is used at boot time?
If not, the suggestion is pointless.
About (C)
=
This is not the same as (B), because the existence of an "application" that uses
old APIs does not mean that sd_booted() will return <= 0.
> But there are applications, which don't use the libc interface for
> reading/writing utmp entries, they have their own implementation.
Which are these applications? The best approach is, obviously, to modernize
these applications by sending them patches. codesearch.debian.net is your
friend.
> It's impossible to find and
> fix all the code writing utmp entries at their own, especially if this
> is 3rd party software and you don't have the source code.
We should not let proprietary 3rd-party software prevent us from modernizing
the GNU system. This is the stance that the Linux kernel has taken more than
20 years ago, by defining a formal interface for kernel modules. If some
proprietary applications cause trouble, then the best approach is to ignore
it and thus increase the users' will to replace these proprietary applications
with free software.
A temporary proposal
It is possible that, to get best results in the situation you describe, as
long as some applications use the old APIs, it is needed for readutmp to
combine (not choose among) the two approaches.
The best way to determine if this is true, IMO, would be to
1) turn the compile-time switch in lib/readutmp.c:942 into a run-time switch.
2) Add a '--traditional' option to 'who', 'pinky', 'uptime'.
3) Wait for users to report problems, that is, situations where `who` and
`who --traditional` disagree, with precise description of the problems.
Paul, Pádraig, what do you think?
Bruno
bug#65599: mv and cp give a pointless warning when moving/copying a directory
Paul Eggert wrote:
> Yes please. If chmod also does not work coreutils might need more
> workarounds.
>
> Just to be clear; please try it on a file where chmod should fail
> because you don't own the file and you're not root. chmod should fail
> with EPERM in this case; it it fails with EACCES it's buggy. Thanks.
chmod fails with EACCES, hence per POSIX
<https://pubs.opengroup.org/onlinepubs/9699919799/functions/chmod.html>
it's buggy.
See: On the server I have 2 files and 2 directories:
# ls -ld dir1 dir1/file1 dir2 dir2/file2
drwxrwxr-x2 brunoeveryone 4096 Sep 2 12:58 dir1/
-rw-rw-r--1 brunoeveryone 4 Sep 2 00:52 dir1/file1
drwxrwxrwx2 adminadminist 4096 Sep 2 00:54 dir2/
-rw-rw-r--1 adminadminist 4 Sep 2 00:54 dir2/file2
The user that exports the mount point is 'bruno'. But since dir2
and dir2/file2 are not owned by 'bruno', an error will (and should)
occur when these two are manipulated through the CIFS client.
chown fails; that's the problem you already committed a workaround for:
$ ./fchownat-test /media/nas/bruno/dir1/file1
lchown: Permission denied
fchownat: Permission denied
chown: Permission denied
fchown: Permission denied
$ ./fchownat-test /media/nas/bruno/dir1
lchown: Permission denied
fchownat: Permission denied
chown: Permission denied
fchown: Permission denied
$ ./fchownat-test /media/nas/bruno/dir2/file2
lchown: Permission denied
fchownat: Permission denied
chown: Permission denied
fchown: Permission denied
$ ./fchownat-test /media/nas/bruno/dir2
lchown: Permission denied
fchownat: Permission denied
chown: Permission denied
fchown: Permission denied
chmod also fails with EACCES instead of EPERM:
$ ./fchmodat-test /media/nas/bruno/dir1/file1
$ ./fchmodat-test /media/nas/bruno/dir1
$ ./fchmodat-test /media/nas/bruno/dir2/file2
lchmod: Permission denied
fchmodat: Permission denied
chmod: Permission denied
fchmod: Permission denied
$ ./fchmodat-test /media/nas/bruno/dir2
lchmod: Permission denied
fchmodat: Permission denied
chmod: Permission denied
fchmod: Permission denied
The same thing with strace:
$ strace ./fchmodat-test /media/nas/bruno/dir2/file2
...
newfstatat(AT_FDCWD, "/media/nas/bruno/dir2/file2", {st_mode=S_IFREG|0664,
st_size=4, ...}, AT_SYMLINK_NOFOLLOW) = 0
openat(AT_FDCWD, "/media/nas/bruno/dir2/file2",
O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 3
newfstatat(3, "", {st_mode=S_IFREG|0664, st_size=4, ...}, AT_EMPTY_PATH) = 0
chmod("/proc/self/fd/3", 0664) = -1 EACCES (Permission denied)
close(3)= 0
dup(2) = 3
fcntl(3, F_GETFL) = 0x402 (flags O_RDWR|O_APPEND)
getrandom("\x68\x8f\xbd\x08\xf8\x4b\xa9\xa5", 8, GRND_NONBLOCK) = 8
brk(NULL) = 0x5563c5d01000
brk(0x5563c5d22000) = 0x5563c5d22000
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x5), ...},
AT_EMPTY_PATH) = 0
write(3, "lchmod: Permission denied\n", 26lchmod: Permission denied
) = 26
close(3)= 0
openat(AT_FDCWD, "/media/nas/bruno/dir2/file2",
O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 3
newfstatat(3, "", {st_mode=S_IFREG|0664, st_size=4, ...}, AT_EMPTY_PATH) = 0
chmod("/proc/self/fd/3", 0664) = -1 EACCES (Permission denied)
close(3)= 0
dup(2) = 3
fcntl(3, F_GETFL) = 0x402 (flags O_RDWR|O_APPEND)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x5), ...},
AT_EMPTY_PATH) = 0
write(3, "fchmodat: Permission denied\n", 28fchmodat: Permission denied
) = 28
close(3)= 0
chmod("/media/nas/bruno/dir2/file2", 0664) = -1 EACCES (Permission denied)
dup(2) = 3
fcntl(3, F_GETFL) = 0x402 (flags O_RDWR|O_APPEND)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x5), ...},
AT_EMPTY_PATH) = 0
write(3, "chmod: Permission denied\n", 25chmod: Permission denied
) = 25
close(3)= 0
openat(AT_FDCWD, "/media/nas/bruno/dir2/file2", O_RDONLY|O_NOFOLLOW) = 3
fchmod(3, 0664) = -1 EACCES (Permission denied)
dup(2) = 4
fcntl(4, F_GETFL) = 0x402 (flags O_RDWR|O_APPEND)
newfstatat(4, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x5), ...},
AT_EMPTY_PATH) = 0
write(4, "fchmod: Permission denied\n", 26fchmod: Permission denied
) = 26
close(4)= 0
exit_group(1) = ?
+++ exited with 1 +++
$ strace ./fchmodat-test /media/nas/bruno/dir2
...
newfstatat(AT_FDCWD, "/media/nas/bruno/dir2", {st_mode=S_IFDIR|0777, st_
bug#65599: mv and cp give a pointless warning when moving/copying a directory
Paul Eggert wrote: > Good, thanks for confirming that the chmod family does not have a > similar bug. Well, you asked me to try it on / but / is an ext4 FS, not a CIFS FS on my system. Should I try it also on a directory inside the CIFS mount? > I installed the attached workaround into coreutils master on Savannah. > Please give it a try. It does remove the diagnostic as intended: Preparations: $ mkdir dir3 dir4 $ echo foo > dir3/file3 $ echo foo > dir4/file4 With coreutils-9.4: $ mv dir3 /media/nas/bruno/ /gnu-inst-coreutils/9.4/bin/mv: failed to preserve ownership for '/media/nas/bruno/dir3': Permission denied With coreutils from git now: $ src/mv dir4 /media/nas/bruno/ I had worries regarding losing diagnostics that would be worth seeing. But in fact, coreutils-9.4 did not actually produce such a diagnostic. So, there is no regression for this case. Preparations: $ mkdir dir1 dir2 $ echo foo > dir1/file1 $ echo foo > dir2/file2 $ chmod a+rwx dir1 dir2 $ sudo chown root:root dir1 dir2 With coreutils-9.4: $ mv dir1 /media/nas/bruno/dir1 With coreutils from git now: $ src/mv dir2 /media/nas/bruno/dir2 Bruno
bug#65599: mv and cp give a pointless warning when moving/copying a directory
Paul Eggert wrote:
> > If you compile and run the attached
> > program on a file that you don't own (e.g., './a.out /'), does it
> > incorrectly issue "Permission denied" instead of "Operation not
> > permitted" diagnostics?
and I replied:
> In this case, the diagnostic is "Operation not permitted" (from EPERM):
>
> $ ./a.out /
> lchown: Operation not permitted
> fchownat: Operation not permitted
> chown: Operation not permitted
> fchown: Operation not permitted
Oops, I mistakenly compiled and ran the wrong test program. Here are the
results for the test program you meant:
$ ./a.out /
lchmod: Operation not permitted
fchmodat: Operation not permitted
chmod: Operation not permitted
fchmod: Operation not permitted
$ strace ./a.out /
...
chmod("/proc/self/fd/3", 0755) = -1 EPERM (Operation not permitted)
close(3)= 0
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
getrandom("\xd5\xae\x46\x10\x0d\x5a\x98\x9a", 8, GRND_NONBLOCK) = 8
brk(NULL) = 0x5575761c2000
brk(0x5575761e3000) = 0x5575761e3000
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0xd), ...},
AT_EMPTY_PATH) = 0
write(3, "lchmod: Operation not permitted\n", 32lchmod: Operation not permitted
) = 32
close(3)= 0
openat(AT_FDCWD, "/", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 3
newfstatat(3, "", {st_mode=S_IFDIR|0755, st_size=4096, ...}, AT_EMPTY_PATH) = 0
chmod("/proc/self/fd/3", 0755) = -1 EPERM (Operation not permitted)
close(3)= 0
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0xd), ...},
AT_EMPTY_PATH) = 0
write(3, "fchmodat: Operation not permitte"..., 34fchmodat: Operation not
permitted
) = 34
close(3)= 0
chmod("/", 0755)= -1 EPERM (Operation not permitted)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0xd), ...},
AT_EMPTY_PATH) = 0
write(3, "chmod: Operation not permitted\n", 31chmod: Operation not permitted
) = 31
close(3)= 0
openat(AT_FDCWD, "/", O_RDONLY|O_NOFOLLOW) = 3
fchmod(3, 0755) = -1 EPERM (Operation not permitted)
dup(2) = 4
fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(4, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0xd), ...},
AT_EMPTY_PATH) = 0
write(4, "fchmod: Operation not permitted\n", 32fchmod: Operation not permitted
) = 32
close(4)= 0
exit_group(1) = ?
+++ exited with 1 +++
bug#65599: mv and cp give a pointless warning when moving/copying a directory
Paul Eggert wrote:
> If you compile and run the attached
> program on a file that you don't own (e.g., './a.out /'), does it
> incorrectly issue "Permission denied" instead of "Operation not
> permitted" diagnostics?
In this case, the diagnostic is "Operation not permitted" (from EPERM):
$ ./a.out /
lchown: Operation not permitted
fchownat: Operation not permitted
chown: Operation not permitted
fchown: Operation not permitted
$ strace ./a.out /
...
lchown("/", 0, 0) = -1 EPERM (Operation not permitted)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
getrandom("\x30\x26\x33\x40\x27\x47\x25\x78", 8, GRND_NONBLOCK) = 8
brk(NULL) = 0x55f0af43
brk(0x55f0af451000) = 0x55f0af451000
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(3, "lchown: Operation not permitted\n", 32lchown: Operation not permitted
) = 32
close(3)= 0
fchownat(AT_FDCWD, "/", 0, 0, AT_SYMLINK_NOFOLLOW) = -1 EPERM (Operation not
permitted)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(3, "fchownat: Operation not permitte"..., 34fchownat: Operation not
permitted
) = 34
close(3)= 0
chown("/", 0, 0)= -1 EPERM (Operation not permitted)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(3, "chown: Operation not permitted\n", 31chown: Operation not permitted
) = 31
close(3)= 0
openat(AT_FDCWD, "/", O_RDONLY|O_NOFOLLOW) = 3
fchown(3, 0, 0) = -1 EPERM (Operation not permitted)
dup(2) = 4
fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(4, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(4, "fchown: Operation not permitted\n", 32fchown: Operation not permitted
) = 32
close(4)= 0
exit_group(1) = ?
+++ exited with 1 +++
Bruno
bug#65599: mv and cp give a pointless warning when moving/copying a directory
Paul Eggert wrote:
> it'd be helpful to know whether the bug is limited to fchownat or
> (as I suspect) applies also to the other chown variants. Could you try
> running the attached program on your platform, to see whether the bug
> affects these other variants?
It applies to all 4 variants, as can be seen from applying it to a directory:
$ ./a.out /media/nas/bruno/dir1
lchown: Permission denied
fchownat: Permission denied
chown: Permission denied
fchown: Permission denied
$ strace ./a.out /media/nas/bruno/dir1
...
lchown("/media/nas/bruno/dir1", 1000, 1000) = -1 EACCES (Permission denied)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
getrandom("\x31\xd2\xe1\x4b\xa3\xa9\x0a\x95", 8, GRND_NONBLOCK) = 8
brk(NULL) = 0x56031b7c9000
brk(0x56031b7ea000) = 0x56031b7ea000
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(3, "lchown: Permission denied\n", 26lchown: Permission denied
) = 26
close(3) = 0
fchownat(AT_FDCWD, "/media/nas/bruno/dir1", 1000, 1000, AT_SYMLINK_NOFOLLOW) =
-1 EACCES (Permission denied)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(3, "fchownat: Permission denied\n", 28fchownat: Permission denied
) = 28
close(3)= 0
chown("/media/nas/bruno/dir1", 1000, 1000) = -1 EACCES (Permission denied)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(3, "chown: Permission denied\n", 25chown: Permission denied
) = 25
close(3)= 0
openat(AT_FDCWD, "/media/nas/bruno/dir1", O_RDONLY|O_NOFOLLOW) = 3
fchown(3, 1000, 1000) = -1 EACCES (Permission denied)
dup(2) = 4
fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(4, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(4, "fchown: Permission denied\n", 26fchown: Permission denied
) = 26
close(4) = 0
exit_group(1) = ?
+++ exited with 1 +++
or to a regular file:
$ ./a.out /media/nas/bruno/dir1/file1
lchown: Permission denied
fchownat: Permission denied
chown: Permission denied
fchown: Permission denied
$ strace ./a.out /media/nas/bruno/dir1/file1
...
lchown("/media/nas/bruno/dir1/file1", 1000, 1000) = -1 EACCES (Permission
denied)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
getrandom("\x59\x38\xf4\xfa\x24\xc4\x89\x64", 8, GRND_NONBLOCK) = 8
brk(NULL) = 0x56337cdfa000
brk(0x56337ce1b000) = 0x56337ce1b000
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(3, "lchown: Permission denied\n", 26lchown: Permission denied
) = 26
close(3)= 0
fchownat(AT_FDCWD, "/media/nas/bruno/dir1/file1", 1000, 1000,
AT_SYMLINK_NOFOLLOW) = -1 EACCES (Permission denied)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(3, "fchownat: Permission denied\n", 28fchownat: Permission denied
) = 28
close(3)= 0
chown("/media/nas/bruno/dir1/file1", 1000, 1000) = -1 EACCES (Permission denied)
dup(2) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(3, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(3, "chown: Permission denied\n", 25chown: Permission denied
) = 25
close(3)= 0
openat(AT_FDCWD, "/media/nas/bruno/dir1/file1", O_RDONLY|O_NOFOLLOW) = 3
fchown(3, 1000, 1000) = -1 EACCES (Permission denied)
dup(2) = 4
fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
newfstatat(4, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0x9), ...},
AT_EMPTY_PATH) = 0
write(4, "fchown: Permission denied\n", 26fchown: Permission denied
) = 26
close(4)= 0
exit_group(1) = ?
+++ exited with 1 +++
> Also, could you let us know the kernel and glibc version? The bug should
> be documente
bug#65674: Build failure with
Sam James wrote:
> Forwarding a downstream report at https://bugs.gentoo.org/913368
> of coreutils-9.4 failing to build with openssl-1.1.x:
> """
> x86_64-pc-linux-gnu-gcc -I. -I./lib -DHASH_ALGO_BLAKE2=1 -DHAVE_CONFIG_H
> -Ilib -I./lib -Isrc -I./src-O2 -march=x86-64 -pipe -pipe
> -frecord-gcc-switches -fno-diagnostics-color -fmessage-length=0 -c -o
> src/b2sum-digest.o `test -f 'src/digest.c' || echo './'`src/digest.c
> In file included from src/digest.c:41:
> ./lib/md5.h:36:12: fatal error: openssl/configuration.h: No such file or
> directory
>36 | # include
> |^
> compilation terminated.
> """
>
> This appears to have been introduced with gnulib commit
> a436f5f498d7e747864d40d4450fa8330dd44d12.
>
> configuration.h is only available with >=openssl-3.
Thanks for the report.
We did test on platforms with OpenSSL 1.1.x before the coreutils-9.4
release (namely, on AIX 7.1 and FreeBSD 13.1 [1]), and did not encounter
this problem. That is because, by default, on such platforms, the
configure test
checking whether openssl is GPL compatible... no
determines that OpenSSL should not be used. More regarding this license
incompatibility at
https://www.gnu.org/licenses/license-list.en.html#OpenSSL
Apparently your user's build is using the configure option
'--with-openssl' or 'with-openssl=yes'.
The patch below fixes the compilation failure.
However, since OpenSSL 1.1.x is out-of-support 11 days from now [2]
and it is quite dangerous to use a security-centered software when
it is no longer supported, I would suggest that you (Gentoo) migrate
to OpenSSL version 3.0.*.
Bruno
[1] https://lists.gnu.org/archive/html/coreutils/2023-08/msg00096.html
[2] https://en.wikipedia.org/wiki/OpenSSL#Major_version_releases
2023-09-01 Bruno Haible
crypto/{sha*,md5,sm3}-buffer: Fix --with-openssl (regr. 2023-08-26).
Reported by Agostino Sarubbo via Sam James in
<https://lists.gnu.org/archive/html/bug-gnulib/2023-09/msg0.html>.
* lib/sha1.h: Test the OpenSSL major version before attempting to
include .
* lib/sha256.h: Likewise.
* lib/sha512.h: Likewise.
* lib/md5.h: Likewise.
* lib/sm3.h: Likewise.
diff --git a/lib/md5.h b/lib/md5.h
index 6ddf009148..b298fc4cc3 100644
--- a/lib/md5.h
+++ b/lib/md5.h
@@ -33,14 +33,18 @@
# define OPENSSL_API_COMPAT 0x10101000L /* FIXME: Use OpenSSL 1.1+ API. */
# endif
/* If would give a compile-time error, don't use OpenSSL.
*/
-# include
-# if (OPENSSL_CONFIGURED_API \
- < (OPENSSL_API_COMPAT < 0x90L ? OPENSSL_API_COMPAT : \
- ((OPENSSL_API_COMPAT >> 28) & 0xF) * 1 \
- + ((OPENSSL_API_COMPAT >> 20) & 0xFF) * 100 \
- + ((OPENSSL_API_COMPAT >> 12) & 0xFF)))
-# undef HAVE_OPENSSL_MD5
-# else
+# include
+# if OPENSSL_VERSION_MAJOR >= 3
+# include
+# if (OPENSSL_CONFIGURED_API \
+< (OPENSSL_API_COMPAT < 0x90L ? OPENSSL_API_COMPAT : \
+ ((OPENSSL_API_COMPAT >> 28) & 0xF) * 1 \
+ + ((OPENSSL_API_COMPAT >> 20) & 0xFF) * 100 \
+ + ((OPENSSL_API_COMPAT >> 12) & 0xFF)))
+#undef HAVE_OPENSSL_MD5
+# endif
+# endif
+# if HAVE_OPENSSL_MD5
# include
# endif
# endif
diff --git a/lib/sha1.h b/lib/sha1.h
index d5a6e72e2c..cf67997f3d 100644
--- a/lib/sha1.h
+++ b/lib/sha1.h
@@ -32,14 +32,18 @@
# define OPENSSL_API_COMPAT 0x10101000L /* FIXME: Use OpenSSL 1.1+ API. */
# endif
/* If would give a compile-time error, don't use OpenSSL.
*/
-# include
-# if (OPENSSL_CONFIGURED_API \
- < (OPENSSL_API_COMPAT < 0x90L ? OPENSSL_API_COMPAT : \
- ((OPENSSL_API_COMPAT >> 28) & 0xF) * 1 \
- + ((OPENSSL_API_COMPAT >> 20) & 0xFF) * 100 \
- + ((OPENSSL_API_COMPAT >> 12) & 0xFF)))
-# undef HAVE_OPENSSL_SHA1
-# else
+# include
+# if OPENSSL_VERSION_MAJOR >= 3
+# include
+# if (OPENSSL_CONFIGURED_API \
+< (OPENSSL_API_COMPAT < 0x90L ? OPENSSL_API_COMPAT : \
+ ((OPENSSL_API_COMPAT >> 28) & 0xF) * 1 \
+ + ((OPENSSL_API_COMPAT >> 20) & 0xFF) * 100 \
+ + ((OPENSSL_API_COMPAT >> 12) & 0xFF)))
+#undef HAVE_OPENSSL_SHA1
+# endif
+# endif
+# if HAVE_OPENSSL_SHA1
# include
# endif
# endif
diff --git a/lib/sha256.h b/lib/sha256.h
index 508bce7de8..5a0b652b78 100644
--- a/lib/sha256.h
+++ b/lib/sha256.h
@@ -31,14 +31,18 @@
# define OPENSSL_API_COMPAT 0x10101000L /* FIXME: Use OpenSSL 1.1+ API. */
# endif
/* If would give a compile-time error, don't use OpenSSL.
*/
-# include
-# if (OPENSSL_CONFIGURED_API \
- < (OPENSSL_API_COMPAT < 0x90L ? OPENSSL_API_COMPAT : \
-
bug#65617: coreutils 9.4: seg.fault in readutmp with systemd
Paul Eggert wrote:
> I installed the attached patch into Gnulib
> and this should appear in the next coreutils release.
Unfortunately, this patch introduces a memory leak: If
num_sessions == 0 and sessions != NULL (which can happen, according
to the man page), we need to call free (sessions).
This patch fixes it.
2023-08-31 Bruno Haible
readutmp: Fix memory leak introduced by last commit.
* lib/readutmp.c (read_utmp_from_systemd): If num_sessions == 0 and
sessions != NULL, do call free (sessions).
diff --git a/lib/readutmp.c b/lib/readutmp.c
index e99158677c..ec09feb59b 100644
--- a/lib/readutmp.c
+++ b/lib/readutmp.c
@@ -795,7 +795,7 @@ read_utmp_from_systemd (idx_t *n_entries, STRUCT_UTMP
**utmp_buf, int options)
{
char **sessions;
int num_sessions = sd_get_sessions ();
- if (num_sessions > 0)
+ if (num_sessions >= 0 && sessions != NULL)
{
char **session_ptr;
for (session_ptr = sessions; *session_ptr != NULL; session_ptr++)
bug#65599: mv and cp give a pointless warning when moving/copying a directory
Thanks for looking into this, Paul. But there's a big misunderstanding. This is not on Android. It's on Linux with glibc (Ubuntu 22.04, to be precise), as can be seen from the log file: openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 and from the fact that I could write to /tmp. The fchownat() failure comes from the combination of a 'cifs' mount of the Linux kernel (mount options: rw,relatime,vers=1.0,cache=strict,username=bruno,uid=1000,forceuid,gid=1000,forcegid,addr=192.168.**.**,soft,unix,posixpaths,serverino,mapposix,acl,rsize=1048576,wsize=1048576,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1 ) and the CIFS/SMB server on my NAS (an old Linux/mips machine). > Although the attached hacky coreutils patch (which I haven't installed) > might solve the immediate problem Unfortunately, this patch would also silence warnings that are justified (e.g. when a normal user moves a directory owned by 'root', whose parent directory has mode drwxrwxrwx). I think the diagnostic should only be supporessed when, despite the failed fchownat() call, the destination has the same uid and gid as the 3rd and 4th argument of the fchownat() invocation. > it sounds like this is something that Gnulib's fchownat Whether this is better done in Gnulib or in coreutils/src/copy.c, I can't judge. Bruno
bug#65599: mv and cp give a pointless warning when moving/copying a directory
When I move a directory from an ext4 file system to a CIFS v1 file system,
I get a diagnostic
mv: failed to preserve ownership for '...': Permission denied
although the owner and group of the directory after and before the move
are the same. So, there is actually nothing to warn about.
$ mv --version
mv (GNU coreutils) 9.3.147-d553ab
Copyright (C) 2023 Free Software Foundation, Inc.
...
$ mkdir dir1
$ echo foo > dir1/file1
$ ls -ld dir1
drwxrwxr-x 2 bruno bruno 4096 Aug 26 19:38 dir1
$ strace -o /tmp/mv.log mv dir1 /media/nas/bruno/dir1
mv: failed to preserve ownership for '/media/nas/bruno/dir1': Permission denied
$ echo $?
0
$ ls -ld /media/nas/bruno/dir1
drwxrwxr-x 2 bruno bruno 0 Aug 26 19:38 /media/nas/bruno/dir1
Find attached the strace log file. The essential line is
fchownat(AT_FDCWD, "/media/nas/bruno/dir1", 1000, 1000, AT_SYMLINK_NOFOLLOW) =
-1 EACCES (Permission denied)
People say that "chown only works for root anyway" [1], but it's certainly
safer to try the chown nevertheless, because
- the current user might not be root but might have non-standard capabilities.
- the target file system may be reporting failure but may do the requested
action nevertheless,
- the target file system might map uids (think of NFS or CIFS [2]).
However, the diagnostic should be omitted if the 'mv' program can verify
that the uid and gid of the directory on the target file system is the same
as on the source file system.
Likewise for the 'cp' program:
$ cp --version
cp (GNU coreutils) 9.3.147-d553ab
...
Written by Torbjorn Granlund, David MacKenzie, and Jim Meyering.
$ mkdir dir2
$ echo foo > dir2/file2
$ ls -ld dir2
drwxrwxr-x 2 bruno bruno 4096 Aug 29 20:15 dir2
$ strace -o /tmp/cp.log cp dir2 /media/nas/bruno/dir2
$ strace -o /tmp/cp.log cp -a dir2 /media/nas/bruno/dir2
cp: failed to preserve ownership for '/media/nas/bruno/dir2': Permission denied
$ ls -ld /media/nas/bruno/dir2
drwx-- 2 bruno bruno 0 Aug 29 20:15 /media/nas/bruno/dir2
Here, in cp.log, the essential line is:
fchownat(AT_FDCWD, "/media/nas/bruno/dir2", 1000, 1000, AT_SYMLINK_NOFOLLOW) =
-1 EACCES (Permission denied)
Bruno
[1] https://lists.gnu.org/archive/html/emacs-devel/2009-10/msg00094.html
[2] https://linux.die.net/man/8/mount.cifs
execve("/media/develdata/devel/build/coreutils-9.3.147-d553ab/build-64/src/mv", ["mv", "dir1", "/media/nas/bruno/dir1"], 0x7ffd93a100d0 /* 76 vars */) = 0
brk(NULL) = 0xc7d000
arch_prctl(0x3001 /* ARCH_??? */, 0x7ffc663bc920) = -1 EINVAL (Invalid argument)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3771429000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=130427, ...}, AT_EMPTY_PATH) = 0
mmap(NULL, 130427, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f3771409000
close(3)= 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=166280, ...}, AT_EMPTY_PATH) = 0
mmap(NULL, 177672, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f37713dd000
mprotect(0x7f37713e3000, 139264, PROT_NONE) = 0
mmap(0x7f37713e3000, 106496, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7f37713e3000
mmap(0x7f37713fd000, 28672, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0x7f37713fd000
mmap(0x7f3771405000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x27000) = 0x7f3771405000
mmap(0x7f3771407000, 5640, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3771407000
close(3)= 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libacl.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=34888, ...}, AT_EMPTY_PATH) = 0
mmap(NULL, 36896, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f37713d3000
mprotect(0x7f37713d5000, 24576, PROT_NONE) = 0
mmap(0x7f37713d5000, 16384, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f37713d5000
mmap(0x7f37713d9000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7f37713d9000
mmap(0x7f37713db000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7f37713db000
close(3)= 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libattr.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
newfstatat(3, "", {st_mod
bug#65310: test failure on Alpine Linux: tests/sort/sort-debug-keys
Pádraig Brady wrote: > The fact that ',' isn't used as the decimal point is surprising, > and is what's causing the test to fail. Ah, I see. Yes, for some tests one only needs a fr_FR.UTF-8 that supports UTF-8 in LC_CTYPE, whereas for other tests it also needs to support the LC_NUMERIC category correctly. > The attached adds an extra guard to the test to skip this portion > if ',' is not in fact the decimal point. Thanks! Bruno
bug#65255: uptime's boot time is inconsistent after VM sleep & resume
Paul Eggert wrote: > Thanks for the further comments. I installed your patch, along with the > attached additional patches. The other patch I mentioned, from https://lists.gnu.org/archive/html/coreutils/2023-08/msg00028.html , is also needed, for the "VM saved/sleep" change on NetBSD, OpenBSD, Minix, as far as I understand. Also, a typo in NEWS: s/Minux/Minix/ Bruno
bug#65310: test failure on Alpine Linux: tests/sort/sort-debug-keys
Hi, Doing "make check" of current coreutils on Alpine Linux 3.18, I see a test failure that I didn't see with the coreutils-9.3 release: FAIL: tests/sort/sort-debug-keys I'm attaching the relevant part of tests/test-suite.log. Bruno == GNU coreutils 2023-08-13: ./tests/test-suite.log == # TOTAL: 645 # PASS: 431 # SKIP: 211 # XFAIL: 0 # FAIL: 3 # XPASS: 0 # ERROR: 0 .. contents:: :depth: 2 FAIL: tests/sort/sort-debug-keys + initial_cwd_=/home/bruno/coreutils-2023-08-13/build + testdir_prefix_ + printf gt + pfx_=gt + mktempd_ /home/bruno/coreutils-2023-08-13/build gt-sort-debug-keys.sh. + destdir_=/home/bruno/coreutils-2023-08-13/build + template_=gt-sort-debug-keys.sh. + MAX_TRIES_=4 + destdir_slash_=/home/bruno/coreutils-2023-08-13/build/ + unset TMPDIR + d=/home/bruno/coreutils-2023-08-13/build/gt-sort-debug-keys.sh.VmIq + : + test -d /home/bruno/coreutils-2023-08-13/build/gt-sort-debug-keys.sh.VmIq + ls -dgo /home/bruno/coreutils-2023-08-13/build/gt-sort-debug-keys.sh.VmIq + perms='drwx--S--- 2 4096 Sep 13 08:45 /home/bruno/coreutils-2023-08-13/build/gt-sort-debug-keys.sh.VmIq' + : + echo /home/bruno/coreutils-2023-08-13/build/gt-sort-debug-keys.sh.VmIq + return + test_dir_=/home/bruno/coreutils-2023-08-13/build/gt-sort-debug-keys.sh.VmIq + cd /home/bruno/coreutils-2023-08-13/build/gt-sort-debug-keys.sh.VmIq + srcdir=../.. + builddir=.. + export srcdir builddir + gl_init_sh_nl_=' ' + IFS=' ' + expr 1 + 128 + eval 'trap '"'"'Exit 129'"'"' 1' + trap 'Exit 129' 1 + expr 2 + 128 + eval 'trap '"'"'Exit 130'"'"' 2' + trap 'Exit 130' 2 + expr 3 + 128 + eval 'trap '"'"'Exit 131'"'"' 3' + trap 'Exit 131' 3 + expr 13 + 128 + eval 'trap '"'"'Exit 141'"'"' 13' + trap 'Exit 141' 13 + expr 15 + 128 + eval 'trap '"'"'Exit 143'"'"' 15' + trap 'Exit 143' 15 + saved_IFS=' ' + IFS=: + new_PATH= + sep_= + test -d /home/bruno/coreutils-2023-08-13/build/src/. + new_PATH=/home/bruno/coreutils-2023-08-13/build/src + sep_=: + test -d /home/bruno/bin/. + new_PATH=/home/bruno/coreutils-2023-08-13/build/src:/home/bruno/bin + sep_=: + test -d /usr/local/bin/. + new_PATH=/home/bruno/coreutils-2023-08-13/build/src:/home/bruno/bin:/usr/local/bin + sep_=: + test -d /usr/sbin/. + new_PATH=/home/bruno/coreutils-2023-08-13/build/src:/home/bruno/bin:/usr/local/bin:/usr/sbin + sep_=: + test -d /usr/bin/. + new_PATH=/home/bruno/coreutils-2023-08-13/build/src:/home/bruno/bin:/usr/local/bin:/usr/sbin:/usr/bin + sep_=: + test -d /sbin/. + new_PATH=/home/bruno/coreutils-2023-08-13/build/src:/home/bruno/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin + sep_=: + test -d /bin/. + new_PATH=/home/bruno/coreutils-2023-08-13/build/src:/home/bruno/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin + sep_=: + IFS=' ' + PATH=/home/bruno/coreutils-2023-08-13/build/src:/home/bruno/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin + export PATH + trap remove_tmp_ EXIT + path_prepend_ ./src + test 1 '!=' 0 + path_dir_=./src + abs_path_dir_=/home/bruno/coreutils-2023-08-13/build/./src + PATH=/home/bruno/coreutils-2023-08-13/build/./src:/home/bruno/coreutils-2023-08-13/build/src:/home/bruno/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin + create_exe_shims_ /home/bruno/coreutils-2023-08-13/build/./src + return 0 + shift + test 0 '!=' 0 + export PATH + print_ver_ sort printf + require_built_ sort printf + skip_=no + test no '=' yes + test yes '=' yes + local i + env sort --version sort (GNU coreutils) 2023-08-13 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Mike Haertel and Paul Eggert. + env printf --version printf (GNU coreutils) 2023-08-13 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by David MacKenzie. + cat + sort -s -k2n --debug sort: text ordering performed using simple byte comparison sort: key 1 is numeric and spans multiple fields sort: note numbers use '.' as a decimal point in this locale + printf '1\n\n44\n33\n2\n' + sort -s -k1.3n --debug sort: text ordering performed using simple byte comparison sort: leading blanks are significant in key 1; consider also specifying 'b' sort: key 1 is numeric and spans multiple fields sort: note numbers use '.' as a decimal point in this locale + printf '1\n\n44\n33\n2\n' + sort -s -k1n --debug sort: text ordering performed using simple byte comparison sort: key 1 is numeric and spans multiple fields sort: not
bug#65255: uptime's boot time is inconsistent after VM sleep & resume
I wrote: > > I'll provide a NEWS > > entry afterwards, that summarizes the changes in coreutils + gnulib on the > > various platforms. > Paul Eggert wrote: > Thanks, this looks good. A NEWS entry would be welcome. Actually, it's 4 entries: * The 'uptime' program is now being built on FreeBSD, Haiku. * 'who -a' now displays the boot time on Alpine Linux, OpenBSD, Cygwin, Haiku, and some distributions of Android. * 'uptime' now succeeds on some distributions of Android. * The "up" time displayed by 'uptime' now includes VM saved/sleep time on Linux, Hurd, GNU/kFreeBSD, NetBSD, OpenBSD, Minix, Cygwin. Comments: Re entry 1: Yes, the 'uptime' program was not being built on FreeBSD. A consequence of the fact that FreeBSD has but no , and of this commit: https://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=2984e47c789ebc39f55a3b1cb20b943de88eeedc Re entry 2, 3: Let's write "some distributions of Android" since for me it works (under Termux, with an LG Android 11) whereas for Po Lu these two system APIs don't work (due to SELinux policies). Re entry 4: The improvement comes from both [1] https://lists.gnu.org/archive/html/coreutils/2023-08/msg00028.html on NetBSD, OpenBSD, Minix, [2] https://lists.gnu.org/archive/html/bug-coreutils/2023-08/msg00068.html on Linux, Hurd, GNU/kFreeBSD, NetBSD, Minix, Cygwin. Find below the detailed test results. > Does this mean Gnulib's 'uptime' module is obsolete? It can be marked obsolete, yes. But it should not be removed immediately, since some other package is using it: dc3dd. See https://codesearch.debian.net/search?q=uptime+path%3Abootstrap.conf=1 Bruno TEST RESULTS with + current gnulib as of today + current coreutils + my patches [1][2]: Fedora Rawhide Cinnamon $ date +"%Y-%m-%d %H:%M" 2023-08-15 11:11 $ coreutils-9.3/src/who -a system boot 2023-08-09 21:03 run-level 5 2023-08-09 21:03 bruno+ tty1 2023-08-09 21:03 old 1072 (:0) pts/32023-08-15 11:11107603 id=ts/3 term=0 exit=0 $ coreutils-9.3/src/uptime 11:11:14 up 5 days 12:57, 1 user, load average: 0.08, 0.03, 0.00 Includes VM sleep time? no $ coreutils-2023-08-13/src/who -a system boot 2023-08-09 21:03 run-level 5 2023-08-09 21:03 bruno+ tty1 2023-08-09 21:03 old 1072 (:0) pts/32023-08-15 11:11107603 id=ts/3 term=0 exit=0 $ coreutils-2023-08-13/src/uptime 11:11:19 up 5 days 14:07, 1 user, load average: 0.16, 0.04, 0.01 Includes VM sleep time? yes CentOS 6 $ date +"%Y-%m-%d %H:%M" 2023-08-15 11:18 $ coreutils-9.3/src/who -a system boot 2023-08-14 04:45 run-level 5 2023-08-14 04:45 LOGIN tty3 2023-08-14 04:45 1889 id=3 LOGIN tty4 2023-08-14 04:45 1891 id=4 LOGIN tty2 2023-08-14 04:45 1887 id=2 LOGIN tty5 2023-08-14 04:45 1893 id=5 LOGIN tty6 2023-08-14 04:45 1901 id=6 bruno+ tty1 2023-08-14 04:46 old 2072 (:0) bruno+ pts/02023-08-14 04:46 . 2466 (:0.0) $ coreutils-9.3/src/uptime 11:18:14 up 1 day 5:26, 2 users, load average: 0.00, 0.00, 0.00 Includes VM sleep time? no $ coreutils-2023-08-13/src/who -a system boot 2023-08-14 04:45 run-level 5 2023-08-14 04:45 LOGIN tty3 2023-08-14 04:45 1889 id=3 LOGIN tty4 2023-08-14 04:45 1891 id=4 LOGIN tty2 2023-08-14 04:45 1887 id=2 LOGIN tty5 2023-08-14 04:45 1893 id=5 LOGIN tty6 2023-08-14 04:45 1901 id=6 bruno+ tty1 2023-08-14 04:46 old 2072 (:0) bruno+ pts/02023-08-14 04:46 . 2466 (:0.0) $ coreutils-2023-08-13/src/uptime 11:18:19 up 1 day 6:33, 2 users, load average: 0.00, 0.00, 0.00 Includes VM sleep time? yes Raspbian $ date +"%Y-%m-%d %H:%M" 2023-08-15 11:21 $ coreutils-9.2/src/who -a system boot 1970-01-01 01:00 pi - tty1 2023-07-09 19:16 old 445 pi + tty7 2023-08-10 14:31 old 7911 (:0) run-level 5 2023-08-10 13:07 $ coreutils-9.2/src/uptime 11:21:36 up 4 days 20:41, 2 users, load average: 0.69, 0.20, 0.06 Includes VM sleep time? no $ coreutils-2023-08-13/src/who -a system boot 2023-08-10 13:07 pi - tty1 2023-07-09 19:16 old 445 pi + tty7 2023-08-10 14:31 old 7911 (:0) run-level 5 2023-08-10 13:07 $ coreut
bug#65294: [PATCH] maint: Fix compilation error on GNU/Hurd
Compiling a current coreutils on Debian GNU/Hurd from 2022, I get this
compilation error:
CC src/copy.o
../src/copy.c: In function 'set_author':
../src/copy.c:984:15: error: 'MACH_PORT_nullptr' undeclared (first use in this
function); did you mean 'MACH_PORT_NULL'?
984 | if (file == MACH_PORT_nullptr)
| ^
| MACH_PORT_NULL
../src/copy.c:984:15: note: each undeclared identifier is reported only once
for each function it appears in
make[2]: *** [Makefile:12489: src/copy.o] Error 1
The attached patch fixes it.
>From 2046827462094dde4ac8391865a3edc74e313126 Mon Sep 17 00:00:00 2001
From: Bruno Haible
Date: Mon, 14 Aug 2023 20:59:41 +0200
Subject: [PATCH] maint: Fix compilation error on GNU/Hurd (regression
2023-06-29)
* src/copy.c (set_author): Revert change from MACH_PORT_NULL to
MACH_PORT_nullptr.
---
src/copy.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/copy.c b/src/copy.c
index 90eebf6bc..a4aad06a8 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -981,7 +981,7 @@ set_author (char const *dst_name, int dest_desc, const struct stat *src_sb)
file_t file = (dest_desc < 0
? file_name_lookup (dst_name, 0, 0)
: getdport (dest_desc));
- if (file == MACH_PORT_nullptr)
+ if (file == MACH_PORT_NULL)
error (0, errno, _("failed to lookup file %s"), quoteaf (dst_name));
else
{
--
2.34.1
bug#65276: Build failure with gcc < 4.8
Building the current coreutils on GNU/kFreeBSD 7, I get link errors:
CCLD src/cksum
src/cksum-cksum.o: In function `pclmul_supported':
/home/bruno/coreutils-2023-08-13/build-64/../src/cksum.c:149: undefined
reference to `__builtin_cpu_supports'
/home/bruno/coreutils-2023-08-13/build-64/../src/cksum.c:150: undefined
reference to `__builtin_cpu_supports'
collect2: error: ld returned 1 exit status
make[2]: *** [src/cksum] Error 1
CCLD src/wc
src/wc.o: In function `avx2_supported':
/home/bruno/coreutils-2023-08-13/build-64/../src/wc.c:150: undefined reference
to `__builtin_cpu_supports'
collect2: error: ld returned 1 exit status
make[2]: *** [src/wc] Error 1
The reason is that the __builtin_cpu_supports function does not exist
(since the gcc version is 4.7.2 and __builtin_cpu_supports was only
introduced in gcc 4.8), but the configure test succeeds: Compiling this file
=== foo.c ===
#include
int
main (void)
{
return __builtin_cpu_supports ("pclmul");
}
=
merely produces warnings:
$ gcc -c -Wall foo.c
foo.c: In function ‘main’:
foo.c:6:3: warning: implicit declaration of function ‘__builtin_cpu_supports’
[-Wimplicit-function-declaration]
The attached patch fixes it.
>From a57d40e3f6997845ce88be6f40813b1b26cb6e16 Mon Sep 17 00:00:00 2001
From: Bruno Haible
Date: Mon, 14 Aug 2023 01:45:39 +0200
Subject: [PATCH] cksum,wc: Fix link errors with gcc < 4.8
* configure.ac: Attempt to link, not only compile, the test programs
with __builtin_cpu_supports.
---
configure.ac | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/configure.ac b/configure.ac
index 786ce81a5..5e5a55dab 100644
--- a/configure.ac
+++ b/configure.ac
@@ -519,7 +519,7 @@ ac_c_werror_flag=$cu_save_c_werror_flag
ac_save_CFLAGS=$CFLAGS
CFLAGS="-mavx -mpclmul $CFLAGS"
AC_MSG_CHECKING([if pclmul intrinsic exists])
-AC_COMPILE_IFELSE(
+AC_LINK_IFELSE(
[AC_LANG_SOURCE([[
#include
@@ -548,7 +548,7 @@ CFLAGS=$ac_save_CFLAGS
CFLAGS="-mavx2 $CFLAGS"
AC_MSG_CHECKING([if avx2 intrinstics exists])
-AC_COMPILE_IFELSE(
+AC_LINK_IFELSE(
[AC_LANG_SOURCE([[
#include
--
2.34.1
bug#65255: uptime's boot time is inconsistent after VM sleep & resume
PS: I wrote: > (print_uptime): Don't read /proc/uptime You might wonder: What is the replacement? Namely, as a fallback (for situations where the current process is running inside a container or jail): - On Linux, the auxiliary function get_linux_boot_time_final_fallback reads the info from /proc/uptime. - On macOS and BSD systems, we don't need to read it, because the sysctl invoked by the auxiliary function get_bsd_boot_time_final_fallback reads the same info, without needing a mounted /proc file system. - On Minix and Cygwin, no replacement is needed, because there is no "container" or "jail" concept in these OSes. Bruno
bug#65255: uptime's boot time is inconsistent after VM sleep & resume
On the following platforms:
- Linux (that includes glibc-based systems, Alpine Linux, Raspbian),
- NetBSD,
- Cygwin,
- Minix,
the "up" duration is inconsistent after the VM in which the OS is running
has been
1. put into sleep / saved / pause mode (terminology depends on the
hypervisor),
2. resumed,
3. a date bump has been done in the VM (either automatically or manually).
VM sleep and a date adjustment after resume is a common operation, as can
be seen
- from the fact that it's automatic in VirtualBox machines with the
"guest extensions" installed [1],
- from the fact that it's automatic in KVM [2],
- from the fact that it's automatic in Haiku as guest,
- from a remark in the libvirt documentation [3],
- from stackoverflow questions such as [4].
After such a VM sleep and a date adjustment, coreutils' "uptime" is
inconsistent in three ways:
* On Linux, the displayed "up" duration includes active time and
hardware suspend time [5], but excludes VM sleep time.
Such a figure is not useful for estimating the boot time, since
(time now) - (that "up" time) is not the boot time.
It is also not useful for estimating when e.g. a file system check
would be necessary on file systems without a journal, or how much
electricity was consumed — since the hardware suspend time was included.
So this figure is useless.
* The displayed "up" duration is inconsistent with the boot time displayed
by "who -a". (Since the Gnulib module 'readutmp' makes efforts to find
the boot time in a way that does not change when the date is adjusted.)
* The behaviour is inconsistent among platforms: On platforms which have
a /proc/uptime file, the displayed "up" time _excludes_ VM sleep time.
On the other platforms it _includes_ VM sleep time.
Seen on Linux (Fedora Rawhide, Alpine Linux 3.18, Raspbian), NetBSD 9.3,
Cygwin 2.9.0, Minix 3.3.
I see this as a bug. Find attached a fix for this bug. I'll provide a NEWS
entry afterwards, that summarizes the changes in coreutils + gnulib on the
various platforms.
Bruno
[1]
https://docs.oracle.com/en/virtualization/virtualbox/6.1/user/guestadditions.html#4.1.-Introduction-to-Guest-Additions
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1115340
[3]
https://libvirt.gitlab.io/libvirt-appdev-guide-python/libvirt_application_development_guide_using_python-Guest_Domains-Lifecycle-Save.html
[4] https://serverfault.com/questions/334698/
[5] This is indicated by glibc's :
/* Monotonic system-wide clock that includes time spent in suspension. */
# define CLOCK_BOOTTIME 7
I also verified this on a laptop.
>From ff6e92f38b560ade54df4c94e9d91657f740bb4b Mon Sep 17 00:00:00 2001
From: Bruno Haible
Date: Sun, 13 Aug 2023 00:57:23 +0200
Subject: [PATCH] uptime: Include VM sleep time in the "up" duration
* src/uptime.c: Don't include c-strtod.h.
(print_uptime): Don't read /proc/uptime, because the value it provides
does not change when a date adjustment occurs.
* bootstrap.conf (gnulib_modules): Remove 'uptime'.
---
bootstrap.conf | 1 -
src/uptime.c | 36 ++--
2 files changed, 6 insertions(+), 31 deletions(-)
diff --git a/bootstrap.conf b/bootstrap.conf
index 6ce27d5dd..d758a9ff6 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -285,7 +285,6 @@ gnulib_modules="
unlocked-io
unsetenv
update-copyright
- uptime
useless-if-before-free
userspec
utimecmp
diff --git a/src/uptime.c b/src/uptime.c
index c29798d32..0cc1dedba 100644
--- a/src/uptime.c
+++ b/src/uptime.c
@@ -22,7 +22,6 @@
#include
#include "system.h"
-#include "c-strtod.h"
#include "long-options.h"
#include "quote.h"
#include "readutmp.h"
@@ -42,33 +41,13 @@ print_uptime (idx_t n, struct gl_utmp const *this)
idx_t entries = 0;
time_t boot_time = 0;
time_t time_now;
- time_t uptime = 0;
+ time_t uptime;
intmax_t updays;
int uphours;
int upmins;
struct tm *tmn;
double avg[3];
int loads;
-#ifdef HAVE_PROC_UPTIME
- FILE *fp;
-
- fp = fopen ("/proc/uptime", "r");
- if (fp != nullptr)
-{
- char buf[BUFSIZ];
- char *b = fgets (buf, BUFSIZ, fp);
- if (b == buf)
-{
- char *end_ptr;
- double upsecs = c_strtod (buf, _ptr);
- if (buf != end_ptr)
-uptime = (0 <= upsecs && upsecs < TYPE_MAXIMUM (time_t)
- ? upsecs : -1);
-}
-
- fclose (fp);
-}
-#endif /* HAVE_PROC_UPTIME */
/* Loop through all the utmp entries we just read and count up the valid
ones, also in the process possibly gleaning boottime. */
@@ -79,16 +58,13 @@ print_uptime (idx_t n, struct gl_utmp const *this)
boot_time = this->ut_ts.tv_sec;
++this;
}
+ /* The gnulib m
bug#64937: boot time on Linux
Natanael Copa wrote:
> There are machines without RTC (Raspberry PI for example), and in
> this case the time stamp may end up to be the same every reboot (if
> correctly set up it should save the shutdown time for the reboot and set
> time to this on next boot, but there is no guarantee).
Indeed, on a Raspi with Raspbian (Debian derivative), /var/run/utmp contains
a BOOT_TIME entry with time = 1970-01-01 00:00:05. Which is unusable.
The clock gets set during the boot process, apparently through NTP. Then,
some files get touched:
/var/lib/systemd/timers/stamp-apt-daily.timer
/var/lib/systemd/timers/stamp-apt-daily-upgrade.timer
/var/lib/apt/daily_lock
But I suspect that they may be touched at other moments than during boot.
Therefore here, the workaround with the file time stamps does not work.
But another workaround works:
2023-08-10 Bruno Haible
readutmp: Fix the boot time returned on Raspbian.
* lib/readutmp.c (read_utmp_from_file): When the time of the BOOT_TIME
entry is very close to the Epoch, replace it with the time from the
"runlevel"/"~" entry.
diff --git a/lib/readutmp.c b/lib/readutmp.c
index b344d8294d..e383531474 100644
--- a/lib/readutmp.c
+++ b/lib/readutmp.c
@@ -369,6 +369,12 @@ read_utmp_from_file (char const *file, idx_t *n_entries,
STRUCT_UTMP **utmp_buf,
SET_UTMP_ENT ();
+# if defined __linux__ && !defined __ANDROID__
+ bool file_is_utmp = (strcmp (file, UTMP_FILE) == 0);
+ /* Timestamp of the "runlevel" entry, if any. */
+ struct timespec runlevel_ts = {0};
+# endif
+
void const *entry;
while ((entry = GET_UTMP_ENT ()) != NULL)
@@ -408,6 +414,13 @@ read_utmp_from_file (char const *file, idx_t *n_entries,
STRUCT_UTMP **utmp_buf,
struct UTMP_STRUCT_NAME const *ut = (struct UTMP_STRUCT_NAME const *)
entry;
# endif
+ struct timespec ts =
+#if (HAVE_UTMPX_H ? 1 : HAVE_STRUCT_UTMP_UT_TV)
+{ .tv_sec = ut->ut_tv.tv_sec, .tv_nsec = ut->ut_tv.tv_usec * 1000 };
+#else
+{ .tv_sec = ut->ut_time, .tv_nsec = 0 };
+#endif
+
a = add_utmp (a, options,
UT_USER (ut), strnlen (UT_USER (ut), UT_USER_SIZE),
#if (HAVE_UTMPX_H ? HAVE_STRUCT_UTMPX_UT_ID :
HAVE_STRUCT_UTMP_UT_ID)
@@ -431,11 +444,7 @@ read_utmp_from_file (char const *file, idx_t *n_entries,
STRUCT_UTMP **utmp_buf,
#else
0,
#endif
-#if (HAVE_UTMPX_H ? 1 : HAVE_STRUCT_UTMP_UT_TV)
-(struct timespec) { .tv_sec = ut->ut_tv.tv_sec, .tv_nsec =
ut->ut_tv.tv_usec * 1000 },
-#else
-(struct timespec) { .tv_sec = ut->ut_time, .tv_nsec = 0 },
-#endif
+ts,
#if (HAVE_UTMPX_H ? HAVE_STRUCT_UTMPX_UT_SESSION :
HAVE_STRUCT_UTMP_UT_SESSION)
ut->ut_session,
#else
@@ -443,6 +452,12 @@ read_utmp_from_file (char const *file, idx_t *n_entries,
STRUCT_UTMP **utmp_buf,
#endif
UT_EXIT_E_TERMINATION (ut), UT_EXIT_E_EXIT (ut)
);
+# if defined __linux__ && !defined __ANDROID__
+ if (file_is_utmp
+ && memcmp (UT_USER (ut), "runlevel", strlen ("runlevel") + 1) == 0
+ && memcmp (ut->ut_line, "~", strlen ("~") + 1) == 0)
+runlevel_ts = ts;
+# endif
}
END_UTMP_ENT ();
@@ -450,9 +465,19 @@ read_utmp_from_file (char const *file, idx_t *n_entries,
STRUCT_UTMP **utmp_buf,
# if defined __linux__ && !defined __ANDROID__
/* On Alpine Linux, UTMP_FILE is not filled. It is always empty.
So, fake a BOOT_TIME entry, by getting the time stamp of a file that
- gets touched only during the boot process. */
+ gets touched only during the boot process.
+
+ On Raspbian, which runs on hardware without a real-time clock, during
boot,
+ 1. the clock gets set to 1970-01-01 00:00:00,
+ 2. an entry gets written into /var/run/utmp, with ut_type = BOOT_TIME,
+ ut_user = "reboot", ut_line = "~", time = 1970-01-01 00:00:05 or so,
+ 3. the clock gets set to a correct value through NTP,
+ 4. an entry gets written into /var/run/utmp, with
+ ut_user = "runlevel", ut_line = "~", time = correct value.
+ In this case, copy the time from the "runlevel" entry to the "reboot"
+ entry. */
if ((options & (READ_UTMP_USER_PROCESS | READ_UTMP_NO_BOOT_TIME)) == 0
- && strcmp (file, UTMP_FILE) == 0)
+ && file_is_utmp)
{
bool have_boot_time = false;
for (idx_t i = 0; i < a.filled; i++)
@@ -460,12 +485,16 @@ read_utmp_from_file (char const *file, idx_t *n_entries,
S
bug#64937: boot time on Linux
Po Lu wrote: > >> Both clock_gettime (CLOCK_BOOTIME, ... sysinfo fail with AVC denial > >> errors and errno set to EACCESS. > > > > Was this inside Termux, or inside the Emacs app? > > Inside the Emacs app. Emacs does not have the following in AndroidManifest.xml, which Termux has: Maybe one of these makes the difference? Bruno
bug#64937: "who" reports funny dates
Thorsten Kukuk wrote: > Not sure how relevant this code still is, but currently I get with this: > > lib/readutmp.c: In function 'get_boot_time_uncached': > lib/readutmp.c:326:35: error: declaration of 'up' shadows a previous local > [-Werror=shadow] > 326 | struct timespec up = > | ^~ > lib/readutmp.c:286:19: note: shadowed declaration is here > 286 | struct timespec up; > | ^~ > cc1: all warnings being treated as errors This is merely a warning, and it's already gone after today's refactorings in Gnulib. To get past it, either remove '-Werror' from the Makefile, or bootstrap against the current Gnulib: $ GNULIB_SRCDIR= $ ./bootstrap --no-git --gnulib-srcdir=$GNULIB_SRCDIR Bruno
bug#64937: boot time on Linux
Po Lu wrote: > Both clock_gettime (CLOCK_BOOTIME, ... sysinfo fail with AVC denial > errors and errno set to EACCESS. Was this inside Termux, or inside the Emacs app? Bruno
bug#64937: boot time on Linux
I wrote:
> > No, it isn't. The attached file, when compiled and run under Termux (which
> > doesn't have particular permissions), prints e.g.:
> >
> > from clock : 1691616762.476870660 = 2023-08-09 21:32:42.476870660
> > from sysinfo: 1691616762.329261637 = 2023-08-09 21:32:42.329261637
> >
> > Note that this uses the kernel's uptime counter, so it will not work well
> > when the user changes the current time manually. But this is rare on
> > Android.
It works well enough, that I'm adding it to Gnulib, through the attached patch.
Po Lu wrote:
> This uses the uptime counter (which also results in an SELinux denial
> for me, but different Android distributions have SELinux policies of
> varying strictness)
How did you run the program, and which of the two calls (clock_gettime, sysinfo)
failed for you? Maybe it depends not only on the Android version and device,
but also on the permissions required by the app?
Bruno
From 9db6a91083ecca9c49bd5353c7624c6388f332fb Mon Sep 17 00:00:00 2001
From: Bruno Haible
Date: Thu, 10 Aug 2023 07:59:19 +0200
Subject: [PATCH] readutmp: Return a boot time also on Android.
* lib/readutmp.c (get_linux_uptime): New function, extracted from
get_boot_time_uncached.
(read_utmp_from_file): Don't look for file time stamps on Android.
Instead, use get_linux_uptime.
(get_boot_time_uncached): Use get_linux_uptime.
---
ChangeLog | 9 +++
lib/readutmp.c | 196 +
2 files changed, 124 insertions(+), 81 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 3c2256a7b2..b167189c03 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,12 @@
+2023-08-10 Bruno Haible
+
+ readutmp: Return a boot time also on Android.
+ * lib/readutmp.c (get_linux_uptime): New function, extracted from
+ get_boot_time_uncached.
+ (read_utmp_from_file): Don't look for file time stamps on Android.
+ Instead, use get_linux_uptime.
+ (get_boot_time_uncached): Use get_linux_uptime.
+
2023-08-09 Bruno Haible
readutmp: Fix a mistake (regression 2023-08-08).
diff --git a/lib/readutmp.c b/lib/readutmp.c
index ec21f5e16f..b344d8294d 100644
--- a/lib/readutmp.c
+++ b/lib/readutmp.c
@@ -31,11 +31,13 @@
#include
#include
+#if READUTMP_USE_SYSTEMD || defined __ANDROID__
+# include
+# include
+#endif
#if READUTMP_USE_SYSTEMD
# include
-# include
# include
-# include
#endif
#include "stat-time.h"
@@ -284,6 +286,60 @@ finish_utmp (struct utmp_alloc a)
return a;
}
+# if READUTMP_USE_SYSTEMD || defined __ANDROID__
+
+/* Store the uptime counter, as managed by the Linux kernel, in *P_UPTIME.
+ Return 0 upon success, -1 upon failure. */
+static int
+get_linux_uptime (struct timespec *p_uptime)
+{
+ /* The clock_gettime facility returns the uptime with a resolution of 1 µsec.
+ It is available with glibc >= 2.14. In glibc < 2.17 it required linking
+ with librt. */
+# if (__GLIBC__ + (__GLIBC_MINOR__ >= 17) > 2) || defined __ANDROID__
+ if (clock_gettime (CLOCK_BOOTTIME, p_uptime) >= 0)
+return 0;
+# endif
+
+ /* /proc/uptime contains the uptime with a resolution of 0.01 sec.
+ But it does not have read permissions on Android. */
+# if !defined __ANDROID__
+ FILE *fp = fopen ("/proc/uptime", "re");
+ if (fp != NULL)
+{
+ char buf[32 + 1];
+ size_t n = fread (buf, 1, sizeof (buf) - 1, fp);
+ fclose (fp);
+ if (n > 0)
+{
+ buf[n] = '\0';
+ /* buf now contains two values: the uptime and the idle time. */
+ char *endptr;
+ double uptime = strtod (buf, );
+ if (endptr > buf)
+{
+ p_uptime->tv_sec = (time_t) uptime;
+ p_uptime->tv_nsec = (uptime - p_uptime->tv_sec) * 1e9 + 0.5;
+ return 0;
+}
+}
+}
+# endif
+
+ /* The sysinfo call returns the uptime with a resolution of 1 sec only. */
+ struct sysinfo info;
+ if (sysinfo () >= 0)
+{
+ p_uptime->tv_sec = info.uptime;
+ p_uptime->tv_nsec = 0;
+ return 0;
+}
+
+ return -1;
+}
+
+# endif
+
# if !HAVE_UTMPX_H && HAVE_UTMP_H && defined UTMP_NAME_FUNCTION && !HAVE_DECL_GETUTENT
struct utmp *getutent (void);
# endif
@@ -391,7 +447,7 @@ read_utmp_from_file (char const *file, idx_t *n_entries, STRUCT_UTMP **utmp_buf,
END_UTMP_ENT ();
-# if defined __linux__
+# if defined __linux__ && !defined __ANDROID__
/* On Alpine Linux, UTMP_FILE is not filled. It is always empty.
So, fake a BOOT_TIME entry, by getting the time stamp of a file that
gets touched only during the boot process. */
@@ -436,6 +492,37 @@ read_utmp_from_file (char const *file, idx_t *n_entries, STRUCT_UTMP **utmp_buf,
}
# endif
+# if defined __ANDROID__
+ /* On Android, there is no /var, and normal processes don't have access
+
bug#64937: boot time on Linux
Po Lu wrote:
> > Also, I don't know how Android records boot time so I'll cc this to Po
> > Lu, the main developer for Emacs on Android.
>
> The boot time is off limits to user programs on Android, for security
> reasons.
No, it isn't. The attached file, when compiled and run under Termux (which
doesn't have particular permissions), prints e.g.:
from clock : 1691616762.476870660 = 2023-08-09 21:32:42.476870660
from sysinfo: 1691616762.329261637 = 2023-08-09 21:32:42.329261637
Note that this uses the kernel's uptime counter, so it will not work well
when the user changes the current time manually. But this is rare on Android.
Bruno
#include
#include
#include
#include
#include
static const char *
as_time_string (time_t tim)
{
struct tm *gmt = gmtime ();
static char timbuf[100];
if (gmt == NULL
|| strftime (timbuf, sizeof (timbuf), "%Y-%m-%d %H:%M:%S", gmt)
== 0)
strcpy (timbuf, "---");
return timbuf;
}
int main ()
{
/* clock() returns the uptime with a resolution of ca. 1 usec. */
struct timespec ts_now;
struct timespec up;
if (clock_gettime (CLOCK_REALTIME, _now) == 0
&& clock_gettime (CLOCK_BOOTTIME, ) == 0)
{
struct timespec result = ts_now;
if (result.tv_nsec < up.tv_nsec)
{
result.tv_nsec += 10;
result.tv_sec -= 1;
}
result.tv_sec -= up.tv_sec;
result.tv_nsec -= up.tv_nsec;
printf ("from clock : %d.%09d = %s.%09d\n",
(int) result.tv_sec, (int) result.tv_nsec,
as_time_string (result.tv_sec), (int) result.tv_nsec);
}
/* /proc/uptime contains the uptime with a resolution of 0.01 sec. */
FILE *fp = fopen ("/proc/uptime", "re");
if (fp != NULL)
{
char buf[32 + 1];
size_t n = fread (buf, 1, sizeof (buf) - 1, fp);
fclose (fp);
if (n > 0)
{
buf[n] = '\0';
/* buf now contains two values: the uptime and the idle time. */
char *endptr;
double uptime = strtod (buf, );
if (endptr > buf)
{
struct timespec result;
if (clock_gettime (CLOCK_REALTIME, ) == 0)
{
time_t uptime_sec = uptime;
struct timespec up =
{
.tv_sec = uptime_sec,
.tv_nsec = (uptime - uptime_sec) * 1e9 + 0.5
};
if (result.tv_nsec < up.tv_nsec)
{
result.tv_nsec += 10;
result.tv_sec -= 1;
}
result.tv_sec -= up.tv_sec;
result.tv_nsec -= up.tv_nsec;
printf ("from /proc : %d.%09d = %s.%09d\n",
(int) result.tv_sec, (int) result.tv_nsec,
as_time_string (result.tv_sec), (int) result.tv_nsec);
}
}
}
}
/* The sysinfo call returns the uptime with a resolution of 1 sec only. */
struct sysinfo info;
if (sysinfo () >= 0)
{
struct timespec result;
if (clock_gettime (CLOCK_REALTIME, ) == 0)
{
result.tv_sec -= info.uptime;
printf ("from sysinfo: %d.%09d = %s.%09d\n",
(int) result.tv_sec, (int) result.tv_nsec,
as_time_string (result.tv_sec), (int) result.tv_nsec);
}
}
return 0;
}
bug#64937: boot time on Linux
Paul Eggert wrote: > This patch does not address the problem for Alpine, nor I suspect for > Android. I suppose Alpine could use the timestamp of /var/run/utmp (or > is that /run/utmp?) but I don't know how 'configure' would reliably > detect it's being built or cross-built for Alpine. I'll cc this to > Natanael Copa, who does the Alpine ports for Emacs, to see whether he > can give advice. For Alpine Linux, in Gnulib, I've used the following approach: Read the entire contents of /var/run/utmp, and if it contains no entry with ut_type == BOOT_TIME, look at the time stamp of some files, in particular /var/run/utmp. It doesn't require a configure test. Bruno
bug#64937: boot time on Linux
I wrote:
> So, all approaches that compute the boot time through a subtraction are
> going to be wrong in these scenarios.
>
> The better approach is really to read the boot time from a time stamp —
> inside a file such as /var/run/utmp or /var/log/wtmp, or attached to
> a file such as
> /var/lib/systemd/random-seed(first file touched during boot)
> /var/log/boot.log (one of the last files touched during boot).
>
> And on Alpine Linux, while /var/run/utmp is empty, its time stamp is
> essentially the boot time.
With the attached patches, the boot time returned by read_utmp() is stable
across 'sudo date MMDDhhmm' invocations.
On Linux distros, I found these files to be touched after boot:
/var/lib/systemd/random-seed (seen on distros with systemd, e.g. CentOS 7,
Fedora 27, Debian 9, Ubuntu 17.10, Arch 19.11,
Manjaro 17)
/var/run/utmp (seen on distros with OpenRC, e.g. Alpine Linux)
/var/lib/random-seed (seen on OpenSUSE 12.1, CentOS 5)
And, while at it, also on OpenBSD. It touches /var/db/host.random and
/var/run/utmp.
Bruno
2023-08-09 Bruno Haible
readutmp: Return a boot time also on OpenBSD.
* lib/readutmp.h (BOOT_TIME, USER_PROCESS): Provide fallback
definitions.
* lib/readutmp.c (read_utmp_from_file) [__OpenBSD__]: Fake a BOOT_TIME
entry by looking at the time stamp of a specific file.
readutmp: Return a boot time also on Alpine Linux.
* lib/readutmp.c: Include stat-time.h.
(SIZEOF): New macro.
(read_utmp_from_file) [__linux__]: Fake a BOOT_TIME entry by looking
at the time stamp of a specific file.
* modules/readutmp (Depends-on): Add stat-time.
readutmp: Fix boot time in VMs after sleep state and date update.
* lib/readutmp.c (read_utmp_from_file): New function, extracted from
read_utmp.
(get_boot_time_uncached): Before all other approaches, try to find the
boot time in the /var/run/utmp file.
(read_utmp): Invoke read_utmp_from_file.
readutmp: Make it easier to filter for/against the boot-time entry.
* lib/readutmp.h (READ_UTMP_BOOT_TIME, READ_UTMP_NO_BOOT_TIME): New
enum items.
* lib/readutmp.c (desirable_utmp_entry, read_utmp_from_systemd):
Implement them.
(read_utmp): If no entries can match the given options, return
immediately.
From 1e4bc08eb8ce3c00f7b8e72dc69e07ccbc63ed20 Mon Sep 17 00:00:00 2001
From: Bruno Haible
Date: Wed, 9 Aug 2023 18:49:22 +0200
Subject: [PATCH 1/4] readutmp: Make it easier to filter for/against the
boot-time entry.
* lib/readutmp.h (READ_UTMP_BOOT_TIME, READ_UTMP_NO_BOOT_TIME): New
enum items.
* lib/readutmp.c (desirable_utmp_entry, read_utmp_from_systemd):
Implement them.
(read_utmp): If no entries can match the given options, return
immediately.
---
ChangeLog | 12 ++-
lib/readutmp.c | 245 +++--
lib/readutmp.h | 12 ++-
3 files changed, 157 insertions(+), 112 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 40274c0a08..c2c9613643 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,7 +1,17 @@
+2023-08-09 Bruno Haible
+
+ readutmp: Make it easier to filter for/against the boot-time entry.
+ * lib/readutmp.h (READ_UTMP_BOOT_TIME, READ_UTMP_NO_BOOT_TIME): New
+ enum items.
+ * lib/readutmp.c (desirable_utmp_entry, read_utmp_from_systemd):
+ Implement them.
+ (read_utmp): If no entries can match the given options, return
+ immediately.
+
2023-08-08 Paul Eggert
readutmp: omit pragma
- * lib/readutmp.c: Omit -Sstringop-overread pragma.
+ * lib/readutmp.c: Omit -Wstringop-overread pragma.
It’s no longer needed now that extract_trimmed_name
no longer calls strnlen.
diff --git a/lib/readutmp.c b/lib/readutmp.c
index 31db4023a1..a7773c4f36 100644
--- a/lib/readutmp.c
+++ b/lib/readutmp.c
@@ -163,6 +163,13 @@ desirable_utmp_entry (STRUCT_UTMP const *ut, int options)
&& ut->ut_line[0] == '\0' && ut->ut_host[0] == '\0')
return false;
# endif
+
+ bool boot_time = UT_TYPE_BOOT_TIME (ut);
+ if ((options & READ_UTMP_BOOT_TIME) && !boot_time)
+return false;
+ if ((options & READ_UTMP_NO_BOOT_TIME) && boot_time)
+return false;
+
bool user_proc = IS_USER_PROCESS (ut);
if ((options & READ_UTMP_USER_PROCESS) && !user_proc)
return false;
@@ -171,6 +178,7 @@ desirable_utmp_entry (STRUCT_UTMP const *ut, int options)
&& 0 < UT_PID (ut)
&& (kill (UT_PID (ut), 0) < 0 && errno == ESRCH))
return false;
+
return true;
}
@@ -441,7 +449,7 @@ read_utmp_from_systemd (idx_t *n_entries, STRUCT_UTMP **utmp_buf, int options)
struct utmp_alloc a = {0};
/* Synthesize a BOOT_TIME entry. */
- if (!(options &
bug#64937: boot time on Linux
More info about this problem: I wrote
> I have a VM running in VirtualBox, that I booted 6 days ago, then "saved"
> it (i.e. all the state gets frozen) and resumed it today around 20:30 UTC.
> ...
> So, both programs show a "boot time" that is just 5 hours ago, at a moment
> when
> the VM was in fact frozen.
1) The issue can be more easily reproduced by simply setting the date
("sudo date MMDDhhmm") in a VM, be it in VirtualBox or QEMU.
The result of
(time now) - (kernel's uptime notion)
becomes wrong at that point.
2) Fedora Rawhide has the 'virtualbox-guest-additions' package installed
by default, and these guest additions contain a "time synchronization"
feature [1]. Within 5 seconds after resuming a VM from sleep, they
do effectively the same as a "sudo date MMDDhhmm" command.
So, all approaches that compute the boot time through a subtraction are
going to be wrong in these scenarios.
The better approach is really to read the boot time from a time stamp —
inside a file such as /var/run/utmp or /var/log/wtmp, or attached to
a file such as
/var/lib/systemd/random-seed(first file touched during boot)
/var/log/boot.log (one of the last files touched during boot).
And on Alpine Linux, while /var/run/utmp is empty, its time stamp is
essentially the boot time.
The approach used by Emacs, namely to look at the time stamp of
/var/run/random-seed, is therefore essentially one of the best approaches.
It just needs to also look at /var/lib/systemd/random-seed and - on Alpine
Linux - /var/run/utmp .
Bruno
[1]
https://docs.oracle.com/en/virtualization/virtualbox/6.1/user/guestadditions.html#4.1.-Introduction-to-Guest-Additions
bug#64937: boot time on Linux
Thorsten Kukuk wrote: > > What API do you suggest we use instead? > > For me clock_gettime works very well: > https://github.com/thkukuk/utmpx/blob/main/utmp-to-logind.md#determine-boot-time Now that I've implemented this method in gnulib's 'readutmp' module and built coreutils with that, I see that it does not work "very well" at all. I have a VM running in VirtualBox, that I booted 6 days ago, then "saved" it (i.e. all the state gets frozen) and resumed it today around 20:30 UTC. In this VM, the test-readutmp program produces this output: Here are the read_utmp results. Flags: B = Boot, U = User Process Termi‐ Flags Time (GMT) User DevicePIDnation Exit B U Host --- -- --- -- -- ---- - - 2023-08-02 16:56:15 bruno seat0 15110 0 X :0 2023-08-08 15:47:48 reboot ~00 0 X Similarly, $ TZ=UTC ./who -a /var/run/utmp system boot 2023-08-08 15:47 bruno? seat02023-08-02 16:56 ? 1511 (:0) So, both programs show a "boot time" that is just 5 hours ago, at a moment when the VM was in fact frozen. Whereas the 'who' program, when it actually reads the /var/run/utmp file (via the symlink /run/utmp), produces the correct time: $ TZ=UTC ./who -a /run/utmp system boot 2023-08-02 16:55 run-level 5 2023-08-02 16:56 bruno+ tty1 2023-08-02 16:56 old 1558 (:0) pts/12023-08-02 17:31 2815 id=ts/1 term=0 exit=0 All three methods from get_boot_time_uncached produce this wrong "boot time", because they are all three based on 1) the kernel's ktime_get_boottime or ktime_get_boottime_ts64 function, 2) subtracting the returned value from the current time. The description CLOCK_BOOTTIME in glibc's is: /* Monotonic system-wide clock that includes time spent in suspension. */ # define CLOCK_BOOTTIME 7 Maybe that includes time spent in suspension. But it does not include time spent in "saved" state of virtual machines. Thus is won't work right in cloud environments. I can't really argue that it's a kernel bug, since VirtualBox and VMs exist for more than 10 years, and /proc/uptime and sysinfo() as well. Therefore I think that - The readutmp module should better pick the boot time from the real /var/run/utmp file, and ignore the rest of this file (in systemd mode). - We still don't have a reliable way of getting the boot time on Alpine Linux, since the /var/run/utmp file there is empty. - The effort to get rid of /var/run/utmp has encountered a stumbling block. Bruno
bug#64937: "who" reports funny dates
I wrote:
> > (2) The readutmp module should use a runtime 'if' rather than a
> > compile-time
> > #if, in order to dispatch between the systemd backend and the
> > file-based
> > backend.
This patch implements it.
2023-08-08 Bruno Haible
readutmp: Use classical implementation for files != /var/run/utmp.
* lib/readutmp.c (read_utmp_from_systemd): Renamed from read_utmp
[READUTMP_USE_SYSTEMD]. Remove file argument.
(read_utmp): Call it when the file argument is "/var/run/utmp".
diff --git a/lib/readutmp.c b/lib/readutmp.c
index f7e43eb4a6..66b25bc7ae 100644
--- a/lib/readutmp.c
+++ b/lib/readutmp.c
@@ -439,17 +439,9 @@ guess_pty_name (uid_t uid, const struct timespec at)
return NULL;
}
-int
-read_utmp (char const *file, idx_t *n_entries, STRUCT_UTMP **utmp_buf,
- int options)
+static int
+read_utmp_from_systemd (idx_t *n_entries, STRUCT_UTMP **utmp_buf, int options)
{
- /* The current implementation can imitate only UTMP_FILE. */
- if (strcmp (file, UTMP_FILE) != 0)
-{
- errno = ENOTSUP;
- return -1;
-}
-
/* Fill entries, simulating what a utmp file would contain. */
struct utmp_alloc a = {0};
@@ -602,7 +594,9 @@ read_utmp (char const *file, idx_t *n_entries, STRUCT_UTMP
**utmp_buf,
return 0;
}
-# elif defined UTMP_NAME_FUNCTION /* glibc, musl, macOS, FreeBSD, NetBSD,
Minix, AIX, IRIX, Solaris, Cygwin, Android */
+# endif
+
+# if defined UTMP_NAME_FUNCTION /* glibc, musl, macOS, FreeBSD, NetBSD, Minix,
AIX, IRIX, Solaris, Cygwin, Android */
# if !HAVE_UTMPX_H && HAVE_UTMP_H && !HAVE_DECL_GETUTENT
struct utmp *getutent (void);
@@ -612,6 +606,12 @@ int
read_utmp (char const *file, idx_t *n_entries, STRUCT_UTMP **utmp_buf,
int options)
{
+# if READUTMP_USE_SYSTEMD
+ if (strcmp (file, UTMP_FILE) == 0)
+/* Imitate reading UTMP_FILE, using systemd and Linux APIs. */
+return read_utmp_from_systemd (n_entries, utmp_buf, options);
+# endif
+
/* Ignore the return value for now.
Solaris' utmpname returns 1 upon success -- which is contrary
to what the GNU libc version does. In addition, older GNU libc
bug#64937: "who" reports funny dates
Thorsten Kukuk wrote:
> > What API do you suggest we use instead?
>
> For me clock_gettime works very well:
> https://github.com/thkukuk/utmpx/blob/main/utmp-to-logind.md#determine-boot-time
Indeed, this provides the boot time with a resolution of 1 µsec. Whereas the
/proc/uptime approach only has a resolution of 1 µsec. Thanks for the
suggestion.
2023-08-08 Bruno Haible
readutmp: Get the boot time with higher precision.
Suggested by Thorsten Kukuk in
<https://github.com/thkukuk/utmpx/blob/main/utmp-to-logind.md#determine-boot-time>.
* lib/readutmp.c (get_boot_time_uncached): Try clock_gettime first.
diff --git a/lib/readutmp.c b/lib/readutmp.c
index 7ef5bfe84c..f7e43eb4a6 100644
--- a/lib/readutmp.c
+++ b/lib/readutmp.c
@@ -284,6 +284,31 @@ finish_utmp (struct utmp_alloc a)
static struct timespec
get_boot_time_uncached (void)
{
+ /* The clock_gettime facility returns the uptime with a resolution of 1 µsec.
+ It is available with glibc >= 2.14. In glibc < 2.17 it required linking
+ with librt. */
+# if __GLIBC__ + (__GLIBC_MINOR__ >= 17) > 2
+ struct timespec up;
+ if (clock_gettime (CLOCK_BOOTTIME, ) >= 0)
+{
+ struct timespec result;
+ /* equivalent to:
+ if (clock_gettime (CLOCK_REALTIME, ) >= 0)
+ */
+ if (timespec_get (, TIME_UTC) >= 0)
+{
+ if (result.tv_nsec < up.tv_nsec)
+{
+ result.tv_nsec += 10;
+ result.tv_sec -= 1;
+}
+ result.tv_sec -= up.tv_sec;
+ result.tv_nsec -= up.tv_nsec;
+ return result;
+}
+}
+# endif
+
/* /proc/uptime contains the uptime with a resolution of 0.01 sec. */
FILE *fp = fopen ("/proc/uptime", "re");
if (fp != NULL)
bug#64937: "who" reports funny dates
I wrote: > > (1) The API of the readutmp module should provide unlimited-length > > ut_user, > > ut_host etc. fields always. No more #ifdef UT_USER_SIZE. > > (2) The readutmp module should use a runtime 'if' rather than a > > compile-time > > #if, in order to dispatch between the systemd backend and the > > file-based > > backend. > > > > I'll work on (1) today. (1) done through the following patch. It does not break coreutils. But coreutils can now be simplified through the attached 0001-maint-Simplify-after-gnulib-changed.patch . 2023-08-08 Bruno Haible readutmp: Return entries with unbounded strings on all platforms. Suggested by Paul Eggert in <https://lists.gnu.org/archive/html/bug-gnulib/2023-07/msg00165.html>. * m4/readutmp.m4 (gl_READUTMP): Test also whether struct utmp has an ut_tv member, and whether struct utmp and struct utmpx have an ut_session member. * lib/readutmp.h (struct gl_utmp): Define always. Add ut_exit field. (HAVE_GL_UTMP): Remove macro. (UT_USER, UT_TIME_MEMBER, UT_PID, UT_TYPE_EQ, UT_TYPE_NOT_DEFINED, UT_EXIT_E_TERMINATION, UT_EXIT_E_EXIT, STRUCT_UTMP): Define w.r.t. struct gl_utmp. (UT_USER_SIZE, UT_ID_SIZE, UT_LINE_SIZE, UT_HOST_SIZE): Define to -1 always. (getutent): Remove declaration. (HAVE_STRUCT_XTMP_UT_EXIT): Remove unused macro. (HAVE_STRUCT_XTMP_UT_ID, HAVE_STRUCT_XTMP_UT_PID, HAVE_STRUCT_XTMP_UT_HOST): Change to match the way coreutils uses these macros. * lib/readutmp.c (UT_USER, UT_TIME_MEMBER, UT_PID, UT_TYPE_EQ, UT_TYPE_NOT_DEFINED, IS_USER_PROCESS, UT_EXIT_E_TERMINATION, UT_EXIT_E_EXIT, UT_USER_SIZE, UT_ID_SIZE, UT_LINE_SIZE, UT_HOST_SIZE): Define w.r.t. struct utmpx or struct utmp. (extract_trimmed_name): Don't use UT_USER or UT_USER_SIZE here. (desirable_utmp_entry): Don't use UT_TIME_MEMBER or UT_USER here. (struct utmp_alloc): Define always. (add_utmp): Likewise. Add user_len, id_len, line_len, host_len, termination, exit arguments. Don't require that user, id, line, host are NUL-terminated. Assume user and host are non-NULL. (finish_utmp): New function, extracted from read_utmp. (read_utmp) [READUTMP_USE_SYSTEMD]: Update add_utmp invocations. Pass a non-NULL user and a non-NULL host. Call finish_utmp. (getutent): Move declaration from readutmp.h to here. (copy_utmp_entry): Remove function. (read_utmp) [UTMP_NAME_FUNCTION]: Replace variables n_read, n_alloc, utmp with a 'struct utmp_alloc'. Use 'struct utmpx32' from copy_utmp_entry here. Invoke add_utmp and finish_utmp. (read_utmp) [!UTMP_NAME_FUNCTION]: Replace variables n_read, n_alloc, utmp with a 'struct utmp_alloc'. Invoke add_utmp and finish_utmp. * NEWS: Mention the API change. From 622d2c0763777eb909a4fda5048238f524cc36f9 Mon Sep 17 00:00:00 2001 From: Bruno Haible Date: Tue, 8 Aug 2023 17:36:10 +0200 Subject: [PATCH] readutmp: Return entries with unbounded strings on all platforms. Suggested by Paul Eggert in <https://lists.gnu.org/archive/html/bug-gnulib/2023-07/msg00165.html>. * m4/readutmp.m4 (gl_READUTMP): Test also whether struct utmp has an ut_tv member, and whether struct utmp and struct utmpx have an ut_session member. * lib/readutmp.h (struct gl_utmp): Define always. Add ut_exit field. (HAVE_GL_UTMP): Remove macro. (UT_USER, UT_TIME_MEMBER, UT_PID, UT_TYPE_EQ, UT_TYPE_NOT_DEFINED, UT_EXIT_E_TERMINATION, UT_EXIT_E_EXIT, STRUCT_UTMP): Define w.r.t. struct gl_utmp. (UT_USER_SIZE, UT_ID_SIZE, UT_LINE_SIZE, UT_HOST_SIZE): Define to -1 always. (getutent): Remove declaration. (HAVE_STRUCT_XTMP_UT_EXIT): Remove unused macro. (HAVE_STRUCT_XTMP_UT_ID, HAVE_STRUCT_XTMP_UT_PID, HAVE_STRUCT_XTMP_UT_HOST): Change to match the way coreutils uses these macros. * lib/readutmp.c (UT_USER, UT_TIME_MEMBER, UT_PID, UT_TYPE_EQ, UT_TYPE_NOT_DEFINED, IS_USER_PROCESS, UT_EXIT_E_TERMINATION, UT_EXIT_E_EXIT, UT_USER_SIZE, UT_ID_SIZE, UT_LINE_SIZE, UT_HOST_SIZE): Define w.r.t. struct utmpx or struct utmp. (extract_trimmed_name): Don't use UT_USER or UT_USER_SIZE here. (desirable_utmp_entry): Don't use UT_TIME_MEMBER or UT_USER here. (struct utmp_alloc): Define always. (add_utmp): Likewise. Add user_len, id_len, line_len, host_len, termination, exit arguments. Don't require that user, id, line, host are NUL-terminated. Assume user and host are non-NULL. (finish_utmp): New function, extracted from read_utmp. (read_utmp) [READUTMP_USE_SYSTEMD]: Update add_utmp invocations. Pass a non-NULL user and a non-NULL host. Call finish_utmp. (getutent): Move declaration from readutmp.h to here. (copy_utmp_entry): Remove function. (read_utmp) [UTMP_NAME_FUNCTION]: Replace variables n_r
bug#64937: "who" reports funny dates
Robert Pluim wrote: > Thorsten> They don't record at all. > Thorsten> Which means tools like who just don't show anything. And emacs > will > Thorsten> never find out the boot time with the current code. > > What API do you suggest we use instead? musl libc runs only on Linux. On Linux, you can use this approach from Gnulib: https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=lib/readutmp.c;h=4e1d7ec26bef6cba4474433f72c9079b61794b97;hb=HEAD#l96 lines 96..150. Part of it may already be contained in emacs/src/filelock.c; I can't tell. Bruno
bug#64937: "who" reports funny dates
Paul Eggert wrote: > 0006-readutmp-switch-new-struct-to-struct-timespec.patch > @@ -150,12 +150,11 @@ struct gl_utmp > ⎣ ut_addr_v6 [u]int[4] glibc, musl, Android > */ > > -# include > # if !HAVE_DECL_GETUTENT > struct utmp *getutent (void); > # endif > # define UTMP_STRUCT_NAME utmp > -# define UT_TIME_MEMBER(UT) ((UT)->ut_time) > +# define UT_TIME_MEMBER(UT) ((UT)->ut_ts.tv_sec) > # define SET_UTMP_ENT setutent > # define GET_UTMP_ENT getutent > # define END_UTMP_ENT endutent This causes a compilation error on OpenBSD, AIX, and Android: On OpenBSD: ../../gllib/readutmp.c:76:20: error: no member named 'ut_ts' in 'struct utmp' bool user_proc = IS_USER_PROCESS (ut); ^~~~ ../../gllib/readutmp.h:297:36: note: expanded from macro 'IS_USER_PROCESS' || (UT_TYPE_NOT_DEFINED && UT_TIME_MEMBER (UT) != 0))) ^~~ ../../gllib/readutmp.h:158:36: note: expanded from macro 'UT_TIME_MEMBER' # define UT_TIME_MEMBER(UT) ((UT)->ut_ts.tv_sec) ^ On AIX: "../../gllib/readutmp.c", line 76.20: 1506-022 (S) "ut_ts" is not a member of "const struct utmp". make: 1254-004 The error code from the last command is 1. On Android: ../../gllib/readutmp.c:76:20: error: no member named 'ut_ts' in 'struct utmp'; did you mean 'ut_tv'? This patch fixes it. 2023-08-08 Bruno Haible readutmp: Fix compilation error on OpenBSD and AIX (regr. 2023-08-03). * lib/readutmp.h (UT_TIME_MEMBER) [HAVE_UTMP_H]: Revert last change. diff --git a/lib/readutmp.h b/lib/readutmp.h index 9f53246597..043ae6df16 100644 --- a/lib/readutmp.h +++ b/lib/readutmp.h @@ -155,7 +155,7 @@ struct gl_utmp struct utmp *getutent (void); # endif # define UTMP_STRUCT_NAME utmp -# define UT_TIME_MEMBER(UT) ((UT)->ut_ts.tv_sec) +# define UT_TIME_MEMBER(UT) ((UT)->ut_time) # define SET_UTMP_ENT setutent # define GET_UTMP_ENT getutent # define END_UTMP_ENT endutent
bug#64937: "who" reports funny dates
I wrote on 2023-08-02:
> I wrote:
> > The proposed patch is attached.
>
> Oops, I missed a sizeof of the ut_id field. A corrected patch is attached.
Oops, this causes a compilation error on OpenBSD:
In file included from ../../gllib/readutmp.c:22:
../../gllib/readutmp.h:216:50: error: no member named 'ut_id' in 'struct utmp'
enum { UT_ID_SIZE = sizeof (((STRUCT_UTMP *) 0)->ut_id) };
~~~ ^
This patch fixes it.
2023-08-08 Bruno Haible
readutmp: Fix compilation error on OpenBSD (regr. 2023-08-02).
* lib/readutmp.h (UT_ID_SIZE): Define to a dummy if there is no ut_id
field.
diff --git a/lib/readutmp.h b/lib/readutmp.h
index 01964d2622..9f53246597 100644
--- a/lib/readutmp.h
+++ b/lib/readutmp.h
@@ -213,7 +213,11 @@ enum { UT_USER_SIZE = sizeof UT_USER ((STRUCT_UTMP *) 0) };
#if HAVE_GL_UTMP
enum { UT_ID_SIZE = -1 };
#else
+# if (HAVE_UTMPX_H ? HAVE_STRUCT_UTMPX_UT_ID : HAVE_STRUCT_UTMP_UT_ID)
enum { UT_ID_SIZE = sizeof (((STRUCT_UTMP *) 0)->ut_id) };
+# else
+enum { UT_ID_SIZE = 1 };
+# endif
# define UT_ID_SIZE UT_ID_SIZE
#endif
bug#64937: "who" reports funny dates
Thorsten Kukuk wrote: > On musl libc systems like Alpine, > you don't have utmp nor wtmp. But on Alpine Linux, I don't see a systemd nor a logind daemon. How are logins meant to be recorded on this system? Bruno
bug#64937: "who" reports funny dates
Paul Eggert wrote:
> Fedora 38 runs
> systemd, for example, and it still maintains /var/log/wtmp. Likewise for
> Ubuntu 23.04.
Well, these are the permissions of these files:
/var/run/utmp /var/log/wtmp /var/log/btmpowner
Ubuntu 23.04 rw-rw-r-- rw-rw-r-- rw-rwroot:utmp
Debian 12 rw-rw-r-- rw-rw-r-- rw-rwroot:utmp
Fedora Rawhide rw-rw-r-- rw-rw-r-- rw-rwroot:utmp
context initrc_var_run_t wtmp_t faillog_t
openSUSE 15.5 rw-rw-r-- rw-rw-r-- rw-rwroot:utmp
Slackware 14 rw-rw-r-- rw-rw-r-- rw---root:utmp,
btmp only root:root
Alpine 3.18rw-rw-r-- rw-rw-r-- rw-rwroot:utmp
Debian Hurdrw-rw-r-- rw-rw-r-- rw-rwroot:utmp
Since the fact that /var/run/utmp and /var/log/wtmp are world-readable
implies that they are world-lockable and thus the DoS bug
https://sourceware.org/bugzilla/show_bug.cgi?id=24492 applies,
to me it's clear that both utmp and wtmp needs to go away rather
sooner than later. My guess is that Fedora and Ubuntu/Debian are only
waiting for 'who' (coreutils) and 'last' (util-linux / wtmpdb) to
stop accessing these two files.
> > Is there somebody really using btmp? Beside that it is really unreliable
> > since nearly no application is writing it, I asked on several mailing
> > lists and nobody answered.
I agree with Paul: When three books/blogs mention /var/log/btmp and the
ability to run "sudo who -a /var/log/btmp", and additionally a command
'lastb' exists, for "sudo lastb", we cannot ignore that.
> Although Ubuntu does not maintain /var/log/btmp
What do you mean by that? On Ubuntu 23.04, I just did a "ssh localhost"
with a wrong password, and then I see:
$ sudo who -a /var/log/btmp
LOGIN ssh:notty 2023-08-07 13:06 2564 id=
$ sudo lastb
brunossh:notty127.0.0.1 Mon Aug 7 13:06 - 13:06 (00:00)
Similarly when there were several failed logins.
("sudo who /var/log/btmp" prints nothing, because it filters out the LOGIN
lines. "who -a /var/log/btmp" prints nothing, because it cannot open the
file.)
So, IMO, the conclusion is:
* The /var/log/wtmp argument to "who" and "users" should become deprecated.
* Whereas for /var/log/btmp we need to make an effort to continue supporting
it, in the same 'who' program that accesses the systemd API for utmp.
And access the time stamp in it as an unsigned 32-bit integer, like
Andreas Schwab proposed (already implemented).
This means:
(1) The API of the readutmp module should provide unlimited-length ut_user,
ut_host etc. fields always. No more #ifdef UT_USER_SIZE.
(2) The readutmp module should use a runtime 'if' rather than a compile-time
#if, in order to dispatch between the systemd backend and the file-based
backend.
I'll work on (1) today.
Bruno
bug#64937: "who" reports funny dates
Paul Eggert wrote: > Thanks for doing all that work. I looked into it, and found a problem: a > command like "who /var/log/wtmp" stops working because the systemd > emulation of read_utmp only supports plain "who" (roughly equivalent to > "who /var/run/utmp" on Fedora). Probably the FILE argument to 'who' and to 'users' should be deprecated? Since in systemd mode, the only supported value for FILE is /var/run/utmp. > Although I toyed with the idea of simplifying readutmp.h > further, by moving most of it into readutmp.c and having just struct > gl_utmp public (this would simplify coreutils quite a bit), I ran out of > time and patience and decided to ship what I had. Yes, I see how much contortions (strncpy etc.) these not-NUL-terminated strings cause. And two separate code paths, one for the systemd mode and one for all other platforms, is not ideal from the code coverage/testing/QA perspective either. I didn't tackle this, because before the add_utmp function existed, it would have caused many extra malloc() calls on all platforms. Bruno
bug#64937: "who" reports funny dates
Paul Eggert wrote: > 0001-maint-Update-after-gnulib-module-readutmp-changed.patch > +/* Work around <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109614>, > + triggered by STREQ_LEN with a negative length. */ > +#if 11 <= __GNUC__ > +# pragma GCC diagnostic ignored "-Wstringop-overread" > +#endif The mentioned GCC bug is about a -Wanalyzer-use-after-free false positive. I guess you meant https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110884 or https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101778 ? Bruno
bug#64937: session leaders pids
Thorsten Kukuk wrote: > > * The pids now refer to the session leader, which is a parent (or ancestor) > > of the process which has allocated the pty. > > Depends on your application. For the majority of applications, which we > evaluated, the PIDs reported by logind were identical to the PID from utmp. Well, for me it's the opposite: For all desktops that I evaluated, the PIDs reported by sd_session_get_leader — which is the replacement that you recommend in <https://github.com/thkukuk/utmpx/blob/main/utmp-to-logind.md> (and I don't see any other one) — are different from the PID in utmp. Only for inbound ssh do I see the same PID. * Wayland-based desktops: - GNOME: systemd returns the PID of the gdm-session-worker process. utmp contains the PID of the gdm-wayland-session process (a child process of the former). - KDE: systemd returns the PID of the sddm-helper process. utmp contains the PID of the startplasma-wayland process (a child process of the former). * X11-based desktops: - Budgie: systemd returns the PID of a lightdm process. utmp contains the PID of the gnome-session-binary process (a child process of the former). - Cinnamon: systemd returns the PID of a lightdm process. utmp contains the PID of the cinnamon-session process (a child process of the former). - MATE: systemd returns the PID of a lightdm process. utmp contains the PID of the mate-session process (a child process of the former). - Xfce: systemd returns the PID of a lightdm process. utmp contains the PID of the xfce4-session process (a child process of the former). - SoaS: systemd returns the PID of a lightdm process. utmp contains the PID of a python3 process (a child process of the former). - LXQt: systemd returns the PID of the sddm-helper process. utmp contains the PID of the lxqt-session process (a child process of the former). - Sway: systemd returns the PID of the sddm-helper process. utmp contains the PID of the sway process (a child process of the former). - LXDE: systemd returns the PID of the lxdm-session process. utmp contains no PID at all! (Oh oh, looks like a bug in lxsession.) Find attached the experiments' outputs. Bruno outputs.tar.gz Description: application/compressed-tar
bug#64937: ssh sessions in systemd
Thorsten Kukuk wrote: > And systemd/logind has a hack to delete this dummy entry, so that a > fallback hack becomes active, which tries to determine the tty in > another way. But this "hack" exists only for the dbus interface, not for > libsystemd... I see some hack in systemd/src/basic/terminal-util.c, indeed... Bruno
bug#64937: ssh sessions in systemd
Thorsten Kukuk wrote: > > The only problem with this mapping is for the ut_line row, which > > used to contain, for inbound ssh, the pty device name. It is not easy > > to find the pty name in this situation; at least, none of the systemd > > APIs and /run/systemd/** files that I looked at helped. > > You mean you don't see a TTY on ssh sessions? Yes, I see a session uid=1001 user=other state=active active=Y remote=Y \ starttime=2023-08-01T15:19:25 seat=(null) leaderpid=40513 uiclass=user \ pamservice=sshd vt=-1 type=tty tty=(null) \ display=(null) desktop=(null) remotehost=::1 remoteuser=(null) "uitype=tty" and "tty=(null)" don't make much sense together. > openssh is really special: it does not need a TTY for all kind of ssh > sessions, and thus only opens a TTY if needed after creating the > logind session. Thus the logind session does not contain the TTY > informations. > systemd-logind v254 provides now an interface for this case, which > allows to set the TTY later. For openssh you need this patch: > > https://github.com/thkukuk/utmpx/blob/main/patches/openssh/logind-set-tty.patch That would make sense, yes. But I wonder: - Why is it possible to set the "type" of the session to "tty", without specifying a value for the "tty"? - While at it: Shouldn't OpenSSH also provide a value for the "remote_user" property? Bruno
bug#64937: "who" reports funny dates
I wrote:
> The proposed patch is attached.
Oops, I missed a sizeof of the ut_id field. A corrected patch is attached.
>From 97be578b107e7b87e32e0c3c2d49dc550489415b Mon Sep 17 00:00:00 2001
From: Bruno Haible
Date: Wed, 2 Aug 2023 01:32:55 +0200
Subject: [PATCH] maint: Update after gnulib module 'readutmp' changed
For year-2038 safety on Linux/{x86,arm}, use systemd APIs.
* configure.ac: Don't test whether 'struct utmp' and 'struct utmpx' have
the ut_host field; this is now done in gnulib's readutmp module.
* src/system.h (STREQ_LEN): Allow passing a third argument with value
-1.
* src/pinky.c: Test HAVE_STRUCT_XTMP_UT_HOST instead of HAVE_UT_HOST.
(print_entry): Support the situation where ut_line is a 'char *' rather
than a 'char[]' of fixed size. Likewise for ut_user and ut_host.
* src/who.c: Test HAVE_STRUCT_XTMP_UT_HOST instead of HAVE_UT_HOST.
(print_user): Support the situation where ut_line is a 'char *' rather
than a 'char[]' of fixed size. Likewise for ut_user and ut_host.
(print_deadprocs, print_login, print_initspawn, scan_entries): Likewise.
(make_id_equals_comment): Likewise for ut_id.
(who): Free resources before returning.
* src/users.c (users): Free resources before returning.
* src/local.mk: Link the programs 'pinky', 'uptime', 'users', 'who' with
$(READUTMP_LIB).
---
configure.ac | 30 --
src/local.mk | 6 ++
src/pinky.c | 41 ++--
src/system.h | 6 +-
src/users.c | 1 +
src/who.c| 59
6 files changed, 83 insertions(+), 60 deletions(-)
diff --git a/configure.ac b/configure.ac
index 33441a82f..afc1098f7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -406,36 +406,6 @@ AC_DEFUN([coreutils_DUMMY_1],
])
coreutils_DUMMY_1
-AC_MSG_CHECKING([ut_host in struct utmp])
-AC_CACHE_VAL([su_cv_func_ut_host_in_utmp],
-[AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include
- #include
- struct utmp ut;
- int s = sizeof ut.ut_host;]])],
- [su_cv_func_ut_host_in_utmp=yes],
- [su_cv_func_ut_host_in_utmp=no])])
-AC_MSG_RESULT([$su_cv_func_ut_host_in_utmp])
-if test $su_cv_func_ut_host_in_utmp = yes; then
- have_ut_host=1
- AC_DEFINE([HAVE_UT_HOST], [1], [FIXME])
-fi
-
-if test -z "$have_ut_host"; then
- AC_MSG_CHECKING([ut_host in struct utmpx])
- AC_CACHE_VAL([su_cv_func_ut_host_in_utmpx],
- [AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include
- #include
- struct utmpx ut;
- int s = sizeof ut.ut_host;]])],
-[su_cv_func_ut_host_in_utmpx=yes],
-[su_cv_func_ut_host_in_utmpx=no])])
- AC_MSG_RESULT([$su_cv_func_ut_host_in_utmpx])
- if test $su_cv_func_ut_host_in_utmpx = yes; then
-AC_DEFINE([HAVE_UTMPX_H], [1], [FIXME])
-AC_DEFINE([HAVE_UT_HOST], [1], [FIXME])
- fi
-fi
-
GNULIB_BOOT_TIME([gl_ADD_PROG([optional_bin_progs], [uptime])])
AC_SYS_POSIX_TERMIOS()
diff --git a/src/local.mk b/src/local.mk
index cb9b39274..4d7df2789 100644
--- a/src/local.mk
+++ b/src/local.mk
@@ -317,6 +317,12 @@ src_who_LDADD += $(GETADDRINFO_LIB)
src_hostname_LDADD += $(GETHOSTNAME_LIB)
src_uname_LDADD += $(GETHOSTNAME_LIB)
+# for read_utmp
+src_pinky_LDADD += $(READUTMP_LIB)
+src_uptime_LDADD += $(READUTMP_LIB)
+src_users_LDADD += $(READUTMP_LIB)
+src_who_LDADD += $(READUTMP_LIB)
+
# for strsignal
src_kill_LDADD += $(LIBTHREAD)
diff --git a/src/pinky.c b/src/pinky.c
index 381e753b6..47abd7758 100644
--- a/src/pinky.c
+++ b/src/pinky.c
@@ -62,7 +62,7 @@ static bool include_home_and_shell = true;
static bool do_short_format = true;
/* if true, display the ut_host field. */
-#ifdef HAVE_UT_HOST
+#if HAVE_STRUCT_XTMP_UT_HOST
static bool include_where = true;
#endif
@@ -206,7 +206,8 @@ print_entry (const STRUCT_UTMP *utmp_ent)
#define DEV_DIR_WITH_TRAILING_SLASH "/dev/"
#define DEV_DIR_LEN (sizeof (DEV_DIR_WITH_TRAILING_SLASH) - 1)
- char line[sizeof (utmp_ent->ut_line) + DEV_DIR_LEN + 1];
+#ifdef UT_LINE_SIZE
+ char line[DEV_DIR_LEN + UT_LINE_SIZE + 1];
char *p = line;
/* Copy ut_line into LINE, prepending '/dev/' if ut_line is not
@@ -214,7 +215,18 @@ print_entry (const STRUCT_UTMP *utmp_ent)
absolute file name in ut_line. */
if ( ! IS_ABSOLUTE_FILE_NAME (utmp_ent->ut_line))
p = stpcpy (p, DEV_DIR_WITH_TRAILING_SLASH);
- stzncpy (p, utmp_ent->ut_line, sizeof (utmp_ent->ut_line));
+ stzncpy (p, utmp_ent->ut_line, UT_LINE_SIZE);
+#else
+ /* If ut_line contains a space, the device name starts after the space,
+ else at the beginning. */
+ char *line = xmalloc (DEV_DIR_LEN + strlen (utmp_ent->ut_line) + 1);
+ char *space = strchr (utmp_ent->ut_line, ' ');
+ char *device = (space != NULL ? space + 1 : utmp_ent->ut_line);
+ if ( ! IS_ABSOLUTE_FILE_NAME (device))
+stpcpy (
bug#65003: pinky's "Idle" column lost the unit
Hi Paul, Two days ago, 'pinky's output looked like this: Login TTY Idle When bruno?seat0? 2023-07-30 11:25 bruno tty2 2d 2023-07-30 11:25 ... Today, it looks like this: Login TTY Idle When bruno?seat0? 2023-07-30 11:25 bruno tty2 2 2023-07-30 11:25 ... In the Idle column, the "d" (which stands for days) is gone. This is a regression. Looks like it was caused by commit 0b2796750ff68eb58616655236d7003f6d30223a , today. Bruno
bug#64937: "who" reports funny dates
> Here are the changes I committed in gnulib.
And here are the proposed changes for coreutils. Tested on a Fedora Rawhide
system, prepared from
https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-Rawhide-20230729.n.0.iso
The user-visible changes introduced by this change (when linking with
libsystemd) are, as far as I can see:
* "uptime", "users", "who -q" reports fewer users (because ptys without login
are not counted).
* "who -d" does not report anything any more.
* "who -r" does not report anything any more.
* "who -t" does not report anything any more.
* "who -u" does not report ptys without login any more. The order is arbitrary.
Example:
Without systemd:
brunoseat02023-07-30 11:25 ? 1663 (login screen)
brunotty2 2023-07-30 11:25 old 1663 (tty2)
brunopts/32023-08-01 01:36 22:33 30619 (:0)
otherpts/42023-08-01 17:19 06:50 40513 (::1)
With systemd:
othersshd pts/4 2023-08-01 17:19 06:50 40513 (::1)
brunoseat02023-07-30 11:25 ? 1593
brunotty2 2023-07-30 11:25 old 1593
* pinky $USER
does not report a host any more (and thus does not spend time trying
to do a DNS lookup of "login screen" and "tty2"). Example:
Without systemd:
LoginName TTY Idle When Where
brunoBruno Haible?seat0? 2023-07-30 11:25 login screen
brunoBruno Haible tty2 2d 2023-07-30 11:25 tty2
brunoBruno Haible pts/322:39 2023-08-01 01:36 :0
With systemd:
LoginName TTY Idle When Where
brunoBruno Haible?seat0? 2023-07-30 11:25
brunoBruno Haible tty2 2d 2023-07-30 11:25
The proposed patch is attached.
Note: Instead of the idiom
#ifdef UT_HOST_SIZE
(code for bounded ut_host)
#else
(code for unbounded ut_host)
#endif
one could also write
if (UT_HOST_SIZE >= 0)
{
(code for bounded ut_host)
}
else
{
(code for unbounded ut_host)
}
It's just a matter of style whether one prefers #ifs or implicit dead code.
>From d80eea8fb087e9504a6dad27e1a880227f67915a Mon Sep 17 00:00:00 2001
From: Bruno Haible
Date: Wed, 2 Aug 2023 01:32:55 +0200
Subject: [PATCH] maint: Update after gnulib module 'readutmp' changed
For year-2038 safety on Linux/{x86,arm}, use systemd APIs.
* configure.ac: Don't test whether 'struct utmp' and 'struct utmpx' have
the ut_host field; this is now done in gnulib's readutmp module.
* src/system.h (STREQ_LEN): Allow passing a third argument with value
-1.
* src/pinky.c: Test HAVE_STRUCT_XTMP_UT_HOST instead of HAVE_UT_HOST.
(print_entry): Support the situation where ut_line is a 'char *' rather
than a 'char[]' of fixed size. Likewise for ut_user and ut_host.
* src/who.c: Test HAVE_STRUCT_XTMP_UT_HOST instead of HAVE_UT_HOST.
(print_user): Support the situation where ut_line is a 'char *' rather
than a 'char[]' of fixed size. Likewise for ut_user and ut_host.
(print_deadprocs, print_login, print_initspawn, scan_entries): Likewise.
(who): Free resources before returning.
* src/users.c (users): Free resources before returning.
* src/local.mk: Link the programs 'pinky', 'uptime', 'users', 'who' with
$(READUTMP_LIB).
---
configure.ac | 30 --
src/local.mk | 6 ++
src/pinky.c | 41 ++---
src/system.h | 6 +-
src/users.c | 1 +
src/who.c| 44 ++--
6 files changed, 72 insertions(+), 56 deletions(-)
diff --git a/configure.ac b/configure.ac
index 33441a82f..afc1098f7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -406,36 +406,6 @@ AC_DEFUN([coreutils_DUMMY_1],
])
coreutils_DUMMY_1
-AC_MSG_CHECKING([ut_host in struct utmp])
-AC_CACHE_VAL([su_cv_func_ut_host_in_utmp],
-[AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include
- #include
- struct utmp ut;
- int s = sizeof ut.ut_host;]])],
- [su_cv_func_ut_host_in_utmp=yes],
- [su_cv_func_ut_host_in_utmp=no])])
-AC_MSG_RESULT([$su_cv_func_ut_host_in_utmp])
-if test $su_cv_func_ut_host_in_utmp = yes; then
- have_ut_host=1
- AC_DEFINE([HAVE_UT_HOST], [1], [FIXME])
-fi
-
-if test -z "$have_ut_host"; then
- AC_MSG_CHECKING([ut_host in struct utmpx])
- AC_CACHE_VAL([su_cv_func_ut_host_in_utmpx],
- [AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include
- #include
- struct utmpx ut;
- int s = sizeof ut.ut_host;]])],
-[su_cv_func_ut_host_in_utmpx=yes],
-[su_cv_func_ut_host_in_utmpx=no])])
- AC_MSG_RESULT([$su_cv_func_ut_h
bug#64937: "who" reports funny dates
Thorsten Kukuk wrote:
> If you haven't seen yet, I made some time ago a mapping
> between utmp struct entries and libsystemd functions:
> https://github.com/thkukuk/utmpx/blob/main/utmp-to-logind.md
Thanks. This is helpful.
The only problem with this mapping is for the ut_line row, which
used to contain, for inbound ssh, the pty device name. It is not easy
to find the pty name in this situation; at least, none of the systemd
APIs and /run/systemd/** files that I looked at helped.
> But be aware:
> 1. you need to extend ut_tv from 32bit time_t to 64bit time_t, or your
> patch will not survive 2038.
This is already done by the 'year2038' module of Gnulib, which
coreutils and many other packages use.
> 2. the string entries are size limited in struct utmp, but are not with
> libsystemd. So could be that you need to truncate device names, user
> names and host names.
Thanks for the hint. Truncating would mean to introduce arbitrary limits,
which is against the GNU Coding Standards
<https://www.gnu.org/prep/standards/html_node/Semantics.html>. Hence
we'll happily use arbitrary-length strings in our code.
Paul Eggert wrote:
> This suggests that Gnulib needs to redo the readutmp API so that it's
> not tied to struct utmpx's fixed string lengths. That's a bit of a
> hassle but should be doable (change char array to char *). An advantage
> is that Gnulib is a source-code library and so we can make binary- (and
> even source-) incompatible changes to the API without too much trouble.
For the moment, I've extended the 'readutmp' module to use arbitrary-
length strings only when compiling with libsystemd.
Here are the changes I committed in gnulib.
2023-08-01 Bruno Haible
readutmp: Small changes to reduce the code size on the coreutils side.
* m4/readutmp.m4 (gl_READUTMP): Test also for the ut_host field in
'struct utmpx' and 'struct utmp'.
* lib/readutmp.h (HAVE_STRUCT_XTMP_UT_HOST): New macro.
(UT_USER_SIZE): Define also as a macro. Set to -1 if
READUTMP_USE_SYSTEMD.
(UT_LINE_SIZE, UT_HOST_SIZE): New constants and macros.
2023-08-01 Bruno Haible
readutmp: For year-2038 safety on Linux/{x86,arm}, use systemd APIs.
Suggested by Thorsten Kukuk in
<https://www.thkukuk.de/blog/Y2038_glibc_utmp_64bit/> and
<https://github.com/thkukuk/utmpx/blob/main/utmp-to-logind.md>.
* m4/systemd.m4: New file.
* m4/readutmp.m4 (gl_READUTMP): Require gl_SYSTEMD_CHOICE. Set
READUTMP_LIB. Conditionally define READUTMP_USE_SYSTEMD.
* lib/readutmp.h: For READUTMP_USE_SYSTEMD, include and
.
(struct gl_utmp): New type.
(UTMP_STRUCT_NAME, UT_TIME_MEMBER, UT_EXIT_E_TERMINATION,
UT_EXIT_E_EXIT, UT_USER, HAVE_STRUCT_XTMP_UT_EXIT,
HAVE_STRUCT_XTMP_UT_ID, HAVE_STRUCT_XTMP_UT_PID): Define differently for
READUTMP_USE_SYSTEMD.
(UT_USER_SIZE): Don't define for READUTMP_USE_SYSTEMD.
(UT_TYPE_EQ, UT_TYPE_NOT_DEFINED, READ_UTMP_SUPPORTED): Define also for
READUTMP_USE_SYSTEMD.
(free_utmp): New declaration.
* lib/readutmp.c: Add new includes for READUTMP_USE_SYSTEMD.
(extract_trimmed_name): Adapt to READUTMP_USE_SYSTEMD.
(get_boot_time_uncached, get_boot_time, guess_pty_name): New functions.
(read_utmp): New implementation for READUTMP_USE_SYSTEMD.
(free_utmp): New function.
* tests/test-readutmp.c (main): At the end, invoke free_utmp.
* modules/readutmp (Files): Add m4/systemd.m4.
(Link): New section.
* modules/readutmp-tests (Makefile.am): Link test-readutmp with
READUTMP_LIB.
* NEWS: Mention the free_utmp function and the READUTMP_LIB link
requirement.
When the output of the 'test-readutmp' program without libsystemd is
==
Here are the read_utmp results.
Flags: B = Boot, U = User Process
Termi‐ Flags
Time (GMT) User DevicePIDnation Exit B U
--- -- --- -- -- - -
2023-07-30 09:25:15 reboot ~00 0 X
2023-07-30 09:25:31 runlevel ~ 530 0
2023-07-30 09:25:48 bruno seat0 16630 0 X
2023-07-30 09:25:48 bruno tty2 16630 0 X
2023-07-30 09:31:17pts/1 32100 0
2023-07-31 23:36:52 bruno pts/3306190 0 X
2023-08-01 15:19:26 other pts/4405130 0 X
2023-08-01 19:02:20pts/5433760 1
===
bug#64954: GNU 'uptime' on OpenBSD always prints "0 users"
I wrote in <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=64954>:
> - Gnulib modules should better provide .h files that can be #included
> on any platform. Thus, it's Gnulib's task to provide a readutmp.h
> and a read_utmp() function that can also be used on native Windows.
Done as follows:
2023-07-31 Bruno Haible
readutmp: Make the header file and function usable on all platforms.
* lib/readutmp.h (struct gl_utmp, UTMP_STRUCT_NAME, UT_TIME_MEMBER,
UT_EXIT_E_TERMINATION, UT_EXIT_E_EXIT, UT_USER): Provide fallback
definitions.
(READ_UTMP_SUPPORTED): New macro.
* lib/readutmp.c (read_utmp) [!READ_UTMP_SUPPORTED]: Provide a dummy
definition.
* modules/readutmp (Depends-on): Add sys_time.
(configure.ac): Remove conditional.
(Makefile.am): Compile readutmp.c on all platforms.
(Include): Include readutmp.h on all platforms.
* tests/test-readutmp.c: Include readutmp.h on all platforms.
(main): Invoke read_utmp on all platforms.
diff --git a/lib/readutmp.c b/lib/readutmp.c
index acffe1000e..af43d1ad6b 100644
--- a/lib/readutmp.c
+++ b/lib/readutmp.c
@@ -61,6 +61,8 @@ extract_trimmed_name (const STRUCT_UTMP *ut)
return trimmed_name;
}
+#if READ_UTMP_SUPPORTED
+
/* Is the utmp entry U desired by the user who asked for OPTIONS? */
static bool
@@ -77,12 +79,12 @@ desirable_utmp_entry (STRUCT_UTMP const *u, int options)
return true;
}
-#ifdef UTMP_NAME_FUNCTION
+# if defined UTMP_NAME_FUNCTION
static void
copy_utmp_entry (STRUCT_UTMP *dst, STRUCT_UTMP *src)
{
-#if __GLIBC__ && _TIME_BITS == 64
+# if __GLIBC__ && _TIME_BITS == 64
/* Convert from external form in SRC to internal form in DST.
It is OK to convert now, rather than earlier, before
desirable_utmp_entry was invoked, because desirable_utmp_entry
@@ -119,9 +121,9 @@ copy_utmp_entry (STRUCT_UTMP *dst, STRUCT_UTMP *src)
dst->ut_tv.tv_sec = s->ut_tv.tv_sec;
dst->ut_tv.tv_usec = s->ut_tv.tv_usec;
memcpy (>ut_addr_v6, s->ut_addr_v6, sizeof dst->ut_addr_v6);
-#else
+# else
*dst = *src;
-#endif
+# endif
}
int
@@ -158,7 +160,7 @@ read_utmp (char const *file, size_t *n_entries, STRUCT_UTMP
**utmp_buf,
return 0;
}
-#else
+# else
int
read_utmp (char const *file, size_t *n_entries, STRUCT_UTMP **utmp_buf,
@@ -198,4 +200,16 @@ read_utmp (char const *file, size_t *n_entries,
STRUCT_UTMP **utmp_buf,
return 0;
}
+# endif
+
+#else /* dummy fallback */
+
+int
+read_utmp (char const *file, size_t *n_entries, STRUCT_UTMP **utmp_buf,
+ int options)
+{
+ errno = ENOSYS;
+ return -1;
+}
+
#endif
diff --git a/lib/readutmp.h b/lib/readutmp.h
index 7d2a628135..d710699525 100644
--- a/lib/readutmp.h
+++ b/lib/readutmp.h
@@ -38,6 +38,7 @@
# endif
# if HAVE_UTMPX_H
+
# if HAVE_UTMP_H
/* HPUX 10.20 needs utmp.h, for the definition of e.g., UTMP_FILE. */
# include
@@ -114,6 +115,24 @@
# endif
# endif
+# else
+
+/* Provide a dummy fallback. */
+
+/* Get 'struct timeval'. */
+# include
+
+struct gl_utmp
+{
+ char ut_user[1];
+ char ut_line[1];
+ struct timeval ut_tv;
+};
+# define UTMP_STRUCT_NAME gl_utmp
+# define UT_TIME_MEMBER(UT_PTR) ((UT_PTR)->ut_tv.tv_sec)
+# define UT_EXIT_E_TERMINATION(U) 0
+# define UT_EXIT_E_EXIT(U) 0
+
# endif
/* Accessor macro for the member named ut_user or ut_name. */
@@ -137,6 +156,10 @@
# define UT_USER(Utmp) ((Utmp)->ut_name)
# endif
+# else /* dummy fallback */
+
+# define UT_USER(Utmp) ((Utmp)->ut_user)
+
# endif
# define HAVE_STRUCT_XTMP_UT_EXIT \
@@ -151,10 +174,14 @@
(HAVE_STRUCT_UTMP_UT_PID \
|| HAVE_STRUCT_UTMPX_UT_PID)
+/* Type of entry returned by read_utmp(). */
typedef struct UTMP_STRUCT_NAME STRUCT_UTMP;
+/* Size of the UT_USER (ut) member. */
enum { UT_USER_SIZE = sizeof UT_USER ((STRUCT_UTMP *) 0) };
+/* Definition of UTMP_FILE and WTMP_FILE. */
+
# if !defined UTMP_FILE && defined _PATH_UTMP
# define UTMP_FILE _PATH_UTMP
# endif
@@ -181,12 +208,15 @@ enum { UT_USER_SIZE = sizeof UT_USER ((STRUCT_UTMP *) 0)
};
# define WTMP_FILE "/etc/wtmp"
# endif
+/* Accessor macro for the member named ut_pid. */
# if HAVE_STRUCT_XTMP_UT_PID
# define UT_PID(U) ((U)->ut_pid)
# else
# define UT_PID(U) 0
# endif
+/* Accessor macros for the member named ut_type. */
+
# if HAVE_STRUCT_UTMP_UT_TYPE || HAVE_STRUCT_UTMPX_UT_TYPE
# define UT_TYPE_EQ(U, V) ((U)->ut_type == (V))
# define UT_TYPE_NOT_DEFINED 0
@@ -207,11 +237,17 @@ enum { UT_USER_SIZE = sizeof UT_USER ((STRUCT_UTMP *) 0)
};
# define UT_TYPE_USER_PROCESS(U) 0
# endif
+/* Determines whether an entry *U corresponds to a user process. */
# define IS_USER_PROCESS(U) \
(UT_USER (U)[0] \
&& (UT_TYPE_USER_PROCESS (U)
bug#64954: GNU 'uptime' on OpenBSD always prints "0 users"
Paul Eggert wrote: > > I'm fine with the change, but we'll also need to adjust > > the sc_prohibit_always_true_header_tests syntax check in gnulib > > I looked into that but it's such a hassle that I came up with the > attached simpler patch to Coreutils. How about installing it instead? Yes, that's better than what I proposed, on two accounts: - Gnulib modules should better provide .h files that can be #included on any platform. Thus, it's Gnulib's task to provide a readutmp.h and a read_utmp() function that can also be used on native Windows. - It gets rid of the horrible hack to pass the values of two uninitialized variables down to a function. Pádraig Brady wrote: > gnulib docs > state the following platforms have neither getutent or getutxent, > and so might have issues with this? OpenBSD 6.7, Minix 3.1.8, mingw, MSVC 14 On these platforms the #include "readutmp.h" would already provoke a mass of syntax errors. Btw, we don't test on Minix any longer (since Minix is dead), and OpenBSD 6.x is end-of-life already [1]. Bruno https://en.wikipedia.org/wiki/OpenBSD#Releases
bug#64937: "who" reports funny dates
Paul Eggert wrote: > If I understand that discussion correctly, the idea is to switch from > utmp/utmpx to the systemd interface once systemd 254 comes out. > > As it happens, systemd 254 was published Friday. It's already contained in Fedora Rawhide. > It'd be good to get it working with coreutils. I've installed that Fedora Rawhide from yesterday and am working on a prototype... > I'm hoping we can simplify this by packaging the fix inside > Gnulib Yes. Since not only coreutils, but also inetutils, uses the gnulib module 'readutmp', it makes sense to put the fix into this gnulib module. Specific users, like 'uptime' in coreutils, can optimize things afterwards when they need only a small subset, rather than all, of the data returned by read_utmp(). Bruno
bug#64233: patch: Ensure that makeinfo ≥ 6.8 checks the @menu structure
Pádraig Brady wrote: > coreutils has a dep on makeinfo >= 6.1 > and that version supports the -c option, > so the change should work everywhere. Yes. makeinfo versions < 6.8 give a warning that they don't know about the customization variable CHECK_NORMAL_MENU_STRUCTURE. You can simply ignore this warning. makeinfo version 6.8 is already pretty widely deployed, since it was released two years ago. Bruno
bug#64233: patch: Ensure that makeinfo ≥ 6.8 checks the @menu structure
Makeinfo versions < 6.7 detected mistakes in the @menu structure of TeXinfo input. Makeinfo versions ≥ 6.8 don't do this any more by default. They need an extra option, for this validation to happen. See <https://lists.gnu.org/archive/html/bug-texinfo/2023-06/msg00015.html>. Since doc/coreutils.texi has a hand-written @menu, it should use this extra option. Here's a patch to that effect. I checked it by running touch doc/coreutils.texi; make doc/coreutils.info V=1 and make sc_makefile_at_at_check From d3f0c8f482d8be6cb3e117331f0dca7e4fa72503 Mon Sep 17 00:00:00 2001 From: Bruno Haible Date: Thu, 22 Jun 2023 23:59:11 +0200 Subject: [PATCH] =?UTF-8?q?build:=20Ensure=20that=20makeinfo=20=E2=89=A5?= =?UTF-8?q?=206.8=20checks=20the=20@menu=20structure?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit See <https://lists.gnu.org/r/bug-texinfo/2023-06/msg00015.html>. * doc/local.mk (MAKEINFO): New variable. * cfg.mk (_makefile_at_at_check_exceptions): Update. --- cfg.mk | 3 ++- doc/local.mk | 4 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/cfg.mk b/cfg.mk index 92f119ee2..6ba8981ab 100644 --- a/cfg.mk +++ b/cfg.mk @@ -52,7 +52,8 @@ export XZ_OPT = -8e old_NEWS_hash = ac6cb1c35890b53e56acbfbb078fcd53 # Add an exemption for sc_makefile_at_at_check. -_makefile_at_at_check_exceptions = ' && !/^cu_install_prog/ && !/dynamic-dep/' +_makefile_at_at_check_exceptions = \ + ' && !/MAKEINFO/ && !/^cu_install_prog/ && !/dynamic-dep/' # Our help-version script is in a slightly different location. _hv_file ?= $(srcdir)/tests/misc/help-version diff --git a/doc/local.mk b/doc/local.mk index 9714d5b18..b5a6c33b6 100644 --- a/doc/local.mk +++ b/doc/local.mk @@ -25,6 +25,10 @@ doc_coreutils_TEXINFOS = \ doc/fdl.texi \ doc/sort-version.texi +# The customization variable CHECK_NORMAL_MENU_STRUCTURE is necessary with +# makeinfo versions ≥ 6.8. +MAKEINFO = @MAKEINFO@ -c CHECK_NORMAL_MENU_STRUCTURE=1 + # The following is necessary if the package name is 8 characters or longer. # If the info documentation would be split into 10 or more separate files, # then this is necessary even if the package name is 7 characters long. -- 2.34.1
bug#38627: uniq -c gets wrong count with non-ascii strings
> However, the 2016 edition of POSIX removed mention of LC_COLLATE from 'uniq' Indeed. The change was done in <http://austingroupbugs.net/view.php?id=963>. Quote: "On Page: 3309 Line: 111067 Section: uniq In the ENVIRONMENT VARIABLES section, delete: LC_COLLATE Determine the locale for ordering rules." Bruno
bug#34817: coreutils 8.31 compile failure on rhel5
Hi Paul, > Making the variable 'static' might cause some compilers to notice that > the variable never changes, so it's a constant, so it can be folded, and > then they (mistakenly) convert -0.0 to 0.0. Originally the variable was > extern and the extern hack prevents most compilers from doing this > incorrect optimization (unless they're doing whole-program optimization, > which typically they aren't). Since making the variable static removed > the hack, I decided it was clearer to use the larger change, which makes > it more obvious that we aren't trying to work around this potential > problem any more. Yes, I perfectly agree. When I said "a simpler fix", I did not mean "a better fix". Your fix is the more reliable one! Bruno
bug#34524: wc: word count incorrect when words separated only by no-break space
Hi Pádraig, > >> In regard to options for enabling various behaviors for wc(1), > >> I'm thinking we might keep the strict POSIX isspace() behavior > >> with LC_CTYPE=C and/or POSIXLY_CORRECT=1, and use iswnbspace() > >> by default Since you plan to add a --words=... option in the future (as suggested by Paul or me), it would make sense to add this option now, instead of testing POSIXLY_CORRECT. If you introduce POSIXLY_CORRECT dependent behaviour now (and need to keep it for backward-compatibility), you'll have a hard to understand interface: What will the following do? env POSIXLY_CORRECT=1 wc --words=unicode wc --words=unicode Bruno
bug#34524: wc: word count incorrect when words separated only by no-break space
andard
On one hand, the Unicode standard makes it clear in several places that
1) NO-BREAK SPACE prohibits line breaking,
2) line breaking and words are related.
See for example, the Unicode standard section 5.12
<http://www.unicode.org/versions/Unicode11.0.0/ch05.pdf>
page 219:
"Line breaking algorithms generally use state machines for determining
word breaks."
Or the Unicode standard section 23.2
<http://www.unicode.org/versions/Unicode11.0.0/ch23.pdf> page 859
"Word Joiner. U+2060 word joiner behaves like U+00A0 no-break space
in that it indicates the absence of line breaks; ..."
On the other hand, in the same section 23.2 it says
"Line breaking and word breaking are distinct text processes.
Although a candidate position for a line break in text often coincides
with a candidate position for a word break, there are also many
situations where candidate break positions of different types do not
coincide."
And in the Unicode TR 29 section 4 "Word boundaries"
<https://www.unicode.org/reports/tr29/tr29-33.html#Word_Boundaries>
it treats NO-BREAK SPACE as a word boundary by default - this can be
verified through the program below - but also says that SPACE and
NO-BREAK SPACE "may be tailored to be in MidNum, depending on the environment".
Here's an example program, that uses GNU libunistring:
==
#include
#include
int main ()
{
printf ("%d\n", uc_wordbreak_property (0x00A0));
{
uint8_t string[] = "Regarde : le voilà";
char p[19];
u8_wordbreaks (string, 19, p);
puts ((char *) string);
for (int i = 0; i < 19; i++)
if (p[i])
printf ("word break at position %d\n", i);
}
{
uint8_t string[] = "Regarde\u00A0: le voilà";
char p[20];
u8_wordbreaks (string, 20, p);
puts ((char *) string);
for (int i = 0; i < 20; i++)
if (p[i])
printf ("word break at position %d\n", i);
}
}
==
and its output:
0(means: WBP_OTHER)
Regarde : le voilà
word break at position 7
word break at position 8
word break at position 9
word break at position 10
word break at position 12
word break at position 13
Regarde : le voilà
word break at position 7
word break at position 9
word break at position 10
word break at position 11
word break at position 13
word break at position 14
V) Implementation issues
> The following change would do that:
>
> diff --git a/src/wc.c b/src/wc.c
> index 179abbe..ca990b4 100644
> --- a/src/wc.c
> +++ b/src/wc.c
> @@ -147,6 +147,13 @@ the following order: newline, word, character, byte,
> maximum line length.\n\
>exit (status);
> }
>
> +static int _GL_ATTRIBUTE_PURE
> +iswnbspace (wint_t wc)
> +{
> + return wc == L'\u00A0' || wc == L'\u2007' \
> + || wc == L'\u202F' || wc == L'\u2060';
> +}
> +
> /* FILE is the name of the file (or NULL for standard input)
> associated with the specified counters. */
> static void
> @@ -455,7 +462,7 @@ wc (int fd, char const *file_x, struct fstatus *fstatus,
> off_t current_pos)
>if (width > 0)
> linepos += width;
> }
> - if (iswspace (wide_char))
> + if (iswspace (wide_char) || iswnbspace (wide_char))
> goto mb_word_separator;
>in_word = true;
> }
>
> ...
> For more sophisticated contextual processing we would need
> to use some of the word break functionality from libunistring.
I don't think you will be able to satisfactorily blend POSIX behaviour
with Unicode behaviour without introducing a command-line option.
On the POSIX side: POSIX says
<https://pubs.opengroup.org/onlinepubs/9699919799/utilities/wc.html>
"The wc utility shall consider a word to be a non-zero-length string
of characters delimited by white space."
and
<https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html>
"space
Define characters to be classified as white-space characters."
So, when operates according to POSIX expectations, it MUST use
iswspace (wide_char)
not
iswspace (wide_char) || iswnbspace (wide_char)
On the Unicode side: It is reasonable to see two "words" in
"Regardez\u00A0:", and the GNU libunistring library implement it like
this. It is also reasonable to expect that 'wc' counts words in the Thai
language, which does not use spaces to delimit words. GNU libunistring
may implement this in the future as well.
For this reason, I would find it best to introduce an option '--unicode'
to 'wc', that would produce Unicode compliant results, at the cost of
- not following POSIX to the letter,
- being slower.
Bruno
bug#34239: build failure on Android, due to S_MAGIC_* symbols
' undeclared (first use in this function) ../src/fs-is-local.h:117: error: 'S_MAGIC_UFS_BYTESWAPPED' undeclared (first use in this function) ../src/fs-is-local.h:118: error: 'S_MAGIC_USBDEVFS' undeclared (first use in this function) ../src/fs-is-local.h:119: error: 'S_MAGIC_V9FS' undeclared (first use in this function) ../src/fs-is-local.h:120: error: 'S_MAGIC_VMHGFS' undeclared (first use in this function) ../src/fs-is-local.h:121: error: 'S_MAGIC_VXFS' undeclared (first use in this function) ../src/fs-is-local.h:122: error: 'S_MAGIC_VZFS' undeclared (first use in this function) ../src/fs-is-local.h:123: error: 'S_MAGIC_WSLFS' undeclared (first use in this function) ../src/fs-is-local.h:124: error: 'S_MAGIC_XENFS' undeclared (first use in this function) ../src/fs-is-local.h:125: error: 'S_MAGIC_XENIX' undeclared (first use in this function) ../src/fs-is-local.h:126: error: 'S_MAGIC_XFS' undeclared (first use in this function) ../src/fs-is-local.h:127: error: 'S_MAGIC_XIAFS' undeclared (first use in this function) ../src/fs-is-local.h:128: error: 'S_MAGIC_ZFS' undeclared (first use in this function) ../src/fs-is-local.h:129: error: 'S_MAGIC_ZSMALLOC' undeclared (first use in this function) make[2]: *** [src/tail.o] Error 1 The Android libc, Bionic, does not define any of these S_MAGIC_* symbols or macros, even in the newest version [1]. Can some #ifdef be used to avoid this build failure? 'defined __ANDROID__' tests for Android. 'defined __linux__' tests for Linux excluding Android. Bruno [1] https://android.googlesource.com/platform/bionic/
bug#8960: stdbuf on bi-arch systems
Hi Assaf, > >> The program 'stdbuf' on bi-arch x86 / x86_64 systems cannot work on all > >> kinds > >> of programs. > [...] > >> I would like to have a single binary that works on both x86 and x86_64 > >> programs. > [...] > > if stdbuf sets both LD_PRELOAD_32 and LD_PRELOAD_64 to the > > appropriate libstdbuf.so, it should just work. > > It's been more than 7 years since last comments/progress on this issue. > Is it still relevant / needed ? Yes, it is still relevant: * Bi-arch systems are still frequent. Large packages are still often provided as 32-bit binaries, from some vendors. * The behaviour of stdbuf in coreutils-8.30 is still the same: $ /arch/x86-linux/gnu/bin/stdbuf -o 0 /bin/pwd ERROR: ld.so: object '/arch/x86-linux/gnu/libexec/coreutils/libstdbuf.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored. /home/bruno $ /arch/x86_64-linux/gnu/bin/stdbuf -o 0 /arch/x86-linux/gnu/bin/pwd ERROR: ld.so: object '/arch/x86_64-linux/gnu/libexec/coreutils/libstdbuf.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored. /home/bruno * The distributor (Ubuntu 16.04 in my case) has not fixed the problem either: $ /usr/bin/stdbuf --version stdbuf (GNU coreutils) 8.25 ... $ /usr/bin/stdbuf -o 0 /arch/x86-linux/gnu/bin/pwd ERROR: ld.so: object '/usr/lib/x86_64-linux-gnu/coreutils/libstdbuf.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored. /home/bruno Bruno
bug#7073: threadlib vs. pthread modules
> Can this bug be closed? > (at least - I believe newer coreutils builds fine on newer Mac OS X) Yes. In my understanding, there is no issue between coreutils and gnulib's threadlib and pthread modules any more. Bruno
bug#6816: df bug on 64-bit Solaris (need to use getextmntent)
Hello Assaf, > > 2018-10-12 Bruno Haible > > > > mountlist: Improve support for Solaris in 64-bit mode. > > Reported by David Wood in > > <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=6816>. > > * m4/ls-mntd-fs.m4 (gl_LIST_MOUNTED_FILE_SYSTEMS): On Solaris 8 or > > newer, define MOUNTED_GETEXTMNTENT instead of MOUNTED_GETMNTENT2. > > * lib/mountlist.c: Add code for MOUNTED_GETEXTMNTENT case. > > Did this solve the issue (any reports from Solaris users)? > can this bug be closed? There was no "how-to-reproduce" recipe, nor did I receive feedback from the reporter. But in my understanding the issue is fixed in the proper way. Therefore yes, it can be closed. Bruno
bug#6816: df bug on 64-bit Solaris (need to use getextmntent)
David Wood wrote:
> >> At this point, me->me_dev contains a wrongly packed (32-bit) device
> >> number, which forces the find_mount_point() code path (causing other
> >> unpleasantries). The following patch against coreutils v8.5 fixes the
> >> problem:
This problem description was to the point, but I needed a bit of time to
understand the issue entirely.
On Solaris, gnulib's mountlist module proceeds by reading the /etc/mnttab
file.
https://docs.oracle.com/cd/E86824_01/html/E54775/mnttab-4.html says:
"The mnttab file system provides the previously undocumented dev=xxx option
in the option string for each mounted file system. This is provided for
legacy applications that might have been using the dev=information option.
Using dev=option in applications is strongly discouraged. The device number
string represents a 32-bit quantity and might not contain correct
information in 64-bit environments.
Applications requiring device number information for mounted file systems
should use the getextmntent(3C) interface, which functions properly in
either 32- or 64-bit environments."
The 'stat' program displays a dev_t. For example, for '/':
A 32-bit 'stat': 4750002 = (0x11d << 18) + (0x10002 << 0)
A 64-bit 'stat': 11d00010002 = (0x11d << 32) + (0x10002 << 0)
So, device numbers in a 32-bit program and in a 64-bit program are
different!
Additionally, reading /etc/mnttab produces the same(!) result when
done by a 64-bit program as by a 32-bit program. The approach that
converts the dev=... strings found in /etc/mnttab therefore produces
dev_t values according to 32-bit programs, even in a 64-bit program.
Now comes GNU 'df' which, as David noted, has logic to compare the
two dev_t values:
./src/df.c:1371: me->me_dev = disk_stats.st_dev;
./src/df.c:1388:if (statp->st_dev == me->me_dev
./src/df.c:1394:|| disk_stats.st_dev != me->me_dev)
So, really, we need to avoid the wrongly encoded dev_t values, and
this means to follow the advice from the mnttab.4 man page.
2018-10-12 Bruno Haible
mountlist: Improve support for Solaris in 64-bit mode.
Reported by David Wood in
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=6816>.
* m4/ls-mntd-fs.m4 (gl_LIST_MOUNTED_FILE_SYSTEMS): On Solaris 8 or
newer, define MOUNTED_GETEXTMNTENT instead of MOUNTED_GETMNTENT2.
* lib/mountlist.c: Add code for MOUNTED_GETEXTMNTENT case.
diff --git a/lib/mountlist.c b/lib/mountlist.c
index 970c611..845c348 100644
--- a/lib/mountlist.c
+++ b/lib/mountlist.c
@@ -111,7 +111,11 @@
# include
#endif
-#ifdef MOUNTED_GETMNTENT2 /* Solaris, also (obsolete) SVR4 */
+#ifdef MOUNTED_GETEXTMNTENT /* Solaris >= 8 */
+# include
+#endif
+
+#ifdef MOUNTED_GETMNTENT2 /* Solaris < 8, also (obsolete) SVR4 */
# include
#endif
@@ -918,10 +922,55 @@ read_file_system_list (bool need_fs_type)
}
#endif /* MOUNTED_GETMNTTBL */
-#ifdef MOUNTED_GETMNTENT2 /* Solaris, also (obsolete) SVR4 */
+#ifdef MOUNTED_GETEXTMNTENT /* Solaris >= 8 */
+ {
+struct extmnttab mnt;
+const char *table = MNTTAB;
+FILE *fp;
+int ret;
+
+/* No locking is needed, because the contents of /etc/mnttab is generated
+ by the kernel. */
+
+errno = 0;
+fp = fopen (table, "r");
+if (fp == NULL)
+ ret = errno;
+else
+ {
+while ((ret = getextmntent (fp, , 1)) == 0)
+ {
+me = xmalloc (sizeof *me);
+me->me_devname = xstrdup (mnt.mnt_special);
+me->me_mountdir = xstrdup (mnt.mnt_mountp);
+me->me_mntroot = NULL;
+me->me_type = xstrdup (mnt.mnt_fstype);
+me->me_type_malloced = 1;
+me->me_dummy = MNT_IGNORE () != 0;
+me->me_remote = ME_REMOTE (me->me_devname, me->me_type);
+me->me_dev = makedev (mnt.mnt_major, mnt.mnt_minor);
+
+/* Add to the linked list. */
+*mtail = me;
+mtail = >me_next;
+ }
+
+ret = fclose (fp) == EOF ? errno : 0 < ret ? 0 : -1;
+/* Here ret = -1 means success, ret >= 0 means failure. */
+ }
+
+if (0 <= ret)
+ {
+errno = ret;
+goto free_then_fail;
+ }
+ }
+#endif /* MOUNTED_GETMNTTBL */
+
+#ifdef MOUNTED_GETMNTENT2 /* Solaris < 8, also (obsolete) SVR4 */
{
struct mnttab mnt;
-char *table = MNTTAB;
+const char *table = MNTTAB;
FILE *fp;
int ret;
int lockfd = -1;
@@ -979,6 +1028,7 @@ read_file_system_list (bool need_fs_type)
}
ret = fclose (fp) == EOF ? errno : 0 < ret ? 0 : -1;
+/* Here ret = -1 means success, ret >= 0 means failure. */
}
if (0 <= lockfd && close (lockfd) != 0)
diff --git a/m4/ls-mntd-
bug#6816: df bug on 64-bit Solaris (need to use getextmntent)
Hi Assaf, > In 2014 gnulib gained the following commit: > https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=502809019bd2ca3ce3d041d18c35ce9420eedb72 > === > commit 502809019bd2ca3ce3d041d18c35ce9420eedb72 > Author: Ben Walton > Date: Tue Jun 3 23:01:14 2014 +0100 > > mountlist: avoid hasmntopt const type warning on solaris > === > > Which seems to address a similar issue as this bug ( > https://bugs.gnu.org/6816 ). I don't think it's the same bug. Gnulib still does not use getextmntent. > I think recent coreutils build well on 64bit solaris. According to the cited man page, the effect of not using getextmntent is visible at run-time, not at compile-time. > If there are no objections I will close this bug in couple of days. Objection. Bruno
bug#32236: df header corrupted with LANG=zh_TW.UTF-8 on macOS
Paul Eggert wrote:
> my earlier patch
> neglected the possibility that mbrtowc can return 0
I wouldn't see this as a bug: You can assume that mbrtowc returns
0 if and only if the multibyte sequence is a NUL byte - but you had
chosen srcend in such a way that this would not happen in the loop.
> and it incorrectly assumed
> wide control characters always have a single-byte representation.
Oops, you're right. My mistake as well.
The new patch looks good.
This will catch (and replace with '?') U+2028 and U+2029 on glibc systems.
On macOS, it will not do this, because iswcntrl(0x2028) and iswcntrl(0x2029)
is 0 on this system; this is consistent with the fact that the 'Terminal'
program displays these characters as simple spaces. So, no need to override
iswcntrl on macOS.
Bruno
2018-07-27 Bruno Haible
iswcntrl: Mention minor problem on macOS.
* doc/posix-functions/iswcntrl.texi: Mention oddity on macOS.
diff --git a/doc/posix-functions/iswcntrl.texi
b/doc/posix-functions/iswcntrl.texi
index 99eaa0e..44dd034 100644
--- a/doc/posix-functions/iswcntrl.texi
+++ b/doc/posix-functions/iswcntrl.texi
@@ -25,4 +25,8 @@ Portability problems not fixed by Gnulib:
@item
On AIX and Windows platforms, @code{wchar_t} is a 16-bit type and therefore
cannot
accommodate all Unicode characters.
+@item
+This function returns 0 for U+2028 (LINE SEPARATOR) and
+U+2029 (PARAGRAPH SEPARATOR) on some platforms:
+Mac OS X 10.13.
@end itemize
bug#32236: df header corrupted with LANG=zh_TW.UTF-8 on macOS
Paul Eggert wrote: > Revised proposed patch(es) attached. Looks good to me, except for one little thing: > memcpy (dst, src, n); src and dst may overlap. Therefore memmove should be used instead of memcpy. Bruno
bug#32236: df header corrupted with LANG=zh_TW.UTF-8 on macOS
Pádraig Brady wrote: > > This patch is correct (because the characters that you test for in c_iscntrl > > are 0x00..0x1F, 0x7F, which don't occur as second or later byte in a > > multibyte > > character in the EUC-JP, EUC-KR, GB2312, EUC-TW, GB18030, SJIS encodings). > > ... It might be worth mentioning this subtle point in the c_iscntrl() docs? > "Note this identifies all single byte control chars even in multibyte > encodings". Only in the multibyte encodings that are currently in use. We never know what kinds of features or misfeatures new multibyte encodings will come up with: Before GB18030 was introduced, it was a common feature of all multibyte encodings (including SJIS) that ASCII characters in the range 0x00..0x3F never occur as second or later byte in a multibyte character. Well, GB18030 broke this assumption. So, it is dangerous to rely on this property. Therefore I wouldn't like to document it in the c_iscntrl() documentation. Bruno
bug#32236: df header corrupted with LANG=zh_TW.UTF-8 on macOS
Pádraig Brady wrote: > but I did want to only avoid \n etc. that might cause issues for > programs that parsed output from df on a line by line basis. The current code (which uses iscntrl) also catches escape sequences that can cause weird output on the screen, in a terminal emulator. This is good (because it can confuse a human reader as much as a '\n' would confuse a line-by-line parser). Now, this feature currently only works for escape sequence that start with an ASCII escape U+001B. It would be useful also for other control characters to be caught, at least: * escape characters U+009B. * other characters that cause a newline in a terminal emulator: U+2028 and U+2029. For example, in konsole, the escape sequence '\u009bf' repositions the cursor. So the effects of $ mkdir /tmp/`printf 'abc\u009bf'` $ sudo mount -r -t iso9660 -o loop /some/iso/image.iso /tmp/abc* $ df ... /dev/loop019860481986048 0 100% /tmp/abc�f is that 'df' produces an U+FFFD. This is less useful than what it produces for an ASCII escape: $ mkdir /tmp/`printf 'abc\u001b[2J'` $ sudo mount -r -t iso9660 -o loop /some/iso/image.iso /tmp/abc* $ df ... /dev/loop0 692828 692828 0 100% /tmp/abc?[2J Bruno
bug#32236: df header corrupted with LANG=zh_TW.UTF-8 on macOS
Chih-Hsuan Yen wrote: > The `c_iscntrl()` patch also fixes the issue on macOS. Please tell me > if you want me to test other patches, thanks! You could test how it behaves with mount points that contain U+2028 or U+2029 characters. On Linux, I'd test it like this. Hope it's similar on macOS: $ mkdir /tmp/`printf 'abc\u2028def\u2029ghi'` $ sudo mount -r -t iso9660 -o loop /some/iso/image.iso /tmp/abc* $ df ... /dev/loop019860481986048 0 100% /tmp/abc�def�ghi $ ls -ld /tmp/abc* dr-xr-xr-x 4 root root 2048 Nov 19 2014 /tmp/abc?def?ghi Bruno
bug#31439: [PATCH] fts: avoid a memory leak edge case
Kamil Dudka wrote: > Why are you removing fflush (stdout) from the test without any explanation? Yes, fflush(stdout) statements are extremely important if you want to understand/debug test failures on native Windows. Bruno
bug#27640: Bug-report
Regarding https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27640 :
Hello Wolfgang,
What type of system is this? You are saying "Linux From Scratch version 8.0".
What type of libc is it using? There are several competing ones [1].
> > FAIL: test-getlogin
> > ===
> >
> > test-getlogin.c:92: assertion 'strcmp (pwd->pw_name, buf) == 0' failed
> > FAIL test-getlogin (exit status: 134)
Please run the 'test-getlogin' program with ltrace (or, alternatively,
insert printf statements that show what's going on). I get (in a regular
login, or with sudo):
$ ltrace ./test-getlogin
__libc_start_main(0x400b46, 1, 0x7ffef8724a78, 0x403db0
getlogin()
= "bruno"
ttyname(0)
= "/dev/pts/13"
__xstat(1, "/dev/pts/13", 0x7ffef87248f0)
= 0
getpwuid(1000, 0x7ffef87248f0, 0x7ffef87248f0, 0x7f3d50d1abb5)
= 0x7f3d50feadc0
strcmp("bruno", "bruno")
= 0
+++ exited (status 0) +++
What are the results for you?
What else, that affects the operation of getlogin(), ttyname(), getpwuid(),
could be special in your environment?
Bruno
[1] http://www.etalabs.net/compare_libcs.html
bug#20114: tr does not support multibyte characters in the first argument
POSIX [1] specifies that the recognition of characters in 'tr' depends on
the environment variables LANG, etc.
But trying to replace a multibyte character by another character does not
work:
$ echo $LANG
de_DE.UTF-8
$ enspace=`printf '\u2002'`
$ echo -n X${enspace}Y | tr ${enspace} ' ' | od -t x1
000 58 20 20 20 59
005
Expected output would be:
$ echo -n X${enspace}Y | tr ${enspace} ' ' | od -t x1
000 58 20 59
003
With 'sed' it works:
$ echo -n X${enspace}Y | sed -e s/${enspace}/ /g | od -t x1
000 58 20 59
003
Bruno
[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/tr.html
bug#11933: ambiguous stty --help text
Stefano Lattarini wrote: What about the following? [-]parodd set parity (even parity with '-', odd parity otherwise) It's correct, but sounds strange, because it suggests that the main purpose of this option is to get even parity. As always, the first thing one mentions is the most important or intended one. Bruno
bug#11933: ambiguous stty --help text
Hi coreutils maintainers, stty --help produces the output [-]parodd set odd parity (even with `-') When looking at the code and man termios, which says: PARODD If set, then parity for input and output is odd; otherwise even parity is used. it is clear that option '-parodd' means the opposite of option 'parodd'. But the German translator misunderstood this line twice (even after being pointed to the fact that it's ambiguous), as if it was [-]parodd set odd parity (believe it or not, also with `-') Can this ambiguity please be fixed at the root, in the English source code? Here is a proposed patch (apply with patch -p0): 2012-07-13 Bruno Haible br...@clisp.org stty: Clarify meaning of '-parodd'. * src/stty.c (usage): Disambiguate explanation of -parodd. Reported by Michael Stummvoll mich...@stummi.org. --- src/stty.c.orig Fri Jul 13 15:44:17 2012 +++ src/stty.c Fri Jul 13 15:40:47 2012 @@ -583,7 +583,7 @@ [-]hupsend a hangup signal when the last process closes the tty\n\ [-]hupcl same as [-]hup\n\ [-]parenb generate parity bit in output and expect parity bit in input\n\ - [-]parodd set odd parity (even with '-')\n\ + [-]parodd set odd parity (or even parity with '-')\n\ ), stdout); fputs (_(\ \n\
bug#11843: acknowledged by developer (date -s with locale-dependent input: notabug)
Jim Meyering wrote:
+static inline unsigned char to_uchar (char ch) { return ch; }
For the use of 'inline', one needs this too:
--- m4/parse-datetime.m4.orig Wed Jul 4 10:04:43 2012
+++ m4/parse-datetime.m4Wed Jul 4 10:04:36 2012
@@ -1,4 +1,4 @@
-# parse-datetime.m4 serial 19
+# parse-datetime.m4 serial 20
dnl Copyright (C) 2002-2006, 2008-2012 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -32,6 +32,7 @@
dnl Prerequisites of lib/parse-datetime.y.
AC_REQUIRE([gl_BISON])
+ AC_REQUIRE([AC_C_INLINE])
AC_REQUIRE([gl_C_COMPOUND_LITERALS])
AC_STRUCT_TIMEZONE
AC_REQUIRE([gl_CLOCK_TIME])
bug#10305: coreutils-8.14, rm -r fails with EBADF
Joachim Schmitz wrote: Also 2 small fixes for C99 Thanks for these. Indeed, the 'argp' and 'regex' modules use strcasecmp() and should therefore depend 'strcase' (already done) and include strings.h (done through patch below). 2012-06-26 Bruno Haible br...@clisp.org argp, regex: Ensure strcasecmp gets declared. * lib/argp-help.c: Include strings.h. * lib/regex_internal.h: Likewise. Reported and suggested by Joachim Schmitz j...@schmitz-digital.de. --- lib/argp-help.c.origTue Jun 26 19:44:50 2012 +++ lib/argp-help.c Tue Jun 26 19:42:04 2012 @@ -29,6 +29,7 @@ #include stddef.h #include stdlib.h #include string.h +#include strings.h #include assert.h #include stdarg.h #include ctype.h --- lib/regex_internal.h.orig Tue Jun 26 19:44:50 2012 +++ lib/regex_internal.hTue Jun 26 19:42:56 2012 @@ -24,6 +24,7 @@ #include stdio.h #include stdlib.h #include string.h +#include strings.h #include langinfo.h #ifndef _LIBC
bug#10305: strcasecmp in regex
Hi Paul, Shouldn't regex be avoiding strcasecmp entirely? That is, couldn't there be a weird locale that considers the lower-case equivalent of U to be uu, or something weird like that? In such a locale, strcasecmp would not consider U and uu as being equivalent; only mbscasecmp would do this. But you're right: for comparing results of nl_langinfo (CODESET), one should not use a locale dependent comparison. You wouldn't want ISO-8859-9 and iso-8859-9 to be considered as different, just because the locale is Turkish. For this particular case c-strcase seems overkill, so how about the following further patch? diff --git a/lib/regcomp.c b/lib/regcomp.c index 7eb003b..6d5525a 100644 --- a/lib/regcomp.c +++ b/lib/regcomp.c @@ -899,8 +899,10 @@ init_dfa (re_dfa_t *dfa, size_t pat_len) != 0); #else codeset_name = nl_langinfo (CODESET); - if (strcasecmp (codeset_name, UTF-8) == 0 - || strcasecmp (codeset_name, UTF8) == 0) + if ((codeset_name[0] == 'U' || codeset_name[0] == 'u') + (codeset_name[1] == 'T' || codeset_name[1] == 't') + (codeset_name[2] == 'F' || codeset_name[2] == 'f') + strcmp (codeset_name + 3 + (codeset_name[3] == '-'), 8) == 0) dfa-is_utf8 = 1; /* We check exhaustively in the loop below if this charset is a diff --git a/modules/regex b/modules/regex index 5371bab..cfc5d07 100644 --- a/modules/regex +++ b/modules/regex @@ -26,7 +26,6 @@ mbsinit [test $ac_use_included_regex = yes] nl_langinfo [test $ac_use_included_regex = yes] stdbool [test $ac_use_included_regex = yes] stdint [test $ac_use_included_regex = yes] -strcase [test $ac_use_included_regex = yes] wchar [test $ac_use_included_regex = yes] wcrtomb [test $ac_use_included_regex = yes] wctype-h[test $ac_use_included_regex = yes] Looks right to me. Please add to this the removal of strings.h from regex_internal.h, since I had already committed the #include strings.h. Bruno
bug#11115: linux date arithmetic
Eric Blake wrote: 10:38 minute or 12:38 minute is not a time designation I have ever heard in spoken nor written English. True, 'minute' in isolation, without a 'plus one' qualifier, is unusual; but we have to continue to parse it in isolation since scripts may now be relying on it. No, we don't have to support syntaxes that are not documented and that were only accidentally supported. Scripts may rely on documented features, not on undocumented ones. The doc in parse-datetime.texi explains that the literal word 'minute' occurs only as part of a displacement. Thus it has to follow a number. It can not occur after a literal time (10:38 or 12:38) nor after a time with displacement (11:38 +1). IMO spaces are irrelevant in this discussion. It's just that getdate.y is a hairy mess to properly implement that change without breaking other worthwhile constructs I would remove the tMINUTE_UNIT alternative from 'relunit' and move it instead to the rules that invoke 'relunit' and that need this alternative. Bruno
bug#11115: linux date arithmetic
Eric Blake wrote: the parser is faced with an ambiguity between: (11:38 +1) minute 11:38 (+1 minute) What is the first interpretation meant to mean? 10:38 minute or 12:38 minute is not a time designation I have ever heard in spoken nor written English. If you ditch this interpretation, there is no ambiguity. Bruno
bug#8391: chmod setuid setguid bits
Ondrej Vasik cited Paul Eggert: recommend leading '@' for future scripts. This use of '@' in a mode string conflicts with the use of '@' on MacOS X 10.5 and newer to designate extended attributes (like quarantine information on MacOS X 10.7). $ /bin/ls -l /etc/ntp.conf -rw-r--r--@ 1 root wheel 27 Apr 5 2008 /etc/ntp.conf $ /bin/ls -l@ /etc/ntp.conf -rw-r--r--@ 1 root wheel 27 Apr 5 2008 /etc/ntp.conf com.apple.TextEncoding 15 You can see that both in the mode string (-rw-r--r--@) as well as among the '/bin/ls' command-line option, the '@' has the meaning extended attributes. Having GNU coreutils interpret @rw-r--r-- in a completely different way would be counterintuive and confusing. Bruno
bug#8391: chmod setuid setguid bits
Paul Eggert wrote: this use of + does not conflict with input usages like chmod +x foo. It's because this use of '+' is easy to remember. chmod +x means add execution permissions. chmod -x means remove execution permissions. You want a symbol for assign exact permissions. IMO the equals sign is a more well-known symbol for assignment than '@'. So, how about a notation chmod =755 or chmod =rw-r--r-- ? Then there will not only be no conflict with MacOS X. It will also be reasonably easy to remember for the users. Bruno
bug#8391: chmod setuid setguid bits
Ondrej Vasik wrote: Therefore @ sign was chosen based on http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8391#59 ... The choice was pretty random: we can choose some otherwise-unused character, such as '@'. By the same argument one could also choose any of '%' '^' ',' '.' '_' However, you are right that = is more intuitive (using exact mode, when specified as first character) - but using it for that will either change behaviour or confuse people for 'chmod =rwx DIR' usecase, if there are special bits on directory. Then how about using == or := to designate the assignment? chmod ==755 DIR or chmod :=755 DIR Both somehow remind assignment. The == choice also is similar with chmod 00755 DIR Bruno
bug#10639: BUG REPORT coreutils-8.15 Solaris 10 64bit
Paul Eggert wrote:
Thanks, can you please try the attached patch?
I find it overkill to change the code for HP-UX and NonStop systems
when the report is about Solaris. Also I think the structure of the loop
is not the problem; it is the code's reaction to
acl(x, ACE_GETACL, 4, 0x3432A16E0)Err#-1
Please, can you also try this patch?
2012-02-01 Bruno Haible br...@clisp.org
* lib/file-has-acl.c (file_has_acl) [Solaris]: Treat a failing
acl()/facl() call for ACE_GETACL like a failing call for ACE_GETACLCNT.
* lib/set-mode-acl.c (qset_acl) [Solaris]: Likewise.
* lib/copy-acl.c (qcopy_acl)[Solaris]: Likewise.
--- lib/copy-acl.c.orig Wed Feb 1 11:15:51 2012
+++ lib/copy-acl.c Wed Feb 1 11:10:55 2012
@@ -235,10 +235,22 @@
return -2;
}
- if ((source_desc != -1
- ? facl (source_desc, ACE_GETACL, ace_count, ace_entries)
- : acl (src_name, ACE_GETACL, ace_count, ace_entries))
- == ace_count)
+ ret = (source_desc != -1
+ ? facl (source_desc, ACE_GETACL, ace_count, ace_entries)
+ : acl (src_name, ACE_GETACL, ace_count, ace_entries);
+ if (ret 0)
+{
+ free (ace_entries);
+ if (errno == ENOSYS || errno == EINVAL)
+{
+ ace_count = 0;
+ ace_entries = NULL;
+ break;
+}
+ else
+return -2;
+}
+ if (ret == ace_count)
break;
/* Huh? The number of ACL entries changed since the last call.
Repeat. */
--- lib/file-has-acl.c.orig Wed Feb 1 11:15:51 2012
+++ lib/file-has-acl.c Wed Feb 1 11:13:17 2012
@@ -588,6 +588,8 @@
for (;;)
{
+int ret;
+
count = acl (name, ACE_GETACLCNT, 0, NULL);
if (count 0)
@@ -618,7 +620,16 @@
errno = ENOMEM;
return -1;
}
-if (acl (name, ACE_GETACL, count, entries) == count)
+ret = acl (name, ACE_GETACL, count, entries);
+if (ret 0)
+ {
+free (entries);
+if (errno == ENOSYS || errno == EINVAL)
+ break;
+else
+ return -1;
+ }
+if (ret == count)
{
if (acl_ace_nontrivial (count, entries))
{
--- lib/set-mode-acl.c.orig Wed Feb 1 11:15:51 2012
+++ lib/set-mode-acl.c Wed Feb 1 11:15:26 2012
@@ -219,6 +219,8 @@
for (;;)
{
+int ret;
+
if (desc != -1)
count = facl (desc, ACE_GETACLCNT, 0, NULL);
else
@@ -234,10 +236,16 @@
errno = ENOMEM;
return -1;
}
-if ((desc != -1
- ? facl (desc, ACE_GETACL, count, entries)
- : acl (name, ACE_GETACL, count, entries))
-== count)
+ret = (desc != -1
+ ? facl (desc, ACE_GETACL, count, entries)
+ : acl (name, ACE_GETACL, count, entries));
+if (ret 0)
+ {
+free (entries);
+convention = -1;
+break;
+ }
+if (ret == count)
{
int i;
bug#9140: [PATCH 2/3] large-inode: New module
Eric Blake wrote: It will probably break any other packages that have picked up the latest gnulib in the last few days. Indeed. It broke libiconv-1.14. But fortunately it's not a regression. Large file support in the 'iconv' program would have been an undocumented improvement. Now it's delayed to libiconv-1.15... Thanks for having discovered and fixed this, Eric!! Bruno -- In memoriam Edward Pimental http://en.wikipedia.org/wiki/Edward_Pimental
bug#8846: coreutils-8.12 on HP-UX 11.31: 3 of 365 tests failed
Hi Jim, I think all issues raised here have been resolved. Can you confirm? I think the du/inaccessible-cwd failure was not tackled. But anyway, feel free to close it, and let's test again when you have a pre-release of coreutils-8.13 that you want to get tested. Bruno -- In memoriam Szczepan Ścibior http://pl.wikipedia.org/wiki/Szczepan_Ścibior http://home.clara.net/clinchy/neeb5.htm
bug#9140: fsusage: add large volume support on AIX
Paul Eggert wrote: Your patches look good to me too. Thanks for the review. I've pushed the 3 patches now. You can add a coreutils/NEWS entry that says: 'df' now supports volumes larger than 4 TiB on MacOS X 10.5 or newer and on AIX 5.2 or newer. Bruno -- In memoriam Ezechiele Ramin http://en.wikipedia.org/wiki/Ezechiele_Ramin
bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Paul Eggert wrote: * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs implementations that use only 32 bits to count blocks. Dealing with the statvfs() / statfs() calls is one thing. But 'df' also relies on the 'mountlist' module, which on MacOS X uses the function getmntinfo(). This function too exists in two variants getmntinfo getmntinfo$INODE64 equivalent to getmntinfo64 and the first one returns 'struct statfs' entities with 32-bit f_blocks. Likewise there is getfsstat getfsstat$INODE64 equivalent to getfsstat64 Here's an excerpt from the manual page: NAME getmntinfo -- get information about mounted file systems ... int getmntinfo(struct statfs **mntbufp, int flags); int getmntinfo64(struct statfs64 **mntbufp, int flags); ... The getmntinfo() function passes its flags argument transparently to getfsstat(2), while the getmntinfo64() function passes its flags argument transparently to getfsstat64(). See [1][2]. So, if the 'mountlist' modules is used outside of coreutils, it also needs the 'largefile' module, so as to map getmntinfo to getmntinfo$INODE64. Here's a proposed patch. [1] http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/getmntinfo.3.html [2] http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man2/getfsstat.2.html 2011-07-24 Bruno Haible br...@clisp.org mountlist: Enable large volume support on MacOS X = 10.5. * m4/ls-mntd-fs.m4 (gl_LIST_MOUNTED_FILE_SYSTEMS): Require AC_SYS_LARGEFILE. * modules/mountlist (Depends-on): Add largefile. --- m4/ls-mntd-fs.m4.orig Sun Jul 24 23:44:01 2011 +++ m4/ls-mntd-fs.m4Sun Jul 24 22:41:08 2011 @@ -1,4 +1,4 @@ -# serial 29 +# serial 30 # How to list mounted file systems. # Copyright (C) 1998-2004, 2006, 2009-2011 Free Software Foundation, Inc. @@ -26,6 +26,12 @@ # gl_LIST_MOUNTED_FILE_SYSTEMS([ACTION-IF-FOUND[, ACTION-IF-NOT-FOUND]]) AC_DEFUN([gl_LIST_MOUNTED_FILE_SYSTEMS], [ +dnl Enable large-file support. This has the effect of changing the size +dnl of field f_blocks in 'struct statfs' from 32 bit to 64 bit on +dnl MacOS X = 10.5 (32-bit mode), allowing file systems = 4 TiB to be +dnl found. +AC_REQUIRE([AC_SYS_LARGEFILE]) + AC_CHECK_FUNCS([listmntent getmntinfo]) AC_CHECK_HEADERS_ONCE([sys/param.h sys/statvfs.h]) --- modules/mountlist.orig Sun Jul 24 23:44:01 2011 +++ modules/mountlist Sun Jul 24 22:30:10 2011 @@ -9,6 +9,7 @@ m4/mountlist.m4 Depends-on: +largefile stdbool stdint strstr-simple -- In memoriam Ezechiele Ramin http://en.wikipedia.org/wiki/Ezechiele_Ramin
bug#9140: Coreutils Bug on OSX 10.7 (Lion)
Paul Eggert wrote: Subject: [PATCH 3/3] fsusage: port to MacOS X 10.7 with 4 TiB file systems * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Reject statvfs implementations that use only 32 bits to count blocks. But this change has also effects on other platforms than MacOS X 10.7: - On glibc/Hurd, the type of f_blocks in 'struct statvfs' is __fsblkcnt64_t or __fsblkcnt_t, depending on __USE_FILE_OFFSET64. So fsusage.m4 ought to require AC_SYS_LARGEFILE. - On MacOS X 10.5, the type of f_blocks is fsblkcnt_t, which is 'unsigned int'. The configure test therefore now fails, STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). And 'struct statfs' contains a f_blocks that has 64 bits if and only if __DARWIN_64_BIT_INO_T is defined. So fsusage.m4 ought to require AC_SYS_LARGEFILE, and a module dependency to 'largefile' is needed, so that it picks up your new AC_SYS_LARGEFILE code. - On FreeBSD 6.4, NetBSD 3.0..5.0, OpenBSD 4.9, the type of f_blocks is fsblkcnt_t, which is already a 64-bit integer. So these platforms continue to use statvfs(). - On AIX 5.1..7.1, the type of f_blocks is fsblkcnt_t, which is 'unsigned long', hence usually 32-bit. The configure test therefore now fails, STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). But statfs() has a 'fsblkcnt_t f_blocks' as well, so the switch from statvfs() to statfs() buys us nothing. What would bring something (on AIX = 5.2) is to use statvfs64 or statfs64, which both have a 64-bit f_blocks field. - On HP-UX 11.00..11.31, the type of f_blocks is fsblkcnt_t, which is 64-bit if and only if _APP32_64BIT_OFF_T is defined. By default it is 32-bit, hence the configure test therefore now fails, STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). But statfs() has an 'int32_t f_blocks', so the switch from statvfs() to statfs() buys us nothing. What would bring something is to define _FILE_OFFSET_BITS to 64, which has the effect of enabling _APP32_64BIT_OFF_T. Or alternatively, define _LARGEFILE64_SOURCE and use statvfs64 instead of statvfs. - On IRIX 6.5, the type of f_blocks is fsblkcnt_t, which is already a 64-bit integer, so these platforms continue to use statvfs(). One could define _LARGEFILE64_SOURCE and use statvfs64, but that appears to be not necessary. - On Solaris 7..10, the type of f_blocks is fsblkcnt_t, which is 64-bit in 64-bit mode builds (_LP64) or if _FILE_OFFSET_BITS is 64. By default it usually is 32-bit, the configure test therefore now fails, STAT_STATFS4 gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). But statfs() has a 'long f_blocks', so the switch from statvfs() to statfs() buys us nothing. What would bring something is to define _FILE_OFFSET_BITS to 64. - On Cygwin (both 1.5.x and 1.7.5), the type of f_blocks is fsblkcnt_t, which is 'unsigned long', that is, 32-bit. The configure test therefore now fails, STAT_STATFS2_BSIZE gets defined to 1, and fsusage.c calls statfs() instead of statvfs(). But statfs() has a 'long f_blocks', so the switch from statvfs() to statfs() buys us nothing. I don't see an API that supports 64-bit f_blocks on Cygwin. - On Interix 3.5, the type of f_blocks is 'unsigned long', which is 32-bit. The configure test therefore now fails, and I don't know which API gets used instead: Interix has no statfs. So it appears the patch is killing portability to Interix. - On BeOS and Haiku, the type of f_blocks is fsblkcnt_t, which is 'long long', i.e. 64-bit, so these platforms continue to use statvfs(). I will therefore propose 1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'. This will make statvfs() use a 64-bit f_blocks on glibc/Hurd, MacOS X = 10.4, HP-UX, Solaris. 2) To undo the patch that introduces check_f_blocks_size, because - on MacOS X 10.7 it is not needed any more after 1), - on AIX and Cywin it causes non-POSIX API to be used for no reason, - it kills portability to Interix 3.5. 3) To make use of statvfs64 on AIX. Bruno -- In memoriam Adam Czerniaków http://en.wikipedia.org/wiki/Adam_Czerniaków
bug#9140: fsusage: add large volume support on glibc/Hurd, HP-UX 11, Solaris, MacOS X
1) To require AC_SYS_LARGEFILE, and a module dependency to 'largefile'. This will make statvfs() use a 64-bit f_blocks on glibc/Hurd, MacOS X = 10.4, HP-UX, Solaris. Here's the first proposed patch. I verified that - on HP-UX 11.00 and 11.31, Solaris 7 and 10, it causes statvfs() to be used again: checking for statvfs function (SVR4)... no becomes checking for statvfs function (SVR4)... yes Since the check_f_blocks_size is in place, it proves that this causes f_blocks to be 64-bit. - on MacOS X 10.5, it causes fsusage.o to call statfs$INODE64 instead of statfs, thus also supporting large volumes. 2011-07-23 Bruno Haible br...@clisp.org fsusage: Enable large volume support on glibc/Hurd, HP-UX, Solaris, MacOS X. * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Require AC_SYS_LARGEFILE. * modules/fsusage (Depends-on): Add largefile. --- m4/fsusage.m4.orig Sat Jul 23 16:25:19 2011 +++ m4/fsusage.m4 Sat Jul 23 16:24:57 2011 @@ -1,4 +1,4 @@ -# serial 28 +# serial 29 # Obtaining file system usage information. # Copyright (C) 1997-1998, 2000-2001, 2003-2011 Free Software Foundation, Inc. @@ -29,6 +29,12 @@ AC_DEFUN([gl_FILE_SYSTEM_USAGE], [ +dnl Enable large-file support. This has the effect of changing the size +dnl of field f_blocks in 'struct statvfs' from 32 bit to 64 bit on +dnl glibc/Hurd, HP-UX 11, Solaris (32-bit mode). It also changes the size +dnl of field f_blocks in 'struct statfs' from 32 bit to 64 bit on +dnl MacOS X = 10.5 (32-bit mode). +AC_REQUIRE([AC_SYS_LARGEFILE]) AC_MSG_NOTICE([checking how to get file system space usage]) ac_fsusage_space=no --- modules/fsusage.origSat Jul 23 16:25:19 2011 +++ modules/fsusage Sat Jul 23 16:06:17 2011 @@ -7,6 +7,7 @@ m4/fsusage.m4 Depends-on: +largefile stdbool stdint full-read [test $gl_cv_fs_space = yes] -- In memoriam Adam Czerniaków http://en.wikipedia.org/wiki/Adam_Czerniaków
bug#9140: fsusage: revert unintended change on AIX, Cygwin, Interix
2) To undo the patch that introduces check_f_blocks_size, because - on MacOS X 10.7 it is not needed any more after 1), - on AIX and Cywin it causes non-POSIX API to be used for no reason, - it kills portability to Interix 3.5. Slight correction: It is still needed after 1), at least on MacOS X 10.5. But only on MacOS X. So here's the proposed patch 2. Verified on AIX and Cygwin; I don't have access to an Interix machine. 2011-07-23 Bruno Haible br...@clisp.org fsusage: Restore previous behaviour on AIX, Cygwin, Interix. * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): Enforce a 64-bit struct statvfs f_blocks field only on MacOS X. *** m4/fsusage.m4.orig Sat Jul 23 16:44:07 2011 --- m4/fsusage.m4 Sat Jul 23 16:42:32 2011 *** *** 62,75 Do not use Tru64's statvfs implementation #endif - #include limits.h #include sys/statvfs.h - /* Reject implementations, such as MacOS X 10.7, where f_blocks is a -32-bit quantity; that commonly limits file systems to 4 TiB, a -ridiculously small limit these days. */ struct statvfs fsd; int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT = 32 ? -1 : 1]; ]], [[statvfs (0, fsd);]])], [fu_cv_sys_stat_statvfs=yes], --- 62,80 Do not use Tru64's statvfs implementation #endif #include sys/statvfs.h struct statvfs fsd; + + #if defined __APPLE__ defined __MACH__ + #include limits.h + /* On MacOS X = 10.5, f_blocks in 'struct statvfs' is a 32-bit quantity; + that commonly limits file systems to 4 TiB. Whereas f_blocks in + 'struct statfs' is a 64-bit type, thanks to the large-file support + that was enabled above. In this case, don't use statvfs(); use statfs() + instead. */ int check_f_blocks_size[sizeof fsd.f_blocks * CHAR_BIT = 32 ? -1 : 1]; + #endif ]], [[statvfs (0, fsd);]])], [fu_cv_sys_stat_statvfs=yes], -- In memoriam Adam Czerniaków http://en.wikipedia.org/wiki/Adam_Czerniaków
bug#9140: fsusage: add large volume support on AIX
3) To make use of statvfs64 on AIX. Here's the proposed patch for it. As expected we get: - On AIX 5.2, 6.1, 7.1: checking whether to use statvfs64... yes - On AIX 5.1, HP-UX, IRIX, Solaris: checking whether to use statvfs64... no 2011-07-23 Bruno Haible br...@clisp.org fsusage: Enable large volume support on AIX = 5.2. * m4/fsusage.m4 (gl_FILE_SYSTEM_USAGE): If 'struct statvfs64' has a larger f_blocks field than 'struct statvfs', define STAT_STATVFS64 instead of STAT_STATVFS. * lib/fsusage.c (get_fs_usage) [STAT_STATVFS64]: Use statvfs64. --- lib/fsusage.c.orig Sat Jul 23 17:15:05 2011 +++ lib/fsusage.c Sat Jul 23 17:13:38 2011 @@ -23,7 +23,7 @@ #include limits.h #include sys/types.h -#if STAT_STATVFS/* POSIX 1003.1-2001 (and later) with XSI */ +#if STAT_STATVFS || STAT_STATVFS64 /* POSIX 1003.1-2001 (and later) with XSI */ # include sys/statvfs.h #else /* Don't include backward-compatibility files unless they're needed. @@ -106,6 +106,18 @@ ? PROPAGATE_ALL_ONES (fsd.f_frsize) : PROPAGATE_ALL_ONES (fsd.f_bsize)); +#elif defined STAT_STATVFS64/* AIX */ + + struct statvfs64 fsd; + + if (statvfs64 (file, fsd) 0) +return -1; + + /* f_frsize isn't guaranteed to be supported. */ + fsp-fsu_blocksize = (fsd.f_frsize +? PROPAGATE_ALL_ONES (fsd.f_frsize) +: PROPAGATE_ALL_ONES (fsd.f_bsize)); + #elif defined STAT_STATFS2_FS_DATA /* Ultrix */ struct fs_data fsd; @@ -223,7 +235,7 @@ #endif -#if (defined STAT_STATVFS \ +#if (defined STAT_STATVFS || defined STAT_STATVFS64 \ || (!defined STAT_STATFS2_FS_DATA !defined STAT_READ_FILSYS)) fsp-fsu_blocks = PROPAGATE_ALL_ONES (fsd.f_blocks); --- m4/fsusage.m4.orig Sat Jul 23 17:15:05 2011 +++ m4/fsusage.m4 Sat Jul 23 17:12:20 2011 @@ -1,4 +1,4 @@ -# serial 29 +# serial 30 # Obtaining file system usage information. # Copyright (C) 1997-1998, 2000-2001, 2003-2011 Free Software Foundation, Inc. @@ -81,8 +81,32 @@ [fu_cv_sys_stat_statvfs=no])]) if test $fu_cv_sys_stat_statvfs = yes; then ac_fsusage_space=yes -AC_DEFINE([STAT_STATVFS], [1], - [ Define if there is a function named statvfs. (SVR4)]) +# AIX = 5.2 has statvfs64 that has a wider f_blocks field than statvfs. +# glibc, HP-UX, IRIX, Solaris have statvfs64 as well, but on these systems +# statvfs with large-file support is already equivalent to statvfs64. +AC_CACHE_CHECK([whether to use statvfs64], + [fu_cv_sys_stat_statvfs64], + [AC_LINK_IFELSE( + [AC_LANG_PROGRAM( +[[#include sys/types.h + #include sys/statvfs.h + struct statvfs64 fsd; + int check_f_blocks_larger_in_statvfs64 +[sizeof (((struct statvfs64 *) 0)-f_blocks) + sizeof (((struct statvfs *) 0)-f_blocks) + ? 1 : -1]; +]], +[[statvfs64 (0, fsd);]])], + [fu_cv_sys_stat_statvfs64=yes], + [fu_cv_sys_stat_statvfs64=no]) + ]) +if test $fu_cv_sys_stat_statvfs64 = yes; then + AC_DEFINE([STAT_STATVFS64], [1], +[ Define if statvfs64 should be preferred over statvfs.]) +else + AC_DEFINE([STAT_STATVFS], [1], +[ Define if there is a function named statvfs. (SVR4)]) +fi fi fi -- In memoriam Adam Czerniaków http://en.wikipedia.org/wiki/Adam_Czerniaków
bug#9141: [PATCH 1/3] extensions: Enable extensions on MacOS X 10.5 and later.
Paul Eggert wrote: Recent versions of MacOS seem to have a _DARWIN_C_SOURCE flag that has roughly the same role that _GNU_SOURCE has for GNU systems. It's not roughly the same. _GNU_SOURCE makes some symbols visible that are not visible by default. Whereas _DARWIN_C_SOURCE makes some symbols visible that are already visible by default but hidden when _POSIX_C_SOURCE is defined. The usual idiom in the MacOS X header files is #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) So, I expect that your change will be a no-op for almost everyone. Bruno -- In memoriam Dmitry Pavlov http://en.wikipedia.org/wiki/Dmitry_Pavlov_(general)
bug#9141: fdatasync module proposal
Hi Paul,
Surely coreutils is not the only program that will have problems
with fdatasync on Mac OS. How about the following gnulib patches?
One is for fdatasync, the other for its tests.
Looks mostly good. Just small comments:
--- a/lib/unistd.in.h
+++ b/lib/unistd.in.h
@@ -483,6 +483,34 @@ _GL_WARN_ON_USE (fchownat, fchownat is not portable -
#endif
+#if @GNULIB_FDATASYNC@
+/* Synchronize data changes to a file.
+ Return 0 if successful, otherwise -1 and errno set.
+ See POSIX:2001 specification
+ http://www.opengroup.org/susv3xsh/fdatasync.html. */
POSIX:2001 is superseded by POSIX:2008. Even if the wording in both standards
is the same for this function, it is good to encourage people to look at and
to refer to the newest standard. So, here I would write
See POSIX:2008 specification
http://www.opengroup.org/onlinepubs/9699919799/functions/fdatasync.html.
*/
diff --git a/m4/fdatasync.m4 b/m4/fdatasync.m4
new file mode 100644
index 000..af17970
--- /dev/null
+++ b/m4/fdatasync.m4
@@ -0,0 +1,34 @@
...
+ else
+gl_saved_libs=$LIBS
+ AC_SEARCH_LIBS([fdatasync], [rt posix4],
+ [test $ac_cv_search_fdatasync = none required ||
+ LIB_FDATASYNC=$ac_cv_search_fdatasync])
+ AC_CHECK_FUNCS([fdatasync])
+LIBS=$gl_saved_libs
Here one could add a comment like:
dnl Solaris = 2.6 has fdatasync() in libposix4.
dnl Solaris 7..10 has it in librt.
Just for reference, because in 5 years nobody would remember.
diff --git a/modules/fdatasync b/modules/fdatasync
new file mode 100644
index 000..94980ec
--- /dev/null
+++ b/modules/fdatasync
...
+fsync [test $HAVE_FDATASYNC = 0]
+unistd
+
+configure.ac:
+gl_FUNC_FDATASYNC
+if test $HAVE_FDATASYNC = 0; then
It is safer (w.r.t. future modifications) and more consistent with the
hundreds of other modules to also test $REPLACE_FDATASYNC here:
[test $HAVE_FDATASYNC = 0 || test $REPLACE_FDATASYNC = 1]
const char *file = test-fsync.txt;
With this definition, the test-fsync and test-fdatasync programs will write
to the same file. That is, when run with make -j2, they may collide.
You need to take a different file name for test-fdatasync.
--- /dev/null
+++ b/tests/test-fdatasync.c
@@ -0,0 +1,2 @@
+#define FSYNC fdatasync
+#include test-fsync.c
diff --git a/tests/test-fsync.c b/tests/test-fsync.c
index 2627d0c..6bac01c 100644
--- a/tests/test-fsync.c
+++ b/tests/test-fsync.c
@@ -18,8 +18,12 @@
#include unistd.h
+#ifndef FSYNC
+# define FSYNC fsync
+#endif
+
#include signature.h
-SIGNATURE_CHECK (fsync, int, (int));
+SIGNATURE_CHECK (FSYNC, int, (int));
#include errno.h
#include fcntl.h
@@ -32,7 +36,7 @@ main (void)
int fd;
const char *file = test-fsync.txt;
- if (fsync (0) != 0)
+ if (FSYNC (0) != 0)
{
ASSERT (errno == EINVAL /* POSIX */
|| errno == ENOTSUP /* seen on MacOS X 10.5 */
@@ -42,7 +46,7 @@ main (void)
fd = open (file, O_WRONLY|O_CREAT|O_TRUNC, 0644);
ASSERT (0 = fd);
ASSERT (write (fd, hello, 5) == 5);
- ASSERT (fsync (fd) == 0);
+ ASSERT (FSYNC (fd) == 0);
ASSERT (close (fd) == 0);
ASSERT (unlink (file) == 0);
Here, like with test-pselect, I would move everything after the signature test
to a separate file, test-fsync.h, that is included by both test-fsync.c and
test-fdatasync.c. This avoids #ifdefs (following the general rule to prefer
C functions over C macros), and gives freedom to add tests that apply to one
of the functions but not both.
Bruno
--
In memoriam Dmitry Pavlov http://en.wikipedia.org/wiki/Dmitry_Pavlov_(general)
bug#9076: coreutils-8.12 uses SA_RESETHAND and SA_RESTART unconditionally
Paul Eggert wrote: --- a/doc/posix-functions/sigaction.texi +++ b/doc/posix-functions/sigaction.texi A documentation update for the header file, in doc/posix-headers/signal.texi, would also be useful. Bruno -- In memoriam Natalya Estemirova http://en.wikipedia.org/wiki/Natalya_Estemirova
bug#8961: stdbuf has no effect on some programs
Pádraig Brady wrote: +Disabling buffering for input will not influence the responsiveness +or blocking behavior of the stream input functions. OK, but then what is the effect of -i0? It has only an effect on programs that use stdio. On these programs, it has no effect on the interface between the program and the FILE buffers, only an effect on the interface between the FILE buffers and the kernel. More precisely: - Without -i0, for refilling the buffers, the stdio makes a call to read(0,buf,N) and receives the available n bytes (n = N). - With -i0, the stdio makes repeated calls to read(0,..,1) and receives the available n bytes one after the other. The only change in behaviour that I can see in such programs is that its throughput is reduced, from 35 MB/sec to 2 MB/sec in my tests. So, -i0 has no useful effect on any program. I therefore suggest that you remove this option. Likewise for -o0. Because if you keep it, other users will - like me - spend time, trying to make use of this option. When in fact they are always useless. Bruno -- In memoriam Yuri Shchekochikhin http://en.wikipedia.org/wiki/Yuri_Shchekochikhin
bug#8961: stdbuf has no effect on some programs
I wrote: Likewise for -o0. Oops, please forget the statement about -o0. With the modified GNU libiconv 'iconv' program, -o0 does make a difference: $ (echo -n abc; sleep 1; echo def) | iconv vs. $ (echo -n abc; sleep 1; echo def) | stdbuf -o0 iconv In the first case, output is line buffered. In the second case, abc occurs immediately. Bruno -- In memoriam Yuri Shchekochikhin http://en.wikipedia.org/wiki/Yuri_Shchekochikhin
bug#8961: stdbuf has no effect on some programs
Hi Pádraig, Essentially, setting non buffered input is to limit what the app reads (so that a subsequent app may further process stdin), rather than changing the responsiveness to input. Oh, really? This ought to be documented and emphasized. I was under the impression - because stdbuf -i 0 translates into a call to setvbuf (stdin, ..., _IONBF) and because the man page of setvbuf says When an output stream is unbuffered, information appears on the destination file or terminal as soon as written - that asking for no buffering would also imply immediate responsiveness. Can you name a single program on which -i0 works? Programs that use getline() for example (like sed). They will change to reading 1 char at a time, as will programs that use [f]getc(). I see. So in summary we can say: - stdbuf works only on programs that use stdio, because setvbuf calls have no effect on the behaviour of read() and write(), - If a program only calls [f]getc and processes input immediately, then stdbuf -i 0 will have the desired effect (unbuffered input, implying no blocking). - If a program only calls [f]getc or getline() and processes input immediately, then stdbuf -i L will have the desired effect (line-buffered input, implying no blocking after a line ends). - If a program calls fread() of an entire block, or has logic to call [f]getc() until a fixed-size buffer is filled, then stdbuf -i 0 and stdbuf -i L will have no effect. Is that right? Bruno -- In memoriam Karl Gröger http://en.wikipedia.org/wiki/Karl_Gröger
